A paper submitted to ACM in 2019 reviewed the privacy implications of accelerometer data as of that time.
Behavior tracking can go beyond the traditional step counter people think of when it comes to behavioral analysis. Eating, drinking, and smoking, can be determined from wrist-mounted accelerometers. Further, gait can be inferred and level of intoxication can be interpreted. This can also give a guess on how heavy of a load a person is carrying.
Taking all of this a bit further, you can even determine what is being said or written via these devices. Determining specifc spoken “hotwords” could be done via accelerometer alone. It’s also possible to figure out what someone has typed on their phone’s virtual keyboard without actually having access to the keyboard through only the movements of the phone.
Inference beyond this can be done to get fuzzier understanding of who someone is without actually know who they are. Someone’s physical activity and timing can infer someone’s socioeconomic status, openness, and extraversion. It can be used to understand someone’s mood, their relationships with others, and overall stress levels. It can also be used to determine age and gender based on gait, movement parameters, and activity.
The short version, though? We really need to enforce the privacy related to wearable devices. It’s an imperative for our safety.