Recognising gym excerises in real-time using a neural network(dilpreetsingh.me) |
Recognising gym excerises in real-time using a neural network(dilpreetsingh.me) |
Gyroscope/accelerometer data might seem like the kind of extremely-noisy, high-dimensional data well-suited for neural nets, but 1) you don't have a lot of data, and 2) intuitively the data isn't actually that noisy; my guess is the data separates out quite nicely (not necessarily linearly).
If I had tons of data (a lot of different people using it), I'd experiment with LSTM neural networks because I think the temporal information is crucial for determining a movement.
I think a killer app would be aimed at weight lifters. If you go to the gym, the serious weight lifters record their reps and weight for each exercise. The app would utilize the iWatch or some other wearable and detect the exercise and count the reps. Then it would prompt the user for how much weight they used.
The repetition counting in my next aim. Once apple opens up the gyro on the Watch, I think there could be a good chance to get something like this out the door.
The system is designed to be personalised. So for example if it's not doing well in recognising your squats for instance, you can immediately 'show' it what they look like. A neural network can immediately integrate that, and spit out great accuracy levels. Whilst i'd have to reconstruct a decision tree each time.
--
Rather than using an accelerometer (OP attached an old iPhone to his person while doing the movements) - you could plausibly point a camera and try to use video as the input.
If that pans out the next step would be using Mechanical Turk to cheaply and quickly build the initial training set (no pun intended) using publicly available videos of people working out.
If we instead simply used video information to track exercises the problem would be scaling that to consumers. They'd require an external camera to watch them.
But the idea of using Mechanical Turk is quite smart. It'd help me get varied form - especially if I can get them to wear a band/phone that has the sensors.
Ah, now I understand why it took so long for someone to do this! ;)
Also, identifying good form--that's an app that could help a lot of amateur athletes.
I can find apps for recording training data, but they all use their own formats. Anyway, if there is one it would be cool if this program could use it as output, instead of making up its own ad-hoc format.
www.focusmotion.io
It works with any open accelerometer (Apple Watch, Android Wear, Microsoft Band, Pebble). We track 50 exercises, auto classify 22, and there's a machine learning tool that let's you add new movements to the system and improve it over time.
www.focusmotion.io
Neural networks aren't really core part of the system. As long as I can get good + quick results, I'm really open to swapping out the learning algorithm.