Gesture Recognition with Line Integrals

Gesture Recognition with Line Integrals(justinmeiners.github.io)

51 points by justinmeiners 5 years ago | 16 comments

mgraczyk 5 years ago |

Not sure what is intended, but it seems to have some bugs?

https://assets.opentoken.com/sha256/N1ShZ7kcVmH_-auZEGaGOS_b...

Ah nevermind, it's matching the curve against template fields so it makes sense that it would show the best match even when the match is poor.

justinmeiners 5 years ago | |

> best match even when the match is poor.

Yeah, a real application could have a score threshold.

rg2004 5 years ago |

I created a method similar to this one for curve extrapolation based on sample curves. I took the second derivative of each point on the training data, added it's contributions to a vector field.

To extrapolate, you can take 'position' and 'velocity' of live data and integrate over the vector field to produce an extrapolation. I enjoyed the project and it worked fairly well. I think there's plenty of room for extension around this method.

zamadatix 5 years ago |

It was ridiculously hard to get 4 to work, is this a limitation of the algorithm or just something with the implementation?

justinmeiners 5 years ago | |

How do you draw your 4? The one I used is the "one stroke" 4 where you draw it from left, top, and then down.

zamadatix 5 years ago | | |

I quickly realized variations of:

    │  │
    └--┤
       │

Weren't going to work so I switched to the style in the font:

     ╱│
    ∠--┼
       │

And it really liked to give 7, 6, or 2 if I went in the left,upRight,down pattern.

I think I found out the trick though: if you make a sharp 4 like the font you get random results but if you just make smooth loop of it (almost looks like a tilted roller coaster) it gets it every time.

agency 5 years ago | | |

I draw mine like a capital H minus the lower left segment and I gave up trying to get it to recognize it. It kept coming up 7 or 9

superkuh 5 years ago | |

Even went I deleted the preset glyph patterns and spent 5 minutes re-training it for 1-9 my training pattern for '4' still didn't work. I tried drawing it 2 different ways. It's neat that you can train it so quickly with the hold-shift feature though.

DennisP 5 years ago |

Cool but how is the math used to recognize gestures?

gnramires 5 years ago | |

It appears to be a dot product of a vector field and the curve vector (i.e. a line integral).

I think this approach can't handle certain cases where a path is retraced in opposite directions, like with the handwritten letter 't' or 'i'. Perhaps curvature integrals could be added as well to improve it.

Overall seems to work quite well!

vecter 5 years ago | | |

I think it works quite poorly: https://imgur.com/a/oDpVkZ6

justinmeiners 5 years ago | | |

> can't handle certain cases where a path is retraced in opposite directions

It does ok if there is only a small part that retraces, but you're exactly right.

justinmeiners 5 years ago | |

This animation might help:

https://en.wikipedia.org/wiki/Line_integral#/media/File:Line...