A few months ago I had a silly idea of making a mobile app that used the accelerometer to track the slide motion for playing a virtual trombone. Just wanted to share the story of bringing it to fruition here on hn.

I started out spending a couple days trying to get something cross-platform going in Flutter, but it soon became clear that wasn't the best fit, seeing as I'd need native hooks for most of what the app needed to do, and it wasn't yet clear it'd be possible at all. So I switched to making it an iOS app in Swift. The accelerometer data turned out to be not nearly accurate enough to do the job, so I switched to using the camera/AR using ARKit... and it worked instantly. Like the very first time I hooked up a slider UI element to the distance function. It felt a bit like magic. And also just ridiculous.

Here's a video I recorded to send to some friends at the time: https://youtu.be/6BIogfGH3IQ

Here's a video of it in action in it's current state: https://youtube.com/shorts/8kS2TRzV4I4?feature=share

Apologies for the non-responsive website (using nextjs on cloudflare). It doesn't look great on mobile, which is kind of inexcusable, I'm working on it: https://www.tromphoneapp.com

Anyway, I'm not sure where I'll take it from here. I have some ideas for more AR content like hats, heart eyes, etc. Possibly a song editor so users can add songs that might have issues with copyright if I included them in the app? Any ideas you guys have would be fun to hear.