Hi HN, I’m Carl Morris. After 10 years as a data scientist at Amazon, JD.com, and Stitch Fix, I decided to build my own product. That journey eventually led to Callio, a real-time AI call coach for iOS. I am working together with two others to build my startup.

It took about 9 months of trial and error to get to the right idea, and 3 months more to build the first version of Callio. I’d love your feedback!

# What does Callio do?

Callio is a VoIP softphone with an AI assistant that listens to your calls in real time and suggests what to say next. We believe it is especially helpful for sales calls, and that is who we are marketing it to right now.

You can customize the AI suggetions by:

- Uploading documents (e.g. product sheets, pricing, objection handling guides)

- Setting your sales goals and common Q&A

You can also practice with simulated calls before real calls. The AI uses your materials to generate relevant suggestions that are usable verbatim.

# How does it work?

Connecting an LLM to a call in real-time was hard. We leveraged a lot of off-the-shelf technologies, but wiring them all together was difficult and took some thoughtful effort.

- WebRTC through Twilio facilitates the VOIP call experience via their React Native library.

- WebSockets enable us to stream audio to our backend and the recommendations to the mobile client.

- The LLM gets live transcripts and returns recommendations as the call progresses. We are, in principle, agnostic to the LLM provider, though we've built around OpenAI's APIs to start.

- We built an incremental recommendation workflow to ensure low latency for the AI suggestions despite the latency involved in the REST API.

The architecture is intentionally lean:

- React Native mobile app

- Media server + data/signal server

- FastAPI (initially), but we migrated parts to Sanic for performance

We can certainly do better. Our backend database and some of our data APIs are unnecessarily tricky to scale. Now that we're live on the APP store, we're rebuilding carefully to enable us to scale. Getting real-time to work well (esp. async and latency) was a huge learning curve. Happy to dive deeper if you’re curious.

# What’s next?

The app is live, and we’re marketing to attract more users. Our focus now is:

- Growing beyond our initial beta testers

- Improving the recommendation quality and in-app experience

- Building better onboarding for non-technical users

Would love your thoughts or questions, especially around architecture, user experience, or distribution strategy. The app does require a sign-up, which I know makes it harder for you to try out. Luckily, we have a free trial that you can use.

Thanks in advance for all your feedback!