Show HN: Babytalk: Offline speech to text and text to speech on ESP32

Show HN: Babytalk: Offline speech to text and text to speech on ESP32(github.com)

7 points by tlack 5 hours ago | 4 comments

I have been playing with a lot of $20 devices with no keyboards or screens. I wanted to use speech to interact with them, but without it having to run through a cloud service or use a fixed command vocabulary.

I worked with Claude to implement high performance 4bit int kernels for the ESP32S3 and P4 and quantize some larger STT and TTS models. Surprisingly, it's quite usable on these tiny boards.

Also includes a finetuned model that can better cope with my diesel drones, hacking coughs, and mumbling. Works with Micropython (RAM scarcity issues) and AtomVM (better).

atmanactive 5 hours ago |

I was hoping this could be a hardware STT-to-HID, but, even after looking at the Git ReadMe, I still don't understand how is the text supposed to exit the device?

tlack 5 hours ago | |

It's a library for C / Micropython / AtomVM, so you could send the result over a lora radio, ESPNOW, WAN, match it in a switch statement to activate a relay, draw on the LCD.. that's up to you.

Many ESP32S3 dev boards have a microphone and speaker header available conveniently on the board, so I have been using that to respond via Babytalk's text to speech capability.

I'll improve the README to make it a little more clear.

atmanactive 5 hours ago | | |

Thank you for the explaination. So it is a library and would require someone to adjust it to their use case. That's what got me confused: wide open use case, but nothing tangible. I mean, if an ESP device is doing STT+TTS, then what am I supposed to be talking to? But if it is an ESP device that can do ANY->TTS and/or STT->ANY with some code customization needed, then I think I get it.