Ask HN: Why Haven't Tech Giants Integrated LLMs in Their Voice Assistants, Yet?

2 points by 0asa 2 years ago | 6 comments

Voice assistants could become much more useful and capable of handling complex tasks, which could revolutionize the way we interact with smart devices like the HomePod. It seems like a logical step for them to integrate LLMs into their voice assistants. It would make voice assistants an even more integral part of our daily lives.

Do you think we'll see LLMs integrated into voice assistants in the near future? Are there technical limitations or concerns around privacy and security that need to be addressed? What's preventing them from doing it now? What might be holding them back?

didntcheck 2 years ago |

I imagine that part of the problem is that so far LLMs have primarily been "language in; language out". They are almost always used with the input and output wired up to a human, whereas for voice control (I.e. not just answering questions) you need it to produce structured output for the computer behind it to understand. There certainly has been some success in that area, like the article about JSON from GPT4 posted yesterday [1]

I'm quite certain that Amazon and the others will be putting a lot of time into investigating this avenue, it's just not ready for rollout yet. I'm looking forward to it - I have an Echo and it definitely does use a lot of fuzzy matching to understand basic rewording of queries, but it's still very much a limit set of defined "functions", just with a fuzzy call site. The user has to handle the breaking down of their high-level intent into small requests from those primitives, and keep all the "variables" in their head and pass them to each "function call". Smarter assistants that can remember context is something that's been worked on and allegedly deployed almost since the start, but ChatGPT is the only thing I've seen that lives up to the promise in the real world

[1] https://news.ycombinator.com/item?id=36330972

senttoschool 2 years ago | |

I'd imagine that it's fairly straightforward for an advanced LLM to produce structured instructions from the user's requests? As long as the LLM knows what the available commands are.

0asa 2 years ago |

https://www.cnn.com/2023/06/15/business/mercedes-benz-chatgp...

senttoschool 2 years ago |

>What might be holding them back?

Probably because LLMs hallucinate and often answer with the wrong information. Can you imagine Siri spitting out wrong information, causing some accident or health risk? Apple's pristine PR image would never allow something like that. Quite frankly, I'm surprised that Google and Microsoft are willing to take that risk.

I totally get why OpenAI said "f it, let's just ignore hallucination and accuracy" that only affects 5% of queries. OpenAI does not have an established worldwide brand.

But could you imagine the news stories if Apple released ChatGPT?

0asa 2 years ago | |

I can imagine conversing with a voice assistant if it could process complex language queries, rather than the current restricted tasks (e.g., setting alarms, playing music, checking the weather, turning on lights).

Research in this area is flourishing, so it is certain that LLMs will improve. The popularity of LLMs is partly due to the fact that it has become natural to interact with an "intelligent" computer program. ChatGPT makes this natural, and combining a smart speaker with LLMs would make it even more so.

Enabling a "closed" conversation mode initially, similar to what the free version of ChatGPT does, would be harmless unless people take language model-generated words as absolute truth. This mode could be used to learn new things while cooking and save long conversations to a notes app for later reference. Wouldn't this be an attractive feature for buyers?

jrepinc 2 years ago |

Additional privacy violations, unless it is all trained and process locally on the assistant itself.