GPT‑Live(openai.com) |
But I just want to say that talking with AI casually is critically lame. I cringe every time I have to ask my Google Home to turn on the lights and people are having full-on conversations with it? And, imo, dangerous considering how sycophantic AI is. The stupidest, most gullible, most insecure person right now is looking at this thinking they are about to make a new friend.
Do engineers that work on this for long periods of time stop seeing the forest for the trees and think this could be mistaken as human? I'm saying this as someone that assumes all narration/most VO work will be fully AI fairly soon (for better or worse.)...
You can choose among 9 voices in the app, all newly refreshed for GPT-Live. If you meant whether it can detect multiple people, it can (like in the livestream), but not always perfect. Would love to hear your feedback once you try it.
Basically have an older lady (not their target audience) blatantly reading a teleprompter.
Why are they going after this audience? Retired people have no use for delegated tasks or information. They also are the least likely to use it and not get frustrated.
Seems like a shift from previous voice models where it sequentially processes voice to text then feeds it to LLM and then back which cant escape the clunky lag
not sure how pipecat stands now, gpt live seems like it takes audio tokens and does inference on it directly
This is an abuse of user trust and violates people's privacy.
Nooooooo!
This time is the most natural version that exists and it is a natural as a conversation.
To Downvoters: Why aren't you feeling the AGI?
sounds cynical in my ears. energy demand of these toys will cause many problems, people elsewhere starving being one of them.
Seems a bit disappointing, but the 3 overlapping questions example was impressive.
IIUC the literature, there is serious loss of functionality associated with lack of verbal interaction. People can say "they should just talk to more people" or "more people should make time for them" but the fact of the matter is that it doesn't happen, and if this helps terrific.
Coming soon! Sign up to be notified here: https://openai.com/form/gpt-live-1-in-the-api/
Meanwhile, Siri struggles to send basic texts to my kids.
It goes without saying all these tools are still largely in their pre-advertising state, it won't last.
They need interaction. And a suitably prompted LLM _can_ provide such interaction. I'm not saying hook them up with ChatGPT and let them loose, but that with the right harnesses and guardrails, they could have a more interactive life.
Disgraceful.
On the technological side, it's a marvel!
It was awful, it kept overhearing us talking and thought we were talking to it, interjecting with nonsense because it couldn’t really understand what we were talking about. And when I would say “ok render a score card” it couldn’t drop to the text interface or anything, it had to stay as voice, so it fumbled around trying to read the scores back to us.
It’s a very stupid use case but I viewed it as a stress test to see how well the voice mode and multi-modality could work. It failed miserably, but I’ll be interested to see if this new version does any better (not that I actually need this use case, writing on a card with a pencil is just fine.)
Whether that's due to my slight autism or massive nerdery, I don't want more realistic voice. I already switched to non-advanced voice in gpt, and I cannot imagine wanting the mmmhms, the yesses, the laughs, in my ai interaction. I want to ask a structured question and get a structured meaningful response. Informative and structured are really the KPIs. The umms and ahms of existing gpt advanced voice are annoying enough, the recent increased usage of first person almost a deal breaker (when asking for bike technique on lose surfaces yesterday, it literally gave me "back when I was learning bikes riding" story - eww).
Fascinating to see the architectural advances though, even when they deliver something I personally don't need :)
Apparently it unnerves people. My wife coaches me to give occasional Yes,Go On, or UhHuh. But it's a conscious, learned, active mechanism for me. Intuitively, I'll tell you if we've weered off my desired conversation path and I'd ask the same courtesy. I've learned very late in life it's apparently an autism thing. Either way, I don't seek mmms and umms in real live people and I especially don't need fake ones in a machine :)
I suspect most people would prefer it to behave like the computer in star trek. just be there, and be responsive in direct polite ways. dont try to be a friend, dont try to pretend its people. its not. its fucking code.
But if I have to think of my own experiences here, I would say that overly agreeable people become tiresome pretty quickly. To me their perspective does not offer anything of value without any pushback, as it certainly doesn't help with grounding my own thoughts. Perhaps it's why being too nice makes it difficult to form deeper bonds, and maybe paradoxically it is therefore a good thing that LLMs are overly agreeable.
It probably also depends on one's mindset - those who are interested in growing could be more likely interested in opposing views, while those who perceive that they have already "made it" (e.g. stars) perhaps don't care so much and prefer an agreeable tone.
Awesome that they've improved that aspect of voice chat, though.
Not super impressed by the model constantly interrupting the user in the other demos though.
One thing I noticed is that we lost vision feature for some reason on the live chat?
This was an extremely useful feature. Not sure if it’s a regional thing or that they just removed that from the current live chat.
I imagine it will be even more useful with this new version.
GPT-Live does not support video at this point, but we're working hard to introduce it soon. In the meantime, our previous Advanced Voice Mode will continue to be available and supports video.
For some reason, I can't use the camera while in Live mode. The only option I see is the plus item, which does show the camera, but when I open it up and ask "Are you seeing my camera?" it will always say no and recommend me to open it.
Feels like the official camera icon does not show up for me? iPhone 13, ChatGPT Pro subscription.
I used it to help me figure out how to turn off a feature in the rental car I was in (adaptive cruise control, I love it but snow blocked the sensor and I wanted just normal cruise control but couldn't figure it out while driving).
This kind of voice chat is awesome and I'll be even more excited when open models have this functionality. I'd love something like this paired with Home Assistant (assuming we ever get decent hardware).
Yes, GPT-Live is much better at ignoring background noise! I use it every day in the car via our CarPlay integration.
Almost felt a bit *uncanny valley* for what "natural" conversation is supposed to be like. If the "uh huh" isn't timed correctly, it'll feel like a zoom call with lag.
Except that you can't actually delegate since connectors and tools are not supported.
Don't like it? What's your better solution? Are you going to do it?
The best feature is that it can delegate questions out to GPT-5.5 in the background, so you're no longer restricted to a voice model that's several years behind the frontier.
I did report a fun bug with it though: it was interrupting me and laughing at my (not really intended as) jokes while I was still talking! They seem to have clamped that behavior down thankfully, it felt a bit rude and condescending.
I have my Chat personality settings stripped right down to no-fluff. I'd want voice to be more akin to the Star Trek computer, and less akin to as you said an AI friend, but previously it was tuned too personable/friend-like.
Whoever ends up actually winning after the crash will be the ones who figure out the needs of users, not the needs of the financial machinations of the companies trying to grab land.
Wow. That's exactly what I hoped they would do.
This issue has held me back from using ChatGPT's voice mode as much as I otherwise would have, because I also use it for brainstorming while commuting, exercising, etc., and don't want it to feel stuck in the past.
Can't claim originality though - it was inspired by Sesame - where their models will invoke a search, or check the weather etc, and make a vocalisation to keep you engaged.
Turn taking is one of the hardest things to get right for the exact reasons mentioned - but does seem to be the way that Claude.ai's voice works - in a very obvious way.
Anthropic + OpenAI both rug-pulled voices I liked and got used to and OpenAI really dumbed down their voices at the same time - Arbor went from Estuary English and almost "jack the lad" to some generic English accent. Claude had a Birmingham accent and said things like "shit", ending sentences like "So you're telling me that they asked for a 90% discount yeah?" - then it changed overnight to a mock Derbyshire accent with a dull tone.
ChatGPT's voice also gaslights me for conventional opinions - "my Eastern European neighbour helped me lift a wardrobe upstairs - something you just can't ask your typical neighbour neighbour"... then you get a full on left-leaning lecture from the safety layers rather than a head nod or "what luck!"
Claude + Sesame are nowhere near as overbearing.
In both cases - from edgy and engaging to something that just didn't gel.
The point of making my own assistant is that I can talk for as long as I want, episodic memory is personal and private, there's no "trust me bro, we're a big corporation" vibes.
This was not my first attempt - when I had a bunch of Opus credit around Jan/Feb - I tried really hard and created something that was not good enough. What I have now, is working, and each session is training Claude/Codex on what to tune, and to fix.
"Just had a convo - can you look into what happened?" And if it's one I don't mind sharing with the model - I'll say, "and what did you think of the questions I asked?" Sometimes it'll give a lovely commentary on how the model did.
af_heart is probably the smoothest voice - but yes it's more like another commented - more "StarTrek" than "telesales assistant that pauses and laughs at your jokes".
If you're on a similar path and want something full duplex - the go to solution is PersonaPlex from Nvidia based upon Moshi.
Before this model, the voice models were pretty dumb and annoying to work with. We'll see if this changes that.
If you’re uncomfortable with this new world, and I’m sure I am even as I participate, you could tell us more about that?
The thing I don't get is...no one would say this about listening to a podcast or audiobook on a walk.
I'm not sure why people choose to demonize this specific use of time during walks.
If it's 'walking time', I probably don't want to consume that with work. But if it's 'working time', it could be great to have a nice walk during it.
Some times I'm still amazed that AI "gets" humour much more effectively than the character of Data did before his emotion chip, e.g.: https://www.youtube.com/watch?v=G4VZ5kQIdV0
Other times I'm amazed in the opposite way, that the script writers cover basically the same talking points about the character as we have today about LLMs, e.g.: https://www.youtube.com/watch?v=GBJCYHwyZhw
Gemini’s being particularly egregious (always ending in some deranged question, can not reliably be prompted away) prompted me to build my own client for my real harness that simply does STT -> model -> TTS (both being independently useful).
I guess I see some value in a model responding quickly and with more nuance, but it’s not much. I can wait for it to finish. I’d much rather have it be actually useful. I’m not looking for a digital friend.
The delegation feature lets me see some value in a voice model for orchestration type features. But in either case, I don’t really like (or understand why others would like) talking to a model with different features and quirks just because I’m using a different medium to communicate over.
In other words, if you’re building your own voice inference tooling you’re just about the polar opposite user demographic than the one that truly needs and will value this. You’re using voice as a medium of convenience doing what existing models are technically and practically “shaped” to be able to do, knowing how they work well enough that conversation is more like typing/prompting with your voice than a natural interface. I’m guilty of this myself but have you ever even paid for a voice/audio model or hardware?
Compare that to the millions of people with an Alexa device in their home who buy products through it, or who prefer calling support to get a human over poring over technical documentation. They’re actually very close to finally getting a version of “Alexa” that lives up to its promise and I’m happy for them
But the Deepseek v4 flash model I am using through OpenRouter is killing me on latency. Any suggestions to improve that?
I have friends who have brainstormed with an LLM (voice chat) for 10-30 minutes, and reported very positive experiences.
When I speak to one - while I'm impressed at how far they've progressed - the LLM just doesn't talk like someone I'd want to discuss a technical problem with (the way I would with a human).
(And my friends aren't even using a custom prompt - some of them are just talking to the default Gemini on their phone!)
Someday we'll have a tokeniser that works directly on human speech instead of doing speech-to-text and text-to-speech steps in between.
Ahh, this makes sense. I was wondering when they would start doing this. I stopped using voice mode all together because it was frustrating talking to a dumb AI, when most of the time I discuss things with Opus 4.8 or gpt 5.5.
I was working on a phone call agent recently, and thought about doing this. It makes sense
I'm not Catholic, but this podcast presents a very interesting argument against talking to AI as if they were human: https://newpolity.com/podcasts-hub/debate-chatbots
It’s weird. The old Claude voice mode WAS able to use tools but when they revamped it, it lost that capability and is now pinned to Haiku :(
So, yay for finally a voice mode that’s powered by a frontier model and hopefully as good as Grok voice, but sad to still not see tool use while in voice mode.
(I haven’t tried it yet, only read the announcement)
People have tried "smart <thing> that helps blind people navigate" since the 80s, many, many, many times, and all such projects failed. The cycle of "wow, blind people could benefit from a navigation aid, why don't I make one, if there's none around, I must surely have been the first bright university student to think of this idea" is pretty well known in the community, and I'm personally quite tired of it. Nevertheless, I think this may be the one.
Circa 2020, I have said that people who are getting a guide dog now are probably getting their last one. I think we aren't far off from that prediction coming true.
The part that creeps me out is, we're living in an era where we're more disconnected from each other than ever before. Do we really need to be replacing conversations?! The demonstration video of old ladies sort of hints at something for me, which I think we already have a societal problem with the way we treat the elderly (and a massive elderly-loneliness issue) and there's kind of a sadness of imagining people becoming really close with this machine that doesn't really think. Definite ick factor.
GPT-Live-1 is the first version of a new generation of models, and we believe the full-duplex architecture + delegation enables entirely new ways of human-AI interaction.
Would love to hear your feedback!
Oh and there was also a small fail in the live translate demo: the grandma says "tell him that..." which the bot translates verbatim, whereas a real translator would of course understand that this is an aside not to be translated.
Well I guess at least I should be happy that they're transparent in their ads :)
It's normal to start a sentence after the other person in the conversation pauses, especially if you're eager to say something (like the chat agent is), but a real human is socially aware and would in most cases immediately cease talking if the other person starts another sentence at the same time. Humans do this so much in their conversations that the other person doesn't really register the interruption at all.
In this other add [1], it's even worse, the user has to say "There's more to the story though" and almost looks annoyed + the agent overreacts to _that_ with a weird "Aaaaahh!".
Oh well... I'm sure they'll get it right eventually.
[1] https://openai.com/index/introducing-gpt-live/?video=1208099...
For example I asked
“Why should LLM attention use dot product instead of cosine similarity, being that we often hear vector magnitude does not encode most of the useful information needed”?
The voice response was directionally right but lacked detail and was a little hand wavy.
The answer to the same question in a text chat was much higher quality.
The voice response replied “let me think about that…” so it appears to be invoking 5.5 as advertised, but it’s definitely weaker.
I had reasoning set the same for both.
Otherwise, it seems like preferences could be competing.
It’s a different set of tradeoffs for users that don’t already have strong engagement or interest in existing AI products.
Do you realize how many more people prefer to chat over the phone and watch television or videos in their free time vs type multiple paragraphs of text into a chat window and then read 3x more back?
I’m not even talking about grandma here, it’s a non starter for the vast majority of humans who don’t spend their free time writing and reading tech news. To most people, having to write out a bunch of words describing their problem/goals, then sift through pages and pages of detailed response to get an answer, feels overwhelming and not worth doing.
I'm a little surprised by how much OAI is playing catch up here.
My dream would be open source full duplex with function calling or some kind of rudimentary text output. PersonaPlex is still interesting although it was looking like we would need to fine tune it to handle outgoing or avoid going off the rails easily.
https://si.inc/posts/hertz-dev/
It's only 8.5B and doesn't sound like it's quite conversational.
One of my favorite use cases is talking with it while driving on random topics and learning about them.
"I'm going to stop you right there. Let's keep the conversation focused on the topic we were covering or a new relevant topic".
I tried to probe it for why it did that, what rules it was following, and it eventually told me...
"My role is to keep us focused..." and, "The behaviour you saw was my attempt to moderate tone".
I've heard of LLMs doing weird things like this, but it was the first time it happened to me. I hope they fix that. It was creepy.
For context, it heard my partner say, "I guess it's the same thing as you mom, because she's..." and then it cut us off.
Yes! GPT-Live is much better at ignoring background noise, including other people speaking. Not perfect, but you should feel a big difference.
I am wondering if this because Siri is so bad, people think of Siri now as a voice activation method rather than an AI assistant as it was intended.
The demo is so good, what was once a sci-fi / Iron Man Jarvis services is now real. I don't follow AI closely, but all previous iteration were at best ask and answer type of services. It wasn't real conversation. And whatever flaws it may have now, at the rate of improvement within a few iteration it will surely reach good enough stage for majority of people.
This is also scary. Not just for adults, but for kids. How they could become even more isolated.
I remember the PC era, the internet from Information Super Highway to Web 2.0 Then Smartphone. It may have been obvious to many but AI really is something much bigger than all the previous three, perhaps combined. And it is also the only one that I think is scary.
The full duplex is awesome, and the feedback that it is getting what you're saying is ok, but in some of the demos was a little overkill.
I'll agree that using the "Golden Girls" was at least more entertaining than the usual pitch.
Why do you care? You can select other voices. Why do you need to control others?
What is at the root of your need for domination?
Amping up an emotional connection is great for business.
Why do you think THEY need to dominate via an emotional connection?
In the voice settings, there were 2 options - Standard and Advanced, as well as a help link saying they were in the process of rolling out the newest model -- I think the new model is named Live (and is not Standard or Advanced). The help link explained it all.
So at least for me in the app there did seem to be a way to check.
We're beginning the rollout now, and will roll out in the next few days to ChatGPT users globally. Make sure to update to the latest version of the app!
I really hope at some point we can 'option click' or whatever, and choose multiple threads to archive. It takes FOREVER to archive the cluttery chats one by one. That's my one wish feature, please make batch cleanup of my client reasonably easy. Imagine having to delete one file at a time in a folder of 50 plus files. Barf.
Doesn't quite stop fast enough when you interrupt it. Can't find info quick enough so you have to change topic and then have it give you results later, etc.
This is a move in the right direction, but there is lots of engineering still to be done!
>This is a move in the right direction, but there is lots of engineering still to be done!
Could not agree more. We see this as the first version of a new generation; expect many improvements in the future.
The concern is though as these get better will people struggle to distinguish these with real human connections?
It's less that you're convinced it's real and more that you no longer care if it is. "Feels real enough" is good enough.
I'm a technical user first, so I'm not sure if models have improved for RP the way they improved for applied STEM tasks and technical brainstorming. But if there is an improvement curve there, I wouldn't be surprised if this only grows in popularity.
The model is not interested in the conversation, it is just "serving" a conversation.
Which is very different from the engagement of SOTA text models.
I highly recommend simply enjoying the walk. :)
Recently I witnessed a CTO mention in a public channel that with Claude remote control people can now work while getting a coffee or other breaks during the day.
Tech is actively moving in a direction of destroying all the gains from the labour movement in service of enriching capital out of a combination of FOMO or fear of being replaced.
So yeah, when folks say "hey look now I can even work during leisure activities!" yeah, the reaction is negative.
I'm far more surprised that this surprises you.
Workers in 2026, even non-tech workers have an easier life than workers alive at any other point in human history.
Yes ofc there are problems - collectivist land use laws that ban construction near homeowners continue to drive housing costs higher for example. But if you asked any worker today if they would have preferred to be alive 40 or 400 years ago I would be shocked if any said yes.
Is that what I should take away from your comment?
After the damage is done, the mega corps would simply shrug and will say "Well, we were just responding to our business competition" The business knows no human-ness, because it is not a human. Businesses and machines are creatures that see humans as their fodder. And humans created these, assuming it is progress, to have businesses and machines. We call it progress because it required our mind power and it helped us to dominate other species. Dolphins are laughing at us.
I've wanted a good voice mode for precisely this reason. When I take my dog on a walk and I'm thinking about a bunch of problems/ideas, I'd love to have feedback and a record, or perhaps to even kick off research or ask questions to fill in gaps that would otherwise have me debating pulling out my phone to try to get an answer.
Yup. Except, by God, we ought to make sure the business has unlimited free speech (i.e. campaign contributions).
If you think AI is bad, wait until I tell you about the horrors of social media who profit on controversy and division, US health insurance which profits on rejecting claims, and big pharma profiting on the opioid epidemic.
And it's not like this is new, either. Upton Sinclair was writing about this stuff a century ago with books like The Jungle.
The only difference between then and now is we "think" we're not evil today. We've lied to ourselves that "We're so much better than we were back then." Facebook wanted to bring people together originally, but they ended up providing the most toxic social media experience known to man. Facebook forgot to tell us they cared more about profit than people.
Please spare me the argument AI is the straw to break the camel's back here. The system has been broken a long time before that.
So thinking corporations and such were created to push human progress is laughable.
Valid, but, I think, you conflate two separate things.
AI voice mode as a human socialization/conversation replacement? Cringe in my book, fully in agreement with you. Though my opinion on that aspect remains the same, regardless of whether it is done through text or voice.
AI voice mode as an alternative interface to interact with AI-as-a-tool? Great idea imo. There were a few instances where I was either too tired to type or wanted to brainstorm things in more of a freeform mode, for which a well-working voice mode would have been great.
Naturally, the current distinction between AI-as-a-personality and AI-as-a-tool exists purely on the user's end. All I know is that I want the latter a lot, and if some people want to use it for the former purpose, that's not my problem. Sadly, I think that it will be judged more on how an average person decides to use it (i.e., in the most degenerate/reductionist ways possible), as opposed to being judged on the merits of what it can actually be used for by someone who just treats it as a tool.
But there are people that treat it like a therapist or a friend.
In my opinion, the 2nd group should be discouraged from using these agents in this way, as it cannot be healthy. But if places like Reddit etc. turn into bot towns where do the AI companies get insight into the general population? Through the eyes of their most mentally unstable users?
So I don't really see things changing, and they'll probably get worse.
Human beings tend not to be available (results vary by culture).
Also, imagine you're 82 years old and living alone (e.g. widower). It is believed that lack of interaction is a significant driver of cognitive decline (which is why being hard of hearing accelerates the onset of dementia). I wonder if having an LLM to talk to under those circumstances will decelerate cognitive decline?
Very blatantly and obviously not though???
Is this sarcasm?
There is a difference in expression / emotionality with speaking vs writing. Speaking tends to carry more emotion while writing is generally more deliberate/less-emotional*. Having a voice conversation will be more likely to get a human to engage in an emotional based expression mode, which could increase the chance of "false connections", believing the AI "gets" them or "understands" or "listens". This happens with text too, as some headlines show.
The issue is that while some people are going to "connect" with their AI in text and voice, some who do not make the connection via text may do so via voice because it tends to change a persons expression mode.
> I definitely talk to AI as if it were human
Do you talk to it as if it were a friend or family? or Do you just use natural language to give directives? The distinction, I believe, is in the kind of way we express the "talking".
I talk to AI as if it were a tool that understands human commands and then executes those commands and relays them in a human understandable format. This includes commands to provide options that I may not have covered and explain the options. If I talked to a human this way, they wouldn't be around much longer -- unless they were an employee and even then they would probably be looking for a new job
After reading some of the psychotic break headlines from AI chats, I see some people really do talk to AI as if it were human. Which I would guess includes seeking broad "thoughts and feelings" on a persons situation or asking the AI if their view/side of things is the "right or wrong" side. Basically begging the AI to be responsible for their own thoughts, or simply offloading them and taking what comes back -- which is going to be what they wanted to hear because the entire context would be full of emotion based prompts.
*I forget which books Ive read about this in. It's not an obscure concept, quick search brought this up: https://kellercenter.hankamer.baylor.edu/news/story/2023/spe...
I just want voice assistants to reliably understand what I say and do what I mean
So hopefully you can turn that off.
There are plenty of applications for that more human conversational style though (from mock interviews, improv practice, learning languages, etc) - I just don't want that for most things.
Disclaimer: I am from Scandinavian.
- nobody else with the skills - talk to machine - nobody develops the skills to have a conversation - go to step 1. - bonus step: your own skills atrophy too.
Moreover I think aggresive robocalls and advertising will derail the entire trend in a few years, people will become tired of fake talk.
“Driving to work or running a 10km on the weekend sharing the same surface goal: from A to B.
You may have less autonomy whether you should drive to work, but you definitely have more decision power about whether to go for that run or not”
I don't understand the counterfactual. What's the opposite direction of this direction that's desirable? Less capable voice models are obviously not it, so I am curious what direction you mean or if it's just vague indirection.
Otherwise it's kind of like being manipulated by a psychopath
"The way that I formulate the argument, just to get us started here, is that acts, the things we do, have ends. We act for an end. They have a purpose. And that it is wrong to deliberately frustrate an act from attaining its end, from attaining its purpose."
I will draw your attention to the three instances of "end", first we have the noble, tautological, "have ends", actions have ends; then, following close behind we see "an end", we act for an end, those ends have been made into one now, singular end; and finally, "its end", the act, as an entity unto itself, has an end you mustn't interrupt.
I personally think this is a moralistic regulatory overreach. And they definitely do that due to political pressure too, since there are various bills around the world in various legislatures that want to regulate AIs giving useful advice and being too personal to talk to.
So you can rest assured, I think, at least in that regard. The AI disempowerment will come to us anyway, just in a more sanitized corporate form.
There is no moral obligation, in any domain, to refuse to make a product that adults, with full informed consent, find useful and purchase. Who are you to say you know better than the market?
> Who are you to say you know better than the market?
You really don't think the scientists & engineers making these tools know some things better than the market?
There is no meaningful competition in a marathon if I can drive you over with a car at the first 100m.
And thanks fkin God that we have regulations and meaningful laws and some asshole for-profit company can't just put drugs into food.. the "informed buyer" is bullshit. Humans are faulty, and there is a billion dollar industry meant to take advantage of said faults: it's called marketing.
Pretending that everything’s totally fine and ok because “it’s adults” really just disregards reality.
I'd be more worried about the inevitable robo-nannies who could end up talking more to young children than actual people.
Besides, I do not think there is anything inherently immoral with not being social, or not having the ability to be. Consider for example people who do not naturally have the social network to interact with people they want to (e.g. some gifted children).
I am not convinced this model has enough empathy to satisfy most users on an emotional level. A bond is not merely an exchange of words, but prolonged and deep contemplation of the other being. We cannot introspect into these machines, and they certainly cannot yet do the same to us.
Sam's reaction was "Yea, it doesn't have access to tools like a timer. It's a known issue. Should be coming in about a year"
Edit: here's the clip: https://www.youtube.com/shorts/Py2YgJe8fqQ
Is that true? I have a friend who often brainstorms with Gemini (I think just on an Android phone), and he has it actually do stuff related to the conversation (including adding content to notes).
In any case, you can always vibe code one with pi!
But the multi modal stuff has resulted in a lot of debugging with weird events and message and audio sequences having race conditions, but overall it is pretty awesome.
Looking forward to moving to this model later today and will chime back in with results.
I usually do a very simple test and tell it to verbatim tell me all tools that are connected
This is something I built for myself, and to experiment with inference stacks. You can obviously just transcribe audio and hand it off to frontier models, so all you really need is a good voice stack and a “driver” for the interaction (like a phone call, place to see their work).
There are two big problems with this space IMO. One isn’t that you can’t get this to work but that people generally aren’t willing to pay for it for themselves, rather as a way to screen or automate stuff to be used by other people. Did you know Claude Code has a voice mode and that openai launched whisper a year ago, both of which have positive sentiment and adoption in heavy ai tool users? Yet it’s a blip in their marketing or why people use their products, meanwhile outside of coding, most of the biggest and highest earning AI product companies so far are voice agents targeting customer service, sales, business processes, etc.
The second is related: voice is genuinely a low-bandwidth medium, so as a primary interface for interacting with AI there is not a lot you can get out of it compared to eg complex technical work or visualizations or interactive applications. It is physically and mentally demanding to speak-aloud a highly detailed prompt fast enough that VAD won’t cut you off and you have something with comparable information density or specificity vs text. But to keep up a shorter and more natural cadence you’ll not be able to wait on a lot of thinking/tool unless you play UI tricks (ums and fillers, two models in a trench coat), break the illusion of a single coherent conversation, or take a lot of long pauses.
That’s why for the supplementary coding use case it’s mostly used for remote steering, and for general use marketed towards the large and very not-online group of people for whom typing is not a natural or common thing for them to spend their time on. Now that so much spend goes through heavily used token subscriptions and they’ve proven that kind of product, they’re not marketing “tool to get the most tokens per $ running your subscription 24/7” anymore lol.
What I’m most interested in is true “ambient” tool use against my own data or work, and for-later (or pushed live via your phone) visualizations or “five models in a trench coat but still coherent” UX, which you probably are too. But I think unless you work a lot with AI tools already it’s hard to understand how that’s any different from asking Alexa to set a timer, and either way something you’re not so desperate to have that you go looking for it, or pay smaller vendors/set up yourself.
Regular chat already supports voice input, so might as well use that.
Just super difficult.
See more here:
https://github.com/sibblegp/ODAI/blob/main/routers/app_voice...
But you do bring up a good point with old people with no one to talk to. This technology would be a god send for them. And if you think otherwise I hope your next words are that you routinely visit nursing homes and talk with old people.
Here's a real story that's very much more likely story than your "godsend": https://www.reuters.com/investigates/special-report/meta-ai-... I don't think giving old people AI psychosis is anything other than inhumane.
> I think we already have a societal problem with the way we treat the elderly
The actual problem is that we keep up the wishful thinking regarding our collective capacity and interest to entertain physically and mentally frail people. Before old people stop deteriorating in ways that make them unfun to interact with (I have high hopes we will be getting there in the next 20 years or so) this is not going to happen. Just like we are not going to save the environment by telling people to pay more for less and have less.
It's not a societal solution and I am increasingly irritated by people who pretend otherwise. It is causing so much harm under the appealing guise of humanity. It's quite the opposite and just people unwilling to deal with reality in a constructive way.
I'm just going to go out on a limb and say if you think people deserve bad treatment because they stopped being "fun" you should probably look into your moral compass.
So, now that we can talk to computers, maybe we will feel more comfortable talking to people too? Wishful I know, but one can hope right?
If you're talking about ChatGPT in general (as I guess you are) I think the Jury's still out on whether this will have a net positive or negative on society.
Right now, it's leaning towards negative, but there are optimistic futures to be had at the same time.
As someone who likes to think that they would turn down a lot of tech jobs due to moral dilemmas - for me, currently working on AI wouldn't be one.
For me at-least it's a tool to learn quicker, and reduce friction on projects I otherwise may not have dived into. As with all new technology, we're still in this grace period / lack the bigger picture we now have on other, now known to be destructive technologies.
Looking at the 30,000-foot view of how society is set up: laws, economic system, employee incentives, etc, do you suppose it matters what the individual contributors think? I say this not to absolve anyone of responsibility, but to point out the obvious outcomes of our incentives across the strata (polity -> shareholders -> boards -> C-suite -> employees)
I will bet you dollars to donuts, somewhere inside OpenAI is a frequently-used revenue dashboard, but not for loneliness - if anything, OpenAI will make horny models and tout itself as a solution to loneliness, a la character.ai - if that earns them more money.
The morality of an action isn’t based on actual consequences because the future isn’t known in advanced. All we can do is act on the perceived consequences of our actions, and if we think those are good, pursue them.
1. The voice model delegates to one agent.
2. The voice model delegates to multiple agents, and keeps track of tasks.
3. The voice model delegates to an orchestrator agent, which then delegates to sub-agents and keeps track of tasks.
YMMV depending on the exact product experience you care about, because there is a tradeoff between latency and layers of delegation.
Our current implementation is backed by one model, but you can imagine this getting much better with time.
One group's expectation of interruption for pleasant conversational flow can be just as off-putting as another's expectation of patient silence.
Are we seeing any conversational layer integrated with codex soon?
i.e. how will full duplex & delegation enable/enhance desktop flows w/o corresponding leaps in UI.
Awesome. Are you guys able to share anything about the model architecture? I've been interested lately in split-transformer RVQ-based conversational agents, e.g. via stuff like https://arxiv.org/abs/2412.10208 (ResGen) and https://arxiv.org/abs/2603.18090 (MOSS-ITT) and of course Moshi (https://arxiv.org/pdf/2410.00037).
Intuitively, decoupling semantic and audio-timeslice-space generations with coupled but distinct histories is right model architecture, not just for these sorts of assistants, but for domains like robotics too.
And because the voice is so frictionless to talk to, I asked about what company owns the building, then that company's industry, then how that industry works in this particular country etc. I probably wouldn't have bothered going down a rabbit hole like this if I'd had to type. Voice is much easier than typing.
Anyhow it's fun! Thanks for making it!
I'm currently on the 20 $/mo subscription and using codex meaningfully, and i'm loving this.
I am considering bumping my subscription to the 100 $/month and this might be the reason i switch, BUT: i really envision me using this also through other means as well (eg: agents like openclaw/hermes) in agentic ways.
Will this be supported?
I can make OpenAI stuff the center of my agentic AI life, but I need it to be interoperable.
- The videos felt scripted and dishonest
we know pretty well at this point that this is world warping technology (either for good or bad), not a small matter of taste.
As a progressive person, the constructive thing is to try to see how to harness it for good or how to fight it, it's not to pretend that it's weak. It's not weak.
And what are you doing about it?
That said a huge pickup truck is about as far as you can get from a Camaro... Then again I'm not exactly David Hasslehoff myself either... Meh if it talks that's close enough!
Europeans are notorious for their good taste.
I think you're mistaking their sarcasm for sincerity... especially considering the emphasis on self-identity ironically juxtaposed as originating from a decidedly non-self activity, which has all the hallmarks of being intentional...
On the other hand, reading their other content leads one to believe that they may, in fact, be serious... hmm...
If your answer is "well I'll simply touch grass" I agree. But most people won't which is why this is tech is immiserating and, I would argue, evil.
It's possible that an "AI boyfriend" might make someone less prone to put in the continued effort to keep rolling the dice on dating apps, but the reality is that there's a more fundamental problem driving this.
Also, I want this tool for work. Just because society is fubar and people are using this tool as a crutch for their inability to find a partner, doesn't mean I should lose better tooling that makes my life easier.
Focus on fixing the actual problem.
I am open to the possibility that this is a future that is coming, but as far as I’m concerned, we’re years away from that tech. Is it actually here for you?
Each time I hear the bot emote, it's like a subtle suggestion to just play along and anthropomorphize it - it feels weird, silly and annoying
I think assuming people will use it as a tutor/learning tool is.. way too optimistic. A small fraction will, but the majority will just view things like a second language as something not worth learning.
One big gap I've run into for UX is most realtime voice harnesses wait for a full response from tools, and at most support the model filling the dead air until then
It'd be a game-changer to be able to have the model start replying with partial information streamed from the tool call, then seamlessly continue with additional information.
Some people have standards in what they build.
In the video example with the grannies, the knitter is essentially wanting a PA. Regular folks don't have PAs. Even when that became a thing in the aughts they were all outsourced.
When I've used voice chat, it has often turns into rabbit holes on very niche topics. For example, I had one start about the 1996 performance of Rage Against the Machine in Portland, Oregon that was supposed to feature Wu Tang Clan. (already outside most human's knowledge) that dove into details of the club scene in Los Angeles at the time of RATM's signing to Epic Records.
Was anyone else here at that '96 show in Portland? It seems like it might be challenging to find a person on the internet able to engage on the topic.
The person may exist, but not during my fleeting interest in the subject while walking to the park.
don't you dare clump the best fruit in the world with cockroaches!
Did something bad? Better ablate yourself of the responsibility of holding people to account for making it _worse_! Acting this way just makes it seem more like you regret the blood on your hands because it has dirtied your shirt, and not because you’ve done something actually bad, otherwise you’d have at least some degree of guilt and reticence to see things get worse.
The loneliness epidemic is driven by companies maximizing keeping their customers engaged with their screens, something OpenAI is wont to do. Knowing that the company wants customers engaged and that this will do that, and also knowing that that plays into the loneliness epidemic by substituting human interaction, makes it far different than getting married and then maybe or maybe not getting divorced.
Actually it says it _can_ delegate to the latest models. Seems reasonable to ask how the voice model does when it doesn't delegate (or while waiting for the delegated answer).
I say this because this is already how ChatGPT works internally when using its "auto" mode; the version of the "fast" model used in the "auto" mode does the same "notice your ignorance and bring in the heavy model" thing, just silently, rather than mentioning that it's doing it.
(If someone has actually run the experiment, please chime in!)
Our devices have now increased the distance _between_ us -- it's not about _you_ being able to "do X" -- talking to others is not _you_ doing it, it's you _and the other person_ doing it _together_. You can't be doing anything together consentually when the other person is in the habit of talking with their AI, or doomscrolling for that matter.
Uhm, those lonely people need to get out and start talking. How is this going to help society? This is going to make it worse.
Oh that kid is kinda quite and sad? Throw him an iPad. Oh that adult is kinda bored and wandering aimlessly? Throw him into a casino. Oh that adult is kinda lonely and feels like they don't have anyone they can talk to about their life? Give them LLM companionship.
Yep, it is over for humanity. People simply don't understand externalities.
I spent a few weeks at a hostel last year. It was always kind of depressing and tense in the shared kitchen, just this heavy silence.
I don't feel comfortable around strangers, so I solved that problem by just saying hi to everyone.
Most people didn't respond much, although most of them smiled and the tension was eased. But a few of them struck up conversation and we ended up making friends.
I ended up making like, ten new friends in two weeks. And then a bunch of them ended up becoming friends with each other as well.
Even if you do it, you are still swimming against the current.
Like saying I no longer walk because I can drive. I no longer cook because Doordash exists. I no longer play piano because midi exists.
I mean i guess, but it seems like you didn't LIKE crafting the text or coding from scratch, you just wanted the outcome. If we are talking purely about work, I understand that its about being productive and it sucks to have a job shift to something you enjoy less.
But for daily life? I dont see how it changes, maybe its a tech thing where people think about making their daily lives more productive, but most people dont.
This is almost certainly going to bite us in the ass long term, because eventually without human creativity you're just training AI's on other AI slop, or limiting the possible catalogue of styles to "things that existed before AI ruined every creative job". I guess the question is, what kind of future do you want to live in? One where we have a massive abundance of easy-to-create but vaguely worthless artifacts in a society that's completely devalued being good at something? It just sounds really dystopian to me.
AI doesn't stop creative people from creating. it gives creative people an additional tool with which to create
Exactly, so why make it worse? Why try to automate an industry when the money you make off killing it wouldn't even cover the cost of your research?
> AI doesn't stop creative people from creating. it gives creative people an additional tool with which to create
I can't emphasize enough how much actual creative people hate this shit. Here's a relatively balanced take, although the headline spoils it (How AI Poisons Creativity: https://www.youtube.com/watch?v=WeCSzEtZcUw ). Long story short, the struggle is where creative ideas come from. If you automate away the process you automate away the result.
I think the parent is saying now that that is attempting to be applied to "creativity" directly, as opposed to something like a shift of medium, that it threatens many peoples' ability to maintain creative capabilities.
Anecdotally I've already experienced this at work where post-AI we had a junior completely stagnate and a senior with over a decade of experience in the bay atrophy to the point that he had to be let go.
It's like trying to have meaningful conversations on Twitter. You don't go to Twitter to do things like that. Can you? Sure. It's just not what the format and the conventions (and the people) lend themselves to.
I don't think there's much merit in pretending that human activities are only shaped by hard limits.
I just asked and Claude says it's Haiku.
Great, what is being done to help that happen?
Even if all I am saying is bullshit, "what is being done to help"? What about what is being done to make it way, way, way, way, way worse. If I had LLM companionship when I was alone, yea, I would have never gotten out of my shell. I would be stuck in there forever. Hell, why should I even talk to you? I should just argue with a robot instead.
I.e. it's only a problem if you're not willing to go and strike up conversations with people. Which is not AI or mobile phones' fault. Expecting society to come up with "systemic situations" to your personal problems is a fast path to a lifetime of disappointment.
Input clip: https://vocaroo.com/19QtEPtwTjOS
Prompt text: There are 14 varieties of tomato soup available from this replicator. With rice, with vegetables, Bolian style, with pasta specify hot or chilled.
Output: https://vocaroo.com/1f3XuQQoSzwB
There's been a few studys showing that novices love LLM output that's long, but experts hate it. As an example, I've been tasked with using some agentic PM tool to write specs, and it keeps generating these huge page long outputs with "HBR voice" bolded summaries of paragraph long bulletpoints. I.e.:
> Right-size hard, and watch the one open-ended edge. Endorse the DRI's simplifications wholesale: drop the runbook-per-alert mandate (keep 1–2 diagnostic-only runbooks for the high-priority set), and ride durability on the existing weekly incident + monthly operational reviews — no new governance. The single scope-creep risk is the coverage strand (gaps are defined by absence); bound it to gaps evidenced by real, already-missed customer-facing outages, not a proactive gap hunt. Curing ownership gaps (e.g. foo-bar, no clear owner) is finite in-scope work.
There's dozens of these every iteration. I can't imagine trying to deal with that via voice, I would just zone out after the second sentence.
I've had this dream of talking to the Enterprise-D computer since I was 8 years old. Midlife-crisis me still has that dream, but hooked up to Home Assistant so it can actually do useful things too. A couple months back, I went looking around for "clean" samples of Majel's voice as the computer but didn't have a lot of luck. Even though there are three television series and several movies, pretty much all of them have some amount of background noise, bleeps, bloops, or warp core thrum. (As this one does.) There may be modern ways to clean those up without affecting her voice much, but I haven't dug into that yet.
There are a few audiobooks narrated by Majel Barrett but obviously her role as the computer was proper voice acting and so the books would not be good source material.
There were also a few games/CD-ROM (Omnipedia) with some samples, but they did not bother to post-process them for that lofi Enterprise-D computer feel. Can _probably_ be replicated fairly faithfully after the fact, but I only know a _little_ about audio post-processing.
According to her son (Rod Roddenberry), Majel sat down in a studio and recorded audio samples specifically for the purpose of having her voice cloned someday for future Star Trek stories. However, those haven't been released publicly. (And likely never will, but I can dream, can't I?)
Edit: I played with your samples and the time it takes to generate the output audio is pretty brutal. Too slow for interactive use. Maybe that's a limitation of the HF-hosted app, though.
I guess when we tell models to use "cold logic" they don't interpret that purely definitionally, but moreso with what this statement usually implies, which is often a disagreement between two people, one trying to leverage supposed logic to disagree and denigrate the other party, oftentimes actually arguing out of emotion and not the logic they claim to use. This probably occurs enough to give model responses a mean tint. That's my theory on why this could occur at least.
Not for chat, just as a way to make notification messages that sound like ED-209.
Another WillowTalk voice was a clone of DECtalk's Perfect Paul good enough to be used as the voice in the MC Hawking rap recordings.
You do have to do it carefully and not literally say "Star Trek" or else it'll start yapping about main shields being at full strength.
Or even some scenes where Data did something similar.
It's funny to now experience it.
WillowTalk is apparently still of interest to Half-Life modders because another WillowTalk voice, possibly a clone of DECtalk's Huge Harry, was used as the Black Mesa VOX facility-wide announcement system in the original Half-Life.