> Additionally, rare hallucinations in Voice Mode persist with this update, resulting in unintended sounds resembling ads, gibberish, or background music. We are actively investigating these issues and working toward a solution.
Would be cool to hear some samples of this. I remember there was some hallucinated background music during the meditation demo in the original reveal livestream but haven't seen much beyond that. Artifact of training on podcasts to get natural intonation.
I use advanced voice a lot and have come across many weird bugs.
1) Every response would be normal except end with a “whoosh” like one of those sound effects some mail clients use when an message is sent, and the model itself either couldn’t or wouldn’t acknowledge it.
2) The same except with someone knocking on a door. Like someone would play on a soundboard.
3) The entire history in the conversation disappearing after several minutes of back and forth, leading to the model having no idea what I’m talking about and acting as if it’s a fresh conversation.
4) Advanced voice mode stuttering because it hears its own voice and thinks it’s me interrupting (on a brand new iPhone 16 Pro, medium-low built in speaker volume and built-in mic).
5) Really weird changes in pronunciation or randomly saying certain words high-pitched, or suddenly using a weird accent.
And all of this was prior to these most recent changes.
It also stutters and repeats sometimes and says poor connection even though I know the connection is near-ideal.
I may know why that first one happens! They’re not correctly padding the latent in their decoder (by default torch pads with zeros, they should pad with whatever their latent’s representation of silence is). You can hear the same effect in songs generated with our music model: https://sonauto.ai/
If you pad your output with something that doesn't represent silence, then any outputs that happen to have a non-standard length (i.e. nearly all outputs) will end with whatever sound your padding bits represent in the model's embedding space. if "0000" represents "Whoosh," then most of your outputs will end in "whoosh."
Here's a non-AI example: If all HN comments had to be some multiple of 50 characters long and comments were padded with the letter "A," then most HN comments would look like the user was screaming at the end. AAAAAAAAAAAAAAAAAA
In addition to what Centigonal said, even if the autoencoder was trained on only speech data, an all zero vector is probably just out of distribution (decoder has never seen it before) and causes weird sounds. However, given the hallucinations we're seeing, the AE has (maybe unintentionally) likely seen a bunch of non-speech data like music and sound effects too.
> 3) The entire history in the conversation disappearing after several minutes
Likely a natural consequence of using native voice mode. It is very probable that its context span is very short. Sesame for instance loses context after less than 20 minutes (not sure when exactly).
> 4) Advanced voice mode stuttering because it hears its own voice and thinks it’s me interrupting
I experience the same issue on an iPhone 15 Pro Max and have to mute the mic whenever I'm listening to a response. I wish they added an option to disable voice interruptions so that it could be interrupted only by touch.
If anyone's wondering, here's a short sample. It quietly updated last night, and I ended up chatting for like an hour. It sounds as smart as before, but like 10x more emotionally intelligent. Laughter is the biggest giveaway, but the serious/empathetic tones for more therapy-like conversations are noticeable, too.
https://drive.google.com/file/d/16kiJ2hQW3KF4IfwYaPHdNXC-rsU...
I find it weird the way it's just lying. Did you tell it to impersonate a living human?
Obviously I get LLMs have no concept of truth and hallucinate all the time, but I would have thought that the model prompt would have told it to acknowledge that it's a chatbot.
Holy moley. Thanks for sharing. I had to work with the API version a lot the last week and it was frustrating how "old" it felt intelligence-wise. This is in another league, I hope it's 4.1 x audio training, I'd love to talk to this. Current one is passable for hands-free RAG, that's it for me.
Would be cool to hear some samples of this. I remember there was some hallucinated background music during the meditation demo in the original reveal livestream but haven't seen much beyond that. Artifact of training on podcasts to get natural intonation.