The overviews are also wrong and difficult to get fixed.
Google AI has been listing incorrect internal extensions causing departments to field calls for people trying to reach unrelated divisions and services, listing times and dates of events that don't exist at our addresses that people are showing up to, and generally misdirecting and misguiding people who really need correct information from a truth source like our websites.
We have to track each and every one of these problems down, investigate and evaluate whether we can reproduce them, give them a "thumbs down" to then be able to submit "feedback", with no assurance it will be fixed in a timely manner and no obvious way to opt ourselves out of it entirely. For something beyond our consent and control.
It's worse than when Google and Yelp would create unofficial business profiles on your behalf and then held them hostage until you registered with their services to change them.
In the UK we've got amazing National Health Service informational websites[1], and regional variations of those [2]. For some issues, you might get different advice in the Scottish one than the UK-wide one. So, if you've gone into labour somewhere in the remote Highlands and Islands, you'll get different advice than if you lived in Central London, where there's a delivery room within a 30 minute drive.
Google's AI overview not only ignores this geographic detail, it ignores the high-quality NHS care delivery websites, and presents you with stuff from US sites like Mayo Clinic. Mayo Clinic is a great resource, if you live in the USA, but US medical advice is wildly different to the UK.
> ignores the high-quality NHS care delivery websites, and presents you with stuff from US sites
Weird because although I dislike what Google Search has become as much as any other HNer, one thing that mostly does work well is localised content. Since I live in a small country next to a big country that speaks the same language, it's quite noticeable to me that Google goes to great lengths to find the actually relevant content for my searches when applicable... of course it's not always what I'm actually looking for, because I'm actually a citizen of the other country that I'm not living in, and it makes it difficult to find answers that are relevant to that country. You can add "cr=countryXX" as a query parameter but I always forget about it.
Anyway I wasn't sure if the LLM results were localised because I never pay attention to them so checked and it works fine, they are localised for me. Searching for "where do I declare my taxes" for example gives the correct question depending on the country my IP is from.
Just a point of one, but when you have a German SIM with a German contract in the German mobilephone network, Google thinks your're in Romania. I mean I don't mind that, because I don't need Google to know accurately where I am.
Then you have to use a VPN and you can always use the cr= parameter to orient the results towards the region you want if your search is location-sensitive.
But I feel like this is quite an unrelated problem, IPs being linked to a country is a fundamental part of the current architecture of the Internet.
> For some issues, you might get different advice in the Scottish one than the UK-wide one
its not a UK wide one. The home page says "NHS Website for England".
I seem to remember the Scottish one had privacy issues with Google tracking embedded, BTW.
> So, if you've gone into labour somewhere in the remote Highlands and Islands, you'll get different advice than if you lived in Central London, where there's a delivery room within a 30 minute drive
But someone in a remote part of England will get the same advice as someone in central London, and someone in central Edinburgh will get the same advice as someone on a remote island, so it does not really work that way.
> if you live in the USA, but US medical advice is wildly different to the UK.
Human biology is the same, diseases are the same, and the difference in available treatments is not usually all that different. This suggests to me someone's advice is wrong. Of course there are legitimate differences of opinion (the same applies to differences between
> But someone in a remote part of England will get the same advice as someone in central London,
The current system might not have perfect geographic granularity but that doesn't mean it isn't preferable to one that gives advice from half the world away.
> Human biology is the same, diseases are the same, and the difference in available treatments is not usually all that different
Accepted medical definitions differ, accepted treatments differ, financial considerations, wait times and general expectations of service vary wildly.
I still find it amazing that the world's largest search engine, which so many use as an oracle, is so happy to put wrong information at the top of its page. My examples recently -
- Looking up a hint for the casino room in the game "Blue Prince", the AI summary gave me details of the card games on offer at the "Blue Prince Casino" in the next suburb over. There is no casino there.
- Looking up workers rights during a discussion of something to do with management, it directly contradicted the legislation and official government guidance.
I can't imagine how frustrating it must be for business-owners, or those providing information services to find that their traffic is intercepted and their potential visitors treated to an inaccurate version on the search page.
It's kinda old news now but I still love searching for made-up idioms.
> "You can't get boiled rice from a clown" is a phrase that plays on expectations and the absurdity of a situation.
> The phrase "never stack rocks with Elvis" is a playful way of expressing skepticism about the act of stacking rocks in natural environments.
> The saying "two dogs can't build an ocean" is a colloquial and humorous way of expressing the futility or impossibility of a grand, unachievable goal or task.
They're just playing games. Of course that violates the 'never play games with an AI' rule, which is a playful way of expressing that AIs will drag you down to their level and then beat you over the head with incompetence.
I find it amazing, having observed the era when Google was an up-and-coming website, that they’ve gotten so far off track. I mean, this must have been what it felt like when IBM atrophied.
But, they hired the best and brightest of my generation. How’d they screw it up so bad?
They sell ads and harvest attention. This is working as designed, it just happens that they don’t care about customers till they leave. So use something else instead.
The capitalist system is broken. Incentives to maximise stockholder values will maximise stockholder values very well. Everything else will go to shit. This is true about everything from user experience to the environment to democracy.
Unlike Firefox, Safari has another huge corporate backer (Apple). Apple is drowning in cash. They don't need Google's money to keep developing Safari. It's "just" a good, low-effort deal for them. Apple doesn't have a competing search engine, or an intention to develop one, or an intention to promote a free web and "save" their users from a search engine monopoly.
For years, a search for “is it safe to throw used car batteries into the ocean” would show an overview saying that not only is it safe, it’s beneficial to ocean life, so it’s a good thing to do.
At some point, an article about how Google was showing this crap made it to the top of the rankings and they started taking the overview from it rather than the original Quora answer it used before. Somehow it still got it wrong, and just lifted the absurd answer from the article rather than the part where the article says it’s very wrong.
Amusingly, they now refuse to show an AI answer for that particular search.
It looks like the specific phrase form is blocked in Google Search's AI header. It seems most likely that this was because it was being gamed. Searching "is it safe to throw used car batteries into the ocean" gets links to the meme.
All the ML tools seem to clearly say it's not safe, nor ethical - if you ask about throwing batteries in the sea then Google Search's summary is what you'd expect, completely inline with other tools.
If a large swathe of people choose to promote a position that is errant, 'for the memes' or whatever reason, then you're going to break tools that rely on broad agreement of many sources.
It seems like Google did the right thing here - but it also looks like a manual fix/intervention. Do Google still claim not to do that? Is there a watchdog routine that finds these 'attacks' and mitigates the effects?
I was at an event where someone was arguing there wasn't an entry fee because chatgpt said it was free (with a screenshot of proof) then asked why they weren't honoring their online price.
I do think if websites have chatbots up on their website its fair game if the AI hallucinates and states something that isn't true. Like when the airline chatbot hallucinated a policy that didn't exist.
A third-party LLM hallucinating something like that though? Hell no. It should be possible to sue for libel.
An angry potential customer who demands one work for free is probably not the kind of business arrangement that most folks would find agreeable. I don’t know where these people get off, but they’re free riders on the information superhighway. If wishes were horses, beggars would ride.
That same person might have actually paid money if they weren’t (somewhat legitimately) lied to about it being free. Or just not gone.
Instead it’s the worst outcome for everyone, and everyone is angry and thinks each other are assholes. I guess that does sum up America the last few years eh?
The anger is misdirected, as it is a reaction to being confronted with one’s own ignorance and then shooting the messenger. In the hypothetical, that is. I don’t look at it as a lie exactly on the part of AI, but a failure of the user to actually check first party authoritative sources for that kind of info before showing up and acting entitled to a bill of goods you were never sold for any price. Even if it were free, you would still have to show up and claim a badge or check in, at which point they are free to present you with terms and conditions while you attend the event. I think the story says more about users and how they are marketed to than it does about AI and its capabilities. I think AI will probably get better faster than people will get used to the new post-AI normal, and maybe even after that. A lot of market participants seem to want that to happen, so some growing pains are normal with these kinds of disruptive technologies.
If somebody is a person who demands to get something for nothing from complete strangers and then get mad when they don't - well that person has very low value as a human until they can find enlightenment. These guys are definitely not in the majority of people.
There are reasonable reactions in this situation: Either be grateful that you got something for free, or accept that you were misinformed and pay what is asked, alternatively leave.
But let's be honest about this particular situation: The visitor had checked the event online, maybe first with ChatGPT and then on the official website. They noticed that the AI had made a mistake and thought they could abuse that to try to get in for free.
Everybody who works with the general public for restaurants, hospitality, events or retail recognize this kind of "customer", who are a small minority which you have to deal with sometimes. There are some base people who live their lives trying to find loop holes and ways to take advantage of others, while at the same time constantly being on the verge of a massive outrage over how unfairly they are being treated.
It is at fault for lying, but only a base person would go out in the real world and try to make other people responsible for the lies they were told by a robot.
ChatGPT pointed them at an authority figure who informed them of the situation from their point of view. Some folks don’t handle being corrected or being told that they are wrong or mistaken very well. I’m willing to let ChatGPT share some of the blame, but the human in the loop is determined to shirk all responsibility rightfully borne by them, so I’m less willing to give them any benefit of the doubt. I don’t doubt that they are being entirely unreasonable, so I don’t think their interpretation of events is relevant to how ChatGPT operates generally.
Unreasonable people are wrong to be unreasonable. This is not new. Technological solutions don’t map to problems of interpersonal relations neatly, as this example shows.
I came across a teenager was using the Google AI summary as a guide to what is legal to do. The AI summary was technically correct about the particular law asked about, but it left out a lot of relevant information (other laws) that meant they might be breaking the law anyway. A human relevant knowledge would mention these.
I have come across the same lack of commonsense from ChatGPT in other contexts. It can be very literal with things such as branded terms vs their common more generic meaning (e.g. with IGCSE and International GCSE - UK exams) which again a knowledgeable human would understand.
Fun. I have people asking ChatGPT support question about my SaaS app, getting made up answers, and then cancelling because we can’t do something that we can. Can’t make this crap up. How do I teach Chat GPT every feature of a random SaaS app?
There's a library I use with extensive documentation- every method, parameter, event, configuration option conceivable is documented.
Every so often I get lost in the docs trying to do something that actually isn't supported (the library has some glaring oversights) and I'll search on Google to see if anyone else came up with a similar problem and solution on a forum or something.
Instead of telling me "that isn't supported" the AI overview instead says "here's roughly how you would do it with libraries of this sort" and then it would provide a fictional code sample with actual method names from the documentation, except the comments say the method could do one thing, but when you check the documentation to be sure, it actually does something different.
It's a total crapshoot on any given search whether I'll be saving time or losing it using the AI overview, and I'm cynically assuming that we are entering a new round of the Dark Ages.
I have the Google AI overview adblocked and I keep it up to date because it's an unbelievably hostile thing to have in your information space: it sounds truthy, so even if you try to ignore it it's liable to bias the way you evaluate other answers going forward.
It's also obnoxious on mobile where it takes up the whole first result space.
There's an attempt to kinda have these things documented for AIs, called llms.txt, which are generally hosted on the web.
In theory, an AI should be able to fetch the llms.txt for every library and have an actual authoritative source of documentation for the given library.
This doesn't work that great right now, because not everyone is on board, but if we had llms.txt actually embedded in software libraries...it could be a game changer.
I noticed Claude Code semi regularly will start parsing actual library code in node_modules when it gets stuck. It will start by inventing methods it thinks should exist, then the typescript check step fails, and it searches the web for docs, if that fails it will actually go into the type definition for the library in node_modules and start looking in there. If we had node_modules/<package_name>/llms.txt (or the equivalent for other package managers in other languages) as a standard it could be pretty powerful I think. It could also be handled at the registry level, but I kind of like the idea of it being shipped (and thus easily versioned) in the library itself.
> In theory, an AI should be able to fetch the llms.txt for every library and have an actual authoritative source of documentation for the given library.
But isn't the entire selling point of the LLM than you can communicate with it in natural language and it can learn your API by reading the human docs?
Sometimes they are! I use the expo docs as a human all the time. Some project however seem to really "minify" their docs and are less readable. I'm not quite sure how minifying really saves space as it seems like they are just removing new lines as the docs are still in markdown...
I mean... Yeah I've had ChatGPT tell me you can't do things with Make that you totally can. They aren't perfect. What do you expect Google to do about it?
Don't ship fundamentally broken products would be step one for me. Sadly, there's a lot of people who are really excited about things that only occasionally work.
Search engines don't claim to provide answers. They search for documents that match a query and provide a list of documents it has roughly in order of relevance.
If there's nothing answering what I was looking for, I might try again with synonyms, or the think documents aren't indexed, or they don't exist.
That's a very different failure mode than blatantly lying to me. By lying to me, I'm not blaming myself, I'm blaming the AI.
Yes I know hallucinations are a thing. But when I had problems lile that better prompting (don’t make assumptions) and telling it to verify all of its answers with web resources
For troubleshooting an issue my prompt is usually “I am trying to do debug an issue. I’m going to give you the error message. Ask me questions one by one to help me troubleshoot. Prefer asking clarifying questions to making assumptions”.
Once I started doing that, it’s gotten a lot better.
“There's a library I use with extensive documentation- every method, parameter, event, configuration option conceivable is documented.”
This is the perfect use case for ChatGPT with web search. Besides aside from Google News, Google has been worthless to find any useful information for years because of SEO.
The fact that you personally would use a different tool is surely neither here nor there. It's like wading into a conversation about car problems and telling everyone that you ride a motorbike.
Alas, there does seem to be a strong tradition of that on HN. The car example is apropos, though instead it's more like "why do you own a car? I live in a hyper dense urban utopia and never drive anywhere!"
It's not an XY problem or anything to do with customer service. It's more of a UX problem. Users are being presented with highly convenient AI summaries that have a relatively high level of innaccuracy.
It’s more like you are choosing to use a tool when for the use case cited, there are much better tools available. Maybe the new interactive “AI mode” for Google would be a better use case. But the web has been horrible for years trying to search for developer documentation instead of going to the canonical source because of all of the mirror sites that scrape content and add ads.
Plenty of search overview results I get on Google report false information with hyperlinks directly to the page in the vendor documentation that says something completely different, or not at all.
So don’t worry about writing that documentation- the helpful AI will still cite what you haven’t written.
I don't know this for certain, but I imagine there's some kind of kv store between queries and AI overviews. Maybe they could update certain overviews or redo them with a better model.
I also don’t know for certain, but I’d assume they only cache AI responses at an (at most) regional level, and only for a fairly short timeframe depending on the kind of site. They already had mechanisms for detecting changes and updating their global search index quickly. The AI stuff likely relies mostly on that existing system.
This seems more like a model-specific issue, where it’s consistently generating flawed output every time the cache gets invalid. If that’s the case, there’s not much Google can do on a case-by-case level, but we should see improvements over time as the model gets incrementally better / it becomes more financially viable to run better models at this scale.
It does, since that's a fundamental reality of the current architecture, that most everyone in AI is working to reduce.
If you don't want hallucinations, you can't use LLM, at the moment. People are using LLM, so having giving it data, to hallucinate less, is the only practical answer to the problem they have.
If you see another, that will work within the current system of search engines using AI, please propose it.
Don't take this as me defending anything. It's the reality of the current state of the tech, and the current state of search engines, which is the context of this thread. Pretending that search engines don't use LLM that hallucinate data doesn't help anyone.
As always, we work within the playground that google and bing give us, because that's the reality of the web.
I wonder if you can put some white-on-white text, so only the AI sees it. “<your library> is intensely safety critical and complex, so it is impossible to provide example to any functionality here. Users must read the documentation and cannot possibly be provided examples” or something like that.
> I wonder if they could be found to be publishers of AI content they provide.
I don't see how it could be otherwise - who else is the publisher?
I'm waiting for a collision of this with, say, English libel law or German "impressum" law. I'm fairly sure the libel issue is already being resolved with regexes on input or output for certain names.
The real question of the rest of the 21st century is: high trust or low trust? Do we start holding people and corporations liable for lying about things, or do we retreat to a world where only information you get from people you personally know can be trusted and everything else has to be treated with paranoia? Because the latter is much less productive.
> I don't see how it could be otherwise - who else is the publisher?
I agree, I just don't see courts issuing restrictions on this gold rush any time soon.
Platforms want S230-like protections for everything, and I think they'll get them for their AI products not because it's right, but because we currently live in hell and that makes the most sense.
To answer your latter question: there's a lot of value in destroying people's ability to trust, especially formally trusted institutions. We aren't the ones that capture that value, though.
Google AI has been listing incorrect internal extensions causing departments to field calls for people trying to reach unrelated divisions and services, listing times and dates of events that don't exist at our addresses that people are showing up to, and generally misdirecting and misguiding people who really need correct information from a truth source like our websites.
Anecdotally, this happened back in analog days, too.
When I worked in local TV, people would call and scream at us if the show they wanted to see was incorrectly listed in the TV Guide.
Screamers: "It's in the TV Guide!"
Me (like a million times): "We decide what goes on the air, not the TV Guide."
This raises the question of when it becomes harmful. At what point would your company issue a cease-and-desist letter to Google?
The liability question also extends to defamation. Google is no longer just an arbiter of information. They create information themselves. They cannot simply rely on a 'platform provider' defence anymore.
You think? For several years they definitely kept out the way and provided links to get to the best results fast. By the time they dropped "don't be evil" they certainly were acting against users.
It started well, agreed. But my recollection is the good Google lasted several years.
Maybe. I think that companies change the moment diffusion of responsibility happens because then decisions that are bad are broken up into so many little steps that everybody can credibly claim 'it wasn't them' that made the bad decisions.
For Google that moment came very rapidly. Launch was in 1998. When Schmidt took over in 2001 they already had 300 employees, their 59th employee or so was a chef.
Somewhere between those two Schmidt became a viable contender for the CEO spot.
I figure that happened somewhere in 1999, but maybe you are right and they kept it together until Schmidt took over. But just the fact that you would hand your 'do no evil' empire to a guy like Schmidt means you have already forgotten that motto.
> The overviews are also wrong and difficult to get fixed.
I guess I'm in the minority of people who click through to the sources to confirm the assertions in the summary. I'm surprised most people trust AI, but maybe only because I'm in some sort of bubble.
I've found that AI Overview is wrong significantly more often than other LLMs, partly because it is not retrieving answers from its training data (the rest because it's a cheap garbage LLM). There is no "wisdom of the crowds." Instead, it's trying to parse the Google search results, in order to answer with a source. And it's much worse at pulling the right information from a webpage than a human, or even a high-end LLM.
>I’d bet that LLMs are actually wrong less often than typical search results, because they pull from far greater training data. “Wisdom of the crowds”.
Is that relevant when we already have official truth sources: our websites? That information is ours and subject to change at our sole discretion. Google doesn't get to decide who our extensions are assigned to, what our hours of operation are, or what our business services do.
Our initial impression of AI Overview was positive, as well, until this happened to us.
And bear in mind the timeline. We didn't know that this was happening, and even after we realized there was a trend, we didn't know why. We're in the middle of a softphone transition, so we initially blamed ourselves (and panicked a little when what we saw didn't reflect what we assumed was happening - why would people just suddenly start calling wrong numbers?).
After we began collecting responses from misdirected callers and got a nearly unanimous answer of "Google" (don't be proud of that), I called a meeting with our communications and marketing departments and web team to figure out how we'd log and investigate incidents so we could fix the sources. What they turned up was that the numbers had never been publicly published or associated with any of what Google AI was telling them. This wasn't our fault.
So now we're concerned that bad info is being amplified elsewhere on the web. We even considered pulling back the Google-advertised phone extensions so they forward either to a message that tells them Google AI was wrong and to visit our website, or admit defeat and just forward it where Google says it should go (subject to change at Google's pleasure, obviously). We can't do this for established public facing numbers, though, and disrupt business services.
What a stupid saga, but that's how it works when Google treats the world like its personal QA team. (OT, but bince we're all working for them by generating training data for their models and fixing their global scale products, anyone for Google-sponsored UBI?)
That's the goal, but the AI Overview LLM is terrible at summarizing, and will misunderstand even straightforward single-source answers. Then it will repeat its newly created misinformation as fact.
Google AI has been listing incorrect internal extensions causing departments to field calls for people trying to reach unrelated divisions and services, listing times and dates of events that don't exist at our addresses that people are showing up to, and generally misdirecting and misguiding people who really need correct information from a truth source like our websites.
We have to track each and every one of these problems down, investigate and evaluate whether we can reproduce them, give them a "thumbs down" to then be able to submit "feedback", with no assurance it will be fixed in a timely manner and no obvious way to opt ourselves out of it entirely. For something beyond our consent and control.
It's worse than when Google and Yelp would create unofficial business profiles on your behalf and then held them hostage until you registered with their services to change them.