The issue with AI isn’t that it simply gets things wrong — as is frequently pointed out, so do humans. The issue is that it gets things wrong in a way that comes out of nowhere and doesn’t even have a post-rationalised explanation.
The big claim about AI systems (especially LLMs) is that they can generalise, but in reality the ‘zone of possible generalisation’ is quite small. They overfit their training data and when presented with input out of distribution they choke. The only reason anyone is amazed by the power of LLMs is because the training set is unimaginably huge.
In fifty years we’ll have systems that make this stuff look as much like ‘AI’ as, say, Djikstra’s algorithm does now.
I first noticed this with Watson (IBM's language processing system) when it played Jeopardy!: when it was right, it was spot on and usually faster than the human contestants; but when it was wrong, it was way, way off base.
Part of that has to do with the fact that language is not the same for an LLM as it is for a person. If I say to you the sentence "The cat sat on the mat", that will evoke a picture, at the very least an abstract sketch, in your mind based on prior experience of cats, mats, and the sitting thereupon. Even aphantasic people will be able to map utterances to aspects of their experience in ways that allow them to judge whether something makes sense. A phrase like "colorless green dreams sleep furiously" is arrant nonsense to just about everybody.
But LLMs have no experiences. Utterances are tokens with statistical information about how they relate to one another. Nodes in a graph with weighted edges or something. If you say to an LLM "Explain to me how colorless green dreams can sleep furiously", it might respond with "Certainly! Dreams come in a variety of colors, including green and colorless..."
I've always found Searle's argument in the Chinese Room thought experiment fascinating, if wrong; my traditional response to it was "the man in the room does not understand Chinese, but the algorithm he's running might". I've been revisiting this thought experiment recently, and think Searle may have been less wrong than I'd first guessed. At a minimum, we can say that we do not yet have an algorithm that can understand Chinese (or English) the way we understand Chinese (or English).
Google’s spam filter is pretty good though and has been for a long, long time. But I guess we need to sprinkle everything with a bit of extra LLM AI these days.
Google's spam filter lets through tons of spam for me. I think some spammers are abusing Google's weird DKIM configuration to send me emails that were supposedly sent from my own email address. No amount of clicking "report spam" will do anything.
It also blocks just about any small domain that emails me for the first time. No amount of SPF or DKIM will convince Google that you're legitimate party, there's some kind of minimal volume you need to send Google to make your emails arrive to Gmail inboxes the first time.
It works when it works, but when it doesn't, it's broken without repair. It works _most of the time_ and it's better than Outlook (though that's not a high bar to clear).
Did not work for me from the very first day. I was one of Gmail Beta testers, unfortunately, my new account started receiving one the very first day I registered it. I asked for a blacklist and kept being pushed back saying their filter was good enough, and I should never have never needed a blacklist. Oh well.
> Did not work for me from the very first day. I was one of Gmail Beta testers
But now its superb - reporting that a mail is spam does a good job of marking future mails from that sender as spam and moving messages from spam folder to inbox does the opposite.
I wish I could say the same. I mark so many emails in my Gmail as phishing attempts but it just never learns.
They’re super obvious ones too with a nonsensical email address, a repeating pattern about mcaffee or Norton in the title and an almost empty body with a pdf attached.
Meanwhile Gmail also happily never learns when I tell it something isn’t spam either.
I used Gmail from the first public signups to their bitchedw Apps for Domains migration. After we switched to Fastmail, the first thing we noticed was how much less spam we were seeing and the second was how many legitimate messages had been incorrectly filtered by their priority inbox system.
> Google’s spam filter is pretty good though and has been for a long, long time.
What? This hasn't been true for at least 15 years. Instead, Google's spam filter is far, far more aggressive than could conceivably be appropriate, and it routinely filters important communications from people you know.
The only OpenAI integration is giving users the opportunity to have their model answer questions. No Apple services, and no general queries rely on OpenAI.
Yeah, the OpenAI integration they demonstrated at WWDC showed a very prominent “do you want to send this question to ChatGPT?” dialog when it kicked in. The email feature absolutely isn’t using OpenAI - plus the OpenAI integration isn’t in the iOS 18.1 beta yet.
In any case, dealing with spam/phishing is always an arms race.
One of the drawbacks of AI, is that I suspect it will have patterns that could be figured out, and folks will learn that (crooks tend to be a lot smarter than most folks seem to think. I'll lay odds that every hacker has an HN account).
The goal of betas is to surface issues. This is an issue; it has been surfaced. What do you do when you find an issue in a beta? Do you cross your arms and say “eh, it’s a beta, it’ll get fixed”? Because it won’t if no one talks about it.
It has been reported. The feedback number is right there in the following post. Talking about a flaw you encountered is not “bitching about it”, it’s making others aware so they can test, verify, correct, find other ways in which it manifests… Not to mention pressuring Apple to actually fix it. No one with relevant experience reporting stuff to Apple believes earnestly that a mere report is the most effective path to fixing an issue.
If you do not wish to engage in good faith, you’re free to skip the submission and carry on with your day. You don’t need to succumb to the impetus of making repeated low-effort replies.
