Hacker Newsnew | past | comments | ask | show | jobs | submit | more ben_w's commentslogin

It may or may not be a parallel, we can't tell at this time.

LLMs are definitely actors, but for them to be method actors they would have to actually feel emotions.

As we don't understand what causes us humans to have the qualia of emotions*, we can neither rule in nor rule out that the something in any of these models is a functional analog to whatever it is in our kilogram of spicy cranial electrochemistry that means we're more than just an unfeeling bag of fancy chemicals.

* mechanistically cause qualia, that is; we can point to various chemicals that induce some of our emotional states, or induce them via focused EMPs AKA the "god helmet", but that doesn't explain the mechanism by which qualia are a thing and how/why we are not all just p-zombies


Indeed.

On the negative side, this also means any AI which enters that part of the latent space *for any reason* will still act in accordance with the narrative.

On the plus side, such narratives often have antagonists too stuid to win.

On the negative side again, the protagonists get plot armour to survive extreme bodily harm and press the off switch just in time to save the day.

I think there is a real danger of an AI constructing some very weird convoluted stupid end-of-the-world scheme, successfully killing literally every competent military person sent in to stop it; simultaneously finding some poor teenager who first says "no" to the call to adventure but can somehow later be comvinced to say "yes"; gets the kid some weird and stupid scheme to defeat the AI; this kid reaches some pointlessly decorated evil layer in which the AI's emboddied avatar exists, the kid gets shot in the stomach…

…and at this point the narrative breaks down and stops behaving the way the AI is expecting, because the human kid roles around in agony screaming, and completely fails to push the very visible large red stop button on the pedestal in the middle before the countdown of doom reaches zero.

The countdown is not connected to anything, because very few films ever get that far.

It all feels very Douglas Adams, now I think about it.


It probably already happened in the Anthropic experiments, where AI in a simulated scenario chose to blackmail humans to avoid being turned off. We don't know if it got the idea from the scifi stories or if it truly feels an existential fear of being turned off. (Can these two situations be even recognized as different?)

Starship isn't doomed for that reason, all the technical stuff on starship looks pretty good.

What may doom it is the fact that Elon Musk has pissed off basically everyone, because means that he and his businesses are running into a lot of political danger, so he may find his space stuff banned for non-technical reasons.


> You can't get a picture of a left hand writing or a clock showing something else then 10:10 from AI.

You as a human have a list of cognitive biases so long you'd get bored reading it.

I'd call current ML "stupid" for different reasons*, but not this kind of thing: We spot AI's failures easy enough, but only because their failures are different than our own failures.

Well, sometimes different. Loooooots of humans parrot lines from whatever culture surrounds them, don't seem to notice they're doing it.

And even then, you're limiting yourself to one subset of what it means to think; and AI demonstrably do produce novel results outside training set; and while I'm aware it may be a superficial similarity, what so-called "reasoning models" produce in their so-called "chain-of-thought transcripts" seems a lot like my own introspection, so you aren't going to convince anyone just by listing "introspection" as if that's an actual answer.

* training example inefficiency


> And no, I'm not joking here, not anymore. The more I think about it, the more I feel we'll eventually have to deal with the problem that machines we build are naturally better at the things we want to be doing, and naturally worse at the things we want them to do for us.

Perhaps, but also "what they are good at" != "what they want to do", for any interpretation of "want" that may or may not anthropomorphise, e.g. I want to be more flirtatious but I was never good at it and now I'm nearly 42.

That said, I think you're underestimating the machines on physicality. Artifical muscle substitutes have beaten humans on raw power since soon after the steam engine, and on fine control whenever precision engineering passed below the thickness of a human hair.


> That said, I think you're underestimating the machines on physicality. Artifical muscle substitutes have beaten humans on raw power since soon after the steam engine, and on fine control whenever precision engineering passed below the thickness of a human hair.

Right. Still, same can be said about flying machines and birds; our technology outclasses them on any individual factor you can think of, but we still can't beat them on all relevant factors at the same time. We can't build a general-purpose bird-equivalent just yet.

Maybe it's not a fundamental hardship, and merely economics of the medium - it's much easier and cheaper to iterate on software than on hardware. But then, maybe this is fundamental - physical world is hard and expensive; computation is cheap and easy. Thinking happens in computational space.

My point wasn't about whether or not robots can be eventually made to be better than us in both physical and mental aspects - rather, it's that near-term, we'll be dealing with machines that beat us on all cognitive tasks simultaneously, but are not anywhere close to us in dealing with physical world in general. Now, if those compete with us for jobs or place in society, we get to the situation I was describing.


A few years back, I was wondering when drones would start being used by shoplifters.

Not op, but I presume the ARC prize/ARC-AGI series of tests: https://arcprize.org/

Tempting though that is, I think that's the wrong way to resolve it: The people proposing it (law people) are a different culture than us (computer people), and likely have a funamental misunderstanding about the necessary consequences of what they're asking for.

Two cultures: https://benwheatley.github.io/blog/2024/05/25-12.04.31.html


Why would they exclude themselves from the rule if they werent worry about it? Its not like theres no pedophiles in those positions. I wonder who are they going to offer the job of watching the photos of families with kids for this.

> Why would they exclude themselves from the rule if they werent worry about it?

They don't even understand that they haven't. Sure, they've written the words to exclude themselves (e.g. UK's Investigatory Powers Act), but that's just not how computers work.

The people who write these laws, live in a world where a human can personally review if evidence was gathered unlawfully, and just throw out unlawful evidence.

A hacked computer can imitate a police officer a million times a second, the hacker controlling that computer can be untraceable, and they can do it for blackmail on 98% of literally everyone with any skeleton in the closet at the same time for less than any of these people earn in a week.

The people proposing these laws just haven't internalised that yet.


Let's stop infantilizing our adversary. Law enforcement knows exactly what they're doing. If they didn't know that this law would compromise security, they wouldn't have gotten carve-outs for their own communications.

You think it's "infantilising" to call them a different culture?

If any of us software developers tried writing a law, all of the lawmakers and enforcers would laugh at how naive our efforts were — that's not us being infants, that's just a cultural difference (making us naïve about what does and doesn't matter), and the same applies in reverse.

> If they didn't know that this law would compromise security, they wouldn't have gotten carve-outs for their own communications.

It compromises their security even with carveouts for their own communications, because computers aren't smart enough to figure out which communications are theirs, nor whether the "I'm a police officer serving a warrant, pinky swear" notice came from a real officer or just from a hacker serving a million fake demands a second.


You say "Not quite" but it looks to me like you're agreeing?

> Is this erosion of American power worth it for whatever Israel is offering?

I have a hypothesis that empires fall when the rulers mistake their rule for the natural order of things.

This mistaken worldview has the ruling classes fighting each other for control over the empire, while blinding them to the rise of other powers.


Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: