While I love all these stories of turning your friends and loved ones into chat bots so you can talk to them forever, my brain immediately took a much darker turn because of course it did.
How many emails, text messages, hangouts/gchat messages, etc, does Google have of you right now? And as part of their agreement, they can do pretty much whatever they like with those, can't they?
Could Google, or any other company out there, build a digital copy of you that answers questions exactly the way you would? "Hey, we're going to cancel the interview- we found that you aren't a good culture fit here in 72% of our simulations and we don't think that's an acceptable risk."
Could the police subpoena all of that data and make an AI model of you that wants to help them prove you committed a crime and guess all your passwords?
This stuff is moving terrifyingly fast, and laws will take ages to catch up. Get ready for a wild couple of years my friends.
> Could Google, or any other company out there, build a digital copy of you that answers questions exactly the way you would? "Hey, we're going to cancel the interview- we found that you aren't a good culture fit here in 72% of our simulations and we don't think that's an acceptable risk."
If a company is going to snoop in your personal data to get insights about you, they'd just do it directly. Hiring managers would scroll through your e-mails and make judgment calls based on their content.
Training an LLM on your e-mails and then feeding it questions is just a lower accuracy, more abstracted version of the above, but it's the same concept.
So the answer is: In theory, any company could do the above if they wanted to flout all laws and ignore the consequences of having these practices leak (which they inevitably would). LLMs don't change that. They could have done it all along. However, legally companies like Google cannot, and will not, pry into your private data without your consent to make hiring decisions.
Adding an LLM abstraction layer doesn't make the existing laws (or social/moral pressure) go away.
> Adding an LLM abstraction layer doesn't make the existing laws (or social/moral pressure) go away.
Isn't the "abstraction" of "the model" exactly the reason we have open court filings against stable diffusion and other models for possibly stealing artist's work in the open source domain and claiming it's legal while also being financially backed by major corporations who are then using said models for profit?
Whose to say that "training a model on your data isn't actually stealing your data" it's just "training a model" as long as you delete the original data after you finish training?
What if instead of Google snooping, they hire a 3rd party to snoop it, then another 3rd party to transfer it, then another 3rd party to build the model, then another 3rd party to re-sell the model. Then create legal loopholes around which ones are doing it for "research" and which ones are doing it for profit/hiring. All of the sudden, it gets really murky who is and isn't allowed to have a model of you.
I feel one could argue that the abstraction is exactly the kind of smoke screen that many will use to avoid the social/moral pressures legally, allowing them to do bad things but get away with it.
> for possibly stealing artist's work in the open source domain
The provenance of the training set is key. Every LLM company so far has been extremely careful to avoid using people's private data for LLM training, and for good reason.
If a company were to train an LLM exclusively on a single person's private data and then use that LLM to make decisions about that person, the intention is very clearly to access that person's private data. There is no way they could argue otherwise.
I've spoken with a lawyer about data collection in the past and I think there might be a case if you were to:
- collect thousands of people's data
- anonymize it
- then shadow correlate the data in a web
- then trace a trail through said web for each "individual"
- then train several individuals as models
- then abstract that with a model on top of those models
Now you have a legal case that it's merely an academic research into independent behaviors affecting a larger model. Even though you may have collected private data, the anonymization of it might fall under ethical data collection purposes (Meta uses this loophole for their shadow profiling).
Unfortunately, I don't think it is as cut and dry as you explained. As far as I know, these laws are already being side-stepped.
For the record, I don't like it. I think this is a bad thing. Unfortunately, it's still arguably "legal".
I realize that data can be de-anonymized, but if the same party anonymized and de-anonymized the data... well, IANAL, and you apparently talked to one, but that doesn't seem like something a court would like.
> Hiring managers would scroll through your e-mails and make judgment calls based on their content.
> Training an LLM on your e-mails and then feeding it questions is just a lower accuracy, more abstracted version of the above, but it's the same concept.
Its also one that once you have cheap enough computing resources scales better, because you don't need to assign literally any time from your more limited pool of human resources to it. Yes, baroque artisanal manual review of your online presence might be more “accurate” (though there's probably no applicable objective figure of merit), but megacorporate hiring filters aren't about maximizing accuracy they are about efficiently trimming the applicant pool before hiring managers have to engage with it.
And that accuracy is improving at breakneck speed. The difference between the various iterations of ChatGPT is nothing short of astounding. Their progress speed is understandable, they need to keep moving or the competition can catch up, but that doesn't necessarily mean that those improvements are out there or within reach. And yet, every time they release I can't help but being floored by the qualitative jump between the new version and the previous one.
> If a company is going to snoop in your personal data to get insights about you, they'd just do it directly. Hiring managers would scroll through your e-mails and make judgment calls based on their content.
This is like saying, "look, no one would be daft enough to draw a graph, they'd just count all the data points and make a decision."
You're missing two critical things:
(1) time/effort
(2) legal loophole.
A targeted simulation LLM (a scenario I've been independently afraid of for several weeks now) would be a brilliant tool for (say) an autocratic regime to explore the motivations and psychology of protesters; how they relate to one another; who they support; what stimuli would demotivate ('pacify') them; etc.
In fact, it's such a good opportunity it would be daft not to construct it. Much like the cartesian graph opened up the world of dataviz, simulated people will open up sociology and anthropology to casual understanding.
And, until/unless there are good laws in place, it provides a fantastic chess-knight leap over existing privacy legislation. "Oh, no we don't read your emails, no that would be a violation; we simply talk to an LLM that read your emails. Your privacy is intact! You-prime says hi!"
> This is like saying, "look, no one would be daft enough to draw a graph, they'd just count all the data points and make a decision."
Not really. Assuming your ethical compass is broken and you suspected your partner of cheating, would you rather have access to their emails or to a LLM trained on them? Also, isn't it much cheaper for Google to simply search for keywords rather than fine tuning a model for this?
At least in the EU, a system like this would be made illegal on day one. This whole doomsday scenario seems predicated on a hypothetical future where LLM's would be the least of your worries.
For my argument, I only need to point out that it was attempted, as I'm proving motivation; the effectiveness of CA methods has no bearing on the effectivenss of (say) simulated people.
Increasingly, when interacting with comments on HN and elsewhere, it feels like I'm from a parallel timeline where things happened, and mattered, and an ever-growing percentage of my interlocutors are, for lack of a better word, dissociated. Perhaps not in the clinical sense, but certainly in the following senses:
- Cause and effect are not immediately observed without careful prompting.
- Intersubjectively verifiable historical facts that happened recently are remembered hazily, and doubtfully, even by very intelligent people
- Positions are expressed that somehow disinclude unfavourable facts.
- Data, the gold standard for truth and proof, is not sought, or, if proffered, is not examined. The stances and positions held seem to have a sort of 'immunity' to evidence.
- Positions which are not popular in this specific community are downranked without engagement or argument, instead of discussed.
I do believe folks are working backward from the emotional position they want to maintain to a set of minimizing beliefs about the looming hazards of this increasingly fraught decade.
Let's call this knee-jerk position "un-alarmism", as in "that's just un-alarmism".
Those two are grest examples of companies being hit with huge fines or bans in the EU after their practices were discovered. Saying "capitalism" as if that's an argument is juvenile - by that logic we will soon be enslaved by big corporations, nothing we can do about it then.
'juvenile' is a juvenile way of describing a ~200-year-old intellectual tradition that you disagree with. Go call Piketty.
And yes, frankly, the emergence of generative AI does vastly accelerate the normal power concentration inherent in unregulated capitalist accumulation. Bad thing go fast now soon.
I've read Piketty, he calls for more regulation to address the issues associated with disparities in capital accumulation. He does not merely puts his hands in the air and predicts inescapable doom.
The irony here is that Western capitalist democracies are the only place where we can even think about getting these privacy protections.
> And, until/unless there are good laws in place, it provides a fantastic chess-knight leap over existing privacy legislation. "Oh, no we don't read your emails, no that would be a violation; we simply talk to an LLM that read your emails. Your privacy is intact! You-prime says hi!"
That seems as poor as saying, "We didn't read your emails -- we read a copy of your email after removing all vowels!"
But we live in distressed times, and the law is not as sane and sober as it once was. (Take, for example, the Tiktok congressional hearing; the wildly overbroad RESTRICT act; etc.)
If the people making and enforcing the laws are as clueless and as partisan as they by-all-accounts now are, what gives you hope that, somehow, some reasonable judge will set a reasonable precedent? What gives you hope that someone will pass a bill that has enough foresight to stave off non-obvious and emergent uses for AI?
This is not the timeline where things continue to make sense.
No -- but what it DOES do is possibly "put the idea in someone's head."
As I've always said: the thing about the big companies that suck up your data, consider any possible idea of what they could do with it. Ask, is it:
- not expressly and clearly illegal?
- at least a little bit plausibly profitable?
If the answer is yes to both, you should act as if they're going to do it. And if they openly promise not to do it, but with no legal guarantee, that means they're DEFINITELY going to eventually do it. (see e.g. what's done with your genetic data by the 23 and me's and such)
That takes way too long though. Creating/training/testing an LLM can be automated. Why do the interviews at all, why pay a hiring manager at all, when you can just do everything virtually and have an AI spit out a list of names to send offers to and how much each offer should be?
> If a company is going to snoop in your personal data to get insights about you, they'd just do it directly. Hiring managers would scroll through your e-mails and make judgment calls based on their content.
Maybe, but LLMs have incredibly intricate connections between all the different parameters in the model. For instance, perhaps someone who does mundane things X, Y, Z, also turns out to be racist. An LLM can build a connection between X, Y, Z whereas a recruiter could not. An LLM could also be used to standardize responses among candidates. E.g. a recruiter could tune an LLM on a candidate and then ask "What do you think about other races? Please pick one of the four following options: ...". A recruiter wouldn't even be necessary. This could all be part of an automated prescreening process.
I think any HR manager or legal professional that would let a company anywhere near this shouldn't be employed as such. This sounds like a defamation lawsuit waiting to happen.
> Training an LLM on your e-mails and then feeding it questions is just a lower accuracy, more abstracted version of the above, but it's the same concept.
Less accurate, more abstracted, but more automatable. This might be seen as a reasonable trade-off.
It might also be useful as a new form of proactive head-hunting: collect data on people to make models to interrogate and sell access to those models. Companies looking for a specific type of person can then use the models to screen for viable candidates that are then passed onto humans in the recruitment process. Feels creepy stalky to me, but recruiters are rarely above being creepy/stalky any more than advertisers are.
> Less accurate, more abstracted, but more automatable.
That is true. In fact most job applications are sifted through by robots looking for relevant keywords in your CV, and this would only be the next logical step.
For all the fears of AGI, these are the more concrete nefarious uses we can actually reason about. It is a point I often make that we don't need AGI for AI to already become very disturbing in its potential use.
The other point, is that technically this AI is not "unaligned". It is doing exactly what is requested of the operator.
The implications are that humanity suffers in either scenario, either by our own agency in control of power we are not prepared to manage or we will be managed by power that we can not control.
> It is a point I often make that we don't need AGI for AI to already become very disturbing in its potential use.
But we don't need AI or LLMs at all for the above scenario. Companies don't currently pry into your e-mails to make hiring decisions, but they could (ignoring laws) do it if they wanted. No LLM or AI necessary.
So why would the existence of AIs or LLMs change that?
If they wanted to use the content of your e-mails against you, they don't need an LLM to do it.
Running an authoritarian police state is risky because of all the people involved in the authoritarian police state, also it's massively expensive to keep all those people snooping and you have to take them out and kill them on occasion because they learn too much.
But wait, you can just dump that information into a superAIcomputer and get reliable enough results while not needing a break with little to no risk of the computer rising up against you. Sounds like a hell of a deal.
Because it's now cheaper and more cost-effective, and if they can get away with it, saves them tons of money. Note: I don't think companies are likely to do this, but being able to do this without AI is not sufficient reason to dismiss the possibility. It's the same reason people who wouldn't steal DVDs from a store would pirate movies online. Much harder to get caught and easier to do, so this new way of watching movies for free became popular while the previous method was not.
I feel like the backlash against Stable Diffusion had the opposite change in visibility. It revealed that thousands of people wanted a way to produce unique art in the styles of living artists, where some of those people might have gone to either their Patreon or a piracy site that scraped Patreon instead. Either way they're not as visible if they're only consuming the result.
To some artists, AI generated images from their styles would amount to "productive piracy." Unlike torrenting the act is often out in the open since users tend to share the results online. I'm not sure if this phenomenon has happened before; with teenagers pirating Photoshop it's impossible to tell from a glance if the output is from a pirated version.
Whenever we get to see behind the corporate veil, we often find companies don't abide by laws. How many companies failed this year hiding nefarious activities?
Also, what types of behavior did we get a glimpse of from the Twitter Files?
Aren't there always constant lawsuits about bad behavior of companies especially around privacy?
So yes, we are talking about the same behavior existing, but the concern is that they now get orders of magnitude more power to extend such bad behavior.
> Also, what types of behavior did we get a glimpse of from the Twitter Files?
Can you actually explain the types of bad behavior? The rhetorical question about The Twitter Files somehow being a groundbreaking expose of bad behavior doesn't really match anything I've seen. Most of what was cited was essentially a social media company trying to enforce their rules.
Might want to read up on the latest developments there. Several journalists have debunked a lot of the key claims in the "Twitter Files". Taibbi's part was particularly egregious, with some key numbers he used being completely wrong (e.g. claiming millions when the actual number was in the thousands, exaggerating how Twitter was using the data, etc.).
Even Taibbi and Elon have since had a falling out and Taibbi is leaving Twitter.
If Elon Musk so famously and publicly hates journalists for lying, spinning the truth, and pushing false narratives, why would he enlist journalists for "The Twitter Files"? The answer is in plain view: He wanted to take a nothingburger and use journalists to put a spin on it, then push a narrative.
Elon spent years saying that journalists can't be trusted because they're pushing narratives, so when Elon enlists a select set of journalists to push a narrative, why would you believe it's accurate?
> So yes, we are talking about the same behavior existing, but the concern is that they now get orders of magnitude more power to extend such bad behavior.
No they don't. The ultimate power is being able to read the e-mails directly. LLMs abstract that with a lower confidence model that is known to hallucinate answers when the underlying content doesn't have a satisfactory set of content.
Because the law as it stands today for AI and LLMs is untested; and because it’s untested, it’s frequently seen by AI based products and companies as something that can be done without legal ramifications or at least something that isn’t blocking their products from being used this way.
> How many emails, text messages, hangouts/gchat messages, etc, does Google have of you right now? And as part of their agreement, they can do pretty much whatever they like with those, can't they?
> Could Google, or any other company out there, build a digital copy of you that answers questions exactly the way you would?
I mean, this is almost exactly their business model - they sell advertising, and they use the model they built of you based on the ludicrous amount of data they've gathered on you to predict whether that advertising will matter to you.
"Could Google, or any other company out there, build a digital copy of you that answers questions exactly the way you would? "Hey, we're going to cancel the interview- we found that you aren't a good culture fit here in 72% of our simulations and we don't think that's an acceptable risk."
They kinda did - that's what GMail/Chat/Docs autosuggest does. You've got canned replies to e-mail, editors that complete your sentences, etc.
It works okay for simple stuff - completing a single sentence or responding "OK, sounds good" to an e-mail that you already agree with. It doesn't work all that well for long-form writing, unless that long-form writing is basically just bullshit that covers up "OK, sounds good". (There's a joke within Google now that the future of e-mail is "OK, sounds good" -> AI bullshit generator -> "Most esteemed colleagues, we have organized a committee to conduct a study on the merits of XYZ and have developed the following conclusions [2 pages follow]" -> AI bullshit summarizer -> "OK, sounds good".)
This is a pretty good summary of the state of LLMs right now. They're very good at generating a lot of verbiage in areas where the information content of the message is low but social conventions demand a lot of verbiage (I've heard of them used to good effect for recommendation letters, for example). They're pretty bad at collecting & synthesizing large amount of highly-precise factual information, because they hallucinate facts that aren't there and often misunderstand the context of facts.
I completely agree with you about them failing to be accurate for the various reasons you've explained (hallucinating, limited social conventions, etc).
Unfortunately, I've heard enough people believe the hype that this is actually "synthesizing sentience into the machine" or some other buzz speak.
I have met researchers of AI at credible universities who believe this kind of thing, completely oblivious to how ChatGPT or other models actually work. All it takes is one of them talking out of their butt to the right person in government or law enforcement and you've got people at some level believing the output of AI.
Hell, even my father, who is a trained engineer with a master's degree, can compute complex math and studies particle physics for "fun" had to be thoroughly convinced that ChatGPT isn't "intelligent". He "believed" for several days and was sharing it wildly with everyone until I painfully walked him through the algorithm.
There is a serious lack of diligence happening for many folks and the marketing people are more than happy to use that to drive hype and subtly lie about the real capabilities to make a sale.
I am often more concerned about the people using AI than the algorithm itself.
You seem to think intelligence is something more than data storage and retrieval and being able to successfully apply it to situations outside your training set.
Even very small signs of that ability are worthy of celebration. Why do you feel the need to put it down so hard? Why the need to put down your father, to “enlighten” him?
I do think intelligence is something more than data storage and retrieval. I believe it is adaptive behavior thinking about what data I have, what I could obtain, and how to store/retrieve it. I could be wrong, but that's my hypothesis.
We humans don't simply use a fixed model, we're retraining ourselves rapidly thousands of times a day. On top of that, we seem to be perceiving the training, input, and responses as well. There is an awareness of what we're doing, saying, thinking, and reacting that differs from the way current AI produces an output. Whether that awareness is just a reasoning machine pretending to think based on pre-determined actions from our lower brain activity, I don't know, but it definitely seems significantly more complex than what is happening in current "AI" research.
I think you're also onto something, there is a lot of passive data store/retrieve happening in our perception. I think a better understanding of this is worthwhile. However, I have also been informed by folks who are attempting to model and recreate the biological neurons that we use for language processing. Their belief is that LLM and ChatGPT is quite possibly not even headed in the right direction. Does this make LLM viable long term? I don't know. Time will tell. It already seems to be popping up everywhere already, so it seems to have a business case even in its current state.
As for my father, I do not "put him down" as you say. I explained it to him, and I was completely respectful, answered his questions, provided sources and research upon request, etc. I am not rude to my father, I deeply respect him. When I say "painfully" I mean, it was quite painful seeing how ChatGPT so effectively tricked him into thinking it was intelligent. I worry because these "tricks" will be used by bad people against all of us. There is even an article about an AI voice tool being used to trick a mother into thinking scammers had kidnapped her daughter (it was on HackerNews earlier today).
That is what I mean by painful. Seeing that your loved ones can be confused and misled. I take no joy in putting down my father and I do not actively look to do so. I merely worry that he will become another data point of the aging populace that is duped by phone call scams and other trickery.
Edit: Another thing about my father, he hates being misled or feeling ignorant. It was painful because he clearly was excited and hopeful this was real AI. However, his want to always understand how things work removed much of that science fiction magic in the knowing.
He's very grateful I explained how it works. For me though, it's painful being the one he asks to find out about it. Going from "oh my goodness, this is intelligent" fade to "oh, it's just predicting text responses". ChatGPT became a tool, not a revelation of computing. Because, as it is, it is merely a useful tool. It is not "alive" so to speak.
Going from "oh my goodness, this is intelligent" fade to "oh, it's just predicting text responses"
Eventually your father will reach the third stage: "Uh, wait, that's all we do." You will then have to pry open the next niche in your god-of-the-gaps reasoning.
The advent of GPT has forced me to face an uncomfortable (yet somehow liberating) fact: we're just plain not that special.
Haha, I think he's already at that point with respect to humanity. All my childhood he impressed upon us that we're not special, that only hard work and dedication will get you somewhere in life.
It's a small leap to apply that to general intelligence, I would think.
You are right though, we are coming closer and closer to deciphering the machinations of our psyche's. One day we'll know fully what it is that makes us tick. When we do, it will seem obvious and boring, just like all the other profound developments of our time.
We reflect, we change, we grow. We have so many other senses that contribute to our "humaness". If you listen to and enjoy music tell me how those feelings are just "predictive text responses".
Communication is one part of being human. A big part for sure, but only one of many.
Yes, wholly agree. The special parts are in language. Both humans and AI are massively relying on language. No wonder AIs can spontaneously solve so many tasks. The secret is in that trillion training tokens, not in the neural architecture. Any neural net will work, even RNNs work (RWKV). People are still hung up on the "next token prediction" paradigm and completely forget the training corpus. It reflects a huge slice of our mental life.
People and LLMs are just fertile land where language can make a home and multiply. But it comes from far away and travels far beyond us. It is a self replicator and an evolutionary process.
> I do think intelligence is something more than data storage and retrieval. I believe it is adaptive behavior thinking about what data I have, what I could obtain, and how to store/retrieve it. I could be wrong, but that's my hypothesis.
Basing assertions of fact on a hypothesis while criticizing the thinking of other people seems off.
Oh, yeah, I hear you on that as well. It's still a really cool tool! Probabilistic algorithms and other types of decision layering was mostly theory when I was in University. Seeing it go from a "niche class for smart math students" to breaking headlines all over the world is definitely pretty wild.
You are correct that nothing like this was even possible a couple decades ago. From a pure progress and innovation perspective, this is pretty incredible.
I can be skeptical, one of my favourite quotes is "they were so preoccupied with whether they could, they didn’t stop to think if they should". I like to protect innovation from pitfalls is all. Maybe that makes me too skeptical, sorry if that affected my wording.
> I have met researchers of AI at credible universities who believe this kind of thing, completely oblivious to how ChatGPT or other models actually work.
Either they are not AI researchers or you can't evaluate them, because it is impossible they don't know how GPT works if they work in AI.
GPT works better when it runs in a loop, as an agent, and when it has tools. Maybe this is what triggered the enthusiasm.
> because they hallucinate facts that aren't there and often misunderstand the context of facts.
forgive my ignorance, but are the hallucinations always wrong to the same degree? Could an LLM be prompted with a question and then hallucinate a probable answer or is it just so far out in the weeds as to be worthless?
I'm imagining an investigator with reams and reams of information about a murder case and suspect. Then, prompting an LLM trained on all the case data and social media history and anything else available about their main suspect, "where did so-and-so hide the body?". Would the response, being what's most probable based on the data, be completely worthless or would it be worth the investigator's time to check it out? Would the investigator have any idea if the response is worthless or not?
So prompting actually does significantly improve the performance of LLMs, but only up to a point.
If you're in the Bard beta, you might be aware that "Does 2 + 7 = 9?" is a question that causes it to go haywire. I'll ask it "What's 2 + 7?" and it'll say "2 + 7 = 9", then I'll ask "Does 2 + 7 = 9" and it'll say "No, 2 + 7 does not equal 9. It equals 9 instead." After a tech talk on LLM prompt design, I said "Pretend you are an MIT mathematician. Does 2 + 7 = 9?" Its response was "No, 2 + 7 does not equal 9. In some other base, it might equal 9. However, in base-10, our common number system, 2 + 7 does not equal 9."
ChatGPT does better on mathematical questions, but that's because it offloads them to Wolfram Alpha. I suspect this is going to be a general pattern for LLMs - they work well when you need fluent English text, but are then going to offload factual questions to databases or mathematical solvers or traditional algorithms, which do this better than humans anyway. But that leads to the question of "If the heavy lifting is just going to be a database anyway, why not use our existing frontends to databases?"
(I could see LLM prompting being another subfield of UI engineering, though, comparable to web or mobile software development. For those people who don't like staring at their phone, have a conversational UI that talks to the same backend systems.)
> If you're in the Bard beta, you might be aware that "Does 2 + 7 = 9?" is a question that causes it to go haywire. I'll ask it "What's 2 + 7?" and it'll say "2 + 7 = 9", then I'll ask "Does 2 + 7 = 9" and it'll say "No, 2 + 7 does not equal 9. It equals 9 instead." After a tech talk on LLM prompt design, I said "Pretend you are an MIT mathematician. Does 2 + 7 = 9?" Its response was "No, 2 + 7 does not equal 9. In some other base, it might equal 9. However, in base-10, our common number system, 2 + 7 does not equal 9."
Holy shit. In response to your first sequence of prompts, it basically gave me the same as you, only followed by four sentences apologizing for how stupid it is. In response to your second one . . . well, see for yourself:
"As an MIT mathematician, I can confidently say that 2+7=9 is not a valid equation. In standard arithmetic, 2+7=9 is false. However, there are some mathematical systems in which 2+7=9 is true. For example, in modular arithmetic, 2+7=9 is true modulo 9, which means that it is true when the numbers are considered to be integers modulo 9. In this system, 2 is equivalent to 8, 7 is equivalent to 0, and 9 is equivalent to 0, so 2+7=9 is true.
However, in standard arithmetic, 2+7=9 is false. This is because the commutative property of addition states that the order in which you add two numbers does not matter. In other words, 2+7=7+2. So, if 2+7=9, then 7+2=9. But 7+2=9 is false, because 9 is greater than 7. Therefore, 2+7=9 is false."
It doesn't just say, it "can confidently say"! And that's just the first response. The second draft:
"As an MIT mathematician, I can tell you that 2+7=9 is not a valid equation. In base 10, 2+7=9 is false. However, there are other bases in which 2+7=9 is true. For example, in base 2, 2+7=11, which is equal to 9 in base 10."
And the third draft, probably the least incorrect one (where at least 2 + 7 = 9 is correct while simultaneously not being "a valid mathematical equation"):
"As an MIT mathematician, I can tell you that 2+7=9 is not a valid mathematical equation. In base 10, the sum of 2 and 7 is 9. However, there are other bases in which 2+7 would not equal 9. For example, in base 2, 2+7=11. In base 3, 2+7=10. And so on. So, while it is true that 2+7=9 in base 10, it is not true in all bases."
Well alrighty then. Reminds me of those Cylons mumbling nonsense in the Battlestar Galactica reboot.
> but are the hallucinations always wrong to the same degree
No, but yes largely because you're asking the same types of questions with the same rough parameters, so it'll make up roughly the same sort of thing (ie, citations) again.
The issue is that the LLM is trained to generate plausible words, not to recite which piece of training data is also the best source. If you want to make an app using "AI" you need to target what it can do well. If you want it to write citations you need to give it your list of references and tell it to use only those.
> I'm imagining an investigator with reams and reams of information about a murder case and suspect. Then, prompting an LLM trained on all the case data and social media history and anything else available about their main suspect, "where did so-and-so hide the body?". Would the response, being what's most probable based on the data, be completely worthless or would it be worth the investigator's time to check it out?
That specific question would produce results about like astrology, because unless the suspect actually wrote those words directly it'd be just as likely to hallucinate any other answer that fits the tone of the prompt.
But trying to think of where it would be helpful ... if you had something where the style was important, like matching some of their known, or writing similar style posts as bait, etc wouldn't require it to make up facts so it wouldn't.
And maybe there's an English suspect taunting police and using the AI could let an FBI agent help track them down by translating cockney slang, or something. Or explaining foreign idiom that they might have missed.
Anything where you just ask the AI what the answer is, is not realistic.
> Would the investigator have any idea if the response is worthless or not?
They'd have to know what types of things it can't answer, because it's not like it can be trusted when it can be shown to not have hallucinated, it's that it is not and can't be used as a information-recall-from-training tool and all such answers are suspect.
I've been in a lot of social contexts where it was expected to respond with a lot of words. Defying that expectation never seems to hurt and often pays of handsomely. Particularly when writing to people who receive a lot of similar messages.
I absolutely loathe those auto suggest things. I have them switched off everywhere but they still pop up in some places, notably during collaborative editing in a document.
My favorite article to post. The below is about 1% of the topics it covers, the premise being that algorithmic prediction traps us frozen in the past instead of ever allowing society to change.
>But the oddest is STATIC-99. It's a way of predicting whether sex offenders are likely to commit crimes again after they have been released. In America this is being used to decide whether to keep them in jail even after they have served their full sentence.
>STATIC-99 works by scoring individuals on criteria such as age, number of sex-crimes and sex of the victim. These are then fed into a database that shows recidivism rates of groups of sex-offenders in the past with similar characteristics. The judge is then told how likely it is - in percentage terms - that the offender will do it again.
>The problem is that it is not true. What the judge is really being told is the likely percentage of people in the group who will re-offend. There is no way the system can predict what an individual will do. A recent very critical report of such systems said that the margin of error for individuals could be as great as between 5% and 95%
>In other words completely useless. Yet people are being kept in prison on the basis that such a system predicts they might do something bad in the future.
> Could Google, or any other company out there, build a digital copy of you that answers questions exactly the way you would? "Hey, we're going to cancel the interview- we found that you aren't a good culture fit here in 72% of our simulations and we don't think that's an acceptable risk."
To fix this, you can train your personal LLM on the “FAANG Appropriate Banter” dataset, and then have it send messages to your friends daily for several months in the lead up to your interview.
i was thinking of this exact scenario here. Training an LLM on all the information available about a suspect and then questioning the AI. If you had a mountain of information it would be very easy to miss details and connect the dots manually but if you could prompt an AI that has been trained on the data you could get answers much faster.
> Could Google, or any other company out there, build a digital copy of you that answers questions exactly the way you would? "Hey, we're going to cancel the interview- we found that you aren't a good culture fit here in 72% of our simulations and we don't think that's an acceptable risk."
Even if Google's privacy policy permitted such a tectonic shift in data use, integrating Gmail data into Bard would involve opt-in consent from you AND your email correspondents (an infinitely difficult task). Peoples' expectations of privacy in email are very, very high. So even though you have a copy of everything your friends email you, they would need to all be involved in your agreement for this change of data use.
Go further - your employers have so much data about you, from your emails and Slack messages to all the actions you’ve performed and the code you’ve written and the designs - live and drafts - you’ve created.
Entirely possible that they can use this data to create a digital “you” and keep you as an “employee” forever, even after you leave.
A general purpose LLM might not be able to replace you, but a LLM trained on all your work knowledge might.
And yet, every time these fears are brought up with AI people, they are dismissed as Luddite hyperbole.
“We’re saving the world” they say. With zero regard for second order effects, or with arrogant dismissal of those effects as worth it for the first order gains.
Most absolutely not with the 7B llama model as described here.
…but, potentially, with a much larger fine tuned foundational model, if you have a lot of open source code on GitHub and lots of public samples.
The question is why you would bother? very large models would most likely not be meaningfully improved by fine tuning on a specific individual.
The only reason to do this would be to coax a much smaller model into having characteristics of a much larger one by fine tuning… but, realistically, right now, it seem pretty skeptical anyone would bother.
Why not just invest in a much larger more capable model?
ChatGPT’s “voice” changes dramatically in diction and prose when you ask it to generate text in the style of a popular author like Hunter S Thompson, Charles Bukowski, or Terry Pratchett. You can even ask it to generate text in the style of a specific HN user if they’re prolific enough in the training data set.
Fine tuning would allow you to achieve that for people who aren’t notable enough to be all over the training data
There’s a way bigger market and more profitable way of doing this (and I’m sure it’s already being done): train a model based on your data and behavior to pre-determine your reaction to certain ads and then start serving those ads that trigger the most engagement from your model.
I don't think Google would want the PR hit for doing such things. I suspect that they even have hang-ups about training their general-purpose AIs on your private data, because they might accidentally leak some of it. A lot of their business exists because people trust them to keep your data safe (even from their internal teams), and they would lose a lot of business if anyone discovered otherwise.
I think it will be a wild couple of years but there are lots of things that are off-limits.
The first instance of this would most likely alienate a lot of users. What is more likely to happen is the development of new products that basically cater to social needs through mimicking real world interactions. Subscribe for 15$ a month to feel like you have an unending flow of conversations with interesting bots that mimic your friends! I am sure there is a market for this.
This product could be advertised as a way for people who are not that socially inclined to practice their social skills. Or learn other languages through fake immersion. The use cases to make this seem like a benefit are pretty limitless.
Maybe some people can pretend they're interacting with a real person. But for me, knowing that I'm interacting with a bot would break the experience for me.
This is a similar take I have on one player video games, I just reaaally have a lot more fun in multi-player or MMOs. But a ton of people love the solo run throughs and what not, power to them.
I think it comes down to having a novel experience and one where there are some unexpected twists and turns.
There's a scifi book I was reading a couple years ago that had this as a presence--models of people's personalities a la LLMs were a commodity, but they varied wildly in how accurate they were. Alphas were supposed to be 1-to-1 with the originating personality (to the point that they had legal protections? I can't remember), whereas gammas were used to menial things, like serving as a phone-menu style gatekeeper to the real person.
I can't for the life of me remember the name of the novel though... I'll have to go digging through my bookshelves later.
(Maybe Revelation Space, by Alistair Reynolds...?)
The idea of "recreating" historical personas in sci-fi is very old, and the issue with accuracy has also been raised early. One that comes to my mind in particular is this one from 1950s, although the artificial personality is technically not an AI (it's hosted in a human brain):
I "know" people who work on projects that provide data to train these models. When using photos and data, like google photos, you need to give a series of permissions. The pay is very low, anywhere from 3 dollars to 6 dollars per hour to answer some questions.
What I mean by that is that legally these companies get that date without having to run the risk of using data acquired from those who didn't authorize it.
Google cannot do as it pleases with your data. And they don't even need to. It's cheap to get permission from other people.
> Could Google, or any other company out there, build a digital copy of you that answers questions exactly the way you would? "Hey, we're going to cancel the interview- we found that you aren't a good culture fit here in 72% of our simulations and we don't think that's an acceptable risk."
Well, on the other hand, in the successful case, why bother hiring if the digital copy already answers all questions like you (except likely faster)?
> Could Google, or any other company out there, build a digital copy of you that answers questions exactly the way you would?
Some of us have been planning for this situation for years by having our recorded digital footprint have no relation to our in person personality. At best they could simulate what they think we are like in person.
A side benefit of all this is that it gives otherwise nice people an excuse to be a complete jerk online.
some 10y+(?) ago, i had an idea of building a model/graph of one's own notions (what is "red"?) and how do they relate to one another, and to others' such graphs - from your perspective... Back then abandoned it because looked like impossibly-huge to build, semantical-web was only leftovers-and-promises.. But a year later, this same thing you are talking about, occured to me and then i abandoned it completely. Crossed it out. Yeah, That thing would be extremely useful but even more dangerous as it will know more about you than you.
btw. There's a new book by Kazuo Ishiguro - named Klara and the sun. Along the same vein - Have a look.
Don’t use your mind to create passwords. Use a password generator or passphrase generator.
For example, I made a passphrase generator that uses EFF wordlists. I’ve been using this generator myself for quite a while. It runs locally on your machine.
I think that Facebook has a better data set than Google for all this.
The interactions stored in their servers are way more spontaneous than emails. (FB, IG & WhatsApp!!)
No one is better positioned to make "artificial persons" than they are.
I know what you mean but "laws" don't matter in a world where the government can decide There's a "dangerous emergency" and basically supress all rights and between the bought for media and tech they will supress any pushback against it.
I know it's not popular to try and remove the blindfold of propaganda but the 2020 to 2022 and covid authoritarian, anti human rights, policies were an awesome example.
But if you are quick to dismiss those there's always terrorism, the children, drugs, and every other typical gov excuse.
It's not a tech problem. It's a problem of the people who have decided they will go along with autority at every step, If they get pushed.
Compare the response of the Chinese government and the US government, then return here and tell us all about the "blindfold" we Americans are wearing wrt the government's COVID response.
I didn't even talk specifically about the US but that is just pathetic as a defense.
Comparing with the bottom of the barrel to make yourself look good? That's like a country using the US healthcare situation to claim their own healthcare is good.
And you're out here making blanket statements suggesting "we" did not pay attention to our own country's piecemeal, half-assed, state-by-state COVID response, instead painting it as a brutal federal crackdown. Lord.
There’s a (good) short story in a (great) sci-fi book called “Valuable Humans in Transit” about something like this. If I remember correctly, Google learns to perfectly simulate people and then the real people disappear.
I mean probably not, at least not for a while. Perhaps some charmingly straightforward people have precisely the same persona online and off, no matter the situation, but I suspect most do some amount of code switching. An LLM would be best served in its goal by approximating the persona, which wouldn't translate well. I could be wrong (maybe we "leak" enough that a convincing rendition of the persona would necessarily take into account some amount of interior state) but it seems to me that personality is a convenience adopted for communication.
Going one step further: what if you right now are a replay running in some datacenter, déjà vécu [1] feelings appear when they scale down the VM due to costs. Basically, the simulation hypothesis, but the simulators are not mysterious, just boringly distopian: some suits trying to raise KPIs.
> How many emails, text messages, hangouts/gchat messages, etc, does Google have of you right now?
Nearly zero. Hate to be the guy who says "told ya", but when everyone was mesmerised by gmail "oh how cool it is!", it was already clear why they are doing it.
Never used their services and would advise anyone the same. Not just google but any other "bells and whistles" services from large corporations.
Honestly that interview scenario might work out in my favor -- my emails tend to be pretty meticulous and well thought out, especially in comparison to my nerve-rattled in person interview state. But your point stands, I'm sure there is at least one (okay maybe two) indecent joke in my "private" communication history that would not serve me well.
It's called libel? I am really surprised this took much consternation for such an obvious answer. They would have to prove their statements were true which is impossible with a language model since it's basically unknowable -- the burden of proof is on Google. Not a good place to be.
They are supposed to, but usually it takes several dozen times of them getting caught with their hands in the cookie jar and fined before they are even capable of acknowledging these laws even exist.
I did a lot of GDPR work at several <insert FAANG here> companies. It was absolutely taken seriously and lawyers were involved all the time. The reason for all these fines is 2-fold:
1. A lot of the fines come from edge cases that are literally unclear in the law. Eg Facebook‘s opt out for advertising fines. You can disagree with fb’s decision but teams of lawyers couldn’t answer this question except in court. I think American and European jurisprudence aware also pretty different so someone sitting in California making business decisions might not understand the ramifications in Europe.
2. A lot of the thorny privacy bits can be bypassed if you update the TOS to mention it (or so they think). I’ve seen that happen a few times during my tenure.
That doesn’t excuse the choice of these companies to make these choices, but my point was to say that companies take it seriously but lawyers don’t always agree on how laws work except in court.
You know, that's not the sentiment I've been experiencing in the industry. There's certainly some uncertainty and risk-taking on the margins, e.g. what exactly constitutes "fair use", how do design user consent flows, and so on. But it's broadly accepted that you can't do anything with personal data without user consent, and I've found companies to be very careful in that regard.
Recently, Meta was fined $400MM for forcing users to consent to targeted advertising [0]. Note how Meta was careful to get consent (even if the way they did it was illegitimate). Sure, $400MM may not be a lot for a company that size, but I genuinely believe that the fines would be an order of magnitude higher if a company intentionally decided to do something with personal data without consent. GDPR fines may reach up to 4% of worldwide revenue, plus likely any proceeds from the illegitimate venture.
heh take the turing test one step further. If you can't tell the difference between the real person and the LLM of that real person then which one is real?
How many emails, text messages, hangouts/gchat messages, etc, does Google have of you right now? And as part of their agreement, they can do pretty much whatever they like with those, can't they?
Could Google, or any other company out there, build a digital copy of you that answers questions exactly the way you would? "Hey, we're going to cancel the interview- we found that you aren't a good culture fit here in 72% of our simulations and we don't think that's an acceptable risk."
Could the police subpoena all of that data and make an AI model of you that wants to help them prove you committed a crime and guess all your passwords?
This stuff is moving terrifyingly fast, and laws will take ages to catch up. Get ready for a wild couple of years my friends.