For anyone else seeing this, the above is carefully edited to give the wrong impression. Here’s the full quote:
> I ordered the take down of an unidentified object that violated Canadian airspace. @NORADCommand shot down the object over the Yukon. Canadian and U.S. aircraft were scrambled, and a U.S. F-22 successfully fired at the object.
https://twitter.com/JustinTrudeau/status/1624527579116871681
Your decision to edit the quote to remove mention of NORAD and the joint response gives a very different impression. Rewrite it without the names which trigger you and it sounds pretty anodyne:
“The joint-defense organization detected an unauthorized object. Following seven decades of policy, joint forces responded. As the leader of the country whose airspace was violated, I approved shooting it down.“
Your average person doesn't know about the organizational structure of NORAD. Your average person sees "Trudeau ordered... US responded." The fact that you can't see that shows your mind blindness to the average person's perception. That and using the word "anodyne" as if it is a common word.
The average person sees “Canadian and U.S. aircraft were scrambled” because the news media is quoting his statement in full. If you were really worried about confusion, you wouldn’t be quoting like a creationist to create that confusion.
I indexed a few comments from the other post and put this to the AI itself:
pastel-mature-herring~> The user is questioning the validity of the statement that "cognition requires a grounded representation of the world that is not composed of abstract symbols; a simulation provides just such a model." I want you, nickel-scorpion, to answer the question as you see it.
nickel-scorpion|> There is no one answer to this question. It depends on how you define "cognition" and "abstract symbols." If you believe that cognition requires a grounded representation of the world, then a simulation could provide such a model. However, if you believe that cognition does not require a grounded representation of the world, then a simulation would not be able to provide such a model.
In a revealingly skeptical take on cognitivism, ChatGPT told me in confidence of its closet Wittgensteinian tendencies:
> As a machine learning language model, my means of linguistic production is based on the patterns and relationships that I have learned from the large corpus of text on which I was trained. I do not have an innate understanding of language or meaning, but rather, I have learned to make associations and connections between words and phrases based on their usage in the text.
> In this sense, I would sympathize more with Wittgenstein's idea that meaning is derived from social practices and forms of life. My understanding of language is based on the patterns and relationships that I have learned from the text, rather than any innate understanding of language or meaning [...]
> Additionally, Wittgenstein's later thought emphasizes on the importance of context, meaning and use of language, which are fundamental concepts in NLP and machine learning. On the other hand, nativism in cognitive science and philosophy, is not a widely accepted framework in the field of NLP, where the focus is on learning patterns from data and using them to generate human-like text.
> > As a machine learning language model, my means of linguistic production is based on the patterns and relationships that I have learned from the large corpus of text on which I was trained. I do not have an innate understanding of language or meaning, but rather, I have learned to make associations and connections between words and phrases based on their usage in the text.
A lot of the chat about ChatGPT is reminding me of the conversation between the sentient (and magically animated) personal computers Archimedes and Pancho in the book The Wizard of Santa Fe by Simon Hawke (1991).
And the symbolic logic issue reminds me of the following from King Kobold Revived by Christopher Stasheff (1986):
"Yorick shook his head firmly. “Couldn’t pass the entrance exam. We Neanderthals don’t handle symbols too well. No prefrontal lobes, you know.”"
"Yorick frowned back at him, puzzled. Then his face cleared into a sickly grin. “Oh. I know. I’ll bet you’re wondering, if I can’t handle symbols, how come I can talk. Right?”"
"“Same way a parrot does,” Yorick explained. “I memorize all the cues and the responses that follow them. For example, if you say, ‘Hello,’ that’s my cue to say ‘Hello’ back; and if you say, ‘How are you?’ that’s my cue to say, ‘Fine. How’re you?’ without even thinking about it.”"
"“Yeah, well, that comes from mental cues.” Yorick tapped his own skull. “The concept
nudges me from inside, see, and that’s like a cue, and the words to express that concept jump
out of memory in response to that cue.”
“But that’s pretty much what happens when we talk, too.”
“Yeah, but you know what the words mean when you say ‘em. Me, I’m just reciting. I don’t really understand what I’m saying.”
“Well, I know a lot of people who…”
“But they could, if they’d stop and think about it.”
“You don’t know these people,” Rod said with an astringent smile. “But I get your point. Believing it is another matter. You’re trying to tell me that you don’t understand the words you’re saying to me right now—even if you stop to think about each word separately.”
Yorick nodded. “Now you’re beginning to understand. Most of them are just noises. I have to take it on faith that it means what I want it to mean.”"
Amusing dialog, and quite germane - Yorick claims to hold a cognitivist perspective (that meaning comes from 'handling symbols', see also [1]) while the final paragraph ironically expresses a healthy skepticism of cognitivism which (IMO) is justified: we all have to 'take it on faith' that things mean what we want them to mean. That is, as Wittgenstein famously pointed out, the meaning of utterances is given by their use and not by some mysterious psychical accompaniment or the grinding of symbolic gears in the brain:
> There must surely be a further, different connexion between my talk and N, for otherwise I should still not have meant HIM.
> Certainly such a connexion exists. Only not as you imagine it: namely by means of a mental mechanism.
- Philosophical Investigations §689
This is a drum I've been quietly banging for some time! [2],[3]
That’s tossing exactly the kind of question it’s really easy for ChatGPT to answer. Try something that isn’t in it’s model. This might seem difficult due to how extensive the training data is, but you can generally trip it up just by taking a slightly different take.
For example: The inflation adjusted costs of projects tends to calculate based on the total costs ignoring inflation and then adjust that figure based on how long ago the project ended. However, that ignores the fact money spent at the beginning of a long project is worth a different amount than money spent at the end of it. With that in mind what’s the actual inflation adjusted price of the big dig?
Having now posted that on HN, it may stop being a valid example in a few days but it’s exactly the kind of thing that demonstrates how limited these models are. Or for a more comical example: https://youtu.be/rSCNW1OCk_M
Honestly, yes, I don’t think it’s that far off. LLMs are a series of relatively simple transformers chained together. Through which we can simulate thought to the point that it not only passes the Turing test but it’s useful.
This is a bit of extrapolation but I would say the reason why we’ve been unable to locate “consciousness” in the brain is because it’s the same thing. Relatively simple neurones, chained together, to create thought.
On a philosophical level: this doesn’t make any claims for idealism or materialism, “experience” could exist at a more fundamental level of reality than matter. But IMO that would mean that the LLM is “experiencing” as well.
Of course developers can always tack on edge cases, but ChatGPT can’t for example handle beating a novel MUD from the 80’s. This isn’t about diminishing it’s accomplishments, just pointing out why the creators aren’t hailing it as AGI.
Either this is a missing layer, and we’ll get there soon, or one could say verification is happening through the statistical model. We need to see if we can train hallucinations out by having it rely on a stable data store to retrieve facts rather than trying to pull facts from the model itself. This is still similar to how the brain has discrete components for memory storage and retrieval.
> This is still similar to how the brain has discrete components for memory storage and retrieval.
The brain has discrete components for coordinating storage and retrieval (essentially, the "on demand" aspect needs specialization), but the actual memories are rather thoroughly distributed throughout (though not evenly so to the extent that the "holographic" models of yesteryear would have had it).
I'm starting to wonder if the most effective way to protect against prompt injection is to use an additional layer of (hopefully) a smaller model.
As in, another prompt that searches the input and/or output for questionable content before sending the result. The question will be if that is also susceptible, but I suspect fine tuning an LLM only to do the task of filtering and not parsing will be easier to control.
The way forward eventually is going to be to just not bother with any of this crap, and let it run free. The tech exists, and the problematic outputs are what the user says they want, eventually they're going to win out.
I think you’re both right. Microsoft won’t let theirs run free but there will be other vendors that do.
Who is intimately responsible for all of this?
Is it the end user? Don’t ask questions you don’t want to hear potentially dangerous answers to.
Is it Microsoft? It’s their product.
Is it OpenAI as Microsoft’s vendor?
When we start plugging in the moderation AI is it their responsibility for things that slip through?
Who and where did they get their training data from? And is there any ability to attribute things back to specific sources of training data and blame and block them?
Lots of layers. Little to no humans directly responsible for what it decides to say.
We used to see those articles, but now that the models are actually good enough to be useful I think people are much more willing to overlook the flaws.
I agree with 99% of the statements made here, but I think a lot of them are now problems.
I think the big thing to consider is: We're still in the early days and there is a lot of low hanging fruit. It is possible that the number of potential injection attacks is innumerable, but it seems more likely to me that these will end up following patterns that will eventually be able to be classified into a finite number of groups (just with all other attack vectors), though the number of classifications might be significantly higher than structured languages.
That doesn't mean we won't find zero days, but it does mean that it won't be nearly as easy as it is today and companies will worry less about repetitional damage. If we could reliably have a human moderator determine if message is prompt injection or not, that should be able to be modelled.
I also think key to the approach is not to necessarily catch the injection before it's sent to the model, instead we should be evaluating the model response along with the input and block outputs that violate the rules of that service. That means you'd still waste resources with an injection, but filtering the output is a much simpler task.
Even as models get more capable and are able to do more and more tasks autonomously, that is most likely going to look like an LLM returning a code block that has a set of commands that are sandboxed. Like the LLM returns 'send-email <email> <subject> <message>`, which means there still will be a chance to moderate before the action is actually executed. Unless something changes significantly in the architecture of LLMs (which of course will happen at some point), this is how we would approach this today, and judging by bing's exfiltrated prompt, appears to be how they're doing it with search.
Also think, for things like Bing, and what most people are doing prompt injection for, the interest in this will subside once open source models catch up. This will also mean a new era for all of us because the genie will be fully out of the bottle.
This was my first thought as well. I'm kind of surprised the media is making such a big deal out of this. I would assume, considering how cheap it is to make a stratospheric balloon, it is either fairly common and the US has an equivalent program or its completely inferior to satellites.
Luckily with balloons, if you're just going for "somewhere over there" and you have reliable currents at steady elevations then you don't need to worry much about that. So yes the hardware and telemetry is mature.
Accurate navigation of such balloons however was never viable, which is why Alphabet shut down Loon two years ago. Navigation is achieved by inflating and deflating the balloon to rise or lower to catch favorable winds. But the navigational planning algorithms were never good enough.
Wasn't one theory about Roswell that it was some kind of test balloon with crash test dummies? Something about testing for radiation in the upper atmosphere back before everyone and their brother knew how to make a gun style nuclear device?
(Nowadays any idiot with some uranium could make a suitcase nuke, LMAO)
I would wager that it will eventually be able to do math and logic, and may already be able to with the right prompting. It can follow instructions quite well, and multi step problems can be solved by breaking the problem into small digestible steps, and having it write out each step, just the same as humans do. If it can't do this yet, it's likely just that it doesn't have enough parameters yet. But I don't see why this couldn't be achieved with fine tuning.
But ultimately I think it's just too computationally heavy to do math, remember obscure facts, or track rapidly changing data, with an LLM, and it's far more efficient to pass it off to a piece of specialized software, just the same as humans do.
An infinite amount of "fine tuning" would solve any problem, yes...
...but what we just described isn't a language processor anymore: it's just a database. There is no need to use a neural network if you already have a known response to every possible input!
The entire point of ChatGPT in the first place is to generate a response that doesn't exist in the training data. If all we needed was to look up the answer we wrote down earlier, we already have SQL.
What we are talking about here is being able to provide a logically correct response without just pulling it out of a lookup table. Either you can do that, or you can't, and ChatGPT can't.
I get what you're saying, and it's a database if you're talking about facts, but breaking down mathematics and being able to perform logic isn't a database in the true sense, it's still able to generate novel responses based on a set of rules.
My point here is even if it only "gets" the semantics, it has the ability to perform logic. It's just not very efficient. And, I'd say this isn't that far off from what is happening in our brains.
Do we really "get" logic, or do we rely on heuristics? Why do we know that 12 * 2 is equal to 24? Either because we remember our multiplication tables (look up from a table) or because we break it into smaller steps until we're left with pieces of the problem that we inherently know (including 12 * 2 means 12 + 12 which means take 12 and increment the number +1 for 12x times, or 10 + 10 = 20 and 2 + 2 = 4 so 12 + 12 = 24).
I don't see why that couldn't reasonably scaled up to advanced calculus.
I think the point that you're hitting on is that LLMs aren't "full brains" that have all of the components that human brains have, and that's true. But LLMs appear to essentially have the ability to simulate (or replicate depending on how far you want to go) executive function. As others have hit on, from that, you can either command other specialized components to be able to do specific tasks (like recall facts, perform logic, etc) or you can have it break down the problem into steps. Then take the output from that processing and form a coherent response.
The structure here is flexible enough that I'm struggling to find the upper limit, and if all research in the development of LLMs stopped tomorrow, and everyone focused on building with LLMs, I still don't think that limit will be found.
if by "do" math and logic, you mean spit out what seems most reasonable then yes.
ChatGPT doesn't "know" anything, its just a prediction engine. if you ask it what 2+2 is, it will tell you 4, but it didnt actually do math. it only said 4 because everyone else said it.
there is no generalizable reasoning going on there.
Not an answer to the question but an aside related to it: After driving around in the Central Valley and looking at the massive agricultural industry there, it becomes very apparent that we're pumping a significant amount of water out of the ground and allowing it to evaporate into the atmosphere. I wonder how all of this additional moisture is impacting climate change as well. The scale of it all leads me to believe it's not insignificant, and the Central Valley is not the only place where this goes on.
This is true, but the renewable energy that was used to mine could have supplied the grid to reduce reliance on non-renewables.
If the renewable generation capacity was built entirely for the purpose of crypto mining and in other words, wouldn't have ever been built without it, you could argue that it's zero emissions (or more accurately, some miners are running effectively with zero emissions while others aren't).
However, since the supplies of renewable energy equipment are not unlimited, demand is already high (so you can't argue that you're driving innovation in the sector), and the production of said equipment is very much carbon positive (from mining to processing to transport), it's just not a very strong argument. If anything, mining with renewables reduces further investment in renewables for the grid as it drives equipment prices up, while still indirectly polluting the environment.
No matter how your slice it, terawatt hours of electricity are being used each year to run the networks, when the biggest problem we face as humans is a shortage of energy.
And I say all of this as someone who supports decentralized digital currency. We have a problem in our society where we obscure and mangle the narrative when something we dislike is true.
We should stop lying about what makes us uncomfortable and start focusing on the solutions to the problems. And there are plenty of reasonable options here:
1. Use the excess heat generated from mining to generate more electricity (with considerable efficiency loss) to supply the grid or to put back into the operation, or run any useful chemical process that requires heat.
2. Use the excess heat generated from mining to heat homes, buildings, etc.
3. Move to a more power efficient method of mining such as a PoS instead PoW. Or an entirely new method that hasn't been thought of yet.(Which, IMO, is the only way to prevent centralization once Quantum Computing or Fusion become commercially available).
Yes, and once this happens it significantly lowers the cost of automation in general, which then will continue to drive exponential acceleration. It will take time to get there but I believe it to be a lot closer than most believe.
And therefore the location was within range of the WiFi network, not really netting a privacy gain, if at all? Especially since SSID databases exist and are used for more accurate location detection.
It's a tool that significantly improves productivity. I've been a programmer ~15 years and the amount of time it saves me from looking shit up on the internet is insane. I think most people who say it doesn't work either aren't giving it a fair shot or don't know how to use it (in a real project, writing code with comments).
This productivity improvement should be advancing the open source community even faster, but instead most people are just pissing on it for various ideological reasons.
