I want to push back against your pushback as someone who’s lived in both NYC and the SF area. I agree with you that Uber barely made sense in Manhattan. I never once used it and taxis were plentiful.
I’ve since realized that in the US, NYC is an exception. When I first came to SF and Seattle for job interview related things, I used taxis, only to find out that the taxis were so terrible I never used them again:
- In the suburbs of Seattle, I was given a taxi chit from the place I was interviewing. I called in for a cab and had to wait over a half an hour for one to pick me up.
- In SF, the airport cab I was using had his GPS unmounted from his dash, and ended up handing me the machine and asking me to help him navigate from the back seat. Then when we got to the hotel, he lamented my choice to pay by credit card as it meant he would get the money much later than if he had cash. After I told him I didn’t have the circa $100 in cash he was charging, he sadly acquiesced, then proceeded to take a literal paper rubbing imprint of the card number before I could leave.
I like to say that the Bay Area made Uber make sense, both in terms of urban planning and in terms of how terrible taxis were.
And I think those may be related: if you can get around well in a place like NYC using public transit or walking, taxis have to be a lot more attractive in order to justify their existence. In SF or Seattle they had much less competition due to the suburban sprawl and worse public transit.
I still believe it's a high effort parody and have never been able to find proof otherwise, but I guess most people stop caring about truth if something is entertaining
Anything that allows AI to scale to superinteligence quicker is going to run into AI alignment issues, since we don’t really know a foolproof way of controlling AI. With the AI of today, this isn’t too bad (the worst you get is stuff like AI confidently making up fake facts), but with a superintelligence this could be disastrous.
It’s very irresponsible for this article to advocate and provide a pathway to immediate superintelligence (regardless of whether or not it actually works) without even discussing the question of how you figure out what you’re searching for, and how you’ll prevent that superintelligence from being evil.
I don't think your response is appropriate. Narrow domain "superintelligence" is around us everywhere-- every PID controller can drive a process to its target far beyond any human capability.
The obvious way to incorporate good search is to have extremely fast models that are being used in the search interior loop. Such models would be inherently less general, and likely trained on the specific problem or at least domain-- just for performance sake. The lesson in this article was that a tiny superspecialized model inside a powerful transitional search framework significantly outperformed a much larger more general model.
Use of explicit external search should make the optimization system's behavior and objective more transparent and tractable than just sampling the output of an auto-regressive model alone. If nothing else you can at least look at the branches it did and didn't explore. It's also a design that's more easy to bolt in varrious kinds of regularizes, code to steer it away from parts of the search space you don't want it operating in.
The irony of all the AI scaremongering is that if there is ever some evil AI with some LLM as an important part of its reasoning process if it is evil it may well be so because being evil is a big part of the narrative it was trained on. :D
Of course "superintelligence" is just a mythical creature at the moment, with no known path to get there, or even a specific proof of what it even means - usually it's some hand waving about capabilities that sound magical, when IQ might very well be subject to diminishing returns.
Do you mean no way to get there within realistic computation bounds? Because if we allow for arbitrarily high (but still finite) amounts of compute, then some computable approximation of AIXI should work fine.
>Do you mean no way to get there within realistic computation bounds?
I mean there's no well defined "there" either.
It's a hand-waved notion that adding more intelligence (itself not very well defined, but let's use IQ) you get to something called "hyperintelligence", say IQ 1000 or IQ 10000, that has what can be described as magical powers, like it can convince any person to do anything, can invent things at will, huge business success, market prediction, and so on.
Whether intelligence is cummulative like that, or whether having it gets you those powers (aside from the succesful high IQ people, we know many people with IQ 145+ that are not inventing stuff left and right, or convincing people with some greater charisma than the average IQ 100 or 120 politician, but e.g. are just sad MENSA losers, whose greatest achievement is their test scores).
>Because if we allow for arbitrarily high (but still finite) amounts of compute, then some computable approximation of AIXI should work fine.
I doubt that too. The limit for LLMs for example is more human produced training data (a hard limit) than compute.
IQ has an issue that is inessential to the task at hand, which is how it is based on a population distribution. It doesn’t make sense for large values (unless there is a really large population satisfying properties that aren’t satisfied).
> I doubt that too. The limit for LLMs for example is more human produced training data (a hard limit) than compute.
Are you familiar with what AIXI is?
When I said “arbitrarily large”, it wasn’t for laziness reasons that I didn’t give an amount that is plausibly achievable. AIXI is kind of goofy. The full version of AIXI is uncomputable (it uses a halting oracle), which is why I referred to the computable approximations to it.
AIXI doesn’t exactly need you to give it a training set, just put it in an environment where you give it a way to select actions, and give it a sensory input signal, and a reward signal.
Then, assuming that the environment it is in is computable (which, recall, AIXI itself is not), its long-run behavior will maximize the expected (time discounted) future reward signal.
There’s a sense in which it is asymptotically optimal across computable environments (... though some have argued that this sense relies on a distribution over environments based on the enumeration of computable functions, and that this might make this property kinda trivial. Still, I’m fairly confident that it would be quite effective. I think this triviality issue is mostly a difficulty of having the right definition.)
(Though, if it was possible to implement practically, you would want to make darn sure that the most effective way for it to make its reward signal high would be for it to do good things and not either bad things or to crack open whatever system is setting the reward signal in order for it to set it itself.)
(How it works: AIXI basically enumerates through all possible computable environments, assigning initial probability to each according to the length of the program, and updating the probabilities based on the probability of that environment providing it with the sequence of perceptions and reward signals it has received so far when the agent takes the sequence of actions it has taken so far. It evaluates the expected values of discounted future reward of different combinations of future actions based on its current assigned probability of each of the environments under consideration, and selects its next action to maximize this. I think the maximum length of programs that it considers as possible environments increases over time or something, so that it doesn’t have to consider infinitely many at any particular step.)
>AIXI doesn’t exactly need you to give it a training set, just put it in an environment where you give it a way to select actions, and give it a sensory input signal, and a reward signal.
That's still a training set, just by another name.
And with the environment being the world we live in, it would be constrained by the local environment's possible states, the actions it can perform to get feedback on, and the rate of environment's response (the rate of feedback).
Add the quick state-space inflation in what it is considering, and it's an even tougher deal than getting more training data for an LLM.
When I said it didn’t require a training set, I meant you wouldn’t need to design one.
I don’t understand what you mean by the comment about state-space inflation. Do you mean that the world is big, or that the number of hypotheses it considers is big, or something else?
If the world is computable, then after enough steps it should include the true hypothesis describing the world among the hypotheses it considers. And, the probability it assigns to hypotheses which make contrary predictions should go down as soon as it sees observations that contradict those other hypotheses. (Of course, “the actual world” including its place in it, probably has a rather long specification (if it even has a finite one), so that could take a while, but similar things should apply for good approximations to the actual world.)
As for “it’s possible actions”, “moving around a robot with a camera” and “sending packets on the internet” seem like they would constitute a pretty wide range of possible actions.
Though, even if you strip out the “taking actions” part, and just consider the Solomonoff induction part (with input being, maybe a feed of pairs of “something specifying a source for some information, like a web-address or a physical location and time, along with a type of measurement, such as video” and “encoding of that data”,
should get very good at predicting what will happen, if not “how to accomplish things”. Though I suppose this would involve some “choosing a dataset”.
AIXI would update its distribution over environments based on its observations even when its reward signal isn’t changing.
I’ve been notified of an “unknown AirTag” while I was home. When I checked the locations it was seen with me, it was a random zigzag within a block or two of my home.
I’m pretty sure what happened is that the AirTag belonged to one of my neighbours, there were some GPS distortions happening that made my phone think it was moving slightly, it kept hearing the AirTag’s signal, and it assumed I was being stalked while wandering near home. This person might have the same thing happening to them.
A better comparison for numerical classifiers would be uncountable nouns.
In English, you don’t say “give me three waters”, you say “give me three glasses of water” or “three bottles of water”. You can think of the classifier words as being that, but for everything:
This is not good news, this means that we could end up with a dangerously superintelligent AI just by scaling up the number of parameters, without increasing the amount of training data.
No, but LLMs require orders of magnitude more language input than humans[1]. It's very reasonable to assume that architectural differences (size among them) is more likely a constraint for performance.
1. Specifically larger than the upper bound on lifetime language input for humans, even assuming 24/7 at max reading speed.
Do they? What is the total size of all visual, audio, touch, locomotive, scent, and taste data collected between birth and when a human reaches IQ 100? There are multiple high-bandwidth feeds running into the brain 24/7.
Yes, but LLMs come out of training as experts in approximately any single thing you can think of, and then some, and all that in dozen of languages. Humans don't achieve even a fraction of this kind of breadth.
> You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them.
> In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know.
This is not quite accurate, but complex because measurement is hard. The things they are being tested on are almost surely within the dataset. Let's take the bar exam for instance. Sure, we don't know what's in GPT data, but we know it has reddit, and we know reddit has many similar if not exact questions on it. We know that the first GPT4 did not have good semantic similarity matching because they just used a 3 substring matching on 50 chararcters (Appendix C) and they only consider the false positive nature. Then there's this line...
The RLHF post-training dataset is vastly smaller than the pretraining set and unlikely to have any particular question contaminated. However we did not check explicitly.
But my favorite is the HumanEval. I'll just remind everyone that this was written by 60 authors, mostly from OpenAI
We evaluate functional correctness on a set of 164 handwritten programming problems, which we call the HumanEval dataset. ... __It is important for these tasks to be hand-written, since our models are trained on a large fraction of GitHub, which already contains solutions to problems from a variety of sources.__
The problems? Well they're leetcode style... Can you tell me you can write leetcode style questions that
Human Eval 2
Prompt:
def truncate_number(number: float) -> float: """ Given a positive floating point number, it can be decomposed into and integer part (largest integer smaller than given number) and decimals (leftover part always smaller than 1). Return the decimal part of the number. >>> truncate_number(3.5) 0.5 """
Solution:
return number % 1.0
Human Eval 4
Prompt:
from typing import List def mean_absolute_deviation(numbers: List[float]) -> float: """ For a given list of input numbers, calculate Mean Absolute Deviation around the mean of this dataset. Mean Absolute Deviation is the average absolute difference between each element and a centerpoint (mean in this case): MAD = average | x - x_mean | >>> mean_absolute_deviation([1.0, 2.0, 3.0, 4.0]) 1.0 """
Solution
mean = sum(numbers) / len(numbers)
return sum(abs(x - mean) for x in numbers) / len(numbers)
You really want to bet that that isn't on github? Because I'll bet you any dollar amount you want that there are solutions in near exact form that are on github prior to their cutoff date (Don't trust me, you can find them too. They're searchable even). Hell, I've poisoned the dataset here!
LLMs are (lossy) compression systems. So they're great for information retrieval. And a lot of what we consider intelligence (and possibly even creativity) is based on information retrieval. Doesn't mean these things are any less impressive but just a note on how we should be interpreting results and understanding the limitations of our tools. Measuring intelligence is a really difficult thing and we need to be aware that the term isn't universally agreed upon and so people are often talking past one another and also some people are conflating the differences as if they are the same.
No, they're not. Corporations have known, concrete impacts on the world, whereas the dangers of AI are, so far, corporations. ASIs are (as yet) fictional.
Another difference: most corporations will avoid doing illegal stuff if the penalties are large enough: the corporation alignment problem is political. Pretty much no extant AI systems can be instructed in this way: we don't know how to align AIs even in theory.
That won't provide any motivation: no AI system yet created fears death (except perhaps some of the really simple, evolved ones – but I'd question whether they're sophisticated enough to fear).
In SF, up-front prices have been comparable to Uber’s, except that you don’t have to pay tip, which automatically makes it 15–20% cheaper.
The few times I’ve tried it, the service has been good and its driving was safe. The only downside is that there seems to sometimes be longer wait times.
Legally, it’s not compulsory. Socially and culturally, it definitely is. In the US, if you don’t tip 15%, you’re either an asshole, or you’re saying there’s severe issues with the service.
I’m not going justify this culture—I don’t like it either—but that is the way it is.
I'm in the US and a frequent Uber users, and I've only tipped once over 10 years. My rating is 4.93, so I don't think the drivers see me as an asshole.
It is possible there exists a table called dbo.user_ratings with a boolean column called IsAsshole, and you might have tons of rows there all with a 1 on the boolean column but the Uber APIs don't return that value. CEO Travis could tell us if it exists
I have friends who tip and tbh when I don't get a ride, they don't get a ride and when they get a ride, I get a ride. So obviously the tipping means nothing.
Occasionally I might tip, but only if I got something out of it. And it's rare. I think I might have tipped on 10 out of the 1500 rides.
I'd only tip a uber/rideshare driver if the experience was above and beyond... I do not expect this kind of service from every driver, nor do I expect them to try to provide it every time.
It should be a reward for exceptional service, not a default assumption for average (or so help me, sub average) service.
How many of them really wanted it? To me when Uber added tipping it seemed to come out of the blue. (And it was stupid and bad for everyone except possibly Uber, because the equilibrium total fare in a marketplace with reputation tracking is not going to be raised by turning part of it into a tip, it just adds uncertainty and friction.)
I don't know how many of the wanted it but here is an article that talks about it. Atleast is New York City they were forced to implement it and then it expanded from there.
> Uber might finally be forced to change its tune after New York City's Taxi and Limousine Commission introduced a proposal this week that would require ride-hailing companies operating in the city to allow riders to tip their drivers. The need to follow a rule like that in one of Uber's biggest and most important markets could force the company to allow tipping across the country or around the world.
> The Independent Drivers Guild lobbied for the New York proposal, which it estimates will lead to over $300 million per year in tips for New York drivers.
Does this mean they can go on the highways? If so that’s a significant upgrade to the Waymo service in the SF Bay Area, as it can now take you between towns.
They’ve always had approval to go up to 65 mph, but it was Waymo’s decision not to go on the highways. They’re starting to do it [1] and do a lot of freeway testing, so it might happen sooner.
I’ve since realized that in the US, NYC is an exception. When I first came to SF and Seattle for job interview related things, I used taxis, only to find out that the taxis were so terrible I never used them again:
- In the suburbs of Seattle, I was given a taxi chit from the place I was interviewing. I called in for a cab and had to wait over a half an hour for one to pick me up.
- In SF, the airport cab I was using had his GPS unmounted from his dash, and ended up handing me the machine and asking me to help him navigate from the back seat. Then when we got to the hotel, he lamented my choice to pay by credit card as it meant he would get the money much later than if he had cash. After I told him I didn’t have the circa $100 in cash he was charging, he sadly acquiesced, then proceeded to take a literal paper rubbing imprint of the card number before I could leave.
I like to say that the Bay Area made Uber make sense, both in terms of urban planning and in terms of how terrible taxis were.
And I think those may be related: if you can get around well in a place like NYC using public transit or walking, taxis have to be a lot more attractive in order to justify their existence. In SF or Seattle they had much less competition due to the suburban sprawl and worse public transit.