I'm not really sure but my recollection from talking to them in 2019 was that it was quite difficult to get features shipped because of e.g. hacking risk.
It's certainly true that iOS's strict sandboxing and aggressive resource management probably made life harder for them, but that doesn't excuse the lack of deep integration for 1p automation. That's the kind of stuff AppleScript allowed two decades prior without any background runtime.
taking into account all the impacts on society, uber is a substantial improvement on what came before. sometimes laws are bad and it is good when you break them
This trick is very useful on Nvidia GPUs for calculating mins and maxes in some cases, e.g. atomic mins (better u32 support than f32) or warp-wide mins with `redux.sync` (only supports u32, not f32).
Assuming the 10M records is ~2000M input tokens + 200M output tokens, this would cost $300 to classify using llama-3.3-70b[1]. If using llama lets you do this in say one day instead of two days for a traditional NLP pipeline, it's worthwhile.
Why 2 days? Machine Learning took over the NLP space 10-15 years ago, so the comparison is between small, performant task-specific models versus LLMs. There is no reason to believe the "traditional" NLP pipelines are inherently slower than Large Language Models, and they aren't.
my claim is not that it would take two days for such a pipeline to run but that it would take two days to make an NLP pipeline whereas an LLM pipeline would be faster to make.
First, this model is yet to be released. This is a momentum "announcement". When the O1 was "announced", it was announced as a "breakthrough" but I use Claude/O1 daily and 80% of the time Claude beats it. I also see it as a highly fine-tuned/targeted GPT-4 rather than something that has complex understanding.
So we'll find out if this model is real or not by 2-3 months. My guess is that it'll turn out to be another flop like O1. They needed to release something big because they are momentum based and their ability to raise funding is contingent on their AGI claims.
Intelligence has not been LLM's major limiting factor since GPT4. The original GPT4 reports in late-2022 & 2023 already established that it's well beyond an average human in professional fields: https://www.microsoft.com/en-us/research/publication/sparks-.... They failed to outright replaced humans at work not because of lacking intelligence.
We may have progressed from a 99%-accurate chatbot to one that's 99.9%-accurate, and you'd have a hard time telling them apart in normal real world (dumb) applications. A paradigm shift is needed from the current chatbot interface to a long-lived stream of consciousness model (e.g. a brain that constantly reads input and produces thoughts at 10ms refresh rate; remembers events for years and keep the context window from exploding; paired with a cerebellum to drive robot motors, at even higher refresh rates.)
As long as we're stuck at chatbots, LLM's impact on the real world will be very limited, regardless of how intelligent they become.
O3 is multiple orders of magnitude more expensive to realize a marginal performance gain. You could hire 50 full time PhDs for the cost of using O3. You're witnessing the blowoff top of the scaling hype bubble.
Like they've been making it all this time? Cheaper and cheaper? Less data, less compute, fewer parameters, but the same, or improved performance? Not what we can observe.
>> Tell me, what has this industry been good at since its birth? Driving down the cost of compute and making things more efficient.
No, actually the cheaper compute gets the more of it they need to use or their progress stalls.
Do you understand the difference between training and inference?
Yes, it costs a lot to train a model. Those costs go up. But once you trained it, it’s done. At that point inference — the actual execution/usage of the model — is the cost you worry about.
Inference cost drops rapidly after a model is released as new optimizations and more efficient compute comes online.
That’s precisely what’s different about this approach. Now the inference itself is expensive because the system spends far more time coming up with potential solutions and searching for the optimal one.
And again, no. The cost of inference is a function of the size of the model and if models keep getting bigger, deeper, badder, the cost of inference will keep going up. And if models stop getting bigger because improved performance can be achieved just by scaling inference, without scaling the model- well that's still more inference; and even if the cost overall falls, there will need to be so much more inference to scale sufficiently to keep AI companies in competition with each other that the money they have to spend will keep increasing, or in other words it's not how much it costs but how much you need to buy.
This is a thing, you should know. It's called Jevon's Paradox:
In economics, the Jevons paradox (/ˈdʒɛvənz/; sometimes Jevons effect) occurs when technological progress increases the efficiency with which a resource is used (reducing the amount necessary for any one use), but the falling cost of use induces increases in demand enough that resource use is increased, rather than reduced.[1][2][3][4]
>> Do you understand the difference between training and inference?
Oh yes indeed-ee-o and I'm referring to training and not inference because the big problem is the cost of training, not inference. The cost of training has increased steeply with every new generation of models because it has to, in order to improve performance. That process has already reached the point where training ever larger models is prohibitively expensive even for companies with the resources of OpenAI. For example, the following is from an article that was posted on HN a couple days ago and is basically all about the overwhelming cost to train GPT-5:
In mid-2023, OpenAI started a training run that doubled as a test for a proposed new design for Orion. But the process was sluggish, signaling that a larger training run would likely take an incredibly long time, which would in turn make it outrageously expensive. And the results of the project, dubbed Arrakis, indicated that creating GPT-5 wouldn’t go as smoothly as hoped.
(...)
Altman has said training GPT-4 cost more than $100 million. Future AI models are expected to push past $1 billion. A failed training run is like a space rocket exploding in the sky shortly after launch.
(...)
By May, OpenAI’s researchers decided they were ready to attempt another large-scale training run for Orion, which they expected to last through November.
Once the training began, researchers discovered a problem in the data: It wasn’t as diversified as they had thought, potentially limiting how much Orion would learn.
The problem hadn’t been visible in smaller-scale efforts and only became apparent after the large training run had already started. OpenAI had spent too much time and money to start over.
"Once you trained it it's done" - no. First, because you need to train new models continuously so that they pick up new information (e.g. the name of the President of the US). Second because companies are trying to compete with each other and to do that they have to train bigger models all the time.
Bigger models means more parameters and more data (assuming there is enough which is a whole other can of worms) more parameters and data means more compute and more compute means more millions, or even billions. Nothing in all this is suggesting that costs are coming down in any way, shape or form, and yep, that's absolutely about training and not inference. You can't do inference before you do training, you need to train continuously, and for that reason you can't ignore the cost of training and consider only the cost of inference. Inference is not the problem.
> What they’ve proven here is that it can be done.
No they haven't, these results do not generalize, as mentioned in the article:
"Furthermore, early data points suggest that the upcoming ARC-AGI-2 benchmark will still pose a significant challenge to o3, potentially reducing its score to under 30% even at high compute"
Meaning, they haven't solved AGI, and the task itself do not represent programming well, these model do not perform that well on engineering benchmarks.
Just to be clear — your position is that the cost of inference for o3 will not go down over time (which would be the first time that has happened for any of these models).
Even if compute costs drop by 10X a year (which seems like a gross overestimate IMO), you're still looking at 1000X the cost for a 2X annual performance gain. Costs outpacing progress is the very definition of diminishing returns.
From their charts, o3 mini outperforms o1 using less energy. I don’t see the diminishing returns you’re talking about. Improvement outpacing cost. By your logic, perhaps the very definition of progress?
You can also use the full o3 model, consume insane power, and get insane results. Sure, it will probably take longer to drive down those costs.
You’re welcome to bet against them succeeding at that. I won’t be.
Yes, that's exactly what I'm implying, otherwise they would have done it a long time ago, given that the fundamental transformer architecture hasn't changed since 2017. This bubble is like watching first year CS students trying to brute force homework problems.
That's a very static view of the affairs. Once you have a master AI, at a minimum you can use it to train cheaper slightly less capable AIs. At the other end the master AI can train to become even smarter.
The high efficiency version got 75% at just $20/task. When you count the time to fill in the squares, that doesn't sound far off from what a skilled human would charge
> how is this a plateau since gpt-4? this is significantly better
Significantly better at what? A benchmark? That isn't necessarily progress. Many report preferring gpt-4 to the newer o1 models with hidden text. Hidden text makes the model more reliable, but more reliable is bad if it is reliably wrong at something since then you can't ask it over and over to find what you want.
I don't feel it is significantly smarter, it is more like having the same dumb person spend more thinking than the model getting smarter.
Fantastic article. I didn't realize dairy cows lactated ~60lbs/day or ~3.5% of their body weight. Totally insane. Chickens appear to be this way too -- from some quick research Rhode Island Reds are ~3kg, lay ~300 eggs/year and each egg is ~62g.
The graph at the end ("US milk yield continues to grow, but falls short of its genetic potential") is interesting. If I saw that graph, I would interpret it as dairy farmers overfitting on the genetic yield potential measure, not as something that needs explanations like climate change.
> America’s cows are now extraordinarily productive. In 2024, just 9.3 million cows will produce 226 billion pounds of milk (about 100 million tons) – enough milk to provide ten percent of 333 million insatiable Americans’ diets, and export for good measure.
Is that all the cows in the US? Why tell us how many cows produce 10 percent of demand?
