But this was very much the case with existing OCR software as well? I guess the LLMs will end up making up plausible looking text instead of text riddled with errors, which makes it much harder to catch the mistakes, in fairness
I usually run the image(s) through more than one converter then compare the results. They all have problems, but the parts they agree on are usually correct.
> Also at one point there were no VMs available anymore in our region, which caused a lot of issues.
I'm not sure if this is a difference between other clouds, at least a few years ago this was a weekly or even daily problem in GCP; my experience is if you request hundreds of VMs rapidly during peak hours, all the clouds struggle.
Right now, we can’t request even a single (1) non-beefy non-GPU VM in us-east on Azure. That’s been going on for over a month now, and that’s after being a customer for 2 years :(
We launch 30k+ VMs a day on GCP, regularly launching hundreds at a time when scheduled jobs are running. That’s one of the most stable aspects of our operation - in the last 5 years I’ve never seen GCP “struggle” with that except during major outages.
At the scale of providers like AWS and even the smaller GCP, “hundreds of VMs” is not a large amount.
If you’re deploying something like 100 m5.4xlarge in us-east-1, sure, AWS’s capacity seems infinite. Once you get into high memory instances, GPU instances, less popular regions etc, it drops off.
Now maybe after the AI demand and waves of purchases of systems appropriate for that things have improved, but it definitely wasn’t the case at the large scale employer I worked at in 2023 (my current employer is much smaller, so doesn’t have those needs, so I can’t comment)
To this point, it is actually wild how we each see a different projection of the chamber depending on which articles and threads we dive into. As much as I feel like I "know" HN, I do marvel at the entire subsections of the discourse I'm completely oblivious to, and this is an interesting cross-pollination.
The whole point of this framework is that a ton of the logic remains on the backend, with live connections to the frontend. At least for me the performance and hosting cost implications of that was the first question that came to my mind when I looked at this, I don’t understand how answering this question isn’t relevant here.
That “only big tech can solve AGI” bit doesn’t make sense to me - the scale argument was made back when people thought just more scale and more training was gonna keep yielding results.
Now it seems clear that what’s missing is another architectural leap like transformers, likely many different ones. That could come from almost anywhere? Or what makes this something where big tech is the only potential source of innovation?
Yup. LLMs can get arbitrarily good at anything with RL, but RL produces spiky capabilities, and getting LLMs arbitrarily good at things they're not designed for (like reasoning, which is absolutely stupid to do in natural language) is very expensive due to the domain mismatch (as we're seeing in realtime).
Neurosymbolic architectures are the future, but I think LLMs have a place as orchestrators and translators from natural language -> symbolic representation. I'm working on an article that lays out a pretty strong case for a lot of this based on ~30 studies, hopefully I can tighten it up and publish soon.
The barrier of entry is too high for traditional SV startups or a group of folks with a good research idea like transformers. You now need hundreds of billions if not trillions to get access to compute. OpenAI themselves have cornered 40% of global output of DRAM modules. This isn't like 2012, where you could walk into your local BestBuy, get a laptop, open an AWS account, and start a SaaS over the weekend. Even the AI researchers themselves are commanding 7- and 8-figure salaries that rival NFL players.
At best, they can sell their IP to BigTech, who will then commercialize it.
Are you saying you disagree that a new architectural leap is needed and just more compute for training is enough? Or are you saying a new architectural leap is needed and that or those new architectures will only be possible to train with insane amounts of compute?
If the latter I dont understand how you could know that about an innovation that’s not yet been made
I’m saying it is highly likely that the next leap in AI technology will require massive amounts of compute. On the order of tens of billions per year. I’m also saying that there are a small number of companies that would have access to that level of compute (or financial capital).
In other words, it’s is MORE likely that an OpenAI/Google/Microsoft/Grok/Anthropic gets us closer to AGI than a startup we haven’t heard of yet. Simply because BigTech has cornered the market and has a de facto monopoly on compute itself. Even if you had raised $10 billion in VC funding, you literally can not buy GPUs because there is not enough manufacturing capacity in the world to fill your order. Thus, investors know this and capital is flowing to BigTech, rather than VC funds. Which creates the cycle of BigTech getting bigger, and squeezing out VC money for startups.
If it comes from anywhere else but it needs a lot of capital to execute, big tech will just acquire them right? They'll have all the data centers and compute contracts locked up I guess.
Aye, was involved in some really messed up outages from New Relics agent libraries generating bogus byte code at runtime, absolute nightmare for the teams trying to debug it because none of the code causing the crashing existed anywhere you could easily inspect it. Replaced opaque magic from new relic with simpler OTEL, no more outages
Answer is pretty vague though, but sounds like it’s about not trying to “reverse” what the compiler did, but rather try and “analytically” work put what source code would likely have yielded the byte code it’s looking at?
If you have a block of code a compiler will compile a language expression or statement into a particular set of assembly/bytecode instructions. For example converting `a + b` to `ADD a b`.
A reversing decompiler will look at the `ADD a b` and produce `a + b` as the output. This is the simplest approach as it is effectively just a collection of these types of mapping. While this works, it can be harder to read and noisier than the actual code. This is because:
1. it does not handle annotations like @NotNull correctly -- these are shown as `if (arg == null) throw ...` instead of the annotation because the if/throw is what the compiler generated for that annotation;
2. it doesn't make complex expressions readable;
3. it doesn't detect optimizations like unrolling loops, reordering expressions, etc.
For (1) an analytical decompiler can recognize the `if (arg == null) throw` expression at the start of the function and map that to a @NotNull annotation.
Likewise, it could detect other optimizations like loop unrolling and produce better code for that.
I'm not sure that @NotNull example is appropriate. Java compiler does not add any checks for @NotNull annotations. Those annotations exist for IDE and linting tools, compiler doesn't care. May be there are Java-like languages like Lombok or non-standard compilers which do add those checks, but I think that Java decompiler shouldn't do assumptions of these additional tools.
A better example for Java would be something like lambda expressions on functional interfaces. There, the compiler is creating an anonymous object that implements the interface. A reversable decompiler will just see the anonymous class instance whereas an analytical decompiler can detect that it is likely a lambda expression due to it being an anonymous class object implementing a single method interface and is being passed to a function argument that takes that interface as a parameter.
In C# yield is implemented as a state machine, so an analytical decompiler could recognise that construct.
And yes, for JVM decompilers it could have language heuristics to detect (or be specifically for) Lombok, Scala, Groovy, Kotlin, etc.
> When you compile your project with IntelliJ IDEA build tool, the IDE adds assertions to all code elements annotated with @NotNull. These assertions will throw an error if the elements happen to be null at runtime.
When using Lombok it will use a compiler plugin for this so maven builds have @nonnull generated as if-statements. I dont know if intellij uses their own plugin but they do support Lombok in maven projects, so maybe thats where this is coming from. Afaik intellij has no built in compiler but relies on java .
I was initially impressed with Lombok and then ran into all the downsides of it and it was institutionally abandoned at one particular firm I was with (100s of devs).
You are right to be sceptical of “dependency bloat” - many many systems could be made simpler to debug and maintain if that advice was followed.
But: I urge you to give constraint solvers a try. This is something very different from some hipster ORM library. The good ones will be a single dependency containing extremely carefully optimised matrix math code that let you solve optimisation problems faster and with much more maintainable code.
Try some toy problem with the HiGHS solver for instance, it has good python bindings and even ships as a wasm package you can run in browsers. “Find the optimal times to run this heat pump given electricity prices”, “find the ideal train departure times that optimise for when people need to travel and minimises transfers”
Ethereum is able to process something like 150 transactions per second, using about 1,000,000 validator machines.
Postgres running on a single Raspberry Pi is something like 200 TPC-B read/write transactions per second.
Saying Ethereum “is not using very much CPU” is baffling to me. It is the state-of-the-art in this regard, and it uses something like six orders of magnitude more CPU than a normal database running a ledger workload?
First things first, I'm a crypto-sceptic - to put it in the mildest terms possible.
You're spot on with CPU usage. However: how would you design a RasPi-efficient, fault-tolerant, decentralised ledger with strict ordering and a transparency log?
Consider CAP. Existing banking systems choose partition tolerance (everyone does their own thing all the time basically), and eventual consistency via peering - which is why all settlements are delayed (in favour of fraud detection / mitigation), but you get huge transaction throughput, very high availability, and power efficiency. (Any existing inefficiencies can and should be optimised away, I guess we can blame complacency.)
The system works based on distributed (each bank) but centralised (customer->bank) authority, held up by regulations, capital, and identity verification.
Online authority works in practice - we collectively trust all the Googles, Apples, etc run our digital lives. Cryptocurrency enthusiasts trust the authors and contributors of the software, CPU/OS vendors, so it's not like we're anywhere near an absolute zero of authority.
Online identity verification objectively sucks, so that is out the window. I guess this could work by individual users delegating to a "host" node (which is what is already happening with managed wallets), and host nodes peering with each other based on mutual trust. Kinda like Mastodon, email, or even autonomous systems - the backbone of the Internet itself.
Why does it have to be decentralised (by which I assume you mean permissionlesss to join as a validator?)
The only reason for this - it would seem to me - is the ability to have nobody in control who can be subject to law enforcement.
If you need this kind of decentralisation blockchain, and all its inefficiency, is the only choice.
Societies should not require such things though. They need to have trustable institutions and intermediaries to function, in finance and many other areas.
> Societies should not require such things though. They need to have trustable institutions and intermediaries to function, in finance and many other areas.
...which is more or less the same conclusion that I've arrived at by the end.
Also the capacity is significantly higher with L2 included, and increasing rapidly.
With zkrollups and a decentralized sequencer, you basically pay no penalty vs. putting transactions on L1. So far I think the sequencers are centralized for all the major rollups, but there's still a guarantee that transactions will be valid and you can exit to L1.
Scaling is improving too. Rollups store compressed data on L1 and only need the full data available for a month or so. That temporary storage is cheaper but currently is still duplicated on all nodes. The next L1 upgrade (in November) will use data sampling, so each node can store a small random fraction of that data, with very low probability of any data being lost. It will also switch to a more efficient data storage structure for L1.
With these in place, they can gradually move into much larger L2 capacity, possibly into the millions per second. For the long term, there's also research on putting zk tech on L1, which could get even the L1 transactions up to 10,000/second.
there's 1,000,000 validators (defined as a public key), but you can run multiple validators per machine. Most estimates that crawl the p2p network to index nodes comes out at like ~20,000 machines
doesn't invalidate your point but it at least shaves off a few orders of magnitude
and a single PG node is not a fair comparison, we're talking 100% uptime global networks. Visa does about 70,000 transactions per second - how many servers do you think they run across their infra?
ZK rollups which greatly compress data on chain without losing security guarantees, combined with temporary storage using data sampling so each node only has to store a small portion of it. The zk rollups are live and support significantly more than 150tps today, and the data sampling goes live in November. There's a lot more work to be done but that puts the major pieces in place.
The person you are responding to is not arguing there is not a use case for crypto in cases like this.
They are arguing that stablecoins, specifically, require an off-chain entity that ultimately control them. And if you have an entity actually in control, why go through the trouble of blockchain? Then you can just have the controlling entity run a normal non-blockchain ledger.
I like the argument elsewhere in this thread that the actual reason is that it allows running a bank while pretending it’s not, bypassing regulation meant to protect depositors.
reply