There were many many statements made that JIT compilers could be faster than AOT compilers because they had more information to use at runtime - originally this was mostly aimed at Java/HotSpot which has not, in practice, significantly displaced languages like C or C++ (or these days Rust) from high-performance work.
Yeah those statements were overly optimistic and I don’t think they’re representative of what most people in the JIT field think. It’s also not what I as a JIT engineer would have promised you.
The actual promise is just: JITs make dynamic languages faster and they are better at doing that than AOTs. I think lots of systems have delivered on that promise.
I concur here. 20 years ago I was a JIT cheerleader and in the intervening time I've realized that you're only going to get the super-optimized hot inner loop perfect after the JIT and runtime has chugged through a ton of other slop that tends to make programs bloated and slow. And the Java ecosystem in particular has a tendency to build a ton of ceremony and abstractions that the runtime system has to boil away, but can only really managed to do so with deep inlining and a lot of optimizations, many of which are speculative.
> JITs make dynamic languages faster and they are better at doing that than AOTs
Yeah, i'm curious how well JIT works on languages with less dynamism. Perhaps a combination of AOT + JIT on a strong statically typed language might provide the best of both worlds. Though I suppose PGO kinda does that.
I think about this a bit in the context of Virgil. Virgil's compiler is a whole-program optimizing compiler that does a lot of devirtualization and constant-folding. In the higher optimizations it does a bit of inlining, but I haven't found the huge 10X speedups that you get in, e.g. Java. More like 10-40% performance improvements from inlining.
I think Virgil could benefit a little from runtime information. For example, it could make better inlining and register allocation decisions, as well as code layout. I have a feeling that Virgil code would benefit a little from guarded inlining, but I don't think full-on speculation would help. In general, a lot of polymorphism can melt away if you can look at the whole program. Couple that also with Virgil's compiler doing monomorphization, which means that using parametric polymorphism costs only code space, and I think the gap is pretty small. I'd expect you could maybe get another 10-20% from these things all together--that's a lot of work to get a small amount.
Yup, agreed, in the case of dynamic languages it's much clearer and the evidence is a lot more favourable.
The linked article doesn't help here because the abstract only mentions Javascript in the context of their work to prove their concept, but the body of the paper is clearer that it is discussing JIT vs AOT in the context of Javascript specifically.
I think their findings are applicable to lots of languages where the fastest known implementation is JIT based.
Not all “JIT dominant” languages rely on ICs as part of the JIT’s performance story, but enough of them do that it’s worth studying.
And JS happens to be the language where ICs have been taken the furthest, in terms of just how many different ways have been investigated and how many person years went into tuning them. So in some sense they’re picking the hardest fight. I think that’s a good thing.
HotSpot definitely has delivered on that too. It's a super dynamic runtime with reflection and randomly loaded jars even if Java the language is terse.
It has in a bunch of places. C# is widely used in video games, and Java is widely used in financial trading including HFT scenarios where every millisecond matters. And obviously in Android it's used to write large parts of the OS.
There are places where it hasn't, but that's more due to missing features than JIT vs AOT. Java only got SIMD support recently and it's still in a preview mode, partly because it's all blocking on Valhalla value types.
PGO can make a big difference to C++ codebases, and as JIT is basically PGO with better deployment/developer ergonomics it could probably also work in C++ too. It's just that the most performance sensitive C++ codebases like Chrome prefer to take the build system complexity hit and get the benefits of PGO without the costs, and most C++ codebases just go without.
What promised advantages are you waiting on?
There are lots of systems that have architectures that are similar to HotSpot, or that surpass it in some way. V8 is just one.