> where we work hard, and get many of the tasks (not just one) to be more efficient, so that our overall process can work faster.
The main way by which we do that is to compile everything and to improve the compiler to make everything faster. And by making faster machines.
After that it's hard work. If you find that your program spends its time in small fractions of percentages all over the code, and it's compiled well, it may be very hard to keep the structure of that computation as-is while obtaining any significant improvement.
And hope you have (or can reformulate into) a programming language that is designed to facilitate ambitious optimizations.
I wonder if there's some better term around than "compiler optimization" for doing really ambitious program transformations, in the vein of program specialization/partial evaluation treating the program and its inputs as a single symbolic system to optimize for evaluation or kolmogorov complexity. I have a feeling that by missing this terminology we're already handicapping work in the area...
We're also largely missing application specific optimisations. Occasionally I hear of people writing their own llvm pass to target their application domain but the potential scope vs exploitation of that strategy seems tragically underexplored.
Yes. I wonder if there's a way to plug the recent GPT-3 ish program synthesis advancements into this so humans could easily specify those by expressing just a hint of what to do. The problem is merciful in that you can handle a fair chunk of the correctness validation by comparing against the unoptimized version...
The main way by which we do that is to compile everything and to improve the compiler to make everything faster. And by making faster machines.
After that it's hard work. If you find that your program spends its time in small fractions of percentages all over the code, and it's compiled well, it may be very hard to keep the structure of that computation as-is while obtaining any significant improvement.