> Hmm, why would you need to optimize a benchmark for something? Generally it's the other way round.
It had always been both ways. This is why there exist(ed) quite a lot of people who have/had serious thoughts whether [some benchmark] actually measures the performance of the e.g. CPU or the quality of the compiler.
The "truce" that was adopted concerning these very heated discussions was that a great CPU is of much less value if programmers are incapable of making use of its power.
Examples that evidence the truth of this "truce" [pun intended]:
- Sega Saturn (very hard to make use of its power)
- PlayStation 3 (Cell processor)
- Intel Itanium, which (besides some other problems) needed a super-smart compiler (which never existed) so that programs could make use of its potential
- in the last years: claims that specific AMD GPUs are as fast as or even faster than NVidia GPUs (also: for the same cost) for GPGPU tasks. Possibly true, but CUDA makes it easier to make use of the power of the NVidia GPU.
> - Intel Itanium, which (besides some other problems) needed a super-smart compiler (which never existed) so that programs could make use of its potential
Well, no such thing is possible. Memory access and branch prediction patterns are too dynamic for a compiler to be able to schedule basically anything ahead of time.
A JIT with a lot of instrumentation could do somewhat better, but it'd be very expensive.
It had always been both ways. This is why there exist(ed) quite a lot of people who have/had serious thoughts whether [some benchmark] actually measures the performance of the e.g. CPU or the quality of the compiler.
The "truce" that was adopted concerning these very heated discussions was that a great CPU is of much less value if programmers are incapable of making use of its power.
Examples that evidence the truth of this "truce" [pun intended]:
- Sega Saturn (very hard to make use of its power)
- PlayStation 3 (Cell processor)
- Intel Itanium, which (besides some other problems) needed a super-smart compiler (which never existed) so that programs could make use of its potential
- in the last years: claims that specific AMD GPUs are as fast as or even faster than NVidia GPUs (also: for the same cost) for GPGPU tasks. Possibly true, but CUDA makes it easier to make use of the power of the NVidia GPU.