Not true. Changes to the OS, particularly to the scheduler, can affect CPU-bound work a great deal. It's "your code", but the OS decides when and where it runs. For example, the changes between Linux 6.5 and Linux 6.6 led to a >20% uplift in TensorFlow and some smaller uplifts to Blender. This is the same software on the same hardware.
Depending on the code you're running, calling convention can matter a lot. The SysV ABI will use xmm/ymm/zmm automatically, whereas on Windows you have to opt into it with __vectorcall.