Haven't seen Mozilla's LocalScore [1] mentioned in the comments yet. It's exactly made for the purpose of finding out how well different models run on different hardware.

[1] https://www.localscore.ai/