Think of L2 cache (small access time, small capacity) vs. memory modules (larger access time, large capacity) on a motherboard. You get the large capacity and an access time somewhere in between, depending on the hit rate.
Yes, a different usage pattern could give you the worse of both worlds.
That's a very deep insight from queue theory and similar topics, I don't know if you wrote what you wrote with that in mind (I'm not being an ass, I just want to highlight this observation).
A system's behavior is highly dependent, to the point of it being fundamental, on how it is going to be used.
If you have two systems with opposite bottlenecks you can build a composite system with the bottlenecks reduced.