But at that point you should be at a higher level of abstraction, looking at the number of sprints that will probably be needed to clear the backlog (hopefully understanding that the backlog itself should always be regarded as incomplete). The discussion then should be something like, "Hey, we're four sprints in, the backlog already has about seven more worth of stories, and we only have four more sprints until our target deployment date. Do we cut features, move the date, or add more team members?"

In the pathological case where management is constantly monitoring the team velocity and converting that to individual productivity, you'll get managers who (often to cover their own recklessness in setting an unrealistic date, or procrastinating on starting the project) will decree that the team is going too slow. I know that happens a lot, but it's not the fault of scrum, it's just poor management. Scrum can't make a bad manager good, but avoiding scrum won't make them good either.