Main limiting factor? Hard to know.

But throw 100x more compute to an LLM (through an increase of both parameters and tokens) and it will undoubtedly beat current models.