You might(?) be thinking of the SunRay stuff, which was certainly not X-windows based, though similar in concept/architecture. There was never a time when "everyone simply had a dumb graphical terminal".

(I never worked directly for Sun, though I did have a hell of a lot of dealings with both the Church and State sides of their business.)