The problem is that the majority of user interaction doesn't need to be "useful" (as in increasing productivity): the majority of users are looking for entertainment, so turning up the sycophancy knob makes sense from a commercial point of view.

It's just like adding sugar in foods and drinks.

You're ... Wait, never mind.

I'm not so sure sycophancy is best for entertainment, though. Some of the most memorable outputs of AI dungeon (an early GPT-2 based dialog system tuned to mimic a vaguely Zork-like RPG) was when the bot gave the impression of being fed up with the player's antics.

> I'm not so sure sycophancy is best for entertainment, though.

I don't think "entertainment" is the right concept. Perhaps the right concept is "engagement". Would you prefer to interact with a chatbot that hallucinated or was adamant you were wrong, or would you prefer to engage with a chatbot that built upon your input and outputted constructive messages that were in line with your reasoning and train of thought?

Not sure anyone's entertained by Claude. It's not really an entertaining model. Smart and enthusiastic, yes.

Some of the open models like kimi k2 do a better job of pushing back. It does feel a bit annoying to use them when they don’t just immediately do what you tell them. Sugar-free is a good analogy!

Well, aren't we at the stage where the service providers are fighting for verbs and brand recognition, rather than technological advances.

If there is no web-search, only googling, it doesn't matter how bad the results are for the user as long as the customer gets what they paid for.

I doubt humanity will figure that out, but maybe I’m too cynical