I've assumed that when AI becomes much more mainstream we'll see multiple levels of services.
The cheapest (free or cash strapped services) will implement several (hidden/opaque) ways to reduce the cost of answering a query by limiting the depth and breadth of its analysis.
Not knowing any better you likely won't realize that a much more complete, thoroughly considered answer was even available.
> The cheapest (free or cash strapped services) will implement several (hidden/opaque) ways to reduce the cost of answering a query by limiting the depth and breadth of its analysis.
This already happens. Many of the cheap API providers aggressively quantize the weights and KV cache without making clear that they do.
I've assumed that when AI becomes much more mainstream we'll see multiple levels of services.
The cheapest (free or cash strapped services) will implement several (hidden/opaque) ways to reduce the cost of answering a query by limiting the depth and breadth of its analysis.
Not knowing any better you likely won't realize that a much more complete, thoroughly considered answer was even available.