You can rent H100s for $1.50/gpu/hr these days.

This is not a result of income inequality. This is a result of measures intended to reduce income inequality, which of course almost universally make things worse.

The other lending that the US (and Western governments in general) massively subsidize is education. And look at what has happened to the cost of housing and education. They have both massively exceeded general rates of inflation.

The cure is worse than the disease.

It's almost as if geopolitical environments evolve over time.

I don't think anyone today would feel any differently about preventing Pakistan from attaining nuclear weapons.

Unfortunately it's very difficult to take nukes away from a country.

It was a whole lot easier to take nukes away from a country until America fed Ukraine to the wolves. Great job...

Ok? I agree with you. It doesn't change anything I said.

People really need to stop glossing over the very real differences between state controlled media, and media that you think is aligned with a certain political group.

You can believe Fox News is the worst entity in human history, but Fox News is not RT.

> "Fine-tuning LLMs for knowledge injection is a waste of time" is true, but IDK who's trying to do that.

Have people who say this ever actually done it? It works. It works pretty well.

I have no clue why this bad advice is so routinely parroted.

It technically works with enough data but it's pretty inefficient compared to RAG. However, changing behavior via prompting/RAG is harder than changing behavior via finetuning; they're useful for different purposes.

> How is power discarded

It isn’t, not at scale in any traditional sense.

> the old one is completely valueless to them

This is of course untrue for the same reason that people are still running Windows 2000.

> This is of course untrue for the same reason that people are still running Windows 2000.

What is the reason?

They’ve built processes around it and don’t feel like/can’t afford to/ don’t know to how change them.

I guess we’ll see how that shakes out.

Because models are getting much better every couple months, I wonder if getting too attached to a process built around one in particular is a bad idea.

I would agree if Windows 2000 had the exact same APIs as the next version, but it doesn't. LLMs are text in -> text out, and you can drop in a new LLM and replace them without changing anything else. If anything, newer LLMs will just have more capabilities.

> LLMs are text in -> text out, and you can drop in a new LLM and replace them without changing anything else. If anything, newer LLMs will just have more capabilities.

I don't mean to be too pointed here, but it doesn't sound like you have built anything at scale with LLMs. They are absolutely not plug n play from a behavior perspective. Yes, there is API compatibility (text in, text out) but that is not what matters.

Even frontier SOTA models have their own quirks and specialties.

When I've built things with them I've mostly considered the quirks defects.

Kind of like how httpds will have quirks but those aren't really a good thing and they're kind of plug and play.

What kind of quirks have you seen that the next model wasn't better at?

A simple example would be when models get better at following instructions, the frantic and somewhat insane-sounding exhortations required to get the crappier model to do what you want can cause the stronger model to be a bit too literal and inflexible.

I think you mean non-deterministic, instead of probabilistic.

And there is no reason that these models need to be non-deterministic.

A deterministic algorithm can still be unpredictable in a sense. In the extreme case, a procedural generator (like in Minecraft) is deterministic given a seed, but you will still have trouble predicting what you get if you change the seed, because internally it uses a (pseudo-)random number generator.

So there’s still the question of how controllable the LLM really is. If you change a prompt slightly, how unpredictable is the change? That can’t be tested with one prompt.

> I think you mean non-deterministic, instead of probabilistic.

My thoughts too. It's more accurate to label LLMs as non-deterministic instead of "probablistic".

Append-only would imply yes. There is no overwriting in append-only. There is only truncate and append.

You have misread I think.

There used to be append-only, they've removed it and suggest using a credential that has no 'delete' permission. The question asked here is whether this would protect against data being overwritten instead of deleted.

Yes, it also disallows overwriting.

It's the same process as regular training, but instead of a single token for your cost function, it's top-k logits.