As the author admits at the end, this is rather misleading. In normal usage, "overfit" is by definition a bad thing (it wouldn't be "over" if it was good). And the argument given does nothing to show that Bayesian inference is doing anything bad.
To take a trivial example, suppose you have a uniform(0,1) prior for the probability of a coin landing heads. Integrating over this gives a probability for heads of 1/2. You flip the coin once, and it lands heads. If you integrate over the posterior given this observation, you'll find that the probability of the value in the observation, which is heads, is now 2/3, greater than it was under the prior.
And that's OVERFITTING, according to the definition in the blog post.
Not according to any sensible definition, however.
I was writing another comment based on that same example and his leaving-one-out calculations (at least based on what I understood).
The posterior vs prior would be the extreme case of a leaving-one-out procedure - leaving the only data point out there is nothing left.
The divergence between the data and the model goes down when we include information about the data in the model. That doesn't seem a controversial opinion. (That's how the blog post is introduced here: https://twitter.com/YulingYao/status/1662284440603619328)
---
If the data consists of two flips they are either equal or different (the former becomes more likely as the true probability diverges from 0.5).
a) If the data is the same, the posterior probability of that result is 3/4. The log score is 2 log(3/4) = -0.6
When we check the out-of-sample log score for each one based on the 2/3 posterior obtained from the other we get in each case a log score log(2/3) = -0.4
b) If the data is different, the posterior probability is still 1/2. The log score is 2 log(1/2) = 2 -0.7 = -1.4
When we check the out-of-sample log score for each one based on the 1/3 posterior for getting that result obtained from the other we get in each case a log score log(1/3) = -1.1
When there is a small amount of information the variance of any estimation is very big and this explains what happens in that example. Overfitting implies a different behavior in training and in test and this is related to a big variance in the estimation of the error. So small amount of information implies that any model suffer overfitting and big variance, so is a general result not related especifically with Bayes.
I'm on my phone so I haven't tried to work through the math to see where the error is, but the author's conclusion is wrong and the counterexample is simple.
> Unless in degenerating cases (the posterior density is point mass), then the harmonic mean inequality guarantees a strict inequality
p
(
y
i
|
y
−
i
)
<
p
(
y
i
|
y
)
, for any point
i
and any model.
Let y_1, ... y_n be iid from a Uniform(0,theta) distribution, with some nice prior on theta (e.g. Exponential(1)). Then the posterior for theta, and hence the predictive density for a new y_i, depends only on max(y_1, ..., y_n). So for all but one of the n observations the author's strict inequality does not hold.
The author mentions he defines over fit as “Test error is always larger than training error”. Is there an algorithm or model where that’s not the case?
We've trained on the training set, but our fit() implementation was, "accept the input, do nothing with it, and then return." When we then evaluate the model on the test set and training set, assuming there isn't a large covariant shift between the two or something, we'll get approximately equal values.
So it's a degenerate case, but not of the training set. (And presumably that's partly what they meant by "pedantic".)
You can see regularly in practice where aggressive data augmentation is used, which obviously is only used on training data. But, of course, you'd still be 'overfit' if you fed in unaugmented training data.
I think they mean "always (statistically) significantly larger". They're probably imagining that something like cross-validation would make test errors approximately equal to training errors, but if you consistently see significantly larger errors, then you've overfit.
When you find the minima of your validation curve, rarely if ever is your test loss lower than your training loss. I don't think this necessarily means you're overfitting.
Under the Probably Approximately Correct framework, there's an x% chance your error on the empirical distribution deviates more than \epsilon away in either direction from that of the actual distribution, so one can assume similar things between two empirical distributions.
Yes and no. Suppose for example you give MNIST to a SVM and fit the model, then test it only on 0 and 1 digits, which are generally well discriminized, you'll get almost 100% accuracy in test, whereas 97% or less in training. (ye probably need some preprocessing, like using PacMAP or UMAP or whatever but the point is the same)
However, that's just because I decided the right data to test it onto. So, you can't really say much on a model using that definition.
I don't see how this simplifies to the harmonic mean expression in the post.
Regardless, the author is asserting that
p(y_N|{y_1...y_{N-1}}) ≤ p(y_N|{y_1...y_N})
which seems intuitively plausible for any trained model — given a model trained on data {y_1...y_N}, performing inference on any datapoint y_1...y_N in the training set will generally be more accurate than performing inference on a datapoint y_{N+1} not in the training set.
It's reassuring to see that I'm not the only one who finds those equations far from obvious. I didn't spend much time trying to understand the derivation though - as you wrote the result doesn't seem interesting anyway.
Their paper is more for the case of "we can't gather more data, so what to do?" but your solution is in line with optimal experimental design and choosing a utility function to distinguish between models using as little data as possible.
This is a pretty silly blog post. The complaint comes down to "a Bayesian model will always place a lower probability on outcomes that have not been observed than outcomes which have been observed already" which... of course it would! In what situation where you're trying to understand an exchangeable set of outcomes would you think you should put more probability on things that you haven't seen than those you have? The only things I can dream up violate exchangeability (ie for instance, a finite bag of different color marbles, where you draw without replacement).
So the argument is essentially that "not only if you pick the best thing fitting your finite data, but even if you take a weighted average over things that fit your finite data proportionally to how well they fit your finite data - you are still almost surely end up with something that fits your finite sample better than the general population (that this sample was drawn from)"?
Typically, we think of overfitting and underfitting as exclusive properties. IMO a large problem here is that the author's definition of overfitting is not inconsistent with underfitting. (Underfit indicates a poor fit on both the training and test sets, in general.)
For example, a simple model might underfit in general, but it may still fit the training set better than the test. If this happens yet both are poor fits, it is clearly underfitting and not overfitting. Yet by the article's definition, it would both be underfitting and overfitting simultaneously. So, I suspect this is not an ideal definition.
Am I missing something or is this argument only as strong as the (perfectly reasonable) claim that all models overfit unless they have a regularisation term?
Bayesian statistics is dumb. What's the point of using prior assumptions that are based on speculation? It's better to admitting your lack of knowledge, not jumping to conclusions without sufficient data, and processing data in an unbiased manner.
The prior represents your current state of knowledge, whatever that may be. If you have a total lack of knowledge, your prior is uniform over all possible states.
It is never proper to fabricate a biased (e.g. non-uniform) prior without any knowledge/information, that is not in any way part of Bayesian Inference. If you do have some extremely weak evidence, you use it accordingly with an extremely weak prior.
To paraphrase E.T. Jaynes, the rules of probability theory (e.g. Bayes Theorem) are the unique logically consistent way to reason about uncertainty.
I'm not sure total lack of knowledge can ever be encoded into a prior distribution, and if it can, I think Jeffreys prior makes a more compelling case than a uniform prior. After all, to believe that all possible values of the prior parameter are equally likely, is to believe that all possible values of its square are not.
Anyway, you can define a sequence of solutions with bounded uniform priors and calculate the limiting solution. For any given data set when the endpoints of the intervals go to +/-infinity the solution will converge to the uniform prior one - if it exist.
I discussed this in another reply in this thread, but I can't personally think of any examples where the hypothesis is unbounded. The laws of physics, computation, the human mind, measuring instruments, etc. all impose bounds on real world problems.
As someone that does Bayesian Inference a lot for my work (computational biology), I very often use uniform priors, but the structure of all real world problems I have ever encountered allows me specify hard bounds to the edges of non-zero probability.
This isn't always possible, but sometimes you can define what's called an improper prior p(µ), such that even if ∫p(µ) is not finite, the posterior distribution p(µ|x) is.
A common example is when p(x|µ,v) is a Gaussian dist. with a prior on the mean set to p(µ)=1.
Can you give an example of a problem where there is no possible prior information, and states all the way from negative to positive infinity are equally plausible? I don't think the laws of physics (not to mention, the practical limits of computation) allow for real world inference problems where there are no bounds on a prior of any kind.
Failing that, I don't think there are any human usable data sources that could report observations over an infinite interval.
It’s easy to use a uniform prior in a suitably huge interval - say from minus one gazillion to plus one gazillion - and you can look at the limit when the endpoints go to minus/plus infinity if you really have doubts about whether the interval was huge enough for your problem to fit comfortably within it.
If you don’t think that this is possible that says more about you than about the shortcomings of Bayesian statistics.
As I said, you can define the improper uniform prior solution as the limit of a sequence of solutions corresponding to a sequence of increasingly wider intervals with endpoints that go to +/-infinity.
(And as I said, you can start with a suitably huge region. Say that you want to determine the position of something and use a uniform prior that extends to a distance of 10^27m - a perfectly bounded prior from a mathematical point of view that covers the whole observable universe. If you observe something outside it, it’s not with the prior that you have a problem.)
To take a trivial example, suppose you have a uniform(0,1) prior for the probability of a coin landing heads. Integrating over this gives a probability for heads of 1/2. You flip the coin once, and it lands heads. If you integrate over the posterior given this observation, you'll find that the probability of the value in the observation, which is heads, is now 2/3, greater than it was under the prior.
And that's OVERFITTING, according to the definition in the blog post.
Not according to any sensible definition, however.