> Even you don't give a simple account of what needs to be done.
Fair enough - though I did mention a few basic things like A/B testing and survival bias (trying to being aware of one's own cognitive biases is the first step in making better decisions; to be aware means you first have to know them).
And yes, it is not trivial at all and dependent on the context.
The best and simplest starting point would be to make a list of potential confounding variables (I mentioned a few of them) and then just looking at the R^2 statistic.
That already should point him in the right direction.
I was less bothered that his methodology wasn't 100% correct, but rather by casually making a statistically unsound decision which has a big effect on people's lives, so getting it right is important - and then when called out dismissing it.
I have a lot of sympathy for people honestly trying to make decisions rigorously and getting it wrong; but you have to try.
Fair enough - though I did mention a few basic things like A/B testing and survival bias (trying to being aware of one's own cognitive biases is the first step in making better decisions; to be aware means you first have to know them). And yes, it is not trivial at all and dependent on the context.
The best and simplest starting point would be to make a list of potential confounding variables (I mentioned a few of them) and then just looking at the R^2 statistic. That already should point him in the right direction.
I was less bothered that his methodology wasn't 100% correct, but rather by casually making a statistically unsound decision which has a big effect on people's lives, so getting it right is important - and then when called out dismissing it. I have a lot of sympathy for people honestly trying to make decisions rigorously and getting it wrong; but you have to try.