> We can all look at various polls and get an assessment of who is generally ahead.
I probably could, but there's a lot of polls to look through and I don't really want to spend the time. Much rather have someone else do it for me.
> if no matter what the result, we can always just say "the candidate who won could always have won given the forecast" - what are we really adding to the conversation here?
Isn't this hypothesis testing? If you have a weighted coin and a guess as to which side is heavier and by how much, you're going to need multiple flips to see if you are right. And it doesn't even really make sense to talk about how right/wrong you are about a single flip, only on the aggregate.
It's possible someone has already compiled FiveThirtyEight's results to get some aggregate accuracy, I haven't checked. If they have and he's wrong on average and that's what you are referencing, my apologies.
The trick here is that if the election is close enough that you'd actually want/need multiple pollsters aggregated, the aggregators will indicate high uncertainty. If it's enough of a blowout for the aggregators to indicate low uncertainty, then the individual polls are going to be showing a large gap.
An aggregator saying "foo has a 65% chance of winning" may seem like it's providing more information than a single historically reliable poll (say Reuters/Ipsos) stating "foo is up by 2 points but there's a 3 point margin of error" - but isn't it just an illusion? High quality pollsters very seldom deviate very much.
And even if you grant that the aggregator is closer to being "right" than any single pollster, is that difference actually meaningful enough to impact any real world behaviors? Would you do anything differently with a 50% chance of victory versus a 70% chance?
I've honestly come to think of them as entertainment, with no real value.
> We can all look at various polls and get an assessment of who is generally ahead.
I probably could, but there's a lot of polls to look through and I don't really want to spend the time. Much rather have someone else do it for me.
> if no matter what the result, we can always just say "the candidate who won could always have won given the forecast" - what are we really adding to the conversation here?
Isn't this hypothesis testing? If you have a weighted coin and a guess as to which side is heavier and by how much, you're going to need multiple flips to see if you are right. And it doesn't even really make sense to talk about how right/wrong you are about a single flip, only on the aggregate.
It's possible someone has already compiled FiveThirtyEight's results to get some aggregate accuracy, I haven't checked. If they have and he's wrong on average and that's what you are referencing, my apologies.