Yes no coin flip

broken image
broken image

Right - I think this is what's at the heart of the original question. (If the distribution is very dispersed, then while the average is less useful as an idea of what to expect, it still minimises prediction error in some loss but that's a different thing and I think less relevant here). (And gets better if you augment it with some measure of dispersion, and so on). Much like how the average is unlikely to be the exact value of a new sample from the distribution, but it's a good way of describing what to expect. However, the idea is that often a lot of the probability mass - an amount that is not small - will be concentrated around the maximum likelihood estimate, and so that's why it makes a good estimate, and worth using. Yes, individual likelihoods are so small, that yes even a MLE solution is extremely unlikely to be correct.

broken image

It is fair to ask why the likelihoods are useful if they are so small, and it's not a good answer to talk about how they could be expressed as logs, or even to talk about the properties of continuous distributions. I think most of the replies, here and on stack exchange, are answering slightly the wrong question.

broken image