are some scientists overstating predictions? or how good are crystal balls?
For those who like to delve into the details:
LANL + Technion | The Good, The Bad, and The Ugly of Predictive Science
[An excellent paper.]
Prediction
Prediction
May the gods preserve and defend me from self-righteous altruists; I can defend myself from my enemies and my friends.
Re: Prediction
"CONCLUSIONTyphoon wrote:are some scientists overstating predictions? or how good are crystal balls?
For those who like to delve into the details:
LANL + Technion | The Good, The Bad, and The Ugly of Predictive Science
[An excellent paper.]
This work studies the relationship between several aspects of prediction accuracy. The main
conclusion is that, in assessing the predictive accuracy of numerical models, one should never focus on a
single aspect only. (like numbers! - SM) Instead, the trade-offs between fidelity-to-data, robustness-to-uncertainty, and
confidence-in-prediction should be explored. One consequence that cannot be emphasized enough is that
the calibration of numerical models—which focuses solely on the fidelity-to-data aspect—is not a sound
strategy for selecting models capable of making accurate predictions. Calibration leaves models
vulnerable to modeling uncertainty. It is further established that models selected for their robustness to
uncertainty will tend to make inconsistent predictions. This finding seems discouraging because one
would like to make accurate predictions while being robust to the sources of uncertainty and lack of knowledge.
It is, however, a fundamental limitation of predictive science that scientists and engineers
should not loose sight of. The trade-off simply expresses that obtaining accurate predictions is
conditioned by the assumptions upon which the models are built."
Thanks for posting. Above text in red is pure gold.
Don't the best Predictive Scientists focus primarily on buying lottery tickets?
Translation: Ignorance is robust. Ignorant people are confident. Being right in the long run is less fun than being ignorant and confident in the short run. Being wrong and popular pays more than being right. Hot chicks dig confidence. Scientists who are ignore uncertainty, and are wrong and confident get laid more than those who question their premises and are right. There is no down side to ignorance!
Last edited by Simple Minded on Wed Feb 11, 2015 2:30 am, edited 3 times in total.
Re: Prediction
cooincidently i was just reading http://www.theregister.co.uk/2015/01/19 ... rd_claims/
ultracrepidarian
Re: Prediction
Over-fitting is a big issue.
If only it were as easy to predict the future as it is to "predict" the past.
If only it were as easy to predict the future as it is to "predict" the past.
cultivate a white rose