When Prediction Helps Create the Future It Claims to Forecast
A risk score can become causal when institutions reorganize access, scrutiny, or opportunity around it.
Prediction feels observational. A model estimates risk, likelihood, performance, fraud, recidivism, default, retention, or success. The number appears to describe a future that already exists somewhere ahead of us. But institutional predictions rarely stop at description.
Once a score changes who receives credit, surveillance, an interview, a developmental assignment, a medical intervention, or the benefit of the doubt, the prediction enters the causal chain. People receive different resources and constraints because of the forecast. Their later outcomes then become new data, which can make the original system look more accurate.
This is the predictive-intelligence paradox: a system can partially create the environment in which its own prediction appears validated. The problem becomes especially serious when error costs are unequal or when people have little ability to challenge the classification.
Responsible predictive systems therefore need more than accuracy metrics. They need recourse, audit, clear decision rights, feedback-loop analysis, protected alternatives, and an explicit account of what the institution itself changes after the score is produced.
This newsroom essay is original site commentary derived from the author’s supplied book arguments and frameworks. It is not a substitute for legal, medical, mental-health, financial, employment, or other regulated professional advice.