Brier Score Calibration Tool — Forecast Accuracy & Model Evaluation
Audit prediction accuracy using the Brier Score metric. Evaluate calibration curve reliability, decomposition variance, and benchmark against superforecaster standards.
Event Forecasts & Quadratic Loss Log
| Event Description | Forecast P(Yes) | Actual Outcome | Squared Error (f - o)² |
|---|
Empirical Calibration Curve (Forecasted vs Observed)
Frequently Answered Questions
What is the Brier Score and how is it interpreted?
The Brier Score measures the mean squared deviation between forecasted probabilities and binary resolution outcomes. A score of 0.0 indicates perfect clairvoyance, 0.25 represents an uninformative random 50/50 baseline, and scores below 0.15 represent elite professional superforecasting.
What is the Brier Skill Score (BSS)?
The Brier Skill Score measures percentage improvement over an uninformative baseline: BSS = 1 - (BS / 0.25). A positive BSS confirms genuine predictive skill, whereas negative scores demonstrate negative alpha.
What is the difference between Calibration and Resolution?
Calibration measures whether events predicted with 70% probability happen 70% of the time. Resolution measures the ability to make distinct, confident predictions away from 50% that still resolve accurately.