Probability Calibration
Calibration can be defined as the process by which the forecast is adjusted / modified based on what has been observed in the past.
For example, if we note that for a specific location, a probabilistic forecast of 80% below normal is actually correct (observed) 60% of the time, the calibrated probabilistic forecast would be 60%.
At the moment, the probabilistic forecasts are NOT calibrated. Why?
One important constraint is that calibration necessitates a large enough database of forecast versus observed values to be statistically significant. The seasonal forecasts database over Canada covers a period of 30 years. While this is sufficient to obtain a statistically significant calibration value for all of Canada, it is insufficient to calibrate for a specific location. This is the reason why the probabilistic forecasts are not calibrated. Many methods for calibration are being evaluated and are the subject of ongoing research, including the calibration for specific regions of Canada (i.e. Provinces).
To obtain a calibrated value for all of Canada, determine the forecast probabilities (see the maps for temperature and precipitation) and then look at the associated reliability diagrams (at the bottom of the web pages). Look at the observed frequency of these diagrams as a function of the forecast probability. There is one curve for each forecast category (above, near and below normal). Error bars are also found on the diagrams to indicate uncertainty on the estimate of the observed frequency. The error bars are mainly a function of the number of observations used in the calculation of the observed frequency. They indicate, with a level of confidence of 95%, that the real observed frequency lies within the values defined by the error bars. By using this information, you can obtain a calibrated probabilistic forecast for Canada. As mentioned previously, this calibration is not suitable for all applications since it was developed over a limited number of years and with data covering Canada as a whole.
The Green Lane™,
Environment Canada's World Wide Web Site.
