Information on Contingency Tables
The verification of the seasonal forecast in category is done using 3x3 contingency tables. The forecast and observed categories are simply classified in a table of 3 rows and 3 columns (see figure 1 below). There is a row for each observed category and a column for each forecast category (above, near and below normal). For each stations or grid points over Canada, 1 is added to the grid element of the contingency table for each event according to the intersection of the forecast category and the observed category (Stanski et al., 1989).
FORECASTS
BELOW NORMAL ABOVE
------------------------------------------------------------------
| | | |
BELOW | A | B | C | D
| | | |
------------------------------------------------------------------
| | | |
OBSERVATIONS NORMAL | E | F | G | H
| | | |
------------------------------------------------------------------
| | | |
ABOVE | I | J | K | L
| | | |
------------------------------------------------------------------
M N O P
Figure 1: Template of the 3x3 contingency tables used in verification of the seasonal and 30-Day forecasts at the Canadian Meteorological Centre (CMC).
Many scores can be calculate from the tables. The percent correct is the sum of the diagonal elements over the total elements (multiply by 100) and it is used as the standard score for the long-range forecasts at CMC (see percent correct maps of the models). You will find below a list of the scores shown on the web page menu Verification of Past Forecast. For a detailed explanation of each score please refer to World Meteorological Organization document of Stanski et al (1989).
Scores from the contingency tables:
PC (PERCENT CORRECT): 100*(A+F+K)/P
HSS1 (HEIDKE SKILL SCORE): ((A+F+K)-C1)/(P-C1)
HSS2 (HEIDKE SKILL SCORE): ((A+F+K)-C2)/(P-C2)
C1 (FOR HSS1): 0.4*(M+O)+0.2*N
C2 (FOR HSS2): (DM+HN+IO)/P
BUSTC (PROPORTION OF ERRORS OF TWO CATEGORIES - COLD) : I/M
BUSTW (PROPORTION OF ERRORS OF TWO CATEGORIES - WARM) : C/O
BUSTT (PROPORTION OF ERRORS OF TWO CATEGORIES - TOTAL) : (C+I)/(M+O)
BIASC (BIAS - COLD) : M/D
BIASN (BIAS - NORMAL) : N/H
BIASW (BIAS - WARM) : O/L
PODC (PROBABILITY OF DETECTION OF A COLD EVENT) : A/D
PODN (PROBABILITY OF DETECTION OF A NORMAL EVENT) : F/H
PODW (PROBABILITY OF DETECTION OF A WARM EVENT) : K/L
FARC (FALSE ALARM RATIO OF COLD FORECASTS) : (E+I)/M
FARN (FALSE ALARM RATIO OF NORMAL FORECASTS) : (B+J)/N
FARW (FALSE ALARM RATIO OF WARM FORECASTS) : (C+G)/O
RELC (RELIABILITY OF COLD FORECASTS : A/M = (1 - FARC)
RELN (RELIABILITY OF NORMAL FORECASTS : F/N = (1 - FARN)
RELW (RELIABILITY OF WARM FORECASTS : K/O = (1 - FARW)
CSIC (CRITICAL SUCCESS INDEX - COLD) : A/(D+M-A)
CSIN (CRITICAL SUCCESS INDEX - NORMAL) : F/(H+N-F)
CSIW (CRITICAL SUCCESS INDEX - WARM) : K/(L+O-K)
Reference
Stanski, H. R., L. J. Wilson, and W. R. Burrows, 1989: Survey of common verification
methods in meteorology. WWW Tech. Rep. 8, WMO/TD 358, 114 pp.
The Green Lane™,
Environment Canada's World Wide Web Site.
