Transcript performance of national weather service forecasts versus model

PERFORMANCE OF NATIONAL WEATHER SERVICE
FORECASTS VERSUS MODEL OUTPUT STATISTICS
Jeff Baars
Cliff Mass
Mark Albright
University of Washington, Seattle, Washington
This work was supported in part by the DoD Multidisciplinary University Research
Initiative (MURI) program administered by the Office of Naval Research Under Grant
N00014-01-10745.
The point of it all…
Vislocky and Fritsch (1997), using 1990-1992
data, saw that an average of 2 or more MOS’s
(CMOS) outperformed individual MOS’s and
many human forecasters in a forecasting
competition.
How has the story changed since then?
And how well do CMOS, MOS & the NWS
perform during extreme conditions? In
different seasons? In different regions?
Data
July 1 2003 – Jan 1
2004 (6 months).
30 stations, all at
major WFO sites.
Maximum and
minimum
temperature, and
POP.
Data (con’t)
Consensus MOS (CMOS)– simply an average of 4
MOS’s: AMOS, EMOS, MMOS, NMOS.
12Z-issued forecast from NWS matched against
previous 00Z forecast from models.
NWS has 00Z model data available, and has added
advantage of watching conditions develop since 00Z.
Models of course can’t look at NWS, but NWS looks at
models.
Forecasts going out 48 (60) hours, so in the analysis
there are:
Two maximum temperatures (MAX-T),
Two minimum temperatures (MIN-T), and
Four 12-hr POP forecasts.
Data (con’t)
NWS MOS definition for MAX-T and MIN-T
and for POP.
Observed precipitation data converted to
binary rain/no-rain data for Brier Score
calculations.
Maximum and Minimum
Temperature
Total MAE for the 6 forecasts
Maximum temperature
Minimum temperature
MAE, temperature, by forecast period
Bias Time Series, all stations
MAE, Maximum temperature period 1, by
station
West
Mtn
West
Southwest
South
Midwest
East
MAE by forecast period, large departure
from climatology
Probability of Precipitation
Total Brier Score for all 6 forecasts
Brier Scores by forecast period
Brier Scores by forecast period, large
departure from climatology
Conclusion
CMOS shows equal or superior forecast skill
compared to NWS and individual MOS’s when
all time periods are considered.
True for max and min temperatures and POPs.
The NWS forecasts show superior forecast
skill for max. temperatures during large
departures from climatology.
http://www.atmos.washington.edu/~jbaars/
mos_vs_nws.html
Future Work
Simple weighting correction to CMOS.
Remove worst model from CMOS.