Transcript STEPS: An empirical treatment of forecast uncertainty

STEPS: An empirical treatment of
forecast uncertainty
Alan Seed
BMRC Weather Forecasting Group
Outline




Where does uncertainty come from?
Can we get rid of it?
How can we quantify it?
Where to from here?
Sources of forecast uncertainty
 Radar rainfall estimation
 Field motion estimation
 Development during forecast period
Radar Measurement Error
 Major contribution to forecast error in the first
hour
10
MSE (mm/h)^2
8
6
4
2
0
0
15
30
45
60
75
Lead Time (min)
Mean square error of rainfall forecast (1km,
15min) as a function of lead time based on a
5-day storm, 200 rain gauges for ground
truth
Radar Measurement Error
14
Mean Std Error (mm/h)
 Radar measurement
errors are highly
variable in time
12
10
8
6
4
2
14
0
12/05/2003
12:00
Mean Std Error (mm/h)
12
12/05/2003
18:00
10
13/05/2003
00:00
13/05/2003
06:00
13/05/2003
12:00
Time
8
Radar QPE
60Min
6
4
2
0
12/05/2003 13/05/2003 14/05/2003 15/05/2003 16/05/2003 17/05/2003 18/05/2003
00:00
00:00
00:00
00:00
00:00
00:00
00:00
Time
Radar QPE
60Min
Errors increase in
significant
convective
weather
Errors Due to Changes in Field
Velocity
Bowler et al 2005; submitted to QJR
Development during the forecast
lead time
 Climatology- topography, diurnal cycle
 Rate of temporal development depends on
scale- predictability limits
 Situation dependent
Effect of topography
Effect of Topography
Predictability is a Function of Scale
Can We Get Rid of Uncertainty?
•No, but
we can reduce it
we can understand it
we can tell our users about it
Quantifying Forecast Uncertainty
 Physical ensembles
 Statistical ensembles
Multi Model Ensemble
3500 campers were evacuated ahead of a
flood at Tamworth after a qualitative
assessment of risk based on the ensemble
mean.
Gordon McKay, Beth Ebert
Short Term Ensemble Prediction
System
 Generate a deterministic nowcast based on
radar data
 Estimate the error for the nowcast and a
NWP forecast over a hierarchy of spatial
scales
 Merge the nowcast with the NWP forecast
using weights that are a function of the
forecast error and spatial scale
 Generate an ensemble by perturbing the
deterministic blend with a stochastic
component
4-2 km
16-8 km
32-16 km
128-64 km
256-128 km
Spectral Decomposition
64-32 km
8-4 km
Temporal Development Model

The Lagrangian temporal development for each level in the
cascade is forecast using an AR(2) model
Xˆ k ,i, j (t  n  1)  k ,1(t ) X k ,i, j (t  n)  k ,2 (t ) X k ,i, j (t  n  1)   k ,i, j (t )


AR(2) parameters are estimated at each time step for each
level
The innovation term  is spatially correlated, temporally
uncorrelated
Forecast Skill
 Model skill is taken to mean the fraction of
the observed variance that is explained by
the model, r2
 Skill of Nowcast is given by the AR-2 model
 Skill of the NWP is calculated as the
correlation between the NWP cascade and
radar cascades
Telling the users
 Observation uncertainty
 Forecast uncertainty
Observation Uncertainty
15-min average interpolated
from gauge network
Error as a fraction
of the rain field
variance
15-min rainfall accumulation
forecast- 20 member stochastic
nowcast ensemble
Ensemble mean
Ensemble standard deviation
Stochastic nowcast model does not yet include
the observation error model
Way Forward:
Heuristic Probabilistic Forecasting?
 Held a workshop in Montreal- presentations can be
found at http://www.radar.mcgill.ca/~cwrp/
• NWP has improved significantly but errors will remain
• Use persistence of NWP errors to develop post-processing
systems to mitigate the error
• Need to model initiation, growth, decay
• Conceptual probabilistic models are likely to be useful
• Would like to develop a common framework and to
collaborate on developing probabilistic forecast models
Thank you