Transcript Slides
NCAR Real-time Ensemble Forecasting System Joint project with MMM/CISL, began early 2015. Produce 48-hour, full-CONUS, 10-member ensemble forecasts every day at 00 UTC. Forecasts started 7 April 2015. State-of-the-art data assimilation and modeling system using NCAR software (WRF and DART). Run daily for long period of time (12-16 months) to facilitate science large sample size, variety of weather events Disseminate forecasts to university and NOAA collaborators. Usage of CISL resources on a given day… Job System Ta s k i n g Wall-clock Chr/dy DA: 50-member Ens Fcst w/ WRF ys n=64; ptile=16 ~15 min/mem 1800 DA: Data assimilation w/ DART ys n=256; ptile=8 ~20 min 600 FCST: 10-member Ens Fcst w/ WRF ys n=512; ptile=16 ~4 hr/mem 20,000 POST: WRF Unified Post-processor ys n=16; ptile=16 ~3 min/job 500 gey/cal n=1 ~1-5 min/job 100 POST: Graphics generation Job Storage/day DA ~100 GB FCST ~3.7 TB* Graphics ~8 GB *much smaller subset archived to glade and HPSS Web presence hosted on webpub (http://ensemble.ucar.edu) Graphics pushed from glade to webpub Novel web displays developed using modern web dev practices Python-based post-processing system Developed python based ensemble forecast graphics library. Uses scientific python stack (matplotlib, numpy, scipy, etc.) Read in wrf output files, compute, and plot ensemble fields (ens mean, max, spread, other unique fields for convective-scale ensemble forecasting). Python dictionary to control plot properties Python-based post-processing system Examples: ./make_webplot.py –d=2015080600 Ensemble forecast date Forecast hour/range -tr=12 Fill field -f=t2_mean Barb field -b=wind10m_mean Contour field -c=mslp_mean Title -t=’Ensemble mean 2-m temperature (fill;F), MSLP (contour; hPa), and 10-m wind (kts)' Python-based post-processing system Python-based post-processing system Other examples: ./make_webplot.py –d=2015080600 -tr=12 -f=t2_var \ -t=’Ensemble spread 2-m Temp' ./make_webplot.py –d=2015080600 –tr=18,24 -f=t2_prob_32 \ -t=’Probability of 2-m Temp > 32F between 18-24hrs' ./make_webplot.py –d=2015080600 -tr=12 -f=t2_stamp \ -t=’2-m T postage stamp' ./make_webplot.py –d=2015080600 –tr=24 –f=cref_neprob_40 \ -t=’Neighborhood Prob of Comp. Refl > 40 dBZ' Use of geyser and caldera bsub –q geyser –W 00:05 –n 1 –P P123456 ./make_webplot.py –d=2015080600 -tr=12 -f=t2_var -t=’Ensemble spread 2-m T’ Produces a plot for each of 9 regions (CONUS, NE US, SE US, etc.) Each product (t2 mean, spread, max, etc.) run as a separate job, submitted by a master script. We produce a lot of products… ~200 individual products for each forecast hour, spread out over 4 hr period ~4,500 jobs per day on geyser/caldera (many jobs grouped together) ~80,000 graphics per day Use of geyser and caldera Job length on geyser/caldera on 3 August 2015 N = 4309 jobs 400 350 Number of jobs 300 250 200 150 100 50 0 15 30 45 60 75 90 105 120 135 150 165 180 195 210 225 240 255 270 285 300 315 Job wall-clock time (seconds) Mean job length ~ 95 seconds Web presence (http://ensemble.ucar.edu) Rsync used to push graphics to the web every minute when model is running. Ran into issues with overlapping rsync jobs, and slow rsync jobs with large number of files. Two solutions: 1) On web, place images in subdirectories per region. Rsync each region separately (~8,000 files instead of ~80,000). 1) Use /usr/bin/flock to ensure previous rsync is complete before beginning next rsync */1 22,23,0,1,2 /usr/bin/flock –n /tmp/rsynclock.lock rsync_files.csh