Israel Jirak Steve Weiss and Chris Melick Storm Prediction Center WoF Workshop April 3 2014 Convectionallowing ensembles 4km grid spacing can provide important information to forecasters regarding the uncertainty of storm intensity mode location timing etc on the outlook t ID: 206757
Download Presentation The PPT/PDF document "Evaluation and Comparison of Multiple Co..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Evaluation and Comparison of Multiple Convection-Allowing Ensembles Examined in Recent HWT Spring Forecasting Experiments
Israel Jirak, Steve Weiss, and Chris Melick Storm Prediction Center
WoF Workshop, April 3, 2014Slide2
Convection-allowing ensembles (
~4-km grid spacing) can provide important information to forecasters regarding the uncertainty of storm intensity, mode, location, timing, etc. on the outlook to watch scaleThese ensembles will play an important role in the ability of SPC to provide a more continuous flow of probabilistic hazard information in support of WoF
2 March 2012
29 June 2012
24-h neighborhood
prob
UH ≥25 m
2
/s
2
24-h ensemble max 10-m Wind Speed
Convection-Allowing Ensembles
OverviewSlide3
An experimental real-time Storm-Scale Ensemble Forecast (SSEF) system has been produced for the NOAA Hazardous Weather Testbed (HWT) Spring Forecasting Experiment (SFE) by OU CAPS since 2007 through CSTAR funding
Comprised of 4-km convection-allowing WRF and ARPS members:2007: 10 members; 2008: 10 members; 2009: 20 members; 2010: 26 members (full CONUS to 30 h); 2011: 50 members (full CONUS to 36 h); 2012: 28 members; 2013: 27 members at 00Z (to 48 hours) and 8 members at 12Z (to 18 hours)Primarily examine explicit storm attributes, especially hourly maximum fields (HMFs): Updraft Helicity, Updraft Speed, 10-m AGL Wind Speed, 1-km AGL Reflectivity
Ensemble display approaches include spaghetti plots, ensemble max, and neighborhood probabilities (more on next slide)SPC began processing deterministic high-resolution runs from EMC and NSSL as the Storm-Scale Ensemble of Opportunity (SSEO) in 2011
The 4-km AFWA ensemble was made available to SPC in 2012
Convection-Allowing Ensembles
HistorySlide4
Traditional ensemble probabilities of HMFs from high-resolution models are not especially useful, owing to poor agreement among members at the grid point of these fields.
Applying a binary neighborhood approach to a storm-scale ensemble improves the statistical results of HMFs in forecasting severe weatherROI=20-40 kmSigma=30 grid points
Same approach can also be applied to observations (e.g., radar reflectivity) for verification purposes
03 May 2008 (
Harless
2010)
Convection-Allowing Ensembles
Neighborhood ProbabilitiesSlide5
Convection-Allowing Ensembles
Neighborhood Probabilities
HM Updraft Helicity > 25 m
2
s
-2
SSEO 24-hr
fcst
valid 00Z 28 April Slide6
Convection-Allowing Ensembles
Neighborhood Probabilities
Grid-Point Probability
HM Updraft Helicity >25 m
2
s
-2
SSEO 24-hr fcst valid 00Z 28 April Slide7
Convection-Allowing Ensembles
Neighborhood Probabilities
40-km Neighborhood Probability HM Updraft Helicity >25 m
2
s
-2
SSEO 24-hr
fcst valid 00Z 28 April Slide8
Convection-Allowing Ensembles
Neighborhood Probabilities
40-km Neighborhood Smoothed
Prob
HM Updraft Helicity >25 m
2
s
-2
SSEO 24-hr fcst valid 00Z 28 April Slide9
OU/CAPS
Storm-Scale Ensemble Forecast (SSEF) System
Since 2007; 36-hr forecasts from 00z; 12z runs began in
2013
Primarily WRF-ARW; 4-km grid spacing; forecasts
to
60hrs in 2014 Multi-physics, multi-initial conditions: applies SREF perturbations to NAM
ICsAdvanced physics, 3DVAR & radar data assimilation; available for HWT/SFE SPC Storm-Scale Ensemble of Opportunity (SSEO)Since 2011; 36-hr forecasts at 00z & 12z; 7 members (2 time-lagged)
Multi-model (ARW, WRF-NMM & NMM-B), multi-physics; ~4-km grid spacingUses available deterministic models at SPC to process as an ensembleBasic data assimilation through NDAS; available year-round in SPC Air Force Weather Agency (AFWA) Ensemble
Since 2012; 60-hr forecasts at 00z & 12z; 10 members; 4-km grid spacingSingle model (WRF-ARW), multi-physics, multi-initial conditionsCold start from downscaled global model forecasts (GFS, UM, CMC)No data assimilation; available year-round in SPC
Convection-Allowing EnsemblesSystem ComparisonSlide10
The Fractions Skill Score (FSS) was calculated for neighborhood probability (ROI=40 km; σ=40 km) of updraft helicity ≥ 25 m
2s-2 for the SSEO/SSEF versus practically perfect hindcasts of preliminary severe weather reports (ROI=40 km; σ=120 km) during SE2011The SSEO had higher fractions skill score (FSS) for neighborhood probabilities of UH ≥25 m
2/s2 during SFE2011 than the SSEF
The number of members included in the SSEF did not seem to have a strong impact on the statistical results for neighborhood probabilities of UH during SE2011 when verified against severe weather reports
Convection-Allowing Ensembles
SFE2011 Results
Nprob
UH ≥25 m2/s2
SSEO
SSEF – 24
mem
FSS = 0.84
FSS = 0.68
3-hr [NPRS]:UH ≥25
m
2
s
-2
valid 06Z on 02 June
2011
w/ verifying reports and practically perfect
hindcast
FSS 3-h periods SFE2011Slide11
Even for 6-h QPF, the SSEO received the highest subjective ratings relative to other operational and experimental models and ensembles during SFE2011
Statistically, the probabilistic QPF forecasts (>0.5”) from the SSEO were typically as good as (if not better than) the SSEF during SFE2011 at various lead times
Convection-Allowing Ensembles
SFE2011 Results
from
Tara
Jensen, DTC
from Dave Novak,
WPC
SSEO favored over CAPS ensemble Slide12
During SFE2012, the SSEO outperformed the SSEF and AFWA in terms of FSS for neighborhood probabilities of reflectivity ≥40
dBZ (bug later found in SSEF)Subjective ratings by the SFE2012 participants of HMF ensemble forecasts tended to favor the SSEO forecasts of UH over the SSEF and AFWA forecasts
Convection-Allowing Ensembles
SFE2012 Results
Hourly FSS
Nprob
Refl
≥40 dBZ 3-hr ensemble forecast ratings (max, nprob) of UHSlide13
The quality of the AFWA forecasts was less consistent than the SSEO forecasts
Some UH forecasts from the AFWA ensemble were very good (bottom left) while others were poor (bottom right)
Convection-Allowing Ensembles
SFE2012 ResultsSlide14
The impact of radar data assimilation in the CAPS SSEF was evident in the first 4 hours of the 12 UTC-initialized forecast.
Otherwise, there was little statistical difference in the FSS among the 00 and 12 UTC SSEO and SSEF.Subjective ratings of 00Z ensemble HMFs were again favorable for the SSEO during SFE2013
Convection-Allowing Ensembles
SFE2013 Results
Hourly FSS
Nprob
Refl ≥40 dBZ
3-hr ensemble HMF forecast ratings (max, nprob)Slide15
Why is a “poor man’s ensemble” (i.e., SSEO) performing as well as formally designed ensembles? Let’s consider some aspects of configuration for convection-allowing ensembles
Single model vs. multi-modelNumber of membersInitial conditions and IC/LBC perturbationsPhysics
Convection-Allowing Ensembles
ConfigurationSlide16
Even with the same initial conditions, clustering often occurs by model core
Generally more confident in a solution if different model cores are in agreementIs a multi-model approach a good way to address uncertainty in a convection-allowing ensemble?
Convection-Allowing Ensembles
Configuration: Single model vs. multi-model
WRF-NMM
WRF-ARW
SSEO
21Z on 16 April 2011
3-hr spaghetti plot of UH
≥25
m
2
s
2Slide17
Is the success of the SSEO a fortuitous balance of members with an
underforecast bias and those with an overforecast bias (Row and Correia, 2014 AMS); not necessarily a result of using multiple model cores?Neighborhood verification of radar reflectivity reveals members with lower biases (e.g., NAM Nest) versus those with higher biases (e.g., NSSL-WRF)
Biases will change with upcoming HiResW upgrade, so we may learn more this spring
Convection-Allowing Ensembles
Configuration: Single model vs. multi-modelSlide18
For the way convection-allowing ensembles are currently configured, there does not appear to be a huge benefit to running more than ~10 members
Clark et al. (2011) objectively identified the “point of diminishing returns” at 9 members for 6-hr QPF at f30 and 2-km scale
Convection-Allowing Ensembles
Configuration: Number of members
Could run additional members to more effectively sample the forecast PDF, but is it worth the additional computational cost? Use a larger neighborhood?
from Clark et al. (2011) Slide19
Currently, all members of the SSEO are initialized from the NAM (including two time-lagged members), so diversity primarily arises from multi-model/physics
AFWA approach (single model, 3 different IC/LBCs) often leads to higher, overconfident probabilities SSEF approach not an obvious improvement over single, unperturbed IC (i.e. SSEO), suggesting ICs not properly perturbed at this scale
Convection-Allowing Ensembles
Configuration: Initial conditions and IC/LBC perturbations
AFWA
OBS
high probs
nothing observedSlide20
In four test runs for May 2013, Kong et al. (2014) found larger domain-average ensemble spread for multiple fields by directly using LBCs from SREF members rather than extracting perturbations and applying to the NAM (current strategy)
NSSL-WRF ensemble this spring will directly utilize IC/LBCs from selected SREF members
Convection-Allowing Ensembles
Configuration: Initial conditions and IC/LBC perturbations
Ensemble Spread
from Kong et al. (2014) Slide21
Though spread in an ensemble with physics-only diversity is less than that from an ensemble that also includes IC/LBC perturbations, the contribution to spread from physics diversity can be large, including for instability fields (Clark et al. 2010)
Convection-Allowing Ensembles
Configuration: Physics diversity
from Clark et al. (2010) Slide22
Convection-allowing ensembles (~4-km grid spacing) can provide useful information to forecasters regarding the uncertainty of storm intensity, mode, location, timing, etc. in generating outlooks on Day 1; setting the stage for the continuous flow of probabilistic hazard information down
toward the warning scaleThe SPC SSEO has proven to be as useful/skillful as formally designed convection-allowing ensembles, which raises questions about effective/proper configuration of these types of systemsNSSL is running eight 00Z members this spring with only IC/LBC diversity directly from 21Z SREF members
CAPS is planning to run an experimental 4-km EnKF SSEF system this year in near real-time for comparison with traditional SSEF forecasts
Convection-Allowing Ensembles
Summary