The National Weather Service Valid Time Event Code (VTEC) - PowerPoint Presentation

Slide1

The National Weather Service Valid Time Event Code (VTEC)and Event ID Reuse

Daryl Herzmann

Iowa Environmental

Mesonet

Iowa State University

@

akrherz

About Me2001, Meteorology Degree from Iowa State

2001, Started Iowa Environmental

Mesonet

https://mesonet.agron.iastate.edu

An environmental data warehousing project with massive/unique archives available freely online (not just data from Iowa

)

2005, co-creator of instant messaging system that became

NWSChat

https://nwschat.weather.gov

2007-2016, contractor with the National Weather Service

Worked on

NWSChat

, NWWS-OI, GIFS, IRIS, RIDGE-II, and other acronyms

10,000ft View of How NWS issues Alerts

Offices typically use AWIPS to generate a formatted ASCII text file

This text file goes on a wild and woolly journey to reach the world via many different routes (deduplication is tough)

Near-zero automated quality

c

ontrol is done on this text product, vendors have to account for GIGODiagram shows the “NOAAPORT” system, but a number of other dissemination systems existMost data services have to parse this text product to get atomic data. There is some “native” CAP emittance from AWIPS

https://www.weather.gov/about/warning-dissemination

Example Warning (circa 2003) prior to VTEC

WFUS53 KDMX 102146

TORDMX

IAC007-185

-102230-

BULLETIN - EAS ACTIVATION REQUESTED

TORNADO WARNING

NATIONAL WEATHER SERVICE DES MOINES IA

446 PM CDT SAT MAY 10 2003

THE NATIONAL WEATHER SERVICE IN DES MOINES HAS ISSUED A* TORNADO WARNING FOR... APPANOOSE COUNTY IN SOUTH CENTRAL IOWA EASTERN WAYNE COUNTY IN SOUTH CENTRAL IOWA* UNTIL 530 PM CDT

* AT 443 PM...RADAR INDICATED A THUNDERSTORM CAPABLE OF PRODUCING A TORNADO 3 MILES SOUTHWEST OF SEYMOUR...OR ABOUT 46 MILES SOUTHWEST OF OTTUMWA...MOVING NORTHEAST AT 35 MPH.

* THE STORM WILL BE...

6 MILES SOUTHEAST OF PROMISE CITY AROUND 450 PM...

NEAR PLANO AROUND 455 PM...

3 MILES NORTHWEST OF NUMA AROUND 500 PM...

NEAR MYSTIC AROUND 505 PM...

NEAR RATHBUN AROUND 510 PM AND

5 MILES NORTHWEST OF UDELL AROUND 520 PM.

THIS WARNING REPLACES THE SEVERE THUNDERSTORM WARNING PREVIOUSLY

ISSUED FOR WAYNE AND APPANOOSE COUNTIES.

A TORNADO WATCH REMAINS IN EFFECT UNTIL 1000 PM SATURDAY EVENING FOR EASTERN IOWA.

LAT...LON 4087 9335 4059 9334 4061 9269 4089 9266

JOHNSON

2005, along came Valid Time Event Code (VTEC)

NWS kept the ASCII text format, but added a special string to the product meant for computers to consume

The period delineated parts

WFUS53 KDMX 282344

TORDMX

IAC125-181-290015-

/O.NEW.KDMX.TO.W.0026.170628T2344Z-170629T0015Z/BULLETIN - EAS ACTIVATION REQUESTED

Tornado Warning

National Weather Service Des Moines IA

644 PM CDT WED JUN 28 2017…truncated…PartDescriptionOOperational/Experimental/Test Flag

NEW

Alert status

KDMX

Issuance Center

TO

Phenomena

W

Significance

0026

Yearly Event ID (1

thru 9999)

170628T2344Z

ISO-

ish

event start UTC

170629T0015Z

ISO-

ish

event end UTC

12.5 years of VTEC parsing experience

VTEC put lipstick on the ASCII-product pig. Folks have a fighting chance at parsing NWS text and getting it mostly right

Event IDs are supposed to be unique for a given year

The concept of a “year” is very tricky

Nothing in VTEC tells you which year the event ID is valid for

Enforcement of uniqueness of Event IDs is local office based and subject to all sorts of failures and errors (based on local AWIPS database table)In theory, VTEC + UGC (Universal Geographic Code) should provide logical and programmatic lifecycles of alerts, GIGO still happens near daily

VTEC has a concept of “indefinite time” (000000T0000Z), which is not pleasant to account for in codeVTEC did not solve the conundrum of NWS Alert Products being designed to be read by humans, but computers/automation are more important

The 2017 VTEC Event ID Re-use Issue

Starting ~summer 2017, an unknown failure with ongoing AWIPS upgrades caused the VTEC event database to be emptied

Subsequently issued alerts for the year all reset to event ID 1

Sadly, I did not catch this until late October.

During November, lots of emails / discussions with NWS, wrote some apps for their contractor to attempt to fix the AWIPS databases

Got to mid December without fixing and NWS HQ made a decision not to fix (why: too late in year, issue is mute once calendar flips)My accounting, impacted 332 alerts

The issue at hand, the test Tsunami Warning

WEXX20 PAAQ 061328

TSUAT1

TEST...Tsunami Message Number 1...TEST

NWS National Tsunami Warning Center Palmer AK

828 AM EST Tue Feb 6 2018/

T.NEW.PAAQ.TS.W.0003.180206T1328Z-180206T1428Z/OK, so the “T” (Test) flag was used, end of story? Sadly, nope.

A number of strange things happened

Two text products were issued at the same time

The first was truncated before the VTEC string appeared

The second contained VTEC, but had no mandatory WMO headers denoting the duplicated product in time

/T.NEW.PAAQ.TS.W.

0003.180206T1328Z-180206T1428Z/Event ID 3 had previously been used for an operational warning in JanuaryNWS Product directives are a bit vague if this reuse is OK given the other Tsunami was in the Pacific basin. IMO: not OK.

An additional Test Tsunami Warning was issued at 11 AM ESTProduct formatting was terrible, issued twice, truncated, missing important parts, used Event ID 4

In Closing

I continue to work closely with the NWS to identify and help correct issues found with VTEC / dissemination.

Presently working on QA/QC of their various CAP feeds to identify latency issues / incorrect information

Please squawk at the NWS when you have issues. For many issues, it appears that I am the only one they hear from

 (Others have given up hope getting responses)

Happy to engage others in this areaTwitter: @akrherz

Snail Mail: akrherz@iastate.eduGithub: https://github.com/akrherz/pyIEM/