NCHRP 08-110 Traffic Forecasting Accuracy Assessment Research - PowerPoint Presentation

NCHRP 08-110Traffic Forecasting Accuracy Assessment Research May 2019 Greg Erhardt & Jawad Hoque University of Kentucky Dave Schmitt Connetics Transportation Group

2 “The greatest knowledge gap in US travel demand modeling is the unknown accuracy of US urban road traffic forecasts.”Hartgen, David T. “Hubris or Humility? Accuracy Issues for the next 50 Years of Travel Demand Modeling.” Transportation 40, no. 6 (2013): 1133–57.

Project Objectives “The objective of this study is to develop a process to analyze and improve the accuracy, reliability, and utility of project-level traffic forecasts.” -- NCHRP 08-110 RFPAccuracy is how well the forecast estimates project outcomes.Reliability is the likelihood that someone repeating the forecast will get the same result.Utility is the degree to which the forecast informs a decision. 3

1. Research Approach

Research Question and Approach 5

Challenges 6 Large-N AnalysisDeep Dives

2. Large N Analysis

How Accurate Are Traffic Forecasts? 8 On average, the actual traffic volume is about 6% lower than forecast. On average, the actual traffic is about 17% different from forecast.  

How Accurate Are Traffic Forecasts? 9 Traffic forecasts are more accurate, in percentage terms, for higher volume roads.  

Estimating Uncertainty 10 The quantile regression models presented in this research provide a means of estimating the range of uncertainty around a forecast.

Large N Results 95% of forecasts reviewed are “accurate to within half of a lane.” Traffic forecasts show a modest bias, with actual ADT about 6% lower than forecast ADT. Traffic forecasts had a mean absolute percent error of 25% at the segment level and 17% at the project level. 11

Large N Results Traffic forecasts are more accurate for: Higher volume roads Higher functional classesShorter time horizonsTravel models over traffic count trendsOpening years with unemployment rates close to the forecast yearMore recent opening & forecast years12

3. Deep Dive Results

Deep Dives Projects selected for Deep Dives Eastown Road Extension Project, Lima, OhioIndian River Street Bridge Project, Palm City, FloridaCentral Artery Tunnel, Boston, MassachusettsCynthiana Bypass, Cynthiana, KentuckySouth Bay Expressway, San Diego, CaliforniaUS-41 (later renamed I-41), Brown County, Wisconsin14

Deep Dive Methodology Collect data:Public DocumentsProject Specific Documents Model RunsInvestigate sources of errors as cited in previous research:Employment, Population projections etc.Adjust forecasts by elasticity analysisRun the model with updated information15

Deep Dives General Conclusions The reasons for forecast inaccuracy are diverse. Employment, population and fuel price forecasts often contribute to forecast inaccuracy.External traffic and travel speed assumptions also affect traffic forecasts.Better archiving of models, better forecast documentation, and better validation are needed.16

5. Recommendations

1. Use a range of forecasts to communicate uncertainty Report a range of forecasts.Quantile regression If the project were at the low/high end of the forecast range, would it change the decision?18

2. Archive your forecasts Bronze: Record basic forecast and actual traffic information in a database Silver: Bronze + document forecast in a semi-standardized report Gold: Silver + make the forecast reproducible 19

3. Periodically Report the Accuracy Provides empirical information on uncertainty. Ensures a degree of accountability and transparency20

4. Use Past Results to improve forecasting method Evaluate past forecasts to learn about weaknesses of existing model Identify needed improvementsTest the ability of the new model to predict those project-level changesDo the improvements help?Estimate local quantile regression modelsIs my range narrower than my peer’s?21We build models to predict change. We should evaluate them on their ability to do so.

Why? Giving a range  more likely to be “right” Archiving forecasts and data  Provides evidence for effectiveness of tools usedData to improve models  Testing predictions is the foundation of science22Together, the goal is not only to improve forecasts, but to build credibility.

Questions & Discussion