Guidebook for Managing Data from Emerging

Transcript:Guidebook for Managing Data from Emerging:
Guidebook for Managing Data from Emerging Technologies for Transportation Transportation Research Board National Cooperative Research Program Project 08-116 March 2020 Overview of Presentation Project background and objectives Challenges to managing data from emerging technologies What is big data? Why should agencies move toward the modern approach to data management? How can agencies make the move toward this approach? Roadmap to Managing Data from Emerging Technologies for Transportation Supporting tools 2 Guidebook for Managing Data from Emerging Technologies for Transportation Project Background and Objectives Background: New, big, and varied datasets are available to transportation agencies at an increasing pace. These data have tremendous potential to offer new insights to transportation agencies. The volume, speed, and granularity of these data are unprecedented and will fundamentally alter the transportation sector. Research Objectives: Develop a framework for managing data from emerging technologies, including data from connected and automated vehicles and data linked to new mobility initiatives. Outline a process for applying this framework to help agencies incorporate these data into the decision-making process. 3 Guidebook for Managing Data from Emerging Technologies for Transportation Transportation Agency Challenges to Managing Data from Emerging Technologies Reliance on traditional database management systems. Data from emerging technologies are too large, too varied in nature, and will change too quickly to be handled by these traditional data systems. Struggle to break down business unit and data silos. Do not fully recognize the value of big data or the eminent need to ready for it. Do not fully understand the uses and benefits of cloud-based architecture conducive to handling data from emerging technologies. Have difficulty hiring and retaining modern data management professionals. “Our big data issues are straightforward, we don’t have the technology, money, or skills.” – CITY DOT Experience a loss of control to vendors over data, technology, and service agreements. Are unequipped to handle this level of big data at an organizational level. 4 Guidebook for Managing Data from Emerging Technologies for Transportation 5 Big Data is More Than a Buzzword or Simply “Lots of Data” Big data may refer to data sets that are so vast and complex that they require new and powerful computational resources to process.1 Big data may encompass all the non-traditional strategies and technologies needed to gather, organize, process, and generate insights from large datasets.2 Big data is an approach to generating knowledge in which advanced techniques are applied to the capture, management, and