From the speech of Armando Cartenì | Osservatorio Sunrise

Join the Data Revolution

Ma preparati a conoscerne benefici e rischi

For decades, transportation systems were studied using static surveys that produced little more than partial snapshots of demand, at very long intervals. Today, however, the availability of continuous streams of real-time data marks a radical change: from traditional planning to adaptive planning, capable of dynamically monitoring, predicting, and correcting services. But what risks does this revolution entail? Professor Cartenì discussed this with us, reflecting on the benefits and challenges of the big data revolution in the field of mobility planning.

Join the Data Revolution

When you consider the speed and “arrogance” with which the rules are being subverted in the field of mobility data analysis, it is no exaggeration to speak of a true revolution. For decades, mobility has been studied and planned on the basis of surveys repeated years apart, sometimes even decades, possibly refined with interviews or one-off traffic counts. The result was little more than a static snapshot of mobility habits. It was unthinkable to talk about seasonality, off-peak hours, or subcategories of users other than those typical of workers and students.

Today, the situation has changed radically: thanks to the increasing availability of data, we have moved from episodic sampling to monitoring platforms, i.e., systems that collect data continuously. Big data, with its two defining characteristics (Volume and Velocity), allows mobility systems and their operation to be monitored in real time, opening the door to adaptive planning that was unimaginable just a few years ago. For example, it is possible to plan an offer that meets real demand and is corrected on the basis of ex-post evaluations, rather than quantitative considerations based on limited data.

But that’s not all: today, it is possible to start managing early warning systems that integrate analysis and monitoring components with forecasting models and decision support, command, and control systems, which were impossible to manage a few years ago with the tools available.

“Today we are inundated with an unprecedented amount of data: the data sources we can draw on to read mobility and develop decision support systems are unprecedented. We analysts are intoxicated by the amount of data available.”

But like all intoxicating things, it is not a risk-free process. We are facing what Prof. Cartenì defines as a veritable flood of data, and it is very important not to be overwhelmed by it, but to develop tools to manage it. How?

“On the one hand, we can continue to do the work of planners and analysts as we did in the past, but with different tools. Or we can ‘attack’ this deluge and structurally change the way we capture, organize, analyze, and validate this data to increase its value.”

Big data, big opportunities

As Cartenì explains, we can distinguish between two generations of data:

• ‘first generation’ big data: collected for other purposes (e.g., data from cell phones, public transport turnstiles, highway toll booths) and then reused for mobility analysis
• ‘second generation’ big data: unlike the former, this data is designed from the outset to support mobility decisions.

It is the latter that we need to invest in, with the support of artificial intelligence, which allows us to clean, validate, and analyze complex patterns, to the point of developing dynamic forecasts.

Among the emerging tools enabled by big data, digital twins stand out, i.e., dynamic digital replicas of real systems. By continuously “feeding” on data from the physical world, they are able to update continuously in real time, allowing for much more effective ex-ante assessments than in the past, or even simulating scenarios, anticipating critical issues, and adaptively managing the evolution of the system, for example by activating early warning systems in the event of accidents or seasonal and exceptional phenomena. This is a leap forward compared to the static simulation models of the past, which only capture a single moment in time:

“Even if they reproduce it in the best possible way, traditional models refer to a moment in the past that is no longer representative. With digital twins, powered by AI and big data, it is possible to develop much more advanced decision support systems.”

Many players, almost always private, are working hard to develop digital twins: we are in the midst of this revolution.

But can we trust this data?

This revolution also opens up crucial challenges that undermine the potential of the new paradigm: the need to validate data quality, address the issue of platform management costs, “digital deserts” that leave entire segments of the population behind, and the risk of creating inaccessible information silos.

Data quality is perhaps the most sensitive issue:

“The speed and quantity of data is such that it is often impossible to validate its quality accurately: this can often lead to clearly erroneous interpretations of the phenomenon. Too much data too fast must be accurately validated by appropriate technical expertise, and above all, predictive algorithms must be governed by transportation system analysts.”

The risk of creating digital silos is also of great importance. The sovereignty of this type of data, in the absence of good governance, can limit access to it, creating so-called silos: places where data is collected, cleaned, and validated in an excellent manner but where it remains accessible only to a limited number of people.

Then there is the issue of costs: processing, cleaning, and storing this enormous amount of data generates significant costs, both in terms of man-hours and electricity consumption, cloud space, and resources to run the machines.

Another problem is the so-called digital deserts, i.e., pockets of population and/or portions of territory that cannot be monitored by big data, which can lead to an incomplete and incorrect reading of the phenomenon observed. For example, vulnerable users (children or elderly people who do not drive cars equipped with black boxes or do not own a cell phone), or rural areas with no or few sensors to read the phenomenon, which may be more prone to so-called hallucinations caused by incorrect data interpretation.

Another challenge is the critical skills gap: those involved in transport analysis should not be expected to do data science, or vice versa, those who collect and manage data at the IT level should not also be expected to interpret transport data. According to Cartenì, the team and the multidisciplinary approach are the only real winning formula.

Consequently, continuous training remains strictly necessary. AI is evolving at a rapid pace: one of the main rules for managing AI is to “not take it for granted that tomorrow we will find artificial intelligence as we left it today.” The evolution is so fast that continuous training is an essential element in order to fully exploit its opportunities.

New frontiers: Explainability & Accountability

But reliability is not just about the technical side. One aspect that has always been overlooked in the field of planning data is the ability to make not only the potential of the tools, but also the results themselves, clear and understandable to all stakeholders.

“The large amount of data can greatly help in communicating performance, functions, and opportunities, and allows us to obtain validation from stakeholders, decision-makers, and people involved in the process in general, beyond just analysts.”

This aspect is very important because we must not repeat the mistake, typical in engineering, of developing tools and results that are important but incomprehensible to most people. This is the moment when data and methods can be conveyed to make third-party stakeholders understand the potential of the data, fully enhancing its potential.

The role of mobility observatories

Finally, the professor recalled the importance of collaborative ecosystems where all the skills and actors involved in this process can converge: the mobility observatories. These are physical places where different subjects and interests can make the most of the opportunities offered by the available data and collect, validate, and normalize this data, thanks to advanced skills in the fields of statistics and information technology for interpreting and cleaning the data. In this context, it is possible to consciously develop forecasting and decision support systems in the field of planning. A place where raw data is stored and where a data warehouse is set up for cleaning, analysis, and validation.

“This is why I imagine a strong, multidisciplinary team, with skills ranging from teleportation system engineer analysts to data scientists, but also lawyers who deal with copyright and privacy issues.”

Cartenì gives two examples in Italy:

• The Observatory on Passenger Mobility Trends, promoted by the MIT Technical Mission Structure and coordinated by Prof. Cartenì himself, aims to monitor the evolution of mobility demand in the country. It draws on data provided by mobility players (research centers, MIT directorates-general, etc.), which supply the Ministry’s data lake with mobility demand data. The data warehouse, which is still being refined, cleans, validates, and identifies anomalous data, and then provides forecasts and representations published in a quarterly report.
• The SUNRISE Observatory – part of the National Center for Sustainable Mobility (MOST) and promoted by universities and major mobility players such as Vodafone, Generali, and GO-Mobility – is more recent and aims to create a collaborative ecosystem to integrate different data sources and skills, enriching itself with exclusive proprietary databases in order to conduct studies more specifically aimed at transport decarbonization.

In conclusion: big data and AI have enormous potential, but if they are not governed by solid expertise, they risk producing misleading results and succumbing to the challenges that threaten their potential. The challenge is not only technical: it is cultural and political, and concerns the ability to use data not as an end in itself, but as a tool for knowledge and shared decision-making. This requires a multidisciplinary approach and continuous training to govern tools that are evolving at a rapid pace.