Elena Colli (GO-Mobility)

Data does not speak for itself

We are the ones who make them speak and decide what to say. The case of MIT data

The case of the MIT Senseable City Lab study, which was widely reported by the media, which for days repeated the slogan: “30 km/h speed limit = more pollution”, gives pause for thought. As is increasingly the case, we are faced with a simplification and distortion of scientific results to suit the needs and communication styles of contemporary media, which would suggest that a 30 km/h speed limit leads to an increase in climate-changing gas and particulate emissions. But what does this story really teach us (apart from the fact that no, the 30 km/h limit does not cause any significant increase in pollution)?

What are we talking about?

On July 8, starting with an article in Rai News, headlines and proclamations began circulating online, on social media, and in other national and popular newspapers, proclaiming that “30 km/h causes more pollution.” The articles refer to a study by the renowned MIT Senseable City Lab on the effects that reducing speed limits from 50 km/h to 30 km/h could have on urban traffic in Milan.

The case itself arises not so much from the results of the study but from how the media’s dissemination of these results was partial and misleading. The much-touted conclusion (30 limit = more pollution) was never even mentioned during the presentation of the study at the Urban Mobility Council Forum by Prof. Carlo Ratti himself (who, on the contrary, highlights the benefits of the intervention). However, in the days that followed, it led to the need for further press releases and statements from Senseable City Lab representatives to clarify what had been inappropriately and hastily reported by the media: “In the twenty years that I have been directing the MIT Senseable City Lab, nothing like this has ever happened” (Carlo Ratti).

Misunderstandings, allusions, omissions

The article in Rai News, which was then picked up by newspapers such as Corriere della Sera, TGCom24, Il Giorno, and many others, states that “If the 30 km/h speed limit were applied throughout the entire municipality of Milan, CO2 emissions would increase by 1.5%, while PM emissions, which are particularly harmful to human health, would increase by 2.7%.”

There are three issues here:

• The hyperbole built on an increase that is actually modest. As Carlo Ratti suggests in subsequent statements, the true interpretation of that 1.5% is that the increases resulting from the application of this measure are essentially negligible; just as it is biased to headline “travel times will increase” when, in the most drastic scenario, the expected increase is less than a minute and a half.
• The non-random selection of the data to be communicated: the data that the press chose for public communication is that relating to the most extreme scenario among the 12 simulated. The increases of 1.5% and 2.7% refer to the application of the 30 km/h limit to non-primary roads throughout the entire municipal area, which is not comparable to the measures adopted in Italy and around the world (the Città 30 in Bologna, for example, affects about 70% of roads). It is therefore likely that in an average scenario, this impact will be even smaller or non-existent.
• The articles omit other data accompanying the study and presented during the conference, which highlight the benefits of the measure, especially in terms of urban road safety, which is the primary objective of Città 30: as documented in the literature, in fact, measurements taken before and after the measure show, on average, a sharp reduction in accidents (-23%), mortality (-37%), injuries (-38%), and noise pollution of 2.5dB.

As Umberto Fugiglando of MIT reiterates, “We ourselves have demonstrated that the introduction of Zone 30 in Paris has had a positive impact on pedestrian and economic-social activity on the streets involved.”

Screen from the Slow Zones project

As the MIT expert states, reiterating what was already expressed in the presentation of the research to the Urban Mobility Council, “considering all the primary and secondary effects, it is expected that the overall impact of reducing speed limits could lead to a net reduction in gas emissions,” in line with the results observed in other European cities and contradicting the misleading reports in the Italian press.

Screenshot of the article published by Rai News

The (professional) distortion of data

Nowadays, the media must adapt to the new mechanisms that dictate competition in the field of journalism: textual adaptations for SEO, misleading headlines to encourage clickbait, simplification and brevity to adapt to lower attention spans.

What is remarkable, if not serious, is that it is not only the media (newspapers, administrators, politicians, professionals) but also science itself that must adapt its methods of dissemination to these standards. The production of scientific articles and reports must adapt to the demands of brevity, bulleted lists, and summaries of summaries, with the near certainty that in the overproduction of content that characterizes this era, few have the time and ability to read thoroughly and even fewer to verify the details of studies, especially technical and methodological details such as the data sources used, the characteristics of the algorithm applied, and the test environment used.

The consequence is that producing a press release by technical figures is now more complex and requires great care to protect against the wide possibility that the data communicated will be misrepresented, taken out of context, or simplified into misleading or partial headlines. Once again, the data is being misrepresented, with the awareness that the rule “The data says so” prevails undisturbed (all the more so if it is MIT data).

But data alone says nothing: it is we who convey specific messages by selecting some numbers over others, adding subjective interpretations not supported by the data, using specific words or tones, omitting other data or methodological details.

On the other hand, Prof. Ratti also admits: “Perhaps we could have been more careful too. […] That slide ended up in the press kit for some reason, and journalists did the rest… I repeat: we should have been even clearer.”

Lessons learned: is partial information the new fake news?

What has this story taught us? First of all, that data alone does not speak. Or at least it does not speak our language. Data is numbers, and it is up to us to apply the right rules of interpretation and communication.

For those who produce technical and scientific communication, it is important first and foremost to verify the accuracy of conclusions by cross-referencing sources, monitoring trends, and combining different points of view, data sources, and methodologies. At the crucial moment of summarizing for the public, it is essential to use clear language that is not open to interpretation and, above all, to be cautious when dealing with assumptions and hypotheses rather than verified correlations and causations.

As an article in The Post headlines, “Misleading news does much more damage than fake news,” because it contributes to misinformation on social media more than “fake news,” especially when it comes from reliable sources (as in this case, the Senseable City Lab). According to the research cited in the aforementioned article (also conducted at MIT), it was found that during the pandemic, content related to vaccines that was factually correct but suggestive and misleading circulated much more than false content.

Our experience (and holiday homework)

We too, as part of the Data Mobility outreach project, have learned that we need to be very careful when communicating our data. In our experience, we have noticed that despite the care taken in all the content we produce, our press releases will inevitably be turned into clickbait headlines to stir up angry crowds on social media and create engagement or, for example, draw hasty conclusions to discredit certain notoriously unpopular measures (paid parking, restricted traffic zones, 30 km/h zones).

Article by Quattroruote that draws on data from Data Mobility 2024, and engagement on social media

When drafting reports and press releases, we have therefore learned that it is necessary to measure our words with increasing care, avoid misunderstandings, not quote statements we are not sure about, and not use ambiguous terms. While on the one hand it is more prudent to simply make raw graphs available for consultation without any interpretation whatsoever, on the other hand this reduces comparison, debate, and the accessibility of the results produced, which is one of the guiding principles of the project.

Sometimes we simply accept the risk, knowing that no matter how careful we are, the data will still be interpreted and distorted by each publication according to its own intentions and preferences. And so a 12% drop in car travel reported in Data Mobility 2023 will become “big cities abandoning the car,” a 7% drop in average speed in metropolitan cities contained in Data Mobility 2024 will become “Italians stuck in traffic,” and so on.

Hyperbole: headline relating to Data Mobility 2023 data (12% drop in car travel)

Other hyperbole and distortions: some of the headlines for the first results of Data Mobility 2024

This therefore invites us, not only those who produce information but also those who read it, to not overlook the effect of true but partial and misleading news, which can lead to very serious communication errors. Perhaps during these holidays, we can try to pay more attention to the citation of sources, to delve deeper into the original studies referenced in articles, to find small errors, hyperbole, and omissions. Because no matter how valid and impeccable the study (and therefore the data) may be, extreme attention must be paid to fact-checking, especially in popular articles, because the mission of science is truly accomplished when it reaches a large audience and policy makers (i.e., when it is well communicated and understood). The effort, commitment, and impeccability of each study can fade away in that final crucial step where everything is at stake, namely when the data is made to speak for itself.

[1] Abstract of the research available here

[2] From a post by Andrea Colombo, strategic expert in sustainable mobility, public space, and the environment at the Fondazione Innovazione Urbana in Bologna.

[3] G. Yannis & E. Michelaraki (2024) “Review of City-Wide 30 km/h Speed Limit Benefits in Europe,” Sustainability, 16(11), 4382. https://doi.org/10.3390/su16114382

[4] Slow Zones study by Senseable City Lab, interactive visualization available here