Go directly to content

Kuva: Esa Kapila

Published November 12, 2019

Impact evaluation from the future 

Morning news on 9 July 2030:
“The effects of the mobility experiment exceeded expectations” 
Writer
CEO, Gaia Consulting Oy

The good results yielded by the autonomous vehicle experiment launched in 2025 have now been confirmed. The CO2 emission reductions in the participating cities were significant. Traffic-related emissions decreased by as much as 50%. City inhabitants’ travel times have become significantly shorter and their satisfaction with urban mobility has increased. As a result of the experiment, mobility behaviour has also changed.

“It was especially delightful to see that elderly people, a group that was originally thought to pose a challenge, changed their mobility behaviour proportionally most and their satisfaction has increased,” says Joy Suresonthe project director responsible for evaluating  the impactof the experiment. 

In 2022, 15 cities decided to join the experimental project. These cities vary a great deal in terms of size, and they are located across Finland. Nevertheless, they share a common goal: to quickly adopt new climate-efficient mobility solutions. Their incentive was to further accelerate the achievement of the carbonneutrality targets set at the turn of the 2020s. In addition to the cities, funding for the experiments came from Sitra, several ministries and companies. 

From the onset, the project invested significantly in impact evaluation. “We wanted to get value for our money and to find out about the effects during the experiment so that we could quickly change the direction the experiment was going in different cities,” sayLiza Preciseness, the CEO of the company responsible for developing a smart application for nature lovers exploring rural areas that was adopted in six of the cities. Impact evaluation was based on mass data that was collected about the city inhabitants’ mobility, the functionality of technology, and related thoughts and feelings. 

The interactive “What’s the feeling?” application became a hit and is now also used abroad. The basis for the analysis of effects that resulted from the experiment and not from other factors was a control group of cities that were involved in the experiment from the beginning but did not adopt the new solutions introduced. 

“We did not mind being in the control group. In 2022, we had yet to set carbonneutrality targets, but participation in this experiment provided us with an enormous amount of useful information about mobility in our city and solutions that have proven functional elsewhere. We are adopting these solutions now,” sayCity Mayor Al Ert. 

Information about actual effects was available right from the start, almost in real time. An essential aspect was the use of artificial intelligence solutions in the quantitative and qualitative data analysis. An active international community of volunteers also contributed to the experiment by suggesting research questions and producing analysis data along the way. 

“At first, we were a bit sceptical: will this work and can text mining really replace evaluation professionals? says Project Director Joy Sureson. “Fortunately, we found brilliant analytics partners in Finland and abroad to complement our own evaluation expertise. Tools for analysing data and verifying effects already existed but we just did not know how to fully utilise them. In the end, we got more validated data with less effort and are using this data in a more versatile manner than in more traditional evaluations. I wish this had been possible in the 2010s when the culture of experimentation was becoming established in Finland. 

 

BLOG SERIES

THE FUTURE OF EVALUATION

This blog post is part of a blog series that tries to predict the future of evaluation. What is the value of evaluation in the society of the future? How should evaluation be developed to ensure that it can tackle the challenges posed by a changing world? The blog posts review evaluation tasks and roles as well as new, diverse ways to use new methods, such as data.

What's this about?