3D models to explore the future of urban digitization

ScanVan, a four-year project to develop new technology to digitize cities, reached its culmination in March 2021. This was a tripartite Swiss collaboration between researchers at the EPFL Digital Humanities Laboratory, the HES-SO Valais/Wallis Institute of Systems Engineering, and the University of Zurich’s Center for Information Technology, Society, and Law, with funding from the SNSF’s Big Data National Research Program. Together the team carried the project from theoretical conception, through practical implementation, to exploration of future social and commercial applications.

ScanVan helps to advance an important new frontier in information and communication technology: the production of 3D urban models. Most existing scanning systems either use expansive laser-based systems or photogrammetric approaches based on standard photographic cameras which are not error-proof and therefore require several rounds of scanning. The ScanVan team developed an omni-directional spherical acquisition system associated with a dedicated algorithm that guaranties high scanning and photogrammetric computing efficiency. The system produces spherical images in regular increments while mounted on a moving vehicle (the ScanVan). In practice, the ScanVan need only pass once through a given street to acquire sufficient visual material for a three-dimensional reproduction. Nils Hamel explains the logic behind the camera:

“The goal is to take advantage of a device that is able to see the full sphere surrounding it to capture every aspect of the scene within a single shot. Such a strategy considerably alleviates the problem of image connectivity because captures that are made near each other guarantee a large overlap in any situation. The connectivity graph behind the image acquisition is therefore simplified, leading to much simpler acquisition campaign and bringing photogrammetry closer to a real-world and state-of-the-art technology for continuous 3D and 4D mapping.”

The ScanVan camera © DHLAB / EPFL

These advances make it possible to envision scanning an entire city at regular intervals, thus creating a 4D representation of the city (3D + time). The ScanVan team has tested this technology by producing a fully aligned model of the city of Sion using a prototype of the vehicle. The EPFL team focused on the initial theoretical and computational models. The HES-SO Valais/Wallis team led by Pierre-André Mudry shouldered responsibility for constructing the innovative spherical camera and equipped the vehicle with the relevant data processing hardware and software. On the development of the ScanVan technology, Mudry reflects:

“Bridging the gap between the theory and its real-world implementation required an integrative methodology where the applied engineering skills of our Institute could complement the techniques developed at EPFL. Within this project, we demonstrated in many instances how a successful synergy between fundamental and applied sciences can be beneficial, for instance with a mathematically defined mirror that could be built with real-world constraints and successfully integrated with its acquisition system, into a driving vehicle.”

Digitalization of the EPFL Place Cosandey © 2021 EPFL

In 2020, the University of Zurich joined the project to work on the potential privacy issues linked with the development of high-resolution 3D models. Florent Thouvenin and his team analyzed how to ensure legal compliance of the ScanVan acquisition techniques with Swiss laws that protect personally identifiable information. A privacy by design approach was used to build the processing pipeline and exploration interfaces. This includes automatic algorithmic processes to blur identifiable information concerning persons and vehicles as well as a further layer of human control to promote public transparency. Thouvenin describes how his team collaborated in the practical implementation of the ScanVan:

“Generating 3D urban models from spherical images is a very interesting technical approach. However, the collection of data in public space using cameras leads to the fact that random people are depicted in the images. Various legal provisions, in particular data protection laws, must therefore be complied with. This compliance can be greatly simplified by observing the privacy by design approach at an early development stage. Applying this approach, we analyzed the technology with regard to Swiss and EU data protection law and offered design considerations and recommended actions to overcome the identified legal issues in the course of the project.”

While the ScanVan is still at the prototype phase, it has future implications for the commercial deployment and the governance of services that depend on urban digitization, such as autonomous car programs: a link which may be explored further with the help of Mobility Lab Sion Valais. The ScanVan is a relatively compact and scalable system, and this opens possibilities for local governments and firms to claim greater data autonomy from the current heavyweights in urban digitization such as Google. This would also make it possible to manage decisions about individual privacy in a more localized and publicly accountable way. For Frédéric Kaplan, director of the EPFL Digital Humanities Laboratory, most cities are now at a crossroads:

“In the near future, there will be a great divide between cities that autonomously manage regularly updated digital twins and those still relying on ancient geographic information systems. The first ones will engage a development trajectory based on self-improving information systems that can be used for myriad new services. Thanks to their 4D model, these cities will be able to deploy augmented reality interfaces, self-optimizing logistics and additional autonomous vehicles that will produce, in turn, even more precise real-time representations. They will continuously connect their past with their future. The others will remain stuck with a static understanding of the urban space from the previous century.”

Read the profile article of Albane Descombes, a young member of EPFL’s Digital Humanities Laboratory (DHLAB), who helped with the final implementation of the ScanVan project.