An interpretable machine learning framework for measuring urban perceptions from panoramic street view images - Research insights

Aug 5, 2026

An interpretable machine learning framework extracts neighborhood-level urban perceptions from panoramic street view images by transforming visual scenes into six perception dimensions. The approach emphasizes interpretability through panoptic segmentation to capture human-recognizable visual elements, Elo-based quantification of crowdsourced image pairwise comparisons, and model explanations using feature importance and accumulated local effects. Using the MIT Place Pulse data, the framework operationalizes wealth, boredom, depression, beauty, safety, and liveliness, and demonstrates practical deployment in Inner London with validation against real-world crime rates.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages24

Reading Time1 h 0 min

AccessFree to download

Summary

Introduction

Framework Overview

Panoptic segmentation for visual elements

Crowdsourced pairwise comparisons via Elo rating

Interpretability with feature importance and local effects

Practical Deployment and Validation

Inner London visualization at OA level

Comparison with real-world crime rates

iScience ll OPEN ACCESS Article An interpretable machine learning framework for measuring urban perceptions from panoramic street view images Yunzhe Liu, Meixu Chen, Meihui Wang, Jing Huang, Fisher Thomas, Kazem Rahimi, Mohammad Mamouei [yunzhe.liu@ic.ac.uk](yunzhe.liu@ic.ac.uk) (Y. L.) [maychen@liverpool.ac.uk](maychen@liverpool.ac.uk)[ ](maychen@liverpool.ac.uk)(M.C.) Highlights An interpretable framework to extract urban perceptions from panoramic SVIs Using panoptic segmentation to identify human-recognizable visual elements Crowdsourced SVI pairwise comparisons are quantiﬁed via the Elo rating system Using feature importance and accumulated local effects to improve interpretability Liu et al. , iScience 26, 106132 March 17, 2023 ª 2023 The Authors. [https://doi.org/10.1016/](https://doi.org/10.1016/)[ ](https://doi.org/10.1016/)[j.isci.2023.106132](j.isci.2023.106132) iScience ll OPEN ACCESS Article An interpretable machine learning framework for measuring urban perceptions from panoramic street view images Yunzhe Liu,1,5,6,* Meixu Chen,2,* Meihui Wang,3 Jing Huang, 1,4 Fisher Thomas,1 Kazem Rahimi, 1 and Mohammad Mamouei1 SUMMARY The proliferation of street view images (SVIs) and the constant advancements in deep learning techniques have enabled urban analysts to extract and evaluate urban perceptions from large-scale urban streetscapes. However, many existing analytical frameworks have been found to lack interpretability due to their endto-end structure and ‘‘black-box’’ nature, thereby limiting their value as a planning support tool. In this context, we propose a ﬁve-step machine learning framework for extracting neighborhood-level urban perceptions from panoramic SVIs, speciﬁcally emphasizing feature and result interpretability. By utilizing the MIT Place Pulse data, the developed framework can systematically extract six dimensions of urban perceptions from the given panoramas, including perceptions of wealth, boredom, depression, beauty, safety, and liveliness. The practical utility of this framework is demonstrated through its deployment in Inner London, where it was used to visualize urban perceptions at the Output Area (OA) level and to verify against real-world crime rate. INTRODUCTION As the environment where most human activities occur, cities can be characterized as an interchange hub for capital, logistics, labor, and information, shaping and inﬂuencing the lives of their residents from multiple perspectives.1 ,2 Numerous studies have shown that the physical appearance of cities plays a pivotal role in residents’ psychological feelings toward the urban built environment, consequently inﬂuencing their behaviors.3–10 Such human-perceived experience of the urban environment is also known as urban perception,11 , 12 together with urban identity, formulating important concepts in urbanism and urban design.13–15 Given the spatial heterogeneity and complexity of the urban built environment in terms of overall environmental quality and physical appearance, urban perceptions vary across different city areas. Therefore, research on urban perception offers a promising perspective that assists urban analysts in gaining insights into urban morphology and metabolism and the way residents perceive their living neighborhood areas, facilitating evidence-based policymaking in urban planning and regeneration. Gathering information about visual surroundings from the urban built environment and evaluating their inﬂuences on human perceptions have a long research history.7 , 16–20 However, most previous studies relied on traditional data collection approaches, such as ﬁeld surveys, questionnaires, and interviews, which are costly, error-prone, and time-consuming. As such, these studies encountered challenges in knowledge discovery and generalization, particularly for large-scale urban regions, due to the lack of the ﬁne-granularity and high throughput of the investigation methods.2

FAQs

What problem does the proposed framework address?

Existing SVI-based analysis often behaves like a black box, limiting planning value. The framework aims to measure urban perceptions while remaining interpretable.

How are human-recognizable visual elements extracted?

The framework uses panoptic segmentation to identify visual elements that are recognizable to humans within panoramic street view images.

How are crowdsourced perceptions quantified and interpreted?

Crowdsourced pairwise comparisons are converted into quantitative scores using the Elo rating system. Interpretability is further improved with feature importance and accumulated local effects.

Aug 5, 2026

Upload by Emma Wilson