Testing machine learning systems in real estate - Applied software system testing for ML models

Aug 3, 2026

Uncertainty about how machine learning (ML) models produce estimates slows down the use of ML-enabled solutions in real estate. The central challenge is model transparency: how practitioners can ensure that ML systems deliver intended outputs and do not violate legal requirements. The article argues for a dedicated software testing framework for applied ML. It then shows how system-testing procedures can confirm that ML image classifiers used in automated valuation models perform as specified.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages25

Reading Time1 h 3 min

AccessFree to download

Introduction

Accountability gap and risks of black-box ML

Software testing framework for applied ML

System testing for ML image classifiers in AVMs

Local explanation and scaling challenges

DOI: 10.1111/1540-6229.12416 ORIGINAL ARTICLE Testing machine learning systems in real estate Wayne Xinwei Wan1 Thies Lindenthal2 1 Department of Banking and Finance, Monash University, Clayton, Victoria, Australia 2 Department of Land Economy, The University of Cambridge, Cambridge, Cambridgeshire, UK Correspondence Wayne Xinwei Wan, Department of Banking and Finance, Monash University, W1025 Menzies Bldg, Wellington Rd, Clayton, VIC 3800, Australia. [Email: wayne.wan@monash.edu](Email: wayne.wan@monash.edu) Abstract Uncertainty about the inner workings of machine learning (ML) models holds back the application of MLenabled systems in real estate markets. How do ML models arrive at their estimates? Given the lack of model transparency, how can practitioners guarantee that ML systems do not run afoul of the law? This article first advocates a dedicated software testing framework for applied ML systems, as commonly found in computer science. Second, it demonstrates how system testing can verify that applied ML models indeed perform as intended. Two system-testing procedures developed for ML image classifiers used in automated valuation models (AVMs) illustrate the approach. KEYWORDS accountability gap, computer vision, explainable machine learning, real estate, system testing 1 INTRODUCTION The black-box nature of machine learning (ML) techniques poses a risk to businesses developing ML-enabled systems: How can they verify that a system is performing in thewayit is meant to perform? Can they ensure that its outcomes are not spurious, biased, or unlawful? Still, ML-enabled systems continue to reshape commerce, personal interactions, entertainment, medicine, government services, state supervision—and research (Simester et al., 2020) . In real estate and urban studies, a rapidly expanding literature explores the potential of ML algorithms, introducing novel measurements of the physical environments or using these estimates to improve the traditional real estate valuation and urban planning processes (e.g., Glaeser et al., 2018; Johnson et al., 2020; Karimi et al., 2019; Lindenthal & Johnson, 2021; Liu et al., 2017; Rossetti et al., 2019; Schmidt & © 2022 American Real Estate and Urban Economics Association. Lindenthal, 2020; Shen & Ross, 2021). These studies, again and again, demonstrate the undisputed power of ML systems as prediction machines. Still, it remains difficult for researchers to establish causality or for end users to understand the internal workings of any models. An “accountability gap”(Adadi & Berrada, 2018) remains: How do the models arrive at their prediction results? Can we trust them not to bend rules or to cut corners? This accountability gap holds back the deployment of ML-enabled systems in real-life situations (Ibrahim et al., 2020). If engineers cannot observe the inner workings ofthe models, how can they guarantee reliable outcomes? Furthermore, the accountability gap also leads to obvious dangers: Flaws in prediction machines are not easily discernible by classic cross-validation approaches (Ribeiro et al., 2016) . More importantly, the opacity of the ML models also gives rise to the legal and ethical concerns for its real-life applications (Mullainathan & Obermeyer, 2017). For instance, anecdotal evidence reports that some ML engines for recruitment have exerted biases again the female applicants (Dastin, 2018) . Traditional ML model validation metrics such as the magnitude of prediction errors or 􀀂1-scores can evaluate the models’predictive performance, but they provide limited insights for addressing the accountability gap. Training ML models is a software development process at heart. We therefore suggest that ML system developers follow best practices and industry standards in software testing. Particularly, the system-testing stage of software test regimes is essential: It verifies whether an integrated system performs the exact function as required in the init

FAQs

为什么房地产场景中应用ML模型会遇到“问责差距”？

ML模型的黑箱特性使得难以追踪预测结果的形成过程，从而难以判断其是否遵守规则或法律要求。问责差距因此阻碍了ML在真实情境中的部署。

文章提出用什么方法提升应用ML系统的可信度？

提出建立面向应用ML的专门软件测试框架，并强调系统测试阶段的重要性。系统测试用于验证集成系统是否按最初设计的要求执行，从而提升过程透明度与结果可信性。

文章如何说明系统测试能验证ML模型是否按预期工作？

通过两类用于自动估值模型（AVM）的ML图像分类器用例，展示系统测试程序如何验证模型确实实现了预期功能。

为什么仅依赖模型可解释性方法不足以进行大规模验证？

许多局部解释工具需要对每个观测进行人工检查，通常以定性方式呈现。由于缺乏可扩展性，这类工具难以在大样本场景下完成模型验证。

Aug 3, 2026

Upload by Lucas Martin