Defining and Evaluating Visual Language Models’ Basic Spatial Abilities - A Perspective from Psychometrics

Sep 17, 2026

The Theory of Multiple Intelligences frames cognitive capability as hierarchical, motivating the design of spatial benchmarks for Visual Language Models (VLMs). A psychometric framework is proposed to define and measure five Basic Spatial Abilities: Spatial Perception, Spatial Relation, Spatial Orientation, Mental Rotation, and Spatial Visualization. Benchmarking 13 mainstream VLMs using nine validated experiments shows sizable gaps with humans and reveals model hierarchy patterns, architecture-limited interventions, and CoT/few-shot constraints, while ToT most effectively improves performance. The work identifies barriers such as weak geometry encoding and missing dynamic simulation, and supplies a unified evaluation benchmark and roadmap for embodied AI.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages20

Reading Time50 min

AccessFree to download

Abstract

1 Introduction

2 Related works

2.1 Psychometric Studies on Human’s BSAs

2.2 Evaluation of VLMs’ BSAs

Defining and Evaluating Visual Language Models’ Basic Spatial Abilities: A Perspective from Psychometrics Wenrui Xu1†, Dalin Lyu1†, Weihang Wang1†, Jie Feng2 , Chen Gao3* , Yong Li2,3* 1 School of Architecture, Tsinghua University 2Department of Electronic Engineering, Tsinghua University 3BNRist, Tsinghua University {xwr23,lvdl24,[wang-wh22}@mails.tsinghua.edu.cn](wang-wh22}@mails.tsinghua.edu.cn)[ ](wang-wh22}@mails.tsinghua.edu.cn){fengjie,chgao96, [liyong07}@tsinghua.edu.cn](liyong07}@tsinghua.edu.cn) Abstract The Theory of Multiple Intelligences underscores the hierarchical nature of cognitive capabilities. To advance Spatial Artificial Intelligence, we pioneer a psychometric framework defining five Basic Spatial Abilities (BSAs) in Visual Language Models (VLMs): Spatial Perception, Spatial Relation, Spatial Orientation, Mental Rotation, and Spatial Visualization. Benchmarking 13 mainstream VLMs through nine validated psychometric experiments reveals significant gaps versus humans, with three key findings: 1) VLMs mirror human hierarchies (strongest in 2D orientation, weakest in 3D rotation) with independent BSAs; 2) Many smaller models surpass larger counterparts, with Qwen leading and InternVL2 lagging; 3) Interventions like CoT and fewshot training show limits from architectural constraints, while ToT demonstrates the most effective enhancement. Identified barriers include weak geometry encoding and missing dynamic simulation. By linking Psychometrics to VLMs, we provide a comprehensive BSA evaluation benchmark, a methodological perspective for embodied AI development, and a cognitive science-informed roadmap for achieving human-like spatial intelligence. 1 Introduction Visual Language Models (VLMs) excel in a wide range of specific tasks (Hong et al., 2023) . However, achieving human-like spatial intelligence for embodied AI applications such as visual navigation and embodied Q&A remains a challenge (Duranteet al., 2024 ; Duan et al., 2022) . Recent studies reveal that even advanced models like GPT-4o fail basic 2D spatial reasoning tasks that humans solve effortlessly (Tang et al., 2024) . Theory of Multiple Intelligences (Bornstein and Gardner, 1986), which is widely accepted across † These authors contributed equally to this work. * Corresponding authors. Figure 1: BSA measuring framework for VLMs 1. disciplines, posits that human intelligence is hierarchical, with general intelligence (g) supported by subordinate intelligences including spatial intelligence. Spatial intelligence is also structured hierarchically with several basic spatial abilities (BSAs), which are crucial for advanced intelligence and provide a comprehensive framework for evaluation. However, existing studies assessing the spatial abilities of VLMs lack a solid theoretical foundation and focus on isolated abilities without a comprehensive framework, making it challenging to compare results across different studies or uncover potential interconnections between abilities. Moreover, most research lacks human baselines, leaving the gap between VLMs and human unexplored. To address these gaps, we propose a psychometric framework using standardized human experiments to systematically evaluate VLMs’ BSAs, thereby establishing benchmarks and pathways for enhancing spatial intelligence in AI systems. 2 Related works 2.1 Psychometric Studies on Human’s BSAs Human spatial intelligence, defined as the ability to mentally model and manipulate spatial environ- 1Image source: Vandenberg and Kuse (1978) . 11571 Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 11571–11590 July 27-August 1, 2025 ©2025 Association for Computational Linguistics ments (Bornstein and Gardner, 1986), has been extensively studied since Spearman (1904) first proposed it. Research then diverges into two streams (Porat and Ceobanu, 2023): Psychometric Classification.

FAQs

What five Basic Spatial Abilities (BSAs) does the paper define for VLMs?

The framework defines Spatial Perception, Spatial Relation, Spatial Orientation, Mental Rotation, and Spatial Visualization as measurable BSAs in Visual Language Models.

How was the evaluation conducted across VLMs and what was compared against?

The paper benchmarks 13 mainstream VLMs using nine validated psychometric experiments, and analyzes their performance gaps and hierarchy relationships relative to human baselines.

Which interventions are reported as limited, and which method shows the strongest enhancement?

The paper reports that interventions such as CoT and few-shot training have limits tied to architectural constraints, while ToT demonstrates the most effective enhancement.

Sep 17, 2026

Uploaded by Arica Lee