Do large language models solve verbal analogies like children do - Verbal analogy research

Sep 14, 2026

Analogy-making is central to human cognition, yet adults and children often solve A:B::C:? verbal analogies using different strategies. This study tests whether large language models (LLMs) rely on associations, mirroring children’s tendency to answer by links such as “egg,” rather than relational mapping. Using Dutch analogies from an online adaptive learning system (14,006 children aged 7–12), six monolingual and multilingual LLMs match child-level performance, but associative controls reduce outcomes by 1–2 years, indicating association-driven solving.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

AccessFree to download

Abstract

Introduction

Theoretical Background

The Analogical Reasoning Process

Factors Affecting People’s Verbal Analogy Solving

Do large language models solve verbal analogies like children do? Claire E. Stevenson⋄ , Mathilde ter Veen⋄ , Rochelle Choenni†, Han L. J. van der Maas⋄ and Ekaterina Shutova† ⋄Psychological Methods, University of Amsterdam, the Netherlands †ILLC, University of Amsterdam, the Netherlands [c.e.stevenson@uva.nl](c.e.stevenson@uva.nl) arXiv :2310 .20384v1 [ cs .CL] 31 Oct 2023 Abstract Analogy-making lies at the heart of human cognition. Adults solve analogies such as Horse belongs to stable like chicken belongs to . . . ? by mapping relations (kept in) and answering chicken coop. In contrast, children often use association, e.g., answering egg. This paper investigates whether large language models (LLMs) solve verbal analogies in A:B::C:? form using associations, similar to what children do. We use verbal analogies extracted from an online adaptive learning environment, where 14,006 7-12 year-olds from the Netherlands solved 622 analogies in Dutch. The six tested Dutch monolingual and multilingual LLMs performed around the same level as children, with M-GPT performing worst, around the 7-year-old level, and XLM-V and GPT-3 the best, slightly above the 11-year-old level. However, when we control for associative processes this picture changes and each model’s performance level drops 1-2 years. Further experiments demonstrate that associative processes often underlie correctly solved analogies. We conclude that the LLMs we tested indeed tend to solve verbal analogies by association with C like children do. 1 Introduction Analogy-making, using what you know about one thing to infer knowledge about a new, somehow related instance, lies at the heart of human intelligence and creativity and forms the core of educational practice (Gentner, 1988 ; Hofstadter, 1997 ; Holyoak, 2012) . Given how important analogical reasoning is to learning and generalization, much research has focused on how this seemingly unique human ability emerges, develops, and can be improved (Goswami, 1991 ; Sternberg and Nigro, 1980 ; Stevenson and Hickendorff, 2018) as well as emulated in machines (Gentner and Forbus, 2011 ; Mitchell, 2021) . Recently, large language models (LLMs), such as GPT-3 (Brown et al., 2020), have demonstrated surprisingly good performance in verbal analogy solving (e.g., table is to legs as tree is to . . . ? chair, leaves, branches or roots?) (Lu et al., 2022 ; Webb et al., 2023) . The question then arises how LLMs solve these analogies. Is the process similar to adult humans using relational mapping? Or perhaps more similar to the associative processes children tend to use? Earlier work shows that language models largely rely on semantic similarity between analogy terms to solve analogies (Rogers et al., 2020 ; Ushio et al., 2021b), which would indicate an associative process. In this paper we investigate whether six LLMs use similar associative processes to solve a set of Dutch children’s verbal analogies. First, we examine how LLM performance compares to children and find that the best models, GPT-3 and XLM-V (Liang et al., 2023) perform around the same level as 11-year-olds. Second, we examine whether LLM performance is influenced by the same item characteristics that affect children’s analogy solving, where results showed that GPT-3’s performance follows a similar pattern to children’s, whereas XLM-V’s performance is more stable across item characteristics. Third, through a series of prompting experiments we show that these LLMs appear to use associative solving processes, similar to children, where performance drops 1-2 years in age level when we control for associative processes. This paper contributes to the study of analogical reasoning in LLMs in three ways: (1) it is the first to directly compare LLM analogy solving performance to that of children; (2) we use experiments to tap into the process of analogy-making and compare this to human processes; and (3) we use Dutch rather than English language ite

FAQs

What strategy do children typically use to solve verbal analogies in A:B::C:? problems?

Children often rely on associations between terms rather than explicit relational mapping from the A:B base to the C target.

How do the tested LLMs compare to children on Dutch verbal analogies?

The best models (GPT-3 and XLM-V) perform around the same level as 11-year-olds, while other models are closer to lower child age levels.

What happens to LLM performance when associative processes are controlled?

Performance drops by about 1–2 years in age level, suggesting that correct analogy solving is frequently driven by associative processes.

Sep 14, 2026

Uploaded by Quinn Holloway