(Too Many) Problems of Analogical Reasoning with Word Vectors - Paper

Sep 14, 2026

This paper investigates analogical reasoning using vector space models, focusing on whether linear vector offsets can reliably solve proportional word analogies such as man:woman :: king:? and related tasks. The work argues that “linguistic regularities” are insufficient as a model for vector-space relations and that the offset-based method 3CosAdd depends heavily on target vector similarity. Experiments on GloVe and Word2Vec compare 3CosAdd with alternative approaches and examine how relation type and vector proximity affect accuracy.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages14

Reading Time35 min

AccessFree to download

Abstract

Introduction

Background: “Relational Similarity” vs “Word Analogies”

The (Too Many) Problems of Analogical Reasoning with Word Vectors Anna Rogers Dept. of Computer Science University of Massachusetts Lowell Lowell, MA, USA [arogers@cs.uml.edu](arogers@cs.uml.edu) Aleksandr Drozd Global Scient. Inf. and Comput. Center Tokyo Institute of Technology Tokyo, Japan [alex@smg.is.titech.ac.jp](alex@smg.is.titech.ac.jp) Bofang Li School of Information Renmin University of China Beijing, China [libofang@ruc.edu.cn](libofang@ruc.edu.cn) Abstract This paper explores the possibilities of analogical reasoning with vector space models. Given two pairs of words with the same relation ([e.g.](e.g. man:woman ::)[ man:woman ::](e.g. man:woman ::) [king:queen](king:queen)), it was proposed that the offset between one pair of the corresponding word vectors can be used to identify the unknown member of the other pair (!g 􀀀 !n + !n = ?!n) . We argue against such “linguistic regularities”as a model for linguistic relations in vector space models and as a benchmark, and we show that the vector offset (as well as two other, better-performing methods) suffers from dependence on vector similarity. 1 Introduction This paper considers the phenomenon of “vectororiented reasoning” via linear vector offset in vector space models (VSMs) (Mikolov et al., 2013c,a) . Given two pairs of words with the same linguistic relation (woman:man :: king:queen), it has been proposed that the offset between one pair of word vectors can be used to identify the unknown member of a different pair of words via 􀀀􀀀! solving proportional analogy problems (king 􀀀 !n + !n = ?!n), as shown in Fig. 1. We will refer to this method as 3CosAdd. This approach attracted a lot of attention, both as the “poster child” of word embeddings, and for its potential practical utility. Given the vital role that analogical reasoning plays in human cognition for discovering new knowledge and understanding new concepts, automated analogical reasoning could become a game-changer in many ﬁelds, providing a universal mechanism for detecting linguistic relations (Turney, 2008) and word sense disambiguation (Federici et al., 1997) . It is already used in many downstream NLP tasks, such as splitting compounds (Daiber et al., 2015), semantic search (Cohen et al., 2015), cross-language relational search (Duc et al., 2012), to name a few. Figure 1: Linguistic relations modeled by linear vector offset (Mikolov et al., 2013c) The idea that linguistic relations are mirrored inneat geometrical relations (as shown in Fig. 1) is also intuitively appealing, and 3CosAdd has become a popular benchmark. Roughly, the current VSMs score between 40%(Lai et al., 2016) and 75%(Pennington et al., 2014) on the Google test set (Mikolov et al., 2013a) . However, in fact performance varies widely for different types of relations (Levy and Goldberg, 2014 ; Kper et al., 2015 ; Gladkova et al., 2016) . One way to explain the current limitations is to attribute them to the imperfections of the current models and/or corpora with which they are built: with this view, in a perfect VSM, any linguistic relation should be recoverable via vector offset. The alternative to be explored in this paper is that perhaps natural language semantics is more complex than suggested by Fig. 1, and there maybe both theoretical and mathematical issues with analogical reasoning with word vectors and its 3CosAdd implementation. We present a series of experiments with two popular VSMs (GloVe and Word2Vec) to show that the accuracy of 3CosAdd depends on the proximity of the target vector to its source (i.e. 135 Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017) , pages 135–148, Vancouver, Canada, August 3-4, 2017 . 􀀍c2017 Association for Computational Linguistics !n should be 􀀀􀀀! quite similar to king) . Since not all linguistic relations can be expected to result in high word vector proximity, the method is limited to those that happen

FAQs

What is the main goal of this paper on analogical reasoning with word vectors?

The paper evaluates whether linear vector offset methods can serve as a reliable mechanism for solving word analogies in vector space models.

Why does the paper criticize using “linguistic regularities” for modeling word-vector relations?

It argues that the regularities are not actually regular enough to explain or guarantee consistent analogy performance in vector space representations.

On what factors does the accuracy of the 3CosAdd method depend?

The accuracy depends on how close the target vector is to its source vector in the embedding space, and performance varies with relation types as well.

Sep 14, 2026

Uploaded by Riley West