[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84742-en":3,"doc-seo-84742-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84742,137441390410,"Hazel","https://ap-avatar.wpscdn.com/avatar/2000252f4ab5702993?_k=1776741390130283984",8,"Research & Report","Language Models Represent and Transform Concepts with Shared Geometry","How neural networks represent concepts is a central machine learning question, and existing approaches often assume stationary geometric representations that persist across contexts. This work reframes concept embeddings as point-cloud manifolds and treats contextual change as vector-field transformations. Experiments across six large language model families show that context displaces each concept differently, yet the displacement organization is shared across models. The shared geometry enables cross-model prediction above chance and outperforms chance baselines, supported by structured variance tied to lexical properties.","Language Models Represent and Transform Concepts with Shared Geometry  \nZhimin Hu 1 Lanhao Niu 2 Sashank Varma 1  \narXiv :2607 .04525v 1 [ cs .CL] 5 Jul 2026  \nAbstract  \nHow concepts are represented in neural networks is a fundamental question in machine learning.  \nThe dominant view treats concept representations as stationary geometric objects. Yet concepts appear in context, and context transforms them.  \nDrawing from neural population geometry, we formalize concept representations as point-cloud manifolds and contextual transformations as vector fields, and instantiate this framework in large language models. Across six model families of varying scales, we find that context moves each concept differently. The variance in these displacements is semantically organized, correlating with lexical concreteness and density. Importantly, both the concepts being transformed and this variance structure are shared across models: displacement structure transported from one model predicts held-out displacements in others significantly above chance. Together, these findings show that models share a common geometry not only in how concepts are represented, but more importantly in how context transforms them, a structure with richer organization than prior work has recognized.  \n1. Introduction  \nHow does a neural network represent concepts? The dominant view in machine learning treats concepts as stable geometric objects, such as vectors in activation space that persist across contexts (Park et al., 2024 ; Huh et al., 2024) . This answer has been productive. It grounds methods (Arditiet al., 2024) to steer model behavior via single vectors and motivates alignment research (Jha et al.) . It also fits the modern cognitive science view of concepts as static vectors (Piantadosi et al., 2024) . However, this is a stationary answer to a question that is fundamentally dynamic (Truman  \n1 Georgia Institute of Technology 2University of Edinburgh. Correspondence to: Zhimin Hu \u003C[zhu41@gatech.edu](zhu41@gatech.edu) >, Sashank Varma \u003C[varma@gatech.edu](varma@gatech.edu) >.  \nProceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026 . Copyright 2026 by the author(s) .  \n& Kutas, 2024) . A concept does not appear in isolation; it appears in diverse contexts, and contexts move it.  \nIn this paper, we examine this stationary view directly and find that it does not hold as prior work implies. Context does not perturb concepts around a stable vector. Rather, each concept moves in its own direction and by its own magnitude, and no single vector captures this variation (see Figure 1) . Yet these variances are not arbitrary. They are systematically structured, correlating with lexical concreteness and density. Despite this heterogeneity, the relational geometry is shared across models for both the concepts and the displacements that transform them: how concepts sit and move relative to one another is consistent even when their absolute positions and displacements are not.  \nWe verify the above findings by showing that a relational procedure transporting displacement structure from one model predicts held-out displacements in another significantly above baseline, and that distorting this relational structure degrades prediction. We further show that the sharing of conceptual representations is not a surface artifact-it cannot be reduced to co-occurrence statistics, and scrambling the context collapses the alignment. To summarize, what is consistent across models includes not only where concepts are, but the geometry of how contexts transform them, a structure that has not, to our knowledge, been systematically characterized before.  \nThese results matter at two scales. At the level of representation engineering, we provide a geometric account of singlevector steering as a useful approximation that nonetheless discards a structured residual that is semantically meaningful. More broadly, our results speak to the l","cbCairyux2irGy9c","https://ap.wps.com/l/cbCairyux2irGy9c","pdf",1456123,1,15,"English","en",105,"# Abstract\n# Introduction\n# Preliminaries\n## Concepts as Point-Cloud Manifolds","[{\"question\":\"What is the main problem this paper addresses about concept representation in neural networks?\",\"answer\":\"It questions the dominant assumption that concept representations are stationary geometric objects that remain stable across contexts. The paper argues that context transforms concepts dynamically rather than merely perturbing a single fixed vector.\"},{\"question\":\"How does the paper model concept representations and context effects?\",\"answer\":\"It models concept representations as point-cloud manifolds in neural representation space, and it formalizes contextual transformations as vector fields that map how concepts move under different contexts.\"},{\"question\":\"What evidence shows that geometry is shared across different language models?\",\"answer\":\"Across six model families, the paper finds that the structure of concept displacement is semantically organized and can be transported from one model to predict held-out displacements in other models significantly above chance. Scrambling context collapses the alignment, indicating the effect is not reducible to co-occurrence statistics.\"}]",1784197995,38,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"language-models-represent-and-transform-concepts-with-shared-geometry","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/language-models-represent-and-transform-concepts-with-shared-geometry/84742/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-20","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the main problem this paper addresses about concept representation in neural networks?","Question",{"text":75,"@type":76},"It questions the dominant assumption that concept representations are stationary geometric objects that remain stable across contexts. The paper argues that context transforms concepts dynamically rather than merely perturbing a single fixed vector.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the paper model concept representations and context effects?",{"text":80,"@type":76},"It models concept representations as point-cloud manifolds in neural representation space, and it formalizes contextual transformations as vector fields that map how concepts move under different contexts.",{"name":82,"@type":73,"acceptedAnswer":83},"What evidence shows that geometry is shared across different language models?",{"text":84,"@type":76},"Across six model families, the paper finds that the structure of concept displacement is semantically organized and can be transported from one model to predict held-out displacements in other models significantly above chance. Scrambling context collapses the alignment, indicating the effect is not reducible to co-occurrence statistics.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]