[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85869-en":3,"doc-seo-85869-105":29,"detail-sidebar-cat-0-en-105":83},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85869,1099514068035,"Ezra","https://ap-avatar.wpscdn.com/davatar_276721f389ce27ea32af1340a28f341c",8,"Research & Report","Language Re-generation An Investigation into Information Locality Effects on Reconstruction","Information locality—the tendency for syntactically related words to occur near each other—shapes both human language processing and how language models learn. The study tests whether LLMs can reconstruct natural language from impossible or perturbed inputs by fine-tuning GPT-2 on three perturbation types. Recovered dependencies shorten, reflecting locality bias, while recovery difficulty increases as locality disruption grows. Structural recovery diverges from surface recovery, fluency can decouple under global shuffling, and sentence length modulates success. Results align with learnability difficulty across perturbations.","Language Re-generation:  \nAn investigation into information locality effects on reconstruction  \nAmirhossein Mohammadi, Laurence E. Frank, Albert Gatt, Robert A. Bagheri  \nUtrecht University,  \n[a.mohammadi@uu.nl](a.mohammadi@uu.nl)  \narXiv :2607 . 10268v 1 [ cs .CL] 11 Jul 2026  \nAbstract  \nInformation locality, the tendency for syntactically related words to appear close together, shapes both human language processing and language model learning. While prior work has examined whether language models can acquire impossible languages, it remains unclear whether they can recover natural language from such input and what this reveals about their inductive biases. We address this by complementing learnability-based approaches with areconstruction framework—fine-tuning GPT- 2 models pre-trained on impossible languages to reconstruct natural English from three perturbation types. Our findings show that the recovered structures exhibit shorter dependency lengths than the original text, mirroring the locality preference observed in unconstrained language model generation and providing a quantitative signature of an architectural bias that learnability experiments alone do not reveal. Recovery difficulty increases with the degree of locality disruption. Structural recovery (dependency Triple F 1 ) dissociates from surface recovery (Exact Match), while fluency dissociates from faithful reconstruction under global shuffling. Sentence length further modulates performance: longer sentences facilitate recovery when local structure is preserved but lead to complete collapse under global shuffling. Finally, recovery difficulty tracks learnability difficulty across perturbation types, suggesting that information locality is the shared constraint governing both.  \n§ Repository  \n1 Introduction  \nThere are many conceivable linguistic systems, but not all langauges are equally learnable given human cognitive constraints (Chomsky, 1957, 1986 ; Jackendoff, 2002 ; Pinker, 1994) . Natural languages share structural properties precisely because those properties keep them within cognitive reach (Hawkins, 1994 ; Futrell et al., 2020) . One such  \nPerturbed sentence  \nOriginal sentence  \nFigure 1: Translation task overview. Three separate LLMs are fine-tuned on specific perturbation types (top to bottom PARTIALREVERSE, LOCALSHUFFLE, FULLSHUFFLE) . Each model translates its corresponding perturbed language input (left) into the original language output (right), recovering disrupted natural structure.  \nproperty is Information locality: the tendency for semantically and syntactically related elements to appear close together in linear order. This is rooted in memory limitations of the human brain (Gibson et al., 2000 ; Musso et al., 2003) . Dependency Locality Theory (DLT) formalizes this principle, predicting that processing difficulty increases asthe linear distance between dependent elements grows, reflecting the limits of working memory during incremental comprehension (Gibson, 1998) . Cross-linguistic research corroborates the information locality principle through dependency length minimization (DLM), demonstrating that natural languages tend to converge on word order strategies that keep grammatically related elements adjacent or close (Hawkins, 1994 ; Liu, 2008 ; Futrell et al., 2020 ; Temperley, 2018) .  \nWhen dependent elements are separated from their heads by intervening and unrelated words, this order is disrupted, and the dependency structure of a sentence becomes harder to recover. Asa result, the text becomes cognitively inaccessible even though its lexical content remains intact.  \nScrambling word order in violation of the syntactic constraints of a given language, through operations such as shuffling or reversal, increases dependency lengths and disrupts local syntactic relationships, producing inputs that are harder for humans to process despite intact vocabulary. Although some studies argue that LMs process such violations similarly to","cbCair3gcv1Khn7A","https://ap.wps.com/l/cbCair3gcv1Khn7A","pdf",484118,1,14,"English","en",105,"# Introduction\n## Information locality and dependency locality theory\n## Perturbation-based translation/reconstruction framework\n## Key questions and evaluation metrics","[{\"question\":\"What main patterns emerge when comparing structural recovery, surface recovery, and fluency across perturbations?\",\"answer\":\"Dependency structures recovered from perturbed inputs tend to show shorter dependency lengths than the original text, indicating locality preference. Recovery difficulty increases with stronger locality disruption, structural recovery (dependency F1) can diverge from exact surface matching, and fluency may decouple from faithful reconstruction under global shuffling.\"}]",1784206817,35,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":78,"head_meta":80,"extra_data":82,"updated_unix":27},"language-re-generation-an-investigation-into-information-locality-effects-on-reconstruction","",{"@graph":35,"@context":77},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/language-re-generation-an-investigation-into-information-locality-effects-on-reconstruction/85869/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71],{"name":72,"@type":73,"acceptedAnswer":74},"What main patterns emerge when comparing structural recovery, surface recovery, and fluency across perturbations?","Question",{"text":75,"@type":76},"Dependency structures recovered from perturbed inputs tend to show shorter dependency lengths than the original text, indicating locality preference. Recovery difficulty increases with stronger locality disruption, structural recovery (dependency F1) can diverge from exact surface matching, and fluency may decouple from faithful reconstruction under global shuffling.","Answer","https://schema.org",{"og:url":51,"og:type":79,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":81,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":84},[85,89,93,97,102,107,112,115,120,123,127],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":86,"show_sort_weight":87,"slug":88},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":90,"show_sort_weight":91,"slug":92},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Exam",70,"exam",{"id":98,"doc_module":4,"doc_module_name":45,"category_name":99,"show_sort_weight":100,"slug":101},5,"Comic",60,"comic",{"id":103,"doc_module":4,"doc_module_name":45,"category_name":104,"show_sort_weight":105,"slug":106},6,"Technology",50,"technology",{"id":108,"doc_module":4,"doc_module_name":45,"category_name":109,"show_sort_weight":110,"slug":111},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":113,"slug":114},30,"research-report",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},9,"Religion & Spirituality",20,"religion-spirituality",{"id":118,"doc_module":4,"doc_module_name":45,"category_name":121,"show_sort_weight":118,"slug":122},"World Cup","world-cup",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":124,"slug":126},10,"Lifestyle","lifestyle",{"id":128,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":98,"slug":130},19,"General","general"]