[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-82837-en":3,"doc-seo-82837-105":29,"detail-sidebar-cat-0-en-105":90},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":4,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},82837,5909877438554,"Maeve","https://ap-avatar.wpscdn.com/avatar/5600025385ad2bf12a7?_k=1778553567797529272",8,"Research & Report","Wrong Before Right: Late Rescue and Interface Failure in Aligned Language Models","The paper studies how correctness is assembled inside aligned language models, focusing on internal preference dynamics rather than only output accuracy. Using layer-wise difference-in-differences trajectories over polarity-controlled minimal pairs, it identifies a repeatable “wrong-dip” where mid layers commit to incorrect, often unsafe answers and are rescued only by late-layer correction. The causal mechanism is verified with activation transplantation and shows strong links to structural compression failures, trainable mitigation, and interface-bound evaluation distortions.","arXiv :2607 .04640v 1 [ cs .CL] 6 Jul 2026  \nWrong Before Right: Late Rescue and Interface Failure in Aligned Language Models  \nJiaqi Deng  \nIndependent Researcher  \n[djq627@163.com](djq627@163.com)  \nAbstract  \nWe study how correctness is assembled inside aligned language models—not only whether the final answer is right. Using layer-wise difference-in-differences (DiD) trajectories over polarity-controlled minimal pairs, we identify a robust phenomenon we call the wrong-dip: in mid layers (25–90% depth), a model’s internal preference transiently commits to the incorrect—often unsafe—answer and is rescued only by late-layer correction. We verify the phenomenon causally with patchscope-style activation transplantation and characterize it across 17 models spanning three families and 64× scale (0.5B–32B) .  \nFour findings follow. (1) Alignment amplification of the causal wrong-dip is recipe-specific and emergent: it emerges at 3B in Qwen2.5, remains high, and peaks at 32B (Δdip +0 . 140 → +0 . 182, paired 􀁃 up to 9.7), reverses significantly in Llama-3-8B (􀁃 = −2 . 31), and sits in between for Mistral-7B—the dip audits alignment recipes, not alignment per se. (2) The dip predicts real compression failures with mechanistic specificity: items with large dips on the intact model are 3–7× more likely to flip under genuine late-layer low-rank compression, block dropping, or structured pruning, while flips under quantization are dip-blind—a double dissociation matching the late-rescue mechanism, causally confirmed by selectively ablating late-layer residual contributions. (3) The dip is trainable: a LoRA fine-tune with a mid-layer wrong-margin hinge penalty matches output-only SFT’s perfect held-out accuracy while cutting the causal internal dip by 67–70%, and transfers to compression robustness (mid-SVD retention 0.943 vs 0.872, per-seed McNemar 􀀿 = 2. 8 × 10−6/0 .013/0 .064); output-only SFT instead worsens the causal dip by up to 2. 8× at perfect surface accuracy. (4) Once the readout interface is controlled, the phenomenon survives natural-language I/O: with semantic-candidate readouts, dip stratification of structural-damage failures is significant on naturalistic vignettes (􀀿 = 10−3–10−4), and free-form evaluation fragility separates into a dip-auditable late-rescue layer and a dip-blind interface layer. Together, these results show that output-level correctness can hide a late-rescue production process—and that this process governs structural compression risk, post-training quality, and natural-language evaluation distortion.  \n1 Introduction  \nSafety evaluation of language models is dominated by output-level testing. Yet deployment pipelines routinely alter model internals—quantization, pruning, low-rank compression, distillation, further fine-tuning. If a model’s correct behavior is maintained by late-layer correction of an internally wrong preference, outputlevel tests will certify a model whose correctness is one compression step away from failing—and whose apparent competence in natural-language evaluation may reflect interface binding rather than stable internal computation. This paper asks: do models internally commit to wrong answers before producing right ones; does this matter causally; and can it be measured, predicted, and trained away?  \nWe give a four-part affirmative answer. Figure 1 previews the account: mid layers transiently prefer the wrong answer; late layers rescue the decision; a readout interface then binds the internal decision to output—and each stage can fail in a measurably different way.  \nFigure 1: Overview. Mid layers transiently prefer the wrong answer (the wrong-dip, audited by two item-level statistics); late layers rescue the decision; a readout interface then binds the internal decision to output. Structural damage removes the rescue and flips high-dip items selectively (§5); the generative interface fails in a separate, dip-blind way (§5.6) .  \nPhenomenon and metric (§3). On polarity-controll","cbCaip3282U1Iutj","https://ap.wps.com/l/cbCaip3282U1Iutj","pdf",976861,1,16,"English","en",105,"# Introduction\n# Phenomenon and Metric\n# Scale and Recipe Structure\n# Consequences\n# Intervention","[{\"question\":\"What does the “wrong-dip” phenomenon describe in aligned language models?\",\"answer\":\"In mid layers, the model’s internal preference transiently commits to an incorrect—often unsafe—answer, and the correct behavior is recovered only in later layers through rescue.\"},{\"question\":\"How is the wrong-dip measured and verified causally?\",\"answer\":\"It is measured using polarity-controlled minimal pairs and layer-wise difference-in-differences (DiD) trajectories, and verified causally via patchscope-style activation transplantation that reproduces the wrong answer in neutral decoding.\"},{\"question\":\"Why can output-level accuracy hide model fragility after compression or other pipeline changes?\",\"answer\":\"Because structural damage can remove the late-layer rescue stage that corrects internally wrong preferences, leading to selective flips that output-only tests may not catch; quantization failures behave differently and are dip-blind.\"}]",1784183329,40,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":85,"head_meta":87,"extra_data":89,"updated_unix":27},"wrong-before-right-late-rescue-and-interface-failure-in-aligned-language-models","",{"@graph":35,"@context":84},[36,53,67],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/wrong-before-right-late-rescue-and-interface-failure-in-aligned-language-models/82837/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":61,"encodingFormat":60,"isAccessibleForFree":62,"interactionStatistic":63},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-16",true,{"@type":64,"interactionType":65,"userInteractionCount":4},"InteractionCounter",{"@type":66},"ViewAction",{"@type":68,"mainEntity":69},"FAQPage",[70,76,80],{"name":71,"@type":72,"acceptedAnswer":73},"What does the “wrong-dip” phenomenon describe in aligned language models?","Question",{"text":74,"@type":75},"In mid layers, the model’s internal preference transiently commits to an incorrect—often unsafe—answer, and the correct behavior is recovered only in later layers through rescue.","Answer",{"name":77,"@type":72,"acceptedAnswer":78},"How is the wrong-dip measured and verified causally?",{"text":79,"@type":75},"It is measured using polarity-controlled minimal pairs and layer-wise difference-in-differences (DiD) trajectories, and verified causally via patchscope-style activation transplantation that reproduces the wrong answer in neutral decoding.",{"name":81,"@type":72,"acceptedAnswer":82},"Why can output-level accuracy hide model fragility after compression or other pipeline changes?",{"text":83,"@type":75},"Because structural damage can remove the late-layer rescue stage that corrects internally wrong preferences, leading to selective flips that output-only tests may not catch; quantization failures behave differently and are dip-blind.","https://schema.org",{"og:url":51,"og:type":86,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":88,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":91},[92,96,100,104,109,114,118,121,126,129,133],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":93,"show_sort_weight":94,"slug":95},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":97,"show_sort_weight":98,"slug":99},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":101,"show_sort_weight":102,"slug":103},"Exam",70,"exam",{"id":105,"doc_module":4,"doc_module_name":45,"category_name":106,"show_sort_weight":107,"slug":108},5,"Comic",60,"comic",{"id":110,"doc_module":4,"doc_module_name":45,"category_name":111,"show_sort_weight":112,"slug":113},6,"Technology",50,"technology",{"id":115,"doc_module":4,"doc_module_name":45,"category_name":116,"show_sort_weight":28,"slug":117},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":119,"slug":120},30,"research-report",{"id":122,"doc_module":4,"doc_module_name":45,"category_name":123,"show_sort_weight":124,"slug":125},9,"Religion & Spirituality",20,"religion-spirituality",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":127,"show_sort_weight":124,"slug":128},"World Cup","world-cup",{"id":130,"doc_module":4,"doc_module_name":45,"category_name":131,"show_sort_weight":130,"slug":132},10,"Lifestyle","lifestyle",{"id":134,"doc_module":4,"doc_module_name":45,"category_name":135,"show_sort_weight":105,"slug":136},19,"General","general"]