[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85216-en":3,"doc-seo-85216-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85216,13056703019662,"Evangeline","https://ap-avatar.wpscdn.com/avatar/be000253a8e92610077?_k=1778726343310543188",8,"Research & Report","Context by Distinct Information An Auditable Dirichlet Process Working Memory for Long Redundant Context Streams","Context engineering governs which information a model carries forward across dialogue turns, retrieved passages, tool outputs, observations, and intermediate state, yet current designs meter memory in tokens via bounded recurrent states, token-level key-value stores, or fixed windows and eviction rules. This work targets a distinct-item unit: allocate-on-novelty caching that scales with distinct inputs. It organizes context by task dependence—recall-, summary-, and locality-carried—mapping them to content-addressed novelty cache, recurrent state, and recency window. Experiments show novelty-gated attention matches full attention at fewer tokens, with inspectable, auditable memory scaling with needed information rather than sequence length.","arXiv :2607 . 1044 1v 1 [ cs .LG] 11 Jul 2026  \nContext by Distinct Information:  \nAn Auditable Dirichlet-Process Working Memory for Long, Redundant Context Streams  \nSiddharth Pal Viktoria Rojkova  \nPreprint  \nAbstract  \nContext engineering decides what information a model carries forward—conversation turns, retrieved passages, tool results, observations, and intermediate state—and current designs meter that context in tokens, whether by compressing the past into a bounded recurrent state, keeping a key–value entry for every token, or imposing a fixed budget through a window or eviction rule. All three make the token the unit of memory even when the stream is redundant and the task depends on the distinct information it carries. Building on a companion mechanism paper that opens a cache slot only when an incoming key is novel, so that memory scales with the number of distinct items rather than tokens, we develop that allocate-on-novelty cache as a working-memory component and organize context by how a task depends on the past: recall-carried information, which a later query may need from a particular earlier item, belongs in a content-addressed novelty cache; summary-carried information belongs in a recurrent state; and locality-carried information belongs in a recency window.  \nThe claim is empirical and bounded. On a matched character-level control, novelty-gated attention reaches the performance of full attention while attending to about half the tokens, and coupling the cache with a state-space summary matches full-attention coupling at that reduced cost; retaining full attention beside the cache improves the read further still, so the novelty-kept view supplies signal rather than merely approximating attention. The advantage grows as context lengthens, while a sliding window remains preferable on short, locality-dominated spans. The same pattern holds end to end on real structured context: on next-code prediction over synthetic Medicare claims the coupled component leads full attention and every fixed-budget eviction policy by roughly a third of a bit per event at a thousand-event horizon, whereas cost forecasting over the same stream is summary-carried and the cache is neutral; and which recurrent summary to pair with the cache is itself domain-dependent, since an input-gated variant that wins on text collapses on the structured stream. Across log, clinical, claims, and mapping streams the retained memory is an inspectable table of templates, codes, drugs, or places rather than an opaque state, though a distribution shift in one log stream defeats every read and scopes the claim. The experiments are small-scale, use only public data, and do not yet evaluate multi-turn assistants or retrieval-augmented agents; what they establish is the architectural primitive, that context can scale with the distinct information a task may need rather than with tokens, in a working memory that is both content-addressable and auditable.  \n1 The problem: context is metered in tokens, but information is not  \nContext engineering is the construction and maintenance of the information exposed to a model at inference time. A context stream may contain natural-language turns, document passages, tool outputs, program state, observations, structured records, or mixtures of them. The stream can be long, hierarchical, and multi-rate: a slow-moving fact or entity can remain relevant across thousands of fast local updates. It is also usually redundant. A name recurs across a conversation, a retrieved passage restates an earlier  \nfact, the same tool emits repeated status records, a code or identifier appears hundreds of times, and a log template fires thousands of times.  \nThe standard memory choices nevertheless scale with tokens. A fixed-state recurrent or state-space model processes the stream in linear time and constant-size inference memory, but folds all prior information into a bounded state. Its content-addressed recall degrades once the num","cbCailiCfAYTDTc9","https://ap.wps.com/l/cbCailiCfAYTDTc9","pdf",335603,1,16,"English","en",105,"# The problem: context is metered in tokens, but information is not\n## Unit of context and redundant streams\n# Theoretical anchor: remembering distinct items, no","[{\"question\":\"What is the main idea behind “auditable Dirichlet-Process working memory” in this paper?\",\"answer\":\"It replaces token-metered context with a distinct-item memory that grows with the number of novel items a task may need. An allocate-on-novelty cache is paired with other memory structures for different dependency types, making retained information inspectable rather than opaque.\"},{\"question\":\"How does the paper categorize different kinds of context dependence?\",\"answer\":\"It separates recall-carried information (dependent on specific earlier items), summary-carried information (dependent on aggregates or smooth history), and locality-carried information (dependent mainly on the immediate past). Each category maps to a different memory component.\"},{\"question\":\"What experimental evidence supports the proposed memory designs?\",\"answer\":\"On character-level control, novelty-gated attention reaches full-attention performance while attending to about half the tokens, and coupling the cache with a state-space summary achieves similar benefits at reduced cost. The end-to-end results on structured context show the coupled component can outperform full attention and fixed-budget eviction in certain settings, while cache effects can be neutral for summary-driven tasks.\"}]",1784201805,40,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"context-by-distinct-information-an-auditable-dirichlet-process-working-memory-for-long-redundant-context-streams","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/context-by-distinct-information-an-auditable-dirichlet-process-working-memory-for-long-redundant-context-streams/85216/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-23","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is the main idea behind “auditable Dirichlet-Process working memory” in this paper?","Question",{"text":75,"@type":76},"It replaces token-metered context with a distinct-item memory that grows with the number of novel items a task may need. An allocate-on-novelty cache is paired with other memory structures for different dependency types, making retained information inspectable rather than opaque.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does the paper categorize different kinds of context dependence?",{"text":80,"@type":76},"It separates recall-carried information (dependent on specific earlier items), summary-carried information (dependent on aggregates or smooth history), and locality-carried information (dependent mainly on the immediate past). Each category maps to a different memory component.",{"name":82,"@type":73,"acceptedAnswer":83},"What experimental evidence supports the proposed memory designs?",{"text":84,"@type":76},"On character-level control, novelty-gated attention reaches full-attention performance while attending to about half the tokens, and coupling the cache with a state-space summary achieves similar benefits at reduced cost. The end-to-end results on structured context show the coupled component can outperform full attention and fixed-budget eviction in certain settings, while cache effects can be neutral for summary-driven tasks.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,119,122,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":28,"slug":118},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]