[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84222-en":3,"doc-seo-84222-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84222,962075114101,"Seraphina","https://ap-avatar.wpscdn.com/avatar/e000253a75eb197efd?x-image-process=image/resize,m_fixed,w_180,h_180&k=1780044092746381165",8,"Research & Report","TF-Engram: A Train-Free Engram with SSD-Backed Memory for Large Language Models","Large Language Models (LLMs) embed factual knowledge and semantic patterns implicitly in Transformer parameters, making knowledge expansion expensive via pretraining, fine-tuning, retrieval augmentation, or longer contexts. Engram-style memory injects compact hidden-state signals, yet GPU-centered implementations often use hash-based compression that causes phrase collisions and degrades semantic fidelity. TF-Engram builds phrase-specific semantic memory offline from external corpora, stores large tables across GPU–DRAM–SSD, and uses Early-Exit Guided Predictive Prefetching to mask latency during decoding. On Qwen3-0.6B, TF-Engram raises average downstream score from 57.6 to 59.4 and preserves throughput with low system overhead.","TF-Engram: A Train-Free Engram with SSD-Backed Memory for  \nLarge Language Models  \nYutang MA  \nThe Chinese University of Hong Kong [ytma2@cse.cuhk.edu.hk](ytma2@cse.cuhk.edu.hk)  \nXikun JIANG  \nThe Chinese University of Hong Kong [xkjiang24@cse.cuhk.edu.hk](xkjiang24@cse.cuhk.edu.hk)  \nKecheng HUANG  \nBeijing Institute of Technology, Zhuhai [huangkecheng@bitzh.edu.cn](huangkecheng@bitzh.edu.cn)  \nZili SHAO  \nThe Chinese University of Hong Kong [shao@cse.cuhk.edu.hk](shao@cse.cuhk.edu.hk)  \narXiv :2607 .07388v 1 [ cs .CL] 8 Jul 2026  \nABSTRACT  \nLarge Language Models (LLMs) store factual knowledge and domainspecific patterns implicitly in dense Transformer parameters, making knowledge expansion costly through pretraining, fine-tuning, retrieval augmentation, or longer contexts. Engram-style memory offers a compact hidden-state injection pathway, but existing GPUresident designs often rely on hash-based compression, causing unrelated phrases to collide in shared slots and weakening phrase-level semantic fidelity. We present TF-Engram, a train-free Engram system that constructs phrase-specific semantic memory offline from external corpora, stores large memory tables across a GPU–DRAM– SSD hierarchy, and uses Early-Exit Guided Predictive Prefetching to hide external-memory latency during autoregressive decoding. On Qwen3-0.6B, TF-Engram improves the average downstream score from 57.6 to 59.4, outperforming both the frozen backbone and a parameter-matched LoRA baseline. System evaluation shows that large TF-Engram tables can be built with moderate offline cost, SSD-backed storage substantially reduces GPU memory demand, and predictive prefetching recovers much of the throughput loss caused by external memory access. These results demonstrate that static phrase memory can be integrated into LLM inference as a scalable, train-free, and low-overhead system component.  \n1 INTRODUCTION  \nLarge Language Models (LLMs) have become the dominant architecture for natural language understanding, reasoning, code generation, and knowledge-intensive question answering [3, 7, 10, 28–30] . Their success is largely driven by scaling dense Transformer parameters and training data [12, 18]. However, this design also tightly couples knowledge capacity with model size: factual knowledge, domain-specific terminology, and frequently occurring semantic patterns are implicitly stored inside billions of neural parameters. Improving such knowledge often requires more pretraining, supervised fine-tuning, retrieval augmentation, or longer context windows [2, 3, 11, 23] . These solutions increase training cost, inference latency, memory consumption, or deployment complexity. As LLMs are increasingly used in domain-specific and knowledge-intensive applications, it becomes important to decouple static knowledge storage from dense Transformer computation.  \nRecent work on memory-augmented LLMs explores this decoupling from several directions. Retrieval-Augmented Generation (RAG) systems maintain an external document corpus and retrieve relevant passages during inference, usually appending the retrieved  \ntext to the prompt [2, 11, 15, 23]. Long-context and KV-cache based methods extend the amount of information accessible to the model by keeping more historical tokens or intermediate states [8, 9, 20] . Adapter-based and parameter-efficient memory methods inject additional trainable modules into the model, allowing new knowledge to be stored in a smaller number of parameters than full fine-tuning [13, 14, 21, 24]. Another line of work, represented by Engram-style memory, introduces a lightweight side memory pathway that stores token-level or phrase-level patterns and injects retrieved memory signals into the hidden states of the backbone model [5] . These categories differ substantially in memory granularity, training cost, inference overhead, and system scalability.  \nAlthough RAG, long-context, and adapter-based methods are effective in many scenarios, they are ","cbCaiphRn9MYg8Bx","https://ap.wps.com/l/cbCaiphRn9MYg8Bx","pdf",3477553,1,13,"English","en",105,"# Introduction\n## Motivation: Decoupling static knowledge from dense parameters\n## Related work: RAG, long-context, adapters, and Engram-style memory\n## Limitations of existing Engram-style systems\n## Key challenges addressed by TF-Engram","[{\"question\":\"What problem does TF-Engram address in Engram-style memory for LLMs?\",\"answer\":\"It addresses practical limitations of existing Engram-style designs, especially hash-based compression that can cause phrase collisions and reduce phrase-level semantic fidelity, along with scalability and external-memory access latency during decoding.\"},{\"question\":\"How does TF-Engram construct phrase-specific semantic memory without additional training?\",\"answer\":\"TF-Engram constructs phrase-specific semantic memory offline from external corpora, enabling a train-free system component that avoids training-dependent and model-specific memory construction.\"},{\"question\":\"How does TF-Engram reduce the performance impact of using SSD-backed memory during autoregressive decoding?\",\"answer\":\"It uses Early-Exit Guided Predictive Prefetching to hide external-memory latency, recovering much of the throughput lost to external memory access.\"}]",1784194130,33,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"tf-engram-a-train-free-engram-with-ssd-backed-memory-for-large-language-models","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/tf-engram-a-train-free-engram-with-ssd-backed-memory-for-large-language-models/84222/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does TF-Engram address in Engram-style memory for LLMs?","Question",{"text":75,"@type":76},"It addresses practical limitations of existing Engram-style designs, especially hash-based compression that can cause phrase collisions and reduce phrase-level semantic fidelity, along with scalability and external-memory access latency during decoding.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does TF-Engram construct phrase-specific semantic memory without additional training?",{"text":80,"@type":76},"TF-Engram constructs phrase-specific semantic memory offline from external corpora, enabling a train-free system component that avoids training-dependent and model-specific memory construction.",{"name":82,"@type":73,"acceptedAnswer":83},"How does TF-Engram reduce the performance impact of using SSD-backed memory during autoregressive decoding?",{"text":84,"@type":76},"It uses Early-Exit Guided Predictive Prefetching to hide external-memory latency, recovering much of the throughput lost to external memory access.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]