[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-86406-en":3,"doc-seo-86406-105":30,"detail-sidebar-cat-0-en-105":84},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},86406,3848291630094,"Emma Wilson","https://eur-avatar.wpscdn.com/davatar_085a072bc5b1113ac321206ff7593b45",8,"Research & Report","Preisach Attention: A Hysteretic Model of Sequential Memory","Preisach Attention introduces the Preisach Attention Layer (PAL), a sequence modelling architecture derived from the classical Preisach hysteresis operator. PAL replaces softmax attention with a learned binary relay γ̂αβ using activation/deactivation thresholds (α, β) while maintaining a stack of local extrema as internal state. A single-layer PAL-Transformer is shown Turing-complete and its computable function classes are incomparable to standard transformers. PAL’s response is rate-independent, depending only on local extrema, and the extremum stack forms a minimal sufficient statistic for all rate-independent functionals, enabling efficient long-episodic memory with total inference cost O(n log n).","arXiv :2605 .23603v1 [ cs .LG] 22 May 2026  \nPreprint  \nPreisach Attention: A Hysteretic Model of Sequential Memory  \nPiotr Frydrych  \nThe Metrology and Biomedical Engineering Institute Faculty of Mechatronics, Warsaw University of Technology [piotr.frydrych@pw.edu.pl](piotr.frydrych@pw.edu.pl)  \nPreprint.  \nAbstract  \nWe introduce the Preisach Attention Layer (PAL), a novel sequence modelling architecture grounded in the classical Preisach hysteresis operator from mathematical physics. PAL replaces the softmax attention mechanism with a binary relay operator γˆαβ parameterised by learned activation and deactivation thresholds (α,β), maintaining a stack of local extrema as its internal state. A single-layer PAL-Transformer with O(1) depth is Turing-complete under arbitrary precision arithmetic, achievable through simulation of a two-stack pushdown automaton—in contrast to the O(log n) depth required by standard hard-attention transformers (Pérez et al., 2021) . Second, we prove that the function classes computable by PAL and by the transformer are incomparable: PAL computes historical range statistics in O(1) layers that require O(log n) layers for transformers, while transformers support random-access retrieval that PAL cannot perform without auxiliary state. The separating property is rate-independence—PAL responds only to the sequence of local extrema, not to absolute token positions or temporal spacing. Third, we show that the extremum stack constitutes a minimal sufficient statistic of the input history for all rate-independent functionals, providing a formal analogue of the wiping property in classical hysteresis theory. PAL is thus an efficient architecture for tasks with long episodic memory and weak positional dependence, with O (nlog n) total inference cost versus O (n2 ) for standard attention.  \nKeywords: Preisach operator, hysteresis, attention mechanism, Turing completeness, expressiveness, sequence modelling, long-range dependence, rate-independence.  \nPreprint  \nContents  \n1 Introduction 4  \n2 Background 5  \n2.1 The Preisach Hysteresis Operator ................................ 5  \n2.2 The Extremum Stack ...................................... 5  \n2.3 Transformer Attention ..................................... 5  \n3 Preisach Attention Layer 6  \n3.1 Definition ............................................ 6  \n3.2 Connection to Classical Preisach Operator ........................... 6  \n3.3 Relationship to Standard Attention ............................... 7  \n4 Turing Completeness 7  \n4.1 Encoding Alphabet Symbols in the Extremum Stack ...................... 7  \n4.2 Main Theorem ......................................... 7  \n5 Expressiveness Separation 10  \n5.1 Functions PAL computes but Transformer cannot (at bounded depth) ............. 10  \n5.2 Functions Transformer computes but PAL cannot ....................... 10  \n5.3 The separating property: rate-independence .......................... 11  \n6 Logical Characterisation 11  \n7 Computational Complexity 12  \n8 Related Work 13  \n9 Connection to the Random-Field Ising Model 13  \n9.1 The Preisach–RFIM Equivalence ................................ 13  \n9.2 PAL as a Learned, Sequential RFIM .............................. 14  \n9.3 Problems where PAL Inherits Ising Expressiveness ...................... 14  \n9.3.1 Sequential Binary Optimisation ............................ 14  \n9.3.2 Associative Memory with Structured Forgetting .................... 15  \n9.3.3 Sequential Belief Propagation in Markov Random Fields ............... 16  \n9.4 Avalanches and Phase Transitions in PAL ........................... 16  \n9.5 Summary: PAL vs. Ising-based Methods ............................ 17  \n10 Conclusion 17  \nPreprint  \nA Full Proof of Theorem 6.2: PAL corresponds to EFO 20  \nA.1 Formal Setup .......................................... 20  \nA.2 The Correspondence ...................................... 21  \nA.3 Inductive Proof of Theorem 6.2 ................................. 22  \nA","cbCaik9KfhRsePQ2","https://ap.wps.com/l/cbCaik9KfhRsePQ2","pdf",498115,5,1,24,"English","en",105,"# Introduction\n# Background\n## The Preisach Hysteresis Operator\n## The Extremum Stack\n## Transformer Attention\n# Preisach Attention Layer\n## Definition\n## Connection to Classical Preisach Operator\n## Relationship to Standard Attention\n# Turing Completeness\n## Encoding Alphabet Symbols in the Extremum Stack\n## Main Theorem\n# Expressiveness Separation\n## Functions PAL computes but Transformer cannot (at bounded depth)\n## Functions Transformer computes but PAL cannot\n## The separating property: rate-independence\n# Logical Characterisation\n# Computational Complexity\n# Related Work\n# Connection to the Random-Field Ising Model\n## The Preisach–RFIM Equivalence\n## PAL as a Learned, Sequential RFIM\n## Problems where PAL Inherits Ising Expressiveness\n## Avalanches and Phase Transitions in PAL\n# Conclusion","[{\"question\":\"What is the “rate-independence” separating property of PAL?\",\"answer\":\"PAL responds only to the sequence of local extrema, not to absolute token positions or temporal spacing. This yields functional classes that differ from standard transformers and supports a minimal sufficient statistic interpretation via the extremum stack.\"}]",1784211548,60,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":79,"head_meta":81,"extra_data":83,"updated_unix":28},"preisach-attention-a-hysteretic-model-of-sequential-memory","",{"@graph":36,"@context":78},[37,54,69],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,51],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":50},"https://docshare.wps.com/document/research-report/",3,{"item":52,"name":13,"@type":43,"position":53},"https://docshare.wps.com/document/preisach-attention-a-hysteretic-model-of-sequential-memory/86406/",4,{"url":52,"name":13,"@type":55,"author":56,"headline":13,"publisher":58,"fileFormat":61,"inLanguage":24,"description":14,"dateModified":62,"datePublished":63,"encodingFormat":61,"isAccessibleForFree":64,"interactionStatistic":65},"DigitalDocument",{"name":9,"@type":57},"Person",{"url":41,"name":59,"@type":60},"DocShare","Organization","application/pdf","2026-07-28","2026-07-16",true,{"@type":66,"interactionType":67,"userInteractionCount":20},"InteractionCounter",{"@type":68},"ViewAction",{"@type":70,"mainEntity":71},"FAQPage",[72],{"name":73,"@type":74,"acceptedAnswer":75},"What is the “rate-independence” separating property of PAL?","Question",{"text":76,"@type":77},"PAL responds only to the sequence of local extrema, not to absolute token positions or temporal spacing. This yields functional classes that differ from standard transformers and supports a minimal sufficient statistic interpretation via the extremum stack.","Answer","https://schema.org",{"og:url":52,"og:type":80,"og:title":13,"og:site_name":59,"og:description":14},"article",{"robots":82,"canonical":52},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":85},[86,90,94,98,101,106,111,114,119,122,126],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":87,"show_sort_weight":88,"slug":89},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":91,"show_sort_weight":92,"slug":93},"Literature",80,"literature",{"id":53,"doc_module":4,"doc_module_name":46,"category_name":95,"show_sort_weight":96,"slug":97},"Exam",70,"exam",{"id":20,"doc_module":4,"doc_module_name":46,"category_name":99,"show_sort_weight":29,"slug":100},"Comic","comic",{"id":102,"doc_module":4,"doc_module_name":46,"category_name":103,"show_sort_weight":104,"slug":105},6,"Technology",50,"technology",{"id":107,"doc_module":4,"doc_module_name":46,"category_name":108,"show_sort_weight":109,"slug":110},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":112,"slug":113},30,"research-report",{"id":115,"doc_module":4,"doc_module_name":46,"category_name":116,"show_sort_weight":117,"slug":118},9,"Religion & Spirituality",20,"religion-spirituality",{"id":117,"doc_module":4,"doc_module_name":46,"category_name":120,"show_sort_weight":117,"slug":121},"World Cup","world-cup",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":123,"slug":125},10,"Lifestyle","lifestyle",{"id":127,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":20,"slug":129},19,"General","general"]