[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84271-en":3,"doc-seo-84271-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84271,1374391974564,"Clementine","https://ap-avatar.wpscdn.com/avatar/14000253aa45c000a9e?x-image-process=image/resize,m_fixed,w_180,h_180&k=1779874745381141002",8,"Research & Report","LLT Local Linear Transformer for PDE Operator Learning","LLT (Local Linear Transformer) is presented as a neural-operator architecture for learning PDE solution maps and accelerating numerical simulation. While transformer-based neural operators can model long-range dependencies, standard attention is costly due to quadratic scaling and lacks an explicit bias toward local interactions common in PDEs. LLT combines linear global attention with local spatial mixing and integrates coordinate and geometry information.","arXiv :2607 .077 18v 1 [ cs .LG] 4 Jul 2026  \nLLT: Local Linear Transformer for PDE Operator Learning  \nOded Ovadia∗1 and Eli Turkel†1  \n1 School of Mathematical Sciences, Tel Aviv University, Tel Aviv, Israel  \nAbstract  \nNeural operators have become a common approach for learning PDE solution maps and accelerating numerical simulations. Transformer-based neural operators are of particular interest, since attention can learn long-range dependencies in the computational domain. However, standard attention has two major limitations when applied to PDEs: it scales quadratically with the number of computational nodes, and it lacks an explicit bias toward local interactions. To address these issues, we introduce Local Linear Transformer (LLT) for PDE operator learning. The architecture combines linear global attention with local spatial mixing, and incorporates coordinate and geometry information. We evaluate LLT on several PDE problems, including elasticity, plasticity, airfoil flow, pipe flow, and Darcy flow. The reference data for these problems span finite-element, finite-volume, and finite-difference discretizations on structured and unstructured meshes. Compared with other neural-operator and transformer baselines from prior studies, LLT achieves competitive or lower relative L2 error across these problems. On matched structured discretizations, wall-clock time per training iteration is reduced by factors of 1.8 to 2.5 relative to Transolver. We also scale the approach and apply it to a three-dimensional car aerodynamics dataset with 32,186 unstructured mesh points per sample. Together, these results indicate that LLT providesan accurate and computationally efficient operator for PDE problems across discretizations, mesh types, and problem settings.  \nKeywords: partial differential equations, neural operators, transformers, scientific machine learning  \n1 Introduction  \nThe use of machine learning (ML) methods for scientific computing has been growing rapidly in recent years, with many successful methods for modeling mathematical problems [Raissi, Perdikaris, and Karniadakis, 2019 , Lu, Jin, Pang, Zhang, and Karniadakis, 2021 , Li, Kovachki, Azizzadenesheli, liu, Bhattacharya, Stuart, and Anandkumar, 2021 , Long, Lu, Ma, and Dong, 2018 , Xu, Chang, and Zhang, 2019 , Takamoto, Praditia, Leiteritz, MacKinlay, Alesiani, Pfl¨uger, and Niepert, 2022 , Gupta and Brandstetter, 2022] . Such methods have shown promise in many different areas, including computational mechanics [Cai, Mao, Wang, Yin, and Karniadakis, 2021 , Zhang, Kahana, Turkel, Ranade, Pathak, and Karniadakis, 2022], wave propagation [Ovadia, Kahana, Turkel, and Dekel, 2021 , Ovadia, Kahana, and Turkel, 2024b], materials science [Dingreville, Stewart, and Chen, 2020 , Oommen, Shukla, Goswami, Dingreville, and Karniadakis, 2022], fluid dynamics [Sharma, Chung, Akoush, and Ihme, 2023 , Zhao, Zhang, Lou, Wang, and Yang, 2024], and turbulent flows [Wu, Xiao, and Paterson, 2018 , Wang, Kashinath, Mustafa, Albert, and Yu, 2020] .  \n∗ Corresponding author. Email: [odedovadia@mail.tau.ac.il](odedovadia@mail.tau.ac.il)[ ](odedovadia@mail.tau.ac.il)†Email: [turkel@tauex.tau.ac.il](turkel@tauex.tau.ac.il)  \nIn many applications, the same class of PDEs must be solved repeatedly for varying coefficients, geometries, initial conditions, or boundary conditions. Classical solvers remain the reference method for accuracy, but they can be expensive when thousands of related solves are required. This has motivated neural operators that learn solution maps directly from data. Neural operators provide one such framework by learning maps between input functions and output solution fields [Kovachki, Li, Liu, Azizzadenesheli, Bhattacharya, Stuart, and Anandkumar, 2023 , Azizzadenesheli, Kovachki, Li, Liu-Schiaffini, Kossaifi, and Anandkumar, 2024] .  \nIn particular, the Transformer architecture is a promising candidate for operator learning because its attention mechanism can learn informati","cbCaillYyJUOu4aA","https://ap.wps.com/l/cbCaillYyJUOu4aA","pdf",10937213,1,21,"English","en",105,"# Introduction\n## Motivation from neural operators\n## Transformer limitations for PDEs\n## Architectural requirements for an improved operator\n# Local Linear Transformer (LLT)","[{\"question\":\"What problems does LLT target in transformer-based neural operators for PDEs?\",\"answer\":\"LLT targets two key issues: quadratic self-attention cost with the number of spatial nodes and the absence of built-in bias toward local interactions that PDE structure often exhibits.\"},{\"question\":\"How does LLT modify the transformer attention mechanism for PDE operator learning?\",\"answer\":\"LLT combines linear global attention with local spatial mixing, and it incorporates coordinate and geometry information to better represent computational domains.\"},{\"question\":\"On which types of PDE tasks and datasets is LLT evaluated?\",\"answer\":\"LLT is evaluated on multiple PDE problems including elasticity, plasticity, airfoil flow, pipe flow, and Darcy flow, using reference data from finite-element, finite-volume, and finite-difference discretizations on both structured and unstructured meshes.\"}]",1784194510,53,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"llt-local-linear-transformer-for-pde-operator-learning","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/llt-local-linear-transformer-for-pde-operator-learning/84271/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problems does LLT target in transformer-based neural operators for PDEs?","Question",{"text":75,"@type":76},"LLT targets two key issues: quadratic self-attention cost with the number of spatial nodes and the absence of built-in bias toward local interactions that PDE structure often exhibits.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does LLT modify the transformer attention mechanism for PDE operator learning?",{"text":80,"@type":76},"LLT combines linear global attention with local spatial mixing, and it incorporates coordinate and geometry information to better represent computational domains.",{"name":82,"@type":73,"acceptedAnswer":83},"On which types of PDE tasks and datasets is LLT evaluated?",{"text":84,"@type":76},"LLT is evaluated on multiple PDE problems including elasticity, plasticity, airfoil flow, pipe flow, and Darcy flow, using reference data from finite-element, finite-volume, and finite-difference discretizations on both structured and unstructured meshes.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]