[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85660-en":3,"doc-seo-85660-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},85660,4398048949847,"Eliana","https://ap-avatar.wpscdn.com/avatar/400002536579ef2da7f?_k=1778318612642679267",8,"Research & Report","Layer Parallel Inference Reduces Encrypted Nonlinear Depth in Transformers","Fully homomorphic encryption (FHE) enables Transformer inference on encrypted inputs, but encrypted execution suffers from excessive cost caused by sequential composition of many nonlinear blocks. The study evaluates Structured Newton Layer Parallelism (SNLP) to make inter-layer computation more FHE-friendly without turning each block FHE-native. Using Chebyshev polynomial simulation, error accumulation and error amplification are compared between sequential and SNLP across eight models and four architectures. On a 0.5B IDN-trained model, SNLP reduces bootstraps and keeps encrypted PPL within 1.2%.","arXiv :2607 .048 19v2 [ cs .LG] 13 Jul 2026  \nLayer-Parallel Inference Reduces Encrypted Nonlinear Depth in Transformers  \nLigong Han1,2,3∗ Kai Xu2,3 Hao Wang2,3 Ruijiang Gao5 Han Gao6 Akash Srivastava3,4  \n1MBZUAI IFM 2Red Hat AI Innovation 3MIT-IBM Watson AI Lab  \n4Core AI, IBM 5University of Texas at Dallas 6Iowa State University  \nAbstract  \nFully homomorphic encryption (FHE) enables computation on encrypted data, but practical encrypted Transformer inference is bottlenecked by the sequential composition of many nonlinear blocks. We study whether Structured Newton Layer Parallelism (SNLP) (Han et al., 2026) can make this inter-layer composition more FHE-friendly: each Transformer block still requires polynomial approximations for operations such as softmax and RMSNorm, but SNLP reduces the layerwise sequential nonlinear depth from L stages to a small number of solver iterations plus linear structured corrections. Using a simulation framework based on Chebyshev polynomial approximations, we measure error accumulation under sequential versus SNLP inference across 8 models and 4 architecture families. On a 0.5B IDN-trained model, SNLP uses 2.65 × fewer bootstraps (20 vs. 53) and produces encrypted PPL within 1.2% of sequential’s encrypted PPL, while exhibiting lower error amplification (1.36× vs. 1.42×) . Across all tested models, SNLP has lower amplification than sequential inference.  \nAblations show that softmax approximation dominates the error budget and CKKS arithmetic noise is negligible in our setting, suggesting that SNLP is complementary to block-level FHE-friendly operator design rather than a replacement for it.  \n1 Introduction  \nTransformer language models (Vaswani et al., 2017) are sequential along the layer axis: the hidden state at layer l + 1 depends on the output of layer l. Under fully homomorphic encryption (FHE), this sequential composition is particularly costly. Each Transformer block contains operations that are not native to FHE arithmetic: softmax requires exponentiation, RMSNorm requires inverse square root, and activations require nonlinear evaluation, all of which must be approximated by polynomials when operating on encrypted data (Chenet al., 2022; Hao et al., 2022; Pang et al., 2024) . These polynomial approximations introduce per-block errors that compound across L sequential layers, consuming the modulus budget and requiring frequent bootstrapping, the most expensive FHE operation, accounting for 50–86% of total inference latency (Agrawal et al., 2024) .  \nWe observe that Structured Newton Layer Parallelism (SNLP) (Han et al., 2026) offers a complementary direction for reducing FHE inference cost. SNLP does not make an individual Transformer block FHE-native: softmax, normalization, and activations still require approximation or redesign. Instead, SNLP changes the inter-layer computation graph. Rather than composing L nonlinear block evaluations sequentially, SNLP evaluates suffix layers in parallel over K solver iterations and propagates information across depth using structured Newton-style corrections. For IDN, the correction is purely additive and has zero FHE multiplicative depth; for HCN, it is a small linear mixing over streams. Thus SNLP targets the sequential composition of FHE-unfriendly blocks, not the local block operators themselves.  \n∗ Work done while at Red Hat AI Innovation.  \nWe introduce the metric NFE (Nonlinear Forward Evaluations) = (L − N) + K, where N is the number of parallel suffix layers. In our symbolic CKKS cost model, NFE tracks the FHE bootstrap count closely (ratio 0.99–1.02×) . Using a simulation framework that replaces nonlinear operations with Chebyshev polynomial approximations, we measure how polynomial errors accumulate under sequential versus SNLP inference. Degree-12 Chebyshev softmax approximation degrades sequential PPL by about 42%; the question is whether SNLP adds to or reduces this shared cost.  \nOur main findings:  \n• SNLP has lower error amplif","cbCaibLwGVGJ3pZF","https://ap.wps.com/l/cbCaibLwGVGJ3pZF","pdf",203972,1,11,"English","en",105,"# Abstract\n# Introduction\n# Related Work\n# Methodology and Simulation Framework\n# Experiments and Results\n# Ablation Studies\n# Discussion and Conclusions","[{\"question\":\"What problem does the paper target in encrypted Transformer inference?\",\"answer\":\"The paper targets the high cost and error growth caused by sequential composition of many nonlinear Transformer blocks under FHE.\"},{\"question\":\"How does Structured Newton Layer Parallelism (SNLP) reduce encrypted inference cost?\",\"answer\":\"SNLP changes the inter-layer computation graph by evaluating suffix layers in parallel over K solver iterations and applying structured Newton-style corrections, rather than chaining all nonlinear block evaluations sequentially.\"},{\"question\":\"What do the experiments show about error and bootstrapping under SNLP versus sequential inference?\",\"answer\":\"Across models, SNLP shows lower error amplification than sequential inference; on a 0.5B IDN-trained model it uses 2.65× fewer bootstraps (20 vs. 53) and achieves encrypted PPL within 1.2% of sequential.\"}]",1784205415,28,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"layer-parallel-inference-reduces-encrypted-nonlinear-depth-in-transformers","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/layer-parallel-inference-reduces-encrypted-nonlinear-depth-in-transformers/85660/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does the paper target in encrypted Transformer inference?","Question",{"text":75,"@type":76},"The paper targets the high cost and error growth caused by sequential composition of many nonlinear Transformer blocks under FHE.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does Structured Newton Layer Parallelism (SNLP) reduce encrypted inference cost?",{"text":80,"@type":76},"SNLP changes the inter-layer computation graph by evaluating suffix layers in parallel over K solver iterations and applying structured Newton-style corrections, rather than chaining all nonlinear block evaluations sequentially.",{"name":82,"@type":73,"acceptedAnswer":83},"What do the experiments show about error and bootstrapping under SNLP versus sequential inference?",{"text":84,"@type":76},"Across models, SNLP shows lower error amplification than sequential inference; on a 0.5B IDN-trained model it uses 2.65× fewer bootstraps (20 vs. 53) and achieves encrypted PPL within 1.2% of sequential.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,128,131,135],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":124,"doc_module":4,"doc_module_name":45,"category_name":125,"show_sort_weight":126,"slug":127},9,"Religion & Spirituality",20,"religion-spirituality",{"id":126,"doc_module":4,"doc_module_name":45,"category_name":129,"show_sort_weight":126,"slug":130},"World Cup","world-cup",{"id":132,"doc_module":4,"doc_module_name":45,"category_name":133,"show_sort_weight":132,"slug":134},10,"Lifestyle","lifestyle",{"id":136,"doc_module":4,"doc_module_name":45,"category_name":137,"show_sort_weight":106,"slug":138},19,"General","general"]