[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-84951-en":3,"doc-seo-84951-105":29,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":20,"is_downloadable":20,"audit_status":20,"page_count":21,"language":22,"language_code":23,"site_id":24,"html_lang":23,"table_of_contents":25,"faqs":26,"seo_title":13,"seo_description":14,"update_tm":27,"read_time":28},84951,7971461740886,"Theodore","https://ap-avatar.wpscdn.com/davatar_3d24733baf745e90a7e4bdd5f77d97b2",8,"Research & Report","Multiple Double Arithmetic on NVIDIA Tensor Cores","A multiple double is an unevaluated sum of nonoverlapping doubles, used to extend numerical accuracy beyond standard 64-bit floating-point. NVIDIA tensor cores are specialized matrix-multiplication hardware, introduced on Ampere A100 with IEEE-compliant FP64 instructions that deliver higher peak performance. Multiple double arithmetic requires renormalization after each operation, which introduces branching that tensor cores are not designed to support. The work applies an Ozaki-scheme-like approach to avoid this bottleneck, enabling GPU-oriented execution, with software released under the GPU GPL license.","arXiv :2607 .0688 1v 1 [ cs .MS] 8 Jul 2026  \nMultiple Double Arithmetic on NVIDIA Tensor Cores  \nHoward Chen∗ Jan Verschelde†  \n1 July 2026  \nAbstract  \nA multiple double is an unevaluated sum of doubles. An NVIDIA tensor core is a specialized high performance compute core for matrix multiplication. The Ampere A100, released in 2020, introduced tensor cores capable of 64-bit floating-point arithmetic. Every multiple double arithmetical operation requires renormalization, which involves branching, for which tensor cores are unsuited.  \nTo solve this problem caused by renormalization, we apply a solution similar to the Ozaki scheme [Ozaki et al, Numerical Algorithms, 2012] . Our software is available under the GPU GPL license on github.  \n1 Introduction  \nAn algorithm is robust if it does not fail for small perturbations of degenerate inputs. With multiple double arithmetic we can obtain more accuracy than what can be provided with 64-bit doubles.  \nDefinition 1.1 (multiple double) A multiple double is an unevaluated sum of nonoverlapping doubles.  \nThe algorithms for multiple double arithmetic originated in the late sixties, in efforts [2] to extend the accuracy of the then commonly used 32-bit floating-point arithmetic. In [3] and [15], algorithms are defined for quad double arithmetic, and also described in [10] . QDlib [3] and CAMPARY [5] are two software packages for quad double and general multiple double arithmetic respectively.  \nIf the result of a computation can be represented exactly by a 64-bit double, then the next double in the sequence of multiple doubles represents the working precision in which the computation was executed. This property justifies the term error-free transformations [14] when referring to multiple double arithmetic. The drawback of working with multiple double arithmetic is its cost overhead. For instance, in order to compensate for the cost of quad double arithmetic, one needs to run computations with teraflop performance.  \nIn order to offset the cost of multiple double algorithm, we examine the application of Graphics Processing Units (GPUs), and in particular in this paper, NVIDIA Tensor Cores.  \n∗ University of Illinois at Chicago, Department of Mathematics, Statistics, and Computer Science, 851 S. Morgan St. (m/c 249), Chicago, IL 60607-7045, Email: [hchen221@uic.edu](hchen221@uic.edu).  \n†University of Illinois at Chicago, Department of Mathematics, Statistics, and Computer Science, 851 S.  \nMorgan St. (m/c 249), Chicago, IL 60607-7045, Email: [janv@uic.edu](janv@uic.edu), URL: [http://www.math.uic.edu/](http://www.math.uic.edu/)∼jan.  \nDefinition 1.2 (NVIDIA Tensor Core) An NVIDIA Tensor Core is a specialized high performance compute core for matrix multiplication.  \nIntroduced in 2020, the NVIDIA “Ampere” A100 Tensor Cores offer IEEE-compliant 64-bit floating-point (FP64) tensor core instructions [12] . An appealing aspect of tensor cores are their increased performance, relative to the regular cores. In particular, the theoretical peak performance of the FP64 (non-tensor) cores is 9.7 TFLOPS (teraflops), whereas 19.5 TFLOPS is the theoretical peak performance of the FP64 tensor cores.  \nThen our question becomes: Are tensor cores useful for multiple double arithmetic? At first sight, the answer is simply no because of the following.  \nProblem 1.1 (renormalization) Multiple double arithmetic requires a renormalization after every arithmetical operation. This renormalization involves branching and tensor cores are specialized for matrix multiplication.  \nOur solution is similar to the application of the Ozaki scheme [13], used in mixed precision computations [1], [4], [6], [9] . In mixed precision, accurate results are computed via lower precision arithmetic. Multiple double arithmetic is similar, using double precision arithmetic to multiple the accuracy of the results. Double precision arithmetic has the benefit of a wider range of exponents compared to what is available on lower prec","cbCaikCHQzcXbXDv","https://ap.wps.com/l/cbCaikCHQzcXbXDv","pdf",449280,1,9,"English","en",105,"# Introduction\n## Multiple double definition and accuracy\n## Renormalization challenge\n## GPU and tensor core motivation\n# Rewriting Products of Double Double Matrices","[{\"question\":\"What is multiple double arithmetic and why is it used?\",\"answer\":\"Multiple double arithmetic represents numbers as an unevaluated sum of nonoverlapping doubles to obtain accuracy beyond what 64-bit doubles can provide. Robustness against small perturbations is emphasized as a motivation for more accurate computations.\"},{\"question\":\"Why are NVIDIA tensor cores not suited to multiple double arithmetic directly?\",\"answer\":\"Each multiple double operation needs renormalization after arithmetic. Renormalization involves branching, while tensor cores are specialized for matrix multiplication and do not handle branching efficiently.\"},{\"question\":\"How does the paper address the renormalization problem on GPUs?\",\"answer\":\"It applies a strategy similar to the Ozaki scheme to restructure the computation to be more compatible with the hardware constraints. The paper also discusses related branch-free extended-precision approaches and explains why existing ones may not map well to tensor cores.\"}]",1784199656,23,{"code":4,"msg":30,"data":31},"ok",{"site_id":24,"language":23,"slug":32,"title":13,"keywords":33,"description":14,"schema_data":34,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":27},"multiple-double-arithmetic-on-nvidia-tensor-cores","",{"@graph":35,"@context":85},[36,53,68],{"@type":37,"itemListElement":38},"BreadcrumbList",[39,43,47,50],{"item":40,"name":41,"@type":42,"position":20},"https://docshare.wps.com","Home","ListItem",{"item":44,"name":45,"@type":42,"position":46},"https://docshare.wps.com/document/","Document",2,{"item":48,"name":12,"@type":42,"position":49},"https://docshare.wps.com/document/research-report/",3,{"item":51,"name":13,"@type":42,"position":52},"https://docshare.wps.com/document/multiple-double-arithmetic-on-nvidia-tensor-cores/84951/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":23,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":40,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-17","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What is multiple double arithmetic and why is it used?","Question",{"text":75,"@type":76},"Multiple double arithmetic represents numbers as an unevaluated sum of nonoverlapping doubles to obtain accuracy beyond what 64-bit doubles can provide. Robustness against small perturbations is emphasized as a motivation for more accurate computations.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"Why are NVIDIA tensor cores not suited to multiple double arithmetic directly?",{"text":80,"@type":76},"Each multiple double operation needs renormalization after arithmetic. Renormalization involves branching, while tensor cores are specialized for matrix multiplication and do not handle branching efficiently.",{"name":82,"@type":73,"acceptedAnswer":83},"How does the paper address the renormalization problem on GPUs?",{"text":84,"@type":76},"It applies a strategy similar to the Ozaki scheme to restructure the computation to be more compatible with the hardware constraints. The paper also discusses related branch-free extended-precision approaches and explains why existing ones may not map well to tensor cores.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":24},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,120,123,127,130,134],{"id":20,"doc_module":4,"doc_module_name":45,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":46,"doc_module":4,"doc_module_name":45,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":45,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":45,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":45,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":45,"category_name":117,"show_sort_weight":118,"slug":119},7,"Healthcare",40,"healthcare",{"id":11,"doc_module":4,"doc_module_name":45,"category_name":12,"show_sort_weight":121,"slug":122},30,"research-report",{"id":21,"doc_module":4,"doc_module_name":45,"category_name":124,"show_sort_weight":125,"slug":126},"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":45,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":45,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":45,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]