[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"doc-detail-85472-en":3,"doc-seo-85472-105":30,"detail-sidebar-cat-0-en-105":91},{"code":4,"msg":5,"data":6},0,"success",{"doc_id":7,"user_id":8,"nickname":9,"user_avatar":10,"doc_module":4,"category_id":11,"category_name":12,"doc_title":13,"doc_description":14,"doc_content":15,"file_id":16,"file_url":17,"file_type":18,"file_size":19,"view_count":20,"is_deleted":4,"is_public":21,"is_downloadable":21,"audit_status":21,"page_count":22,"language":23,"language_code":24,"site_id":25,"html_lang":24,"table_of_contents":26,"faqs":27,"seo_title":13,"seo_description":14,"update_tm":28,"read_time":29},85472,962075006959,"Anda","https://ap-avatar.wpscdn.com/avatar/e0002397efbe92a78e?_k=1776741047341049297",8,"Research & Report","CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning","Matrix multiplication (matmul) is a core operation in LLM workloads, yet kernel optimization is difficult because different (M, N, K) shapes need different strategies and optimizations do not transfer well across GPU architectures. CUDA-L2 combines LLMs and reinforcement learning to automatically optimize Half-precision General Matrix Multiply (HGEMM) CUDA kernels using CUDA execution speed as reward, covering all 1,000 relevant (M, N, K) configurations and accelerating performance versus torch.matmul and cuBLAS/cuBLASLt baselines in both offline and server scenarios.","arXiv :2512 .0255 1v 3 [ cs .LG] 13 Jul 2026  \nCUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication  \nthrough Reinforcement Learning  \nSongqiao Su, Xiaoya Li, Albert Wang, Guoyin Wang, Jiwei Li and Chris Shum  \nDeepReinforce Team  \n [github.com/deepreinforce-ai/CUDA-L2](github.com/deepreinforce-ai/CUDA-L2)  \nAbstract  \nMatrix multiplication (matmul) is one of the most fundamental operations in LLMs. However, manually optimizing Matmul kernels is challenging due to the fact that different matrix dimension (M, N, K) require different optimization strategies and that optimizations rarely transform across different GPU architectures, which make comprehensive manual tuning hard at scale. In this paper, we propose CUDA-L2, a system that combines large language models (LLMs) and reinforcement learning (RL) to automatically optimize Half-precision General Matrix Multiply (HGEMM) CUDA kernels. Using CUDA execution speed as the RL reward, CUDA-L2 automatically optimizes HGEMM kernels across 1,000 configurations. These configurations represent all 103 combinations of M, N, K values from {64, 128, 256, 512, 1024, 2048, 4096, 8192, 12288, 16384}, and already covers those used in attention and FFN layers of widely open-sourced models like Qwen, Llama and DeepSeek.  \nCUDA-L2 systematically outperforms major matmul baselines to date, from the widely-used torch.matmul to state-of-the-art Nvidia’s closed-source libraries, i.e., cuBLAS, cuBLASLt. In offline mode, where kernels are executed consecutively without time intervals, CUDA-L2 yields +22.0% over torch.matmul on average; +19.2% over cuBLAS using the optimal layout configuration (normal-normal NN and transposed-normal TN); +16.8% over cuBLASLt-heuristic, which queries cuBLASLt library and selects the algorithm based on the heuristic’s suggestion; and +11.4% over the most competitive cuBLASLt-AutoTuning model, which selects the fastest algorithm from up to 100 candidates from cuBLASLt’s suggestions. In server mode, where kernels are executed at random intervals simulating real-time inference, the speedups further increase to +28.7%, +26.0%, +22.4%, and +15.9% for torch.matmul, cuBLAS, cuBLASLt-heuristic, and cuBLASLt-AutoTuning respectively.  \nCUDA-L2 shows that even the most performance-critical, heavily-optimized kernels like HGEMM can be improved through LLM-guided RL automation by systematically exploring configuration spaces at scales impractical for humans. While the current version of CUDA-L2 only focuses on A100 GPUs, the framework is designed for broad applicability, with ongoing work to extend it to other GPU architectures, including Ada Lovelace, Hopper and Blackwell. B  \n torch.matmul  cuBLAS  cuBLASLt-heuristic  cuBLASLt-AutoTuning  \nSpeed Comparison (%)  \n50%  \n40%  \n30%  \n20%  \n10%  \n\n|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |\n| +22.0% |  | +20.0%+21.4%\u003Cbr>+19.2%+17.3%\u003Cbr>+19.1%\u003Cbr>.8% |  |  |  |  |  |  |  |  |  |  |  |  |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |\n|  |  |  |  |  | +16 |  |  |  |  | +12.1%\u003Cbr>+13.3% |  |  | +11.4% |  |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |\n\ntorch matmul  \nNN TN max NN TN max NN TN max  \n(a) Offline  \nSpeed Comparison (%)  \n50%  \n40%  \n30%  \n20%  \n10%  \n\n|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| +28.7%+28.8%\u003Cbr>+30.2% |  |  |  |  | 0% | 9% |  |  |  |  |  |  |  |  |\n|  |  |  |  |  | +26 . | +24.4%\u003Cbr>+25 . |  |  | +22.4% | +17.9%+19.1%\u003Cbr>9% |  |  |  |  |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |\n|  |  |  |  |  |  |  |  |  |  |  |  |  | +15 . |  |\n|  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |","cbCaiaicIyHUzqpE","https://ap.wps.com/l/cbCaiaicIyHUzqpE","pdf",694039,3,1,16,"English","en",105,"# Abstract\n# Introduction\n## Problem: matmul optimization challenges\n## Approach: LLM-guided RL for CUDA kernels\n## Performance evaluation: offline vs server\n# Related concepts and setup: configurations, layouts, and benchmarks","[{\"question\":\"What problem does CUDA-L2 address in matrix multiplication kernel optimization?\",\"answer\":\"CUDA-L2 targets the difficulty of manually tuning matmul kernels across many (M, N, K) configurations and the fact that optimizations often do not transfer across GPU architectures or even between configurations on the same GPU.\"},{\"question\":\"How does CUDA-L2 train and decide kernel configurations?\",\"answer\":\"CUDA-L2 uses reinforcement learning with CUDA execution speed as the reward signal, automatically exploring kernel configurations for HGEMM to find faster options.\"},{\"question\":\"How does CUDA-L2 perform compared with torch.matmul and cuBLAS/cuBLASLt?\",\"answer\":\"In offline execution, CUDA-L2 delivers +22.0% over torch.matmul on average, and also improves over cuBLAS and cuBLASLt variants; in server-mode randomized timing, speedups increase further to +28.7%, +26.0%, +22.4%, and +15.9% respectively.\"}]",1784203845,40,{"code":4,"msg":31,"data":32},"ok",{"site_id":25,"language":24,"slug":33,"title":13,"keywords":34,"description":14,"schema_data":35,"social_meta":86,"head_meta":88,"extra_data":90,"updated_unix":28},"cuda-l2-surpassing-cublas-performance-for-matrix-multiplication-through-reinforcement-learning","",{"@graph":36,"@context":85},[37,53,68],{"@type":38,"itemListElement":39},"BreadcrumbList",[40,44,48,50],{"item":41,"name":42,"@type":43,"position":21},"https://docshare.wps.com","Home","ListItem",{"item":45,"name":46,"@type":43,"position":47},"https://docshare.wps.com/document/","Document",2,{"item":49,"name":12,"@type":43,"position":20},"https://docshare.wps.com/document/research-report/",{"item":51,"name":13,"@type":43,"position":52},"https://docshare.wps.com/document/cuda-l2-surpassing-cublas-performance-for-matrix-multiplication-through-reinforcement-learning/85472/",4,{"url":51,"name":13,"@type":54,"author":55,"headline":13,"publisher":57,"fileFormat":60,"inLanguage":24,"description":14,"dateModified":61,"datePublished":62,"encodingFormat":60,"isAccessibleForFree":63,"interactionStatistic":64},"DigitalDocument",{"name":9,"@type":56},"Person",{"url":41,"name":58,"@type":59},"DocShare","Organization","application/pdf","2026-07-25","2026-07-16",true,{"@type":65,"interactionType":66,"userInteractionCount":20},"InteractionCounter",{"@type":67},"ViewAction",{"@type":69,"mainEntity":70},"FAQPage",[71,77,81],{"name":72,"@type":73,"acceptedAnswer":74},"What problem does CUDA-L2 address in matrix multiplication kernel optimization?","Question",{"text":75,"@type":76},"CUDA-L2 targets the difficulty of manually tuning matmul kernels across many (M, N, K) configurations and the fact that optimizations often do not transfer across GPU architectures or even between configurations on the same GPU.","Answer",{"name":78,"@type":73,"acceptedAnswer":79},"How does CUDA-L2 train and decide kernel configurations?",{"text":80,"@type":76},"CUDA-L2 uses reinforcement learning with CUDA execution speed as the reward signal, automatically exploring kernel configurations for HGEMM to find faster options.",{"name":82,"@type":73,"acceptedAnswer":83},"How does CUDA-L2 perform compared with torch.matmul and cuBLAS/cuBLASLt?",{"text":84,"@type":76},"In offline execution, CUDA-L2 delivers +22.0% over torch.matmul on average, and also improves over cuBLAS and cuBLASLt variants; in server-mode randomized timing, speedups increase further to +28.7%, +26.0%, +22.4%, and +15.9% respectively.","https://schema.org",{"og:url":51,"og:type":87,"og:title":13,"og:site_name":58,"og:description":14},"article",{"robots":89,"canonical":51},"index,follow",{"doc_id":7,"site_id":25},{"code":4,"msg":5,"data":92},[93,97,101,105,110,115,119,122,127,130,134],{"id":21,"doc_module":4,"doc_module_name":46,"category_name":94,"show_sort_weight":95,"slug":96},"Story & Novel",90,"story-novel",{"id":47,"doc_module":4,"doc_module_name":46,"category_name":98,"show_sort_weight":99,"slug":100},"Literature",80,"literature",{"id":52,"doc_module":4,"doc_module_name":46,"category_name":102,"show_sort_weight":103,"slug":104},"Exam",70,"exam",{"id":106,"doc_module":4,"doc_module_name":46,"category_name":107,"show_sort_weight":108,"slug":109},5,"Comic",60,"comic",{"id":111,"doc_module":4,"doc_module_name":46,"category_name":112,"show_sort_weight":113,"slug":114},6,"Technology",50,"technology",{"id":116,"doc_module":4,"doc_module_name":46,"category_name":117,"show_sort_weight":29,"slug":118},7,"Healthcare","healthcare",{"id":11,"doc_module":4,"doc_module_name":46,"category_name":12,"show_sort_weight":120,"slug":121},30,"research-report",{"id":123,"doc_module":4,"doc_module_name":46,"category_name":124,"show_sort_weight":125,"slug":126},9,"Religion & Spirituality",20,"religion-spirituality",{"id":125,"doc_module":4,"doc_module_name":46,"category_name":128,"show_sort_weight":125,"slug":129},"World Cup","world-cup",{"id":131,"doc_module":4,"doc_module_name":46,"category_name":132,"show_sort_weight":131,"slug":133},10,"Lifestyle","lifestyle",{"id":135,"doc_module":4,"doc_module_name":46,"category_name":136,"show_sort_weight":106,"slug":137},19,"General","general"]