GPU Tile Sim A Tile-Centric GPU Simulation Framework for LLM Hardware-Software Co-Design

Jul 16, 2026

Modern LLM workloads increasingly depend on optimized GPU kernels created through hardware-software co-design. Achieving performance requires fine-grained dependency scheduling and overlap between computation and memory movement, which makes existing GPU performance models harder to use. Instruction-driven simulators are expensive to adapt, while analytical models are too coarse. GPU-Tile-Sim (GTSim) models kernel execution using a warp-level tile graph, capturing tile operations and explicit dataflow and ordering constraints. Evaluations on GEMM, attention, and end-to-end inference show high modeling accuracy across A100 and H100, and early validation on Blackwell.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages14

Reading Time35 min

Abstract

Keywords

Introduction

GPU-Tile-Sim: A Tile-Centric GPU Simulation Framework for LLM Hardware-Software Co-Design Yitong Ding Shanghai Jiao Tong University China [dingyitong@sjtu.edu.cn](dingyitong@sjtu.edu.cn) Yangjie Zhou∗ National University of Singapore Singapore [yj_zhou@nus.edu.sg](yj_zhou@nus.edu.sg) Jiawei Huang Shanghai Jiao Tong University China [huangjiawei@sjtu.edu.cn](huangjiawei@sjtu.edu.cn) Zihan Liu∗ Shanghai Jiao Tong University China [altair.liu@sjtu.edu.cn](altair.liu@sjtu.edu.cn) Renyang Guan Shanghai Jiao Tong University China [guanrenyang@alumni.sjtu.edu.cn](guanrenyang@alumni.sjtu.edu.cn) Yu Feng Shanghai Jiao Tong University China [y-feng@sjtu.edu.cn](y-feng@sjtu.edu.cn) arXiv :2607 . 11262v1 [ cs .DC] 13 Jul 2026 Shixuan Sun Shanghai Jiao Tong University China [sunshixuan@sjtu.edu.cn](sunshixuan@sjtu.edu.cn) Mingyi Guo Shanghai Jiao Tong University China [guo-my@cs.sjtu.edu.cn](guo-my@cs.sjtu.edu.cn) Jingwen Leng Shanghai Jiao Tong University China [leng-jw@cs.sjtu.edu.cn](leng-jw@cs.sjtu.edu.cn) Jian Weng King Abdullah University of Science and Technology (KAUST) Saudi Arabia [jian.weng@kaust.edu.sa](jian.weng@kaust.edu.sa) Abstract Modern LLM (large language model) workloads increasingly rely on optimized GPU kernels through hardware-software co-design. These kernels achieve high-performance through fine-grained dependency scheduling and computation-memory overlap. As such, they incur new challenges on existing GPU performance models. Instruction-driven simulators are costly to adapt to evolving architectures, while analytical models are too coarse to capture kernels’characteristics. We propose GPU-Tile-Sim (GTSim), a tile-centric GPU simulation framework for LLM hardware-software co-design. The key insight is that modern LLM kernel performance is governed less by individual instruction latency than by the dependency structure that controls execution order and overlap. Accordingly, GTSim represents kernel execution as a warp-level tile graph whose nodes capture tile-level operations and whose edges encode data and ordering constraints. Using this representation, we design an automatic tile-graph frontend and a graph-driven simulation backend. We evaluate GTSim on representative GEMM, attention, and end-to-end LLM inference workloads. On A100 and H100 across both conventional and highly optimized kernels, GTSim achieves high performance-modeling accuracy (MAPE, Mean Absolute Percentage Error, 1.22%–8.71%). We further extend GTSim to Blackwell with preliminary validation, and demonstrate its effectiveness in analyzing software and architectural design choices. Keywords GPU simulation, LLM workloads, tile graph, hardware-software co-design, performance modeling ∗ Corresponding authors. 1 Introduction Modern GPUs are key platforms for LLM workload execution. To better exploit their peak performance, recent GPUs introduce features enabled by hardware-software co-design, such as Tensor Memory Accelerator, distributed share memory, and Tensor Memory [28– 30]. As Tensor Core throughput continues to increase and execution becomes coarser-grained, sustaining high utilization requires kernels to keep compute pipelines fed through tiling, kernel fusion, and multi-stage software pipelines that overlap data movement with matrix computation. At the same time, increasingly asynchronous data-movement and on-chip sharing mechanisms encourage programming styles based on producer–consumer coordination, warp specialization, and explicit data placement [7, 10, 22, 25, 36, 47]. As a result, performance increasingly depends on co-designing software execution with these evolving hardware mechanisms. To guide such hardware-software co-design, GPU performance models must predict how software and architectural choices interact to affect performance. Existing approaches fall into two broad categories: instruction-driven simulators and analytical models, with the latter further divided into interval-

FAQs

What problem does GPU-Tile-Sim (GTSim) address in LLM GPU hardware-software co-design?

GTSim targets the difficulty of accurately modeling how modern GPU kernels achieve performance through dependency scheduling and computation-memory overlap, which existing instruction-driven simulators and coarse analytical models struggle to capture efficiently.

How does GTSim represent GPU kernel execution?

GTSim represents execution as a dependency-driven warp-centric tile graph, where nodes correspond to tile-level operations executed by warps or cooperating warp groups and edges encode dataflow and ordering constraints.

What evidence of effectiveness does the document provide?

Experiments on GEMM, attention, and end-to-end LLM inference workloads show high performance-modeling accuracy on A100 and H100 across conventional and highly optimized kernels, measured by MAPE of 1.22%–8.71%, with additional preliminary validation on Blackwell.

Jul 16, 2026

Upload by Evangeline