NVIDIA DGX A100 System Architecture - Technical White Paper - WP-10083-001 v01 - July 2020

Nov 2, 2026

A technical white paper describing the NVIDIA DGX A100 system architecture as a universal foundation for AI infrastructure. It explains why traditional compute is inefficient for AI workloads, then details the design built around the NVIDIA A100 Tensor Core GPU, including AI performance features, multi-GPU and multi-node scalability via DGX reference architectures, and high-throughput interconnects. The document also covers networking, PCIe Gen4 foundations, security mechanisms, the optimized DGX software stack, and performance results such as MLPerf v0.7 training records.

LanguageEnglish

CategoryTechnology

FormatPDF

Pages23

Reading Time58 min

AccessFree to download

1 Introduction

2 System Architecture

3 NVIDIA A100 GPU-8th Generation Data Center GPU for the Age of Elastic Computing

3.1 Third-Generation Tensor Cores

3.2 Tensor Float-32 (T F32) Uses Tensor Cores by Default

3.3 Fine-grained Structured Sparsity

3.4 Multi-Instance GPU (MIG)

4 Third-Generation NVLink and NVSwitch to Accelerate Large Complex Workloads

5 Highest Networking Throughput with Mellanox ConnectX-6

6 First Accelerated System With All PCIe Gen4

7 Security

7.1 Self-Encrypted Drives

7.2 Trusted Platform Module (T PM) Technology

8 Fully Optimized DGX Software Stack

9 Game Changing Performance

10 Breaking AI Performance Records for MLPerf v0.7 Training

11 Direct Access to NVIDIA DGX perts

12 Summary

13 Appendix: Graph Details

13.1 Details for Figure 7: Inference Throughput with MIG

13.2 Details for Figure 12: DGX A100 AI Training and Inference Performance

NVIDIA DGX A100 System Architecture The Universal System for AI Infrastructure Technical White Paper Contact Sky Blue Microsystems GmbH Geisenhausenerstr. 18 81379 Munich, Germany +49 89 780 2970, [info@skyblue.de](info@skyblue.de)[ ](info@skyblue.de)[www.skyblue.de](www.skyblue.de) In Great Britain: Zerif Technologies Ltd. Winnington House, 2 Woodberry Grove Finchley, London N12 0DR +44 115 855 7883, [info@zerif.co.uk](info@zerif.co.uk)[ ](info@zerif.co.uk)[www.zerif.co.uk](www.zerif.co.uk) WP-10083-001 _v01 | July 2020 Table of Contents 1 Introduction ............................................................................................................................. 1 2 System Architecture................................................................................................................ 2 3 NVIDIA A100 GPU-8th Generation Data Center GPU for the Age of Elastic Computing .... 3 3.1 Third-Generation Tensor Cores ...................................................................................... 3 3.2 Tensor Float-32 (T F32) Uses Tensor Cores by Default ................................................... 4 3.3 Fine-grained Structured Sparsity ................................................................................... 6 3.4 Multi-Instance GPU (MIG) ............................................................................................... 7 4 Third-Generation NVLink and NVSwitch to Accelerate Large Complex Workloads........... 10 5 Highest Networking Throughput with Mellanox ConnectX-6 .............................................. 11 6 First Accelerated System With All PCIe Gen4 ...................................................................... 12 7 Security.................................................................................................................................. 13 7.1 Self-Encrypted Drives ................................................................................................... 13 7.2 Trusted Platform Module (T PM) Technology ................................................................ 13 8 Fully Optimized DGX Software Stack .................................................................................... 14 9 Game Changing Performance .............................................................................................. 16 10 Breaking AI Performance Records for MLPerf v0.7 Training ........................................... 17 11 Direct Access to NVIDIA DGX perts ..................................................................................... 18 12 Summary ............................................................................................................................. 18 13 Appendix: Graph Details ..................................................................................................... 19 13.1 Details for Figure 7: Inference Throughput with MIG ............................................... 19 13.2 Details for Figure 12: DGX A100 AI Training and Inference Performance ................ 20 NVIDIA DGX A100 System Architecture WP-10083-001 _v01 | 1 1 Introduction Organizations of all kinds are incorporating AI into their research, development, product, and business processes. This helps them meet and exceed their particular goals, and also helps them gain experience and knowledge to take on even bigger challenges. However, traditional compute infrastructures are not suitable for AI due to slow CPU architectures and varying system requirements for different workloads and project phases. This drives up complexity, increases cost, and limits scale. To help organizations overcome these obstacles and succeed in a world that desperately needs the power of AI to solve big challenges, NVIDIA designed the world's first family of systems purpose-built for AI—NVIDIA DGX™ systems. By leveraging powerful NVIDIA GPUs and designing from the ground up for

FAQs

Why does the paper position the DGX A100 as a universal AI infrastructure system?

It argues that traditional compute infrastructures are unsuitable for AI because of slow CPU architectures and shifting requirements across workloads and project phases. The DGX family is designed to reduce integration complexity while delivering strong performance and scalable deployment options.

What are key hardware elements emphasized in the DGX A100 architecture?

The paper highlights the NVIDIA A100 Tensor Core GPU as the core compute engine, along with features such as high-bandwidth HBM2 memory, third-generation tensor cores, structured sparsity, and Multi-Instance GPU (MIG). It also covers system acceleration through NVLink/NVSwitch and high networking throughput.

What performance and validation topics does the document include?

It discusses performance claims and includes a section on breaking AI performance records for MLPerf v0.7 training. It also references figure-level details related to inference throughput with MIG and training/inference performance.

Nov 2, 2026

Uploaded by cimolpedes