Toward a Unified GPU-Aware OpenSHMEM Specification

Jul 16, 2026

Upload by Riley

Leadership-class HPC systems are increasingly accelerator-centric, with GPUs delivering most computation and memory bandwidth. As accelerators become integrated through high-speed fabrics and interconnects, PGAS models like OpenSHMEM help express fine-grained remote memory operations across tightly coupled device domains. OpenSHMEM 1.x is lean for irregular communication, atomics, synchronization, and collectives, but its memory model lacks accelerator-portable semantics, fragmenting implementations and coherence across vendors. This work proposes a backward-compatible OpenSHMEM Auxiliary Specification adding GPU-scoped memory semantics, capability queries, and defined remote access, atomic, synchronization, and collective behavior for GPU-attached buffers.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages15

Reading Time38 min

Abstract

Motivation for GPU-aware PGAS

Proposed OpenSHMEM Auxiliary Specification

GPU-scoped memory semantics and operation rules

Goals: portability and unification

Toward a Unified GPU-Aware OpenSHMEM Specification Naveen Ravi Nathan Wichmann Md. Wasi-ur-Rahman [nravi@hpe.com](nravi@hpe.com)[ ](nravi@hpe.com)[wichmann@hpe.com](wichmann@hpe.com)[ ](wichmann@hpe.com)[md.rahman@hpe.com](md.rahman@hpe.com)[ ](md.rahman@hpe.com)Hewlett Packard Enterprise (HPE) Bloomington, Minnesota, USA Aurelien Bouteiller Yıltan Hassan Temuçin Avinash Kethineedi Johnathan Alsop Brandon Potter [Aurelien.Bouteiller@amd.com](Aurelien.Bouteiller@amd.com)[yiltan@amd.com](yiltan@amd.com)[avinash.kethineedi@amd.com](avinash.kethineedi@amd.com)[ ](avinash.kethineedi@amd.com)[Johnathan.Alsop@amd.com](Johnathan.Alsop@amd.com)[ ](Johnathan.Alsop@amd.com)[Brandon.Potter@amd.com](Brandon.Potter@amd.com)[ ](Brandon.Potter@amd.com)[Advanced Micro Devices](Advanced Micro Devices), [Inc](Inc). Austin, Texas, USA Shubhendra Pal Singhal Jun Shirako Akihiro Hayashi Vivek Sarkar [ssinghal74@gatech.edu](ssinghal74@gatech.edu)[ ](ssinghal74@gatech.edu)[shirako@gatech.edu](shirako@gatech.edu)[ ](shirako@gatech.edu)[ahayashi@gatech.edu](ahayashi@gatech.edu)[vsarkar@gatech.edu](vsarkar@gatech.edu)[ ](vsarkar@gatech.edu)Georgia Institute of Technology Atlanta, Georgia, USA arXiv :2607 .08006v 1 [ cs .DC] 9 Jul 2026 Lawrence C. Stewart [stewart@serissa.com](stewart@serissa.com)[ ](stewart@serissa.com)Serissa Research Wayland, Massachusetts, USA Michael Beebe [michael.beebe@ttu.edu](michael.beebe@ttu.edu)[ ](michael.beebe@ttu.edu)Texas Tech University Lubbock, Texas, USA Benjamin Michalowicz [michalowicz.2@osu.edu](michalowicz.2@osu.edu)[ ](michalowicz.2@osu.edu)The Ohio State University Columbus, Ohio, USA Jeongnim Kim Thiago Teixeria Mark F. Brown [jeongnim.kim@intel.com](jeongnim.kim@intel.com)[ ](jeongnim.kim@intel.com)[thiago.teixeira@intel.com](thiago.teixeira@intel.com)[ ](thiago.teixeira@intel.com)[mark.f.brown@intel.com](mark.f.brown@intel.com)[ ](mark.f.brown@intel.com)Intel Corporation Hillsboro, Oregon, USA Aaron Welch Oscar Hernandez [welchda@ornl.gov](welchda@ornl.gov)[ ](welchda@ornl.gov)[oscar@ornl.gov](oscar@ornl.gov)[ ](oscar@ornl.gov)Oak Ridge National Laboratory Oak Ridge, Tennessee, USA Wendy Poole Steve Poole [wkpoole@lanl.gov](wkpoole@lanl.gov)[ ](wkpoole@lanl.gov)[swpoole@lanl.gov](swpoole@lanl.gov)[ ](swpoole@lanl.gov)Los Alamos National Laboratory Los Alamos, New Mexico, USA Abstract Leadership-class HPC systems are now accelerator-centric, with GPUs providing most floating-point throughput and memory bandwidth. As next-generation systems increasingly integrate accelerators through high-speed memory fabrics and system interconnects, exposing larger tightly coupled device domains, partitioned global address space (PGAS) models such as OpenSHMEM provide a natural abstraction for expressing fine-grained remote memory operations across these devices. While OpenSHMEM 1.x offers a lean PGAS model for irregular communication, atomics, finegrained synchronization, and collectives, its memory model lacks portable semantics for accelerator architectures. As a result, existing GPU-enabled OpenSHMEM implementations differ in memory Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission [and/or a fee. Request permissions from permissions@acm.org](and/or a fee. Request permissions from permissions@acm.org). CUG’26, Nice, France © 2026 Copyright held by the owner/author(s) . Publication rights licensed to ACM. ACM ISBN 978-x-xxxx-xxxx-x/YYYY/MM [https://doi.org/10.1145/nnnnnnn.nnnnnnn](https://doi.org/10.1

FAQs

Why does OpenSHMEM 1.x need an auxiliary specification for GPU-aware behavior?

Its memory model lacks portable semantics for accelerator architectures, so GPU-enabled implementations diverge in management, capability discovery, and operation semantics, reducing portability and community unification.

What does the proposed OpenSHMEM Auxiliary Specification add?

A lightweight, backward-compatible extension that introduces a minimal memory model via a GPU-scoped memory space abstraction, plus capability queries and well-defined semantics for using GPU-attached buffers in remote access, atomic, synchronization, and collective operations.

How is the specification intended to be supported across devices and vendors?

It is initially conceived through a host-initiated interface, while providing general semantics that can also enable optional device-initiated support, aiming to specify and implement unified GPU-aware semantics across GPUs from multiple vendors.

Jul 16, 2026

Upload by Riley