Hierarchical Reinforcement Learning for Multi-Layer Multi-Service Non-Terrestrial Vehicular Edge Computing - Paper

Aug 5, 2026

Upload by Noah

Vehicular Edge Computing (VEC) struggles to meet escalating Vehicle User (VU) service demands when deployed only via Road Side Units, leading to insufficient computational and communication resources. Integrated Terrestrial and Non-Terrestrial (T-NT) edge computing within 6G expands coverage using onboard edge facilities on Non-Terrestrial Networks (NTN), but multi-node mobility creates high complexity. The work models latency and energy under partial computation offloading as a multi-layer MDP, then applies hierarchical reinforcement learning with a deep Q network for edge selection and offloading policies, validated through simulations against benchmarks.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages17

Reading Time43 min

AccessFree to download

Abstract

Index Terms

I. Introduction

Vehicle Edge Computing Challenges

Limitations of Terrestrial VEC

Role of Non-Terrestrial Networks in 6G

Received 18 December 2023; revised 16 May 2024 and 19 July 2024; accepted 22 July 2024 . Date of publication 25 July 2024; date of current version 31 July 2024 . The associate editor coordinating the review of this article and approving it for publication was H. Ko. Digital Object Identifier 10.1109/TMLCN.2024.3433620 Hierarchical Reinforcement Learning for Multi-Layer Multi-Service Non-Terrestrial Vehicular Edge Computing SWAPNIL SADASHIV SHINDE1,2 (Student Member, IEEE), AND DANIELE TARCHI2 (Senior Member, IEEE) 1 Consorzio Nazionale Interuniversitario delle Telecomunicazioni (CNIT), University of Bologna Research Unit, 40136 Bologna, Italy 2 Department of Electrical, Electronic and Information Engineering ‘‘Guglielmo Marconi,’’ University of Bologna, 40136 Bologna, Italy CORRESPONDING AUTHOR: D. TARCHI ([daniele.tarchi@unibo.it](daniele.tarchi@unibo.it)) This work was supported in part by European Commission under the ‘‘5G-STARDUST’’ Project, which received funding from the Smart Networks and Services Joint Undertaking (SNS JU) under the European Union’s Horizon Europe Research and Innovation Programme under Grant 101096479; in part by the European Union under the Italian National Recovery and Resilience Plan (NRRP) of NextGenerationEU, partnership on ‘‘Telecommunications of the Future under Grant PE00000001-program ‘‘RESTART’’; and in part by the Swiss State Secretariat for Education, Research and Innovation (SERI) . The views expressed are those of the authors and do not necessarily represent the project. The Commission is not liable for any use that may be made of any of the information contained therein. ABSTRACT Vehicular Edge Computing (VEC) represents a novel advancement within the Internet of Vehicles (IoV) . Despite its implementation through Road Side Units (RSUs), VEC frequently falls short of satisfying the escalating demands of Vehicle Users (VUs) for new services, necessitating supplementary computational and communication resources. Non-Terrestrial Networks (NTN) with onboard Edge Computing (EC) facilities are gaining a central place in the 6G vision, allowing one to extend future services also to uncovered areas. This scenario, composed of a multitude of VUs, terrestrial and non-terrestrial nodes, and characterized by mobility and stringent requirements, brings in a very high complexity. Machine Learning (ML) represents a perfect tool for solving these types of problems. Integrated Terrestrial and Nonterrestrial (T-NT) EC, supported by innovative intelligent solutions enabled through ML technology, can boost the VEC capacity, coverage range, and resource utilization. Therefore, by exploring the integrated T-NT EC platforms, we design a multi-EC-enabled vehicular networking platform with a heterogeneous set of services. Next, we model the latency and energy requirements for processing the VU tasks through partial computation offloading operations. We aim to optimize the overall latency and energy requirements for processing the VU data by selecting the appropriate edge nodes and the offloading amount. The problem is defined as a multi-layer sequential decision-making problem through the Markov Decision Processes (MDP) . The Hierarchical Reinforcement Learning (HRL) method, implemented through a Deep Q network, is used to optimize the network selection and offloading policies. Simulation results are compared with different benchmark methods to show performance gains in terms of overall cost requirements and reliability. INDEX TERMS Vehicular networks, edge computing, non-terrestrial networks, computation offloading, reinforcement learning. I. INTRODUCTION WITH the integration Edge Computing (EC), Vehic ular Networks (VNs) are rapidly converging into a highly reliable, intelligent, and complex network system that serves vehicular users (VUs) with new services and applications [1] . However, Vehicular Edge Computing (VEC), enabled by the deployment of roadside units (RSUs) along road ne

FAQs

Why does Vehicular Edge Computing using only roadside units fail to satisfy vehicle service demands?

Roadside-unit-based VEC faces increasing latency and data-rate requirements, limited coverage, fixed deployment constraints, vulnerability to disasters, and data security threats.

How does integrating non-terrestrial edge computing address coverage and capacity needs?

Non-terrestrial networks with onboard edge computing can extend services to uncovered areas and act as relay nodes to route computation loads toward ground cloud facilities.

What approach is used to optimize edge node selection and offloading decisions?

The problem is formulated as a multi-layer sequential decision-making task via Markov Decision Processes (MDP), and hierarchical reinforcement learning using a deep Q network optimizes network selection and offloading policies.

Aug 5, 2026

Upload by Noah