Who Responds When the Driver Is Gone A Framework for Human Intent Understanding

Jul 16, 2026

Upload by Aria

As autonomous vehicles move toward fully driverless mobility, a key question emerges: who understands and responds to passengers when the human driver is absent. The document introduces Intent2Drive, a unified framework for holistic human intent understanding and human-aligned planning. It models passenger intent as a latent cognitive state shaped by language, personal attributes, emotional/physical conditions, behavioral signals, and situational context. A Holistic Intent Dataset and a theory-of-mind-inspired reasoner infer a latent human state, which is converted into a planning-ready human intent objective for a hierarchical planner.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages11

Reading Time28 min

Introduction

Problem: passenger response without a human driver

Limitations of existing machine-centered intent representations

Theory-of-Mind framing for intent inference

Proposed approach: Intent2Drive

Holistic Intent Dataset (HID)

Human Intent Reasoner (HIR) and latent human state

Hierarchical Intent-Conditioned Planner (HICP)

Experimental results and outcomes

Who Responds When the Driver Is Gone? A Framework for Human Intent Understanding Xuewen Luo, Ding Fan, Ruiqi Chen, Ye Cao, Xiujin Liu, Bo Yu, Fengze Yang, Chenxi Liu University of Utah, Monash University arXiv :2607 .04670v 1 [ cs .HC] 6 Jul 2026 Abstract As autonomous vehicles progress toward fully driverless mobility, a critical question emerges: who understands and responds to passengers when the human driver is absent? Existing autonomous driving systems primarily optimize predefined navigation and control objectives from external scene observations, but they remain limited in perceiving and reasoning about in-cabin human intent. In this paper, we propose Intent2Drive, a unified framework for holistic human intent understanding and human-aligned planning. Instead of treating passenger intent as explicit commands alone, Intent2Drive models intent as a latent cognitive state shaped by language, personal attributes, emotional and physical conditions, behavioral signals, and situational context. To support this formulation, we construct a Holistic Intent Dataset (HID) that provides structured supervision over both explicit and implicit intent cues. Built upon HID, our Theoryof-Mind-inspired Human Intent Reasoner (HIR) infers a Latent Human State (LHS) and further translates it into a planner-compatible Human Intent Objective (HIO) . We then introduce a Hierarchical Intent-Conditioned Planner (HICP) that incorporates HIO into route-level and trajectory-level planning, enabling driving behaviors to remain aligned with passenger needs across different planning horizons. Extensive experiments show that Intent2Drive improves structured human intent inference and HIO construction while preserving competitive closed-loop planning performance. These results demonstrate a promising step toward passenger-responsive autonomous driving systems that can reason about, interpret, and act upon human intent in driverless mobility. Introduction Autonomous driving (AD) has achieved remarkable progress in perception, reasoning, and planning, bringing large-scale real-world deployment increasingly within reach (Chen et al. 2024; Zhao et al. 2025; Ding et al. 2026) . However, as autonomous vehicles move from driver-assistance systems toward fully driverless services such as robotaxis, the absence of a human driver raises a new challenge beyond navigation and control: who understands and responds to the passenger when no driver is present? In conventional driving, the driver does more than operate the vehicle. They Copyright © 2026, Association for the Advancement of Artificial Intelligence ([www.aaai.org](www.aaai.org)). All rights reserved. continuously observe, interpret, and respond to passengers’needs, discomfort, hesitation, and situational intent, such as whether a passenger wants to stop, change the destination or request assistance. This passenger-responsive role, though often implicit in human driving, becomes a critical missing capability in driverless mobility. Existing AD systems, however, are still largely designed as machine-centered closed-loop systems that optimize predefined driving objectives based primarily on external sensor observations (Cui et al. 2024) . While such systems are increasingly capable of perceiving traffic scenes and executing safe maneuvers, they remain limited in their ability to perceive, interpret, and respond to in-cabin human intent. Bridging this gap requires AD systems to move beyond objective optimization toward human-centered cooperative intelligence, where vehicles can understand human needs, provide timely feedback, and adapt their behavior accordingly (Huang et al. 2025) . Such a human-in-the-loop perspective introduces an essential cognitive dimension to autonomy, improving interpretability, safety, and societal trust in future driverless transportation systems (Wu et al. 2021; Huang et al. 2025; Luo et al. 2025) . However, existing human-in-the-loop AD approaches remain limite

FAQs

What challenge does driverless mobility introduce regarding passenger interaction?

It raises the question of who can understand and respond to passenger needs when no human driver is present, going beyond navigation and low-level control.

How does Intent2Drive represent passenger intent compared with prior methods?

Instead of treating intent as only explicit commands or direct language-to-action instructions, it models intent as a latent cognitive state influenced by multimodal cues and context.

What components does the framework use to turn intent into driving behavior?

It uses a Holistic Intent Dataset for supervision, a theory-of-mind-inspired Human Intent Reasoner to infer a latent human state, and a Hierarchical Intent-Conditioned Planner that converts the result into planning-compatible objectives across horizons.

Jul 16, 2026

Upload by Aria