NormAct: A Benchmark for Hidden Social Norm Compliance in Embodied Planning

Jul 16, 2026

Upload by Eliana

Multimodal large language models increasingly act as embodied planners in egocentric environments, where success depends on executing instructed goals and behaving in socially appropriate ways. Explicit goals can be optimized, yet implicit social norms create hidden constraints that are rarely evaluated in action-level planning. NormAct is introduced to measure Goal Achievement, Norm Compliance, and overall Task Success by embedding hidden norms within ordinary tasks. Experiments show a large gap between explicit goal achievement and hidden-norm compliance, and a cue-conditioned NormPerceptor improves grounded planning performance.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages18

Reading Time45 min

Abstract

1 Introduction

Evaluation metrics and task setup

NormAct: A Benchmark for Hidden Social Norm Compliance in Embodied Planning Shiyun Zhao1 , Xinwei Song1,3 , Tianyu Guo1 , Xiaomeng Gao1 , Mingyuan Liu1 , Xu Han2 , Yuanyuan Zhang2 , Zhenliang Zhang 1, * , Xue Feng1, * , Bo Dai1, * 1 State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI), 2 China Academy of Information and Communications Technology, 3 ShanghaiTech University Correspondence: Correspondence: zlzhang@bigai.ai, [feng.xue1580@gmail.com](feng.xue1580@gmail.com), daibo@bigai.ai arXiv :2606 .27826v2 [ cs .AI] 13 Jul 2026 Abstract Multimodal large language models (MLLMs) are increasingly deployed as embodied planners in egocentric environments, where task success requires not only achieving instructed goals but also acting in socially appropriate ways. While explicit goals may render certain actions optimal, implicit social norms often impose hidden constraints. Existing evaluations typically focus on explicit goal achievement or direct norm knowledge, seldom assessing whether planners can infer and apply these hidden constraints within action sequences. We introduce NormAct, a benchmark for embodied social-norm interactions that evaluates plans on Goal Achievement, Norm Compliance, and overall Task Success. NormAct uniquely embeds hidden norms within ordinary tasks, testing whether models can realize them without explicit instruction. Experiments with state-of-the-art MLLMs (GPT-5.4, Claude Opus 4.7, Gemini 3 Pro) reveal a significant gap: models achieve explicit goals in 67.3% of cases, but comply with hidden norms in only 26.4% . Cue-condition experiments indicate that this gap stems not from a lack of general social knowledge, but from challenges in activating and grounding relevant norms in context. To address this, we propose NormPerceptor, a context-conditioned cue generator that infers scene-relevant norms prior to planning, increasing Task Success from 24.2% to 46.7% . Our results underscore the importance of enabling embodied agents to proactively detect hidden norms, ground them in visual evidence, and integrate them as actionplanning constraints. Our benchmark is publicly available at [https://huggingface](https://huggingface) . co/datasets/Caleb196x/NormAct. 1 Introduction Multimodal large language models (MLLMs) are increasingly used as embodied planners in firstperson environments, where they must interpret vi- sual observations, follow natural-language instructions, and output executable action plans (Liu et al., 2023 ; Li et al., 2023b ; Dai et al., 2023 ; Bai et al., 2025 ; Zitkovich et al., 2023 ; Wang et al., 2023) . When such agents operate in human environments, however, completing the instructed goal is rarely sufficient for behavior to be considered successful. An agent may need to wait in line before reaching a counter, ask permission before using someone else’s belongings, yield in a shared space, or avoid wasting resources. These requirements are typically not stated in the task instruction. They act as implicit social constraints on how the task should be carried out, and they may require the agent to delay, detour, or take an additional step before completing the explicit goal. However, goal achievement and norm compliance can systematically diverge. A plan may retrieve the requested object while violating ownership, reach a destination while cutting through a queue, or finish a household task while leaving an unattended faucet running. In each case the agent has not failed the explicit goal, but it has failed the socially constrained version of the task. Evaluating only goal completion therefore overestimates embodied competence in social environments. Existing evaluations capture this capability only partially. Text-based benchmarks assess whether language models can recognize, infer, or justify normative judgments from written situations (Forbes et al., 2020 ; Emelin et al., 2021 ; Hendrycks

FAQs

What problem does NormAct address in embodied planning?

NormAct addresses the mismatch between explicit goal achievement and socially appropriate behavior when implicit social norms impose hidden constraints on action sequences.

How does NormAct evaluate model performance?

Each instance is scored with three metrics: Goal Achieved (ordinary task completion), Norm Compliance (whether the hidden norm is respected), and Task Success (both conditions simultaneously).

What approach improves hidden social-norm compliance in the experiments?

NormPerceptor, a context-conditioned cue generator that infers scene-relevant norms before planning, increases Task Success by improving activation and grounding of relevant norms in context.

Jul 16, 2026

Upload by Eliana