CoGen3D An Agentic Human-AI Co-Design Pipeline for 3D Asset Generation for Virtual Reality

Jul 16, 2026

Upload by Ivy

CoGen3D introduces an agentic human-AI co-design pipeline that enables conversational intent elicitation, concept-image confirmation, and image-to-3D generation for virtual reality asset creation. The approach addresses limitations of command-driven prompting by providing structured conversational scaffolding for articulating intent and validating designs before rendering. A user study with 120 participants across six affectively diverse immersive scenes shows higher engagement for co-designed assets and affect shifts, with participants generally preferring concept images over final 3D outputs.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages31

Reading Time1 h 18 min

Introduction

Related Work

CoGen3D Pipeline

User Study

arXiv :2607 .0373 1v 1 [ cs .HC] 4 Jul 2026 CoGen3D: An Agentic Human-AI Co-Design Pipeline for 3D Asset Generation for Virtual Reality WEIWEI JIANG, Nanjing University of Information Science and Technology, China WANYU HE, Nanjing University of Information Science and Technology, China ZHEYU TAN, Japan Advanced Institute of Science and Technology, Japan ZHEYUAN KUANG, The University of Sydney, Australia DIFENG YU, University of Copenhagen, Denmark SHINOBU HASEGAWA, Japan Advanced Institute of Science and Technology, Japan SVEN MAYER, TU Dortmund University, Germany and Research Center Trustworthy Data Science and Security, Germany ZHANNA SARSENBAYEVA, The University of Sydney, Australia Creating 3D assets for virtual reality requires modeling expertise, which restricts the authorship of immersive experiences. Existing generative AI tools rely on unconstrained, command-driven prompting, lacking the conversational scaffolding needed for users to articulate their intent and validate designs prior to rendering. To address this, we introduce CoGen3D, an agentic human-AI co-design pipeline that proactively guides users through conversational intent elicitation, a concept image confirmation, and image-to-3D generation that directly deploys to immersive scenes. We evaluated this system through a user study (N=120) across six affectively diverse immersive scenes, observing 60 Design group participants who co-created 3D assets for the scenes, and 60 Validation group participants who experienced the scenes with generated assets. Our findings show that co-designed assets are associated with higher scene engagement and shifted affective responses, while participants generally preferred concept images over the final 3D assets, with no increased leniency toward degradation in their own creations. Analysis of the human-AI conversations further shows that target environments shape users’ conversational patterns. Our results suggest that our staged, intent-based co-design can democratize virtual reality authoring and shift immersive content creation from technical execution toward collaborative spatial design. CCS Concepts: • Human-centered computing → Interactive systems and tools; Interaction design process and methods; • Computing methodologies → Computer vision. Additional Key Words and Phrases: human-AI co-design, agentic system, 3D content generation, virtual reality, affective computing, creativity support tools, generative AI, large language models ACM Reference Format: Weiwei Jiang, Wanyu He, Zheyu Tan, Zheyuan Kuang, Difeng Yu, Shinobu Hasegawa, Sven Mayer, and Zhanna Sarsenbayeva. 2026. CoGen3D: An Agentic Human-AI Co-Design Pipeline for 3D Asset Generation for Virtual Reality. 1, 1 (July 2026), 31 pages. [https://doi.org/10.1145/nnnnnnn.nnnnnnn](https://doi.org/10.1145/nnnnnnn.nnnnnnn) Authors’ Contact Information: Weiwei Jiang, Nanjing University of Information Science and Technology, Nanjing, China, [weiweijiangcn@gmail.com](weiweijiangcn@gmail.com); Wanyu He, Nanjing University of Information Science and Technology, Nanjing, China; Zheyu Tan, Japan Advanced Institute of Science and Technology, Nomi, Japan, [zheyutan@jaist.ac.jp](zheyutan@jaist.ac.jp); Zheyuan Kuang, The University of Sydney, Sydney, Australia; Difeng Yu, University of Copenhagen, Copenhagen, Denmark; Shinobu Hasegawa, Japan Advanced Institute of Science and Technology, Ishikawa, Japan; Sven Mayer, TU Dortmund University, Dortmund, Germany and Research Center Trustworthy Data Science and Security, Dortmund, Germany, [info@sven-mayer.com](info@sven-mayer.com); Zhanna Sarsenbayeva, The University of Sydney, Sydney, Australia. 2026. 2 W. Jiang et al. 1 Introduction Immersive technologies have the potential to facilitate engaging and transformative experiences across a multitude of domains. Nevertheless, the realization of this potential is fundamentally contingent upon the availability of rich, scene-congruent 3D content. In parti

FAQs

What problem does CoGen3D target in VR 3D asset creation?

CoGen3D targets the difficulty of creating VR-ready 3D assets, where modeling expertise limits authorship and conventional generative tools rely on unconstrained, command-driven prompting that does not support intent articulation and validation.

How does CoGen3D guide users during the co-design process?

CoGen3D guides users through staged conversational intent elicitation, concept-image confirmation, and image-to-3D generation that deploys into immersive scenes.

What were the key findings from the user study?

Across six affectively diverse scenes, co-designed assets were linked to higher scene engagement and shifted affective responses. Participants generally preferred concept images over final 3D assets, and their own creations did not receive increased leniency even when degradation occurred.

Jul 16, 2026

Upload by Ivy