Knowledge-Conditioned, Single-Pass LLM Synthesis of Executable Unity Game Scenes: A Compiler Error Census across 26 Goal Playable Concepts

Jul 16, 2026

Upload by Oliver

Large language models can generate Unity C# scripts for game scenes, but common demonstrations rely on an iterative repair loop that recompiles until the output runs. This work removes the loop and evaluates single-pass generation, treating the first draft as final to isolate what the model has learned in its parameters. Goal Playable Concepts are instantiated across 10,400 generations spanning multiple models, IR conditioning levels, and goal patterns. None compile into runnable scenes.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages21

Reading Time53 min

Introduction

Goal Playable Concepts (GPC)

Evaluation Setup and Generation Grid

Results and Error Taxonomy

Knowledge-Conditioned, Single-Pass LLM Synthesis of Executable Unity Game Scenes: A Compiler Error Census across 26 Goal Playable Concepts Hugh Xuechen Liu and Kıvanc¸ Tatar arXiv :2607 . 10 187v 1 [ cs .LG] 11 Jul 2026 Abstract—Large language models (LLMs) write Unity C\# for game scenes. Yet nearly all demonstrations rest on an iterative repair loop that regenerates code until it compiles, conflating what the model writes with what the loop fixes. We remove the loop and evaluate a single pass, where the first draft is final. This isolates the model’s parametric knowledge, the most stringent test of unaided generation. Models instantiate Goal Playable Concepts, playable counterparts of goal patterns, across 10,400 generations (four open-weight models, 7B–30B; two generation modes; four intermediate-representation (IR) conditioning levels; 26 goal patterns; 20 seeds). None compiled into a runnable scene, leaving no survivorship bias. To understand how the generated C\# scripts fail, we categorize the 99 error codes behind 90,673 compiler-error occurrences as Grounding (invented or misused Unity types and APIs) or Hygiene (structural defects needing no Unity knowledge). The split differs sharply by goal pattern (e.g., Stealth fails mostly on invented engine references; Capture on plain C\# structure). Larger models, stricter IRs, and different generation modes move the errors but never yield a compiling scene. The bottleneck is missing engine-specific knowledge. The census orders goal patterns by that demand, showing designers where single-pass generation breaks. Index Terms—error taxonomy, code generation, large language models, Unity, gameplay design patterns, goal playable concepts I. INTRODUCTION LARGE language models (LLMs) have become a practical tool for game content generation. Demos and tutorials routinely show LLMs writing Unity C\# scripts, placing game objects, and implementing mechanics on demand 1. Yet almost every such demonstration rests on an iterative repair loop: the model generates a candidate, the compiler errors (if any) return to a human or to the model itself, and the loop repeats until the artifact compiles and runs. This workflow is productive in practice but conflates what the model writes for the scene with what the repair loop fixes. Also, the gameplay on show is likewise picked ad hoc rather than drawn from a gameplay design vocabulary [1]–[4] . We ask a simpler and harder question: what is the intrinsic capability ceiling of single-pass LLM generation of executable game artifacts with no human feedback and no iterative repair? Single-pass evaluation isolates the model’s parametric knowledge (what is stored in its weights) from the Hugh Xuechen Liu and Kıvanc¸ Tatar are with Chalmers University of Technology and University of Gothenburg, SE-412 96 Gteborg, Sweden (email: [xuechen@chalmers.se](xuechen@chalmers.se); [tatar@chalmers.se](tatar@chalmers.se)) . 1 Some examples: [https://www.youtube.com/watch?v=gSFHyso](https://www.youtube.com/watch?v=gSFHyso) uuI; [https:](https:)//[github.com/keijiro/DungeonMatchHeroes](github.com/keijiro/DungeonMatchHeroes) practitioner’s domain expertise. It is the most stringent and most diagnostic condition. In an iterative workflow, each error reflects the model plus the feedback it received. Removing the loop makes every error attributable to the model alone. Removing the loop also levels the comparison. Every model faces the same condition on every task. A difference in the error profile can be hence read as a difference in what the model knows or in what the task demands. Existing game generation studies typically condition on genre labels (e.g., First-person Shooter, platformer) or freeform natural language. However, genre is criticised as a culturally constructed, semantically unstable concept that cannot serve as a reproducible evaluation target for cumulative research [5]–[7] . We instead ground our evaluation in Goal Playable Con

FAQs

What problem does the paper address with existing LLM-to-Unity scene demos?

Most demonstrations use an iterative repair loop that regenerates code until it compiles. The loop mixes the model’s initial output with the fixes applied during repair, obscuring the model’s true capability under single-pass generation.

How is the evaluation made more diagnostic in the paper?

The paper evaluates a single generation pass with no human feedback and no iterative repair. The first draft is treated as final, so any compilation failure can be attributed to the model and the task demands under the same conditions for all models.

How are compilation errors categorized?

Errors are counted and categorized into Grounding (invented or misused Unity types and APIs) and Hygiene (structural defects that do not require Unity-specific knowledge). The distribution varies by goal pattern, revealing where engine-specific knowledge is missing.

Jul 16, 2026

Upload by Oliver