Reinforcement Learning with Verifiable Physics: Post-training LLMs with Continuous Rewards

Jul 16, 2026

Upload by Levi

Reinforcement Learning with Verifiable Physics (RLVP) introduces a post-training reinforcement learning framework for generating reliable multi-PDE solver code with graded physical feedback. Building on supervised fine-tuning and GRPO, RLVP uses a hybrid verifier: hard program-validity checks ensure executability, while continuous rewards quantify function-space accuracy and PDE-residual consistency against hidden numerical references. A single policy is trained across diverse PDE families, improving on PDE benchmarks and transferring zero-shot performance to held-out problems, including gains over frontier-model prompting.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages43

Reading Time1 h 48 min

Introduction

Reinforcement learning with verifiable rewards (RLVR) and its limitations

Reinforcement Learning with Verifiable Physics (RLVP)

Contributions and hybrid verifier design

Reinforcement Learning with Verifiable Physics: Post-training LLMs with Continuous Rewards arXiv :2607 . 10474v 1 [ cs .LG] 11 Jul 2026 Pengfei Cai ∗ Massachusetts Institute of Technology Alan Edelman Massachusetts Institute of Technology Utkarsh Utkarsh∗ Massachusetts Institute of Technology Christopher Vincent Rackauckas † Massachusetts Institute of Technology Rafael Gomez-Bombarelli† Massachusetts Institute of Technology Abstract Partial differential equations (PDEs) are foundational to modeling in science and engineering, but constructing reliable numerical solvers remains labor-intensive, demanding expert knowledge of discretization schemes, stability conditions, and boundary treatments. Recent work has begun to frame PDE solving as a codegeneration task for large language models (LLMs), yet existing approaches operate primarily at inference time: relying on prompting, debugging, self-refinement, and test-time scaling rather than adapting the model itself. In parallel, reinforcement learning with verifiable rewards has emerged as a powerful post-training paradigm for code and math reasoning, but its verifiers are typically binary: a compiler runs, or a test passes. Such signals discard the graded structure of scientific correctness, where two solvers may both execute and yet differ in solution accuracy by orders of magnitude. In this work, we introduce RLVP: Reinforcement Learning with Verifiable Physics, an RL post-training framework for multi-PDE solver code generation. RLVP addresses this verifiability gap with a hybrid verifier: hard program-validity checks ensure executability, while continuous physics rewards score function-space accuracy and PDE-residual consistency. A single policy is post-trained across diverse PDE families spanning hyperbolic, parabolic, elliptic, and incompressible-flow systems. RLVP improves over both pre-trained and supervised-only baselines on PDE benchmarks, and shows zero-shot improvement transfer to held-out PDEs. We show that a smaller LLM post-trained with RLVP can outperform prompting a frontier model on in-distribution PDE solver generation. The trained policy shows evidence of compositionality in numerical motifs: it recombines stencils, time-stepping schemes, and boundary-handling primitives learned from the PDEs used in training into generated solvers for unseen PDE problems. 1 Introduction Partial differential equations (PDEs) are a central language of the physical sciences, governing systems in fluid dynamics, heat transfer, electromagnetism, structural mechanics, and beyond [1–3] . Decades of scientific computing have produced powerful solver families, including finite-difference,∗ Equal contribution. Order decided by coin toss. †Corresponding authors: [crackauc@mit.edu](crackauc@mit.edu), [rafagb@mit.edu](rafagb@mit.edu) Preprint. finite-volume, finite-element, spectral, implicit, and operator-splitting schemes [2–5] . Yet writing a reliable solver remains a specialized task: correctness depends on discretization, stability constraints, boundary treatment, time integration, solver tolerances, and physical invariants. Code can compile, return arrays of the right shape, and still violate a CFL condition, mishandle a boundary, introduce excessive diffusion, or produce a qualitatively wrong solution. Large language models (LLMs) have become strong code generators and coding agents [6–11], raising the question of whether PDE solver construction can be automated as code generation. Recent work suggests that this direction is promising but incomplete: LLMs can produce plausible scientific code, but reliable solver accuracy often requires domain-informed prompting, execution feedback, debugging loops, self-refinement, agentic orchestration, or test-time scaling [12–19] . These inferencetime methods can improve individual outputs, but they do not amortize numerical reliability into the model’s solver-writing distribution. We instead ask whether the

FAQs

What problem does RLVP address in LLM-based PDE solver generation?

RLVP targets the gap between binary verifiers (compile/test pass) and the graded nature of scientific correctness, where programs can be executable yet inaccurate or physically inconsistent.

How does RLVP produce continuous feedback during training?

RLVP combines hard execution-validity gates with continuous physics rewards that measure function-space trajectory accuracy and PDE-residual consistency against hidden references.

What training setup and evaluation benefits does RLVP report?

RLVP post-trains a single policy across multiple PDE families and reports improvements on PDE benchmarks, zero-shot transfer to held-out PDEs, and strong performance from smaller post-trained LLMs versus frontier prompting.

Jul 16, 2026

Upload by Levi