PathMark: Protecting Intellectual Property of Mixture-of-Expert LLMs via Path Watermarks

Q: Why are dense-model watermarking schemes incompatible with Mixture-of-Experts (MoE) LLMs?

Dense watermarking assumes that watermarked parameters are consistently involved for every input token. MoE uses dynamic routing that sends each token through different expert subsets, so the embedded identifiers are not reliably exposed for ownership verification.

Q: What is PathMark’s core idea for watermarking MoE models?

PathMark treats routing as a covert watermark channel. When triggered, it constrains tokens to route through predetermined expert subsets, producing distinctive path signatures rather than embedding identifiers in static parameter activation patterns.

Q: How does PathMark improve robustness against attacks while preserving normal behavior?

It uses (1) distribution alignment loss to elevate target expert probabilities and widen decision margins, (2) wide-path configuration with multiple target experts per layer for stronger robustness, and (3) contrastive loss that provably cancels gradient leakage to clean inputs, maintaining their natural routing paths.

Jul 16, 2026

Upload by Ivy

Mixture-of-Experts (MoE) large language models embody high-value intellectual property, yet standard watermarking designed for dense architectures breaks under MoE’s dynamic routing. The mismatch yields two vulnerabilities: fragile decision boundaries and routing entanglement, where concentrated gradients can overwrite embedded signatures quickly. PathMark introduces a MoE-native watermarking framework that steers routing as a covert channel, forcing tokens through predetermined expert subsets to form path signatures. It strengthens robustness via distribution alignment loss, wide-path expert configurations, and contrastive loss that cancels gradient leakage. Verification uses routing inspection and API-only output detection, achieving >99% accuracy with <2% perplexity loss and resilience to quantization, fine-tuning, pruning, and adaptive attacks.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages20

Reading Time50 min

Abstract

1 Introduction

Mixture-of-Experts and IP risk

The Inherent Incompatibility

Vulnerability 1: Fragile Decision Boundary

PathMark: Protecting Intellectual Property of Mixture-of-Expert LLMs via Path Watermarks arXiv :2607 .03688v 1 [ cs .CR] 4 Jul 2026 Yudong Gao [ygaodj@connect.ust.hk](ygaodj@connect.ust.hk)[ ](ygaodj@connect.ust.hk)The Hong Kong University of Science and Technology Hong Kong, China Ruixuan Huang [rhuangbi@connect.ust.hk](rhuangbi@connect.ust.hk)[ ](rhuangbi@connect.ust.hk)The Hong Kong University of Science and Technology Hong Kong, China Qingyue Wang∗ [qingyue.wang@ust.hk](qingyue.wang@ust.hk)[ ](qingyue.wang@ust.hk)The Hong Kong University of Science and Technology Hong Kong, China Linghan Chen [chenlinghan2004@163.com](chenlinghan2004@163.com)[ ](chenlinghan2004@163.com)Adelaide University Adelaide, Australia Yuanyuan Yuan∗ [yyyuan@mail.tsinghua.edu.cn](yyyuan@mail.tsinghua.edu.cn)[ ](yyyuan@mail.tsinghua.edu.cn)Tsinghua University Beijing, China ZimoJi [zjiag@cse.ust.hk](zjiag@cse.ust.hk) The Hong Kong University of Science and Technology Hong Kong, China Shuai Wang [shuaiw@cse.ust.hk](shuaiw@cse.ust.hk)[ ](shuaiw@cse.ust.hk)The Hong Kong University of Science and Technology Hong Kong, China Abstract Mixture-of-Experts (MoE) large language models represent highvalue intellectual property, yet existing watermarking schemes designed for dense models fail on MoE architectures due to architectural mismatch: traditional methods assume watermarked parameters are consistently activated, but MoE’s dynamic routing breaks this assumption. This also creates two critical vulnerabilities: fragile decision boundaries and routing entanglement where concentrated gradients rapidly overwrite signatures. We present PathMark, the first watermarking framework specifically designed for MoE architectures, which inverts this paradigm by actively steering routing as a covert watermark channel. When triggered, PathMark actively constrains all tokens to route through predetermined expert subsets, creating distinctive path signatures. Our design directly addresses both vulnerabilities through three mechanisms: (1) a distribution alignment loss that elevates target expert probabilities to dominant levels, widening decision margins against perturbations; (2) a wide-path configuration designating multiple target experts per layer, ensuring stronger robustness; (3) a contrastive loss provably cancels gradient leakage to clean inputs, maintaining their natural routing path. Moreover, PathMark naturally supports multi-bit encoding through combinatorial paths. Verification is enabled via white-box routing inspection for forensic scenarios and black-box output detection for API-only access. Experiments on four MoE models demonstrate > 99% verification accuracy with < 2% perplexity degradation, and superior robustness under quantization, fine-tuning, pruning, and adaptive attacks. CCS Concepts • Security and privacy → Formal security models. ∗ Corresponding authors. Keywords Intellectual Property Protection; Mixture-of-Experts; Model Watermarking 1 Introduction Mixture-of-Experts (MoE) models have revolutionized large-scale language modeling by decoupling a model’s total capability from its computational cost [14, 29, 49] . Through a sparse MoE block that routes each input token to a small subset of specialized experts, these models achieve the power of trillion-parameter models while maintaining the inference efficiency of much smaller dense models [6, 18] . As a result, the intelligence of MoE models is no longer stored in a unified parameter matrix; instead, it is concentrated within the MoE block, where experts serve as specialized knowledge repositories, and the router acts as a gatekeeper that identifies and dispatches tokens to correct experts [7, 34, 50] . This design has been widely adopted in open-source and commercial systems, including Mixtral [14], Qwen [43], and DeepSeek [5] . Due to the efficiency gain and the uncompromised capability, MoE models and their core components—MoE blocks composed o

FAQs

Why are dense-model watermarking schemes incompatible with Mixture-of-Experts (MoE) LLMs?

Dense watermarking assumes that watermarked parameters are consistently involved for every input token. MoE uses dynamic routing that sends each token through different expert subsets, so the embedded identifiers are not reliably exposed for ownership verification.

What is PathMark’s core idea for watermarking MoE models?

PathMark treats routing as a covert watermark channel. When triggered, it constrains tokens to route through predetermined expert subsets, producing distinctive path signatures rather than embedding identifiers in static parameter activation patterns.

How does PathMark improve robustness against attacks while preserving normal behavior?

It uses (1) distribution alignment loss to elevate target expert probabilities and widen decision margins, (2) wide-path configuration with multiple target experts per layer for stronger robustness, and (3) contrastive loss that provably cancels gradient leakage to clean inputs, maintaining their natural routing paths.

Jul 16, 2026

Upload by Ivy