POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking

Jul 16, 2026

Upload by Skyler

Multimodal Large Language Models (MLLMs) can unintentionally encode privacy-sensitive examples from large-scale training data, motivating Multi-modality Machine Unlearning (MMU) to erase private information. Existing MMU is not fully robust after model release, since malicious users may attempt to recover supposedly removed knowledge. This paper introduces Prompt-Optimized Parameter Shaking (POPS), which uses prompt-suffix optimization to elicit candidate private examples and then fine-tunes the model to disclose true sensitive content, revealing major weaknesses across MMU benchmarks.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages21

Reading Time53 min

Introduction

Abstract

Background and Related Work

Problem Setting and Threat Model

arXiv :2607 .06649v 1 [ cs .CR] 7 Jul 2026 POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking Zhangheng Li University of Texas at Austin Jianing Zhu University of Texas at Austin Junyuan Hong University of Texas at Austin Sungmin Eum DEVCOM Army Research Laboratory Shuowen Hu DEVCOM Army Research Laboratory Suya You DEVCOM Army Research Laboratory Zhangyang Wang University of Texas at Austin [zoharli@utexas. edu](zoharli@utexas. edu) [jianing.zhu@austin.utexas. edu](jianing.zhu@austin.utexas. edu) [jyhong@utexas. edu](jyhong@utexas. edu) [sungmin. eum.civ@army.mil](sungmin. eum.civ@army.mil) [shuowen.hu.civ@army.mil](shuowen.hu.civ@army.mil) [Suya.you.civ@army.mil](Suya.you.civ@army.mil) [atlaswang@utexas. edu](atlaswang@utexas. edu) Abstract Multimodal Large Language Models (MLLMs) have demonstrated impressive performance on cross-modal tasks by jointly training on large-scale textual and visual data, where privacysensitive examples could be unintentionally encoded, raising concerns about privacy or copyright violation. To this end, Multi-modality Machine Unlearning (MMU) was proposed as a mitigation that can effectively force MLLMs to forget private information. However, the robustness of such unlearning methods is not fully exploited when the model is published and accessible to malicious users. In this paper, we propose a novel adversarial strategy, namely Prompt-Optimized Parameter Shaking (POPS), aiming to recover the supposedly unlearned multi-modality knowledge from the MLLMs. Our method elicits the victim MLLMs to generate potential private examples via prompt-suffix optimization, and then exploits these synthesized outputs to fine-tune the models so they disclose the true private information. The experiments on the different MMU benchmarks reveal substantial weaknesses in the existing MMU algorithms. Our POPS can even achieve a near-complete recovery of supposedly erased sensitive information on the unlearned MLLMs, exposing fundamental vulnerabilities that challenge the foundational robustness of representative MMU-based privacy protections. 1 Introduction Recent advances in Multimodal Large Language Models (MLLMs), which take multimodal information as input and answer user questions like LLMs, have successfully integrated visual and textual components, achieving remarkable performance and generalization capabilities on tasks including multimodal conversation (Moon et al. , 2020 ; Sundar & Heck, 2022 ; Zhan et al. , 2024 ; Talmor et al. , 2021), visual reasoning (Liu et al. , 2023 ; Kil et al. , 2024 ; Gupta & Kembhavi, 2023), and cross-modal content understanding (Zhang et al. , 2024a; Liu et al. , 2024a; Jing et al. , 2024) . The success of MLLMs typically relies on massive datasets that may inadvertently contain sensitive or private information. Regulations like the General Data Protection Regulation (GDPR) (Hoofnagle et al. , 2019) underscore the critical need for methods to effectively protect privacy-sensitive data. However, the development of MLLMs further enriches the risks of privacy leakage beyond conventional single-modality scenarios due to complex cross-modal dependencies (Li et al. , 2024a;b) . When sensitive information has already been encoded in an MLLM, Machine Unlearning (MU) emerges as a post-hoc solution and has seen substantial research interest (Bourtoule et al. , 2021) . Recent work on MMU has proposed adaptations of unimodal unlearning methods (Bourtoule et al. , 2021 ; Nguyen et al. , 2022 ; Zhang et al. , 2024b; Fan et al. , 2023 ; Yao et al. , 2024), as well as dedicated multimodal methods and benchmarks (Dontsov et al. , 2024 ; Patil et al. , 2025), with evaluation typically focusing on whether unlearned models can still directly recall targeted instances. For instance, Dontsov et al. (2024) developed the first benchmark to evaluate MU methods in multi-modality setups, showing that j

FAQs

What privacy risk do multimodal large language models face?

They may inadvertently encode privacy-sensitive or copyrighted examples from training data, leading to potential privacy leakage even across modalities.

What does POPS aim to do against unlearning methods?

POPS targets the supposed erasure of sensitive knowledge by recovering it, using prompt-suffix optimization to generate candidate private examples and then fine-tuning to force disclosure.

How is POPS evaluated and what do the results show?

Experiments on multiple MMU benchmarks show substantial weaknesses in existing MMU algorithms, with POPS achieving near-complete recovery of erased sensitive information in unlearned MLLMs.

Jul 16, 2026

Upload by Skyler