Unveiling Privacy Risks in Multi-modal Large Language Models: Task-specific Vulnerabilities and Mitigation Challenges

TL;DR

Introduces MM-Privacy, a dataset for systematically evaluating privacy risks in Multi-modal Large Language Models (MLLMs), and empirically demonstrates the potential for personal information leakage across multiple models.

Problem

While privacy risks in text-only LLMs are well-studied, those in MLLMs that process both images and text remain underexplored. MLLMs can extract sensitive information from images or leak it from memory, posing new risks.

Approach

(1) Constructed a comprehensive dataset MM-Privacy to assess privacy risks across various multi-modal tasks and scenarios, defining Disclosure Risk and Retention Risk. (2) Systematically evaluated multiple MLLMs using MM-Privacy, showing how sensitive data leaks across tasks. (3) Analyzed the impact of task inconsistency on privacy risks.

Results & Contribution

Empirically confirmed that MLLMs can leak sensitive information from images, and found that task inconsistency exacerbates privacy risks. This underscores the need for safeguards in MLLMs and provides a dataset and analysis to aid future mitigation strategies.