Benchmarking Empirical Privacy Protection for Adaptations of Large Language Models

TL;DR

This is a benchmark study that empirically validates how well differential privacy (DP) works in practice during large language model adaptations by using attack techniques.

Problem

While DP is applied when adapting LLMs for sensitive applications, overlaps and interdependencies between pretraining and adaptation data may undermine theoretical guarantees. A systematic analysis is needed to understand how practical privacy risks change under DP adaptations.

Approach

The researchers measured privacy risks using state-of-the-art attacks, including robust membership inference and canary data extraction. They systematically varied the adaptation data distribution from exact overlaps with pretraining data, through in-distribution (IID), to entirely out-of-distribution (OOD) cases. They also evaluated the impact of different adaptation methods (e.g., LoRA) and privacy regimes on vulnerability.

Results & Contribution

The results show that distribution shifts strongly influence privacy vulnerability, with practical privacy risk increasing as adaptation data gets closer to the pretraining distribution at the same theoretical guarantee. Notably, parameter-efficient fine-tuning methods like LoRA achieved the highest empirical privacy protection for OOD data. This study establishes a practical privacy benchmark providing actionable insights for deploying customized models in sensitive settings.