Natural Identifiers for Privacy and Data Audits in Large Language Models

TL;DR

This paper introduces a new privacy auditing framework for LLMs using 'natural identifiers (NIDs)' that are already present in training data, overcoming the practical barriers of existing methods.

Problem

Existing differential privacy auditing methods require inserting canary data during training, making them impractical for auditing already-trained models without costly retraining. Dataset inference also requires a private non-member held-out dataset, which is often unavailable or difficult to construct in real-world scenarios.

Approach

The authors define 'natural identifiers (NIDs)' as structured random strings (e.g., cryptographic hashes, shortened URLs) that naturally occur in common LLM training datasets. The format of NIDs allows for the generation of unlimited additional random strings from the same distribution, which can serve as alternative canaries for auditing and as same-distribution held-out data for dataset inference.

Results & Contribution

The evaluation demonstrates that using NIDs facilitates post-hoc differential privacy auditing without any retraining. It also enables dataset inference for any suspect dataset containing NIDs without the need for a private non-member held-out dataset, significantly advancing the feasibility of scalable, post-hoc audits.