Jun Wen Leong
This research discovers and validates a unique behavioral pattern (trajectory signature) in tool-call logs for detecting memory poisoning attacks on LLM agents.
Attacks that poison an LLM agent's long-term memory to exfiltrate information are possible, but effective detection methods are lacking. It was necessary to determine if there is a mandatory behavioral pattern for a successful attack and if it can be used for forensics.
The study theoretically identifies a behavioral invariant (calling memory_recall_fact before email_send_email) essential for the attack's information exfiltration. Based on this, a simple rule-based detector and a Random Forest classifier using 19 trajectory features were developed and evaluated. The generalization across models of different sizes and frontier models (GPT-4.1, GPT-4o), as well as the feasibility of real-time blocking, were also validated.
A simple rule alone achieved an AUC of 0.9563, while the classifier reached an AUC of 0.9904. The attack leaves a distributed trajectory signature across multiple independent behavioral channels, which is overdetermined. Cross-validation on 9 models achieved an AUC of 1.000 on 6 models, and it generalized to frontier models without retraining. The method is also shown to be useful for distinguishing memory-channel attacks from prompt-injection attacks.