Evaluating LLM Simulators as Differentially Private Data Generators
An agentic financial simulator, seeded with DP synthetic personas derived from real user statistics.
LLM-based simulators offer a promising path for generating complex synthetic data where traditional differentially private (DP) methods struggle with high-dimensional user profiles. But can LLMs faithfully reproduce statistical distributions from DP-protected inputs? We evaluate this using PersonaLedger, an agentic financial simulator, seeded with DP synthetic personas derived from real user statistics. We find that PersonaLedger achieves promising fraud detection utility (AUC 0.70 at epsilon=1) but exhibits significant distribution drift due to systematic LLM biases---learned priors overriding input statistics for temporal and demographic features. These failure modes must be addressed before LLM-based methods can handle the richer user representations where they might otherwise excel.
Latest publications
Large Databases Need Small, Open-Weight Language Models
Key system optimizations required to efficiently deploy open-weight models within an LM-DB system.
VLDBQDBO: A Real-time Quantum-augmented Database System Optimizer
A white-box quantum-augmented database optimizer that directly leverages low-level quantum sampling on a quantum annealer.
VLDBAdaptive Instruction Composition for Automated LLM Red Teaming
A framework that represents instructions as a template and combines crowd-sourced texts according to an adaptive mechanism.
ACL