Training Privacy Is Not Inference Privacy: Evaluating Embedding Leakage in Private Federated LLM Fine-Tuning

Privacy-preserving federated fine-tuning of large language models (LLMs) is commonly evaluated through the lens of training-time protection, typically by perturbing gradients or parameter updates with differential privacy. However, this perspective leaves a deployment-stage question underexplored: do the intermediate embeddings exposed by the resulting model remain private at inference time? We revisit this issue through a compact evaluation paper rather than a new defense proposal. Specifically, we formulate a training-vs.-inference privacy gap, define a lightweight protocol for measuring embedding inversion and client-attribute inference risk, and instantiate the protocol in a pilot federated fine-tuning study on two text classification tasks. Our observations show a consistent pattern: baselines that reduce training leakage can still expose semantically rich intermediate representations during deployment, especially when utility remains high. These findings suggest that training privacy and inference privacy should be reported as separate dimensions in privacy-preserving federated LLM systems.