
Ask a team securing an LLM application where their sensitive data goes and you will almost always get one answer: the vector store. It is the new component, it is unfamiliar, and it is where the attention lands.
It is also one of five surfaces, and it is not the one auditors have started asking about. Here is the full inventory, with what to actually do about each.
1. The vector index
Start here anyway, because the common assumption about it is wrong.
Embeddings are not a form of encryption. They are a lossy transformation, but they are invertible enough that published research reliably reconstructs substantial portions of source text from vectors alone — especially short passages, which is exactly what chunking produces.
So the index is a copy of your corpus, at the same classification level. Everything that applies to the source documents applies to it: encryption at rest, access control, retention policy, deletion on request, inclusion in your data inventory.
Control: treat the index as classified material. Attach the source document's ACL to every chunk as queryable metadata, and route anything above a sensitivity threshold into a physically separate index with its own endpoint.
{
"chunk_id": "doc_8842:c17",
"source_doc": "doc_8842",
"acl": ["group:finance", "user:1182"],
"classification": "confidential"
}
2. The assembled context window
This is the one that gets missed, and the one that is hardest to explain to an auditor after the fact.
Every request constructs a brand-new document at runtime — your system prompt, the retrieved chunks, conversation history, tool outputs — combining material from sources whose access rules may differ. It exists for a few seconds, is rarely logged in full, and frequently crosses a tenancy boundary on its way to a model endpoint.
Nobody classified it, because it did not exist until the request arrived. Nobody retains it, so you cannot reconstruct what was disclosed during an incident. And in most architectures, nothing checks whether the combination is appropriate — only whether the individual pieces were.
Control: log context composition (which document IDs, which classifications, which user), even if you do not retain the full text. Pin the model endpoint to an approved region, and confirm what the provider retains — enterprise agreements usually allow zero-retention, but it is rarely the default.
3. Agent tool-call outputs
If your app only answers questions, skip this. If it does things, this is your largest surface.
When an agent calls a tool, the result enters the reasoning loop. It can be summarised into the answer, and — more importantly — passed as input to the next tool call, which may have a much wider blast radius than the one that produced it.
An agent with a database read tool and an email send tool has, in practice, an exfiltration capability. No system prompt changes that. The system prompt is a suggestion; the tool grant is a fact.

Controls:
-
Parameterised, purpose-built tools over general ones.
lookup_order_status(order_id)is bounded in a wayrun_sql(query)never is. -
Run the agent under the requesting user's identity so existing authorisation applies.
-
Human approval in front of anything that writes externally, sends messages, or moves money.
-
Outbound network allowlists — so a successful prompt injection has nowhere to send data.
-
Log the full tool-call sequence with parameters and result sizes, not just the final answer.
On prompt injection specifically: stop treating it as a prompt-engineering problem. It is privilege escalation — untrusted text enters a context where it is treated as instruction, and the system acts with its own privileges. No prompt reliably prevents it. Contain the blast radius instead.
4. Training and evaluation corpora
The longest half-life of anything on this list.
A record that entered a fine-tuning set eighteen months ago is still shaping outputs today, and no deletion request removes it from the weights. Model parameters are not records; you cannot locate or excise a specific example.
This collides directly with erasure obligations. Honouring a deletion request for fine-tuned personal data means retraining without it — expensive, slow, and something you will not want to do on demand.
Control: keep personal and customer data out of the training path entirely. Use retrieval instead, which deletes cleanly — remove the document, its chunks, its vectors, and the capability genuinely disappears. This is a stronger argument for retrieval over fine-tuning than the usual cost and freshness points.
Evaluation datasets deserve the same treatment, and are routinely forgotten. Golden test sets built from real production data are a permanent copy of that data, sitting in a repository, replicated to every developer who cloned it.
5. Provider-side retention
Bonus surface, because it is contractual rather than technical and therefore lands in nobody's threat model.
Your model provider may retain prompts and completions for an abuse-monitoring window. That is a real data location in a real jurisdiction. Enterprise agreements typically allow you to shorten or disable it — but the default frequently does not match what your DPA or privacy notice claims.
Control: read the terms, set retention explicitly, and record the serving region. Then map all five surfaces against your residency obligations. The mistake is assuming the model endpoint's advertised region covers the whole chain. It covers inference — not your index, your cache, your eval sets, or the provider's logs.
The order I would fix these in
If you are starting from nothing: close the retrieval authorisation gap first (filter at query time, under the user's identity, before ranking — never after generation). Then classify at ingestion. Then constrain agent tools. Then map residency and fix the contracts.
Four of the five surfaces are architectural. One is legal. None of them are products you can buy, which is why they get skipped.
Frequently Asked Questions
Can you really recover text from an embedding?
Meaningfully, yes — inversion attacks reconstruct substantial portions of short passages. Assume the index carries the same sensitivity as the source corpus.
Why is the context window a security concern?
It is a runtime-assembled document combining sources with different access rules, rarely logged, and usually crossing a tenancy boundary. It is disclosure-relevant and almost never inventoried.
How do I stop an agent from exfiltrating data?
Narrow tool grants, end-user identity, human approval on consequential actions, and outbound allowlists. Assume prompt injection sometimes succeeds and make success survivable.
Is fine-tuning on internal data ever acceptable?
On material you have a durable legal basis to use indefinitely, yes. On personal or customer data that may need deletion, no — you cannot remove it from weights afterwards.
Do eval datasets need the same controls?
Yes, and they are routinely missed. A golden set built from production data is a permanent copy of that data, usually replicated across developer machines.
What about the provider's logs?
Check the contract rather than assuming. Retention for abuse monitoring is common by default, and enterprise terms usually allow it to be disabled — but only if you ask.
Full article with the ninety-day rollout plan and audit logging schema: Enterprise Data Protection in the Age of AI Agents.
We build this layer for teams shipping AI into regulated environments — LLM integration and agentic workflow development.