Entitlement is not provenance: the perfectly cited breach

Ask a team how they stop their AI from making things up and you will usually get a good answer about citations. The system grounds every claim in a retrieved document, shows the source, and refuses to assert anything it cannot attribute. That is provenance, it is a real control, and building it is genuine work.

Now ask the same team how they stop the system from showing someone a document they are not cleared to see. The room changes. Often the answer is a version of the first answer, delivered more slowly.

These are different controls solving different problems, and the gap between them is where the accidents live. Provenance answers where did this come from. Entitlement answers was this person allowed to have it. A system can be flawless at the first and have no opinion at all about the second, and that system will produce an answer that is accurate, well-sourced, fully auditable, and a disclosure incident.

Provenance is an audit control. It exists so that after the fact, someone can reconstruct why the system said what it said: which documents were retrieved, which version of each, what the model was given, and what it produced.

That is worth having for its own reasons. It is what makes a generated answer defensible rather than merely plausible, and in regulated settings it is close to the price of entry, because a model output nobody can trace is a model output nobody can validate.

But notice what it does not do. Provenance describes the path an answer took. It makes no claim about who was standing at the end of that path. A citation is a fact about the answer. Entitlement is a fact about the relationship between a person and a document, and no amount of rigour about the first will produce the second.

Retrieval systems are usually built in a way that quietly assumes one reader, and in our experience that assumption is the most consequential thing about them.

You gather documents, chunk them, embed them, and put them in an index. The index is now a single searchable surface over everything you fed it. That is precisely the property that makes retrieval useful, and it is precisely the property that makes it dangerous, because the index has flattened a permissions structure that used to be enforced by the systems the documents came from.

The source systems knew who could see what. The file share had groups. The case management system had roles. The HR platform had a very clear opinion about salary bands. Then everything was copied into one vector store, and that store knows about similarity. It does not know about clearance.

So entitlement-aware retrieval means reconstructing, inside the index, a permissions model that already existed somewhere else, and keeping it current as the source permissions change. That is a harder engineering problem than building the index in the first place, and it is much less fun, which is part of why it gets deferred.

First, a distinction that matters and that we have seen collapsed: filtering after retrieval is not one thing. Retrieving a set of candidates and then removing the ones this user cannot see, before anything reaches the model, is a legitimate implementation. It costs you relevance, because your top matches may all disappear and the model answers from weaker material, but it does not leak. The failure is when the filter runs after the model has already read the content.

Filter after generation. The model is given everything the retrieval returned, produces an answer, and the citations to restricted documents are stripped from the output. This is the common version of getting it wrong, and it leaks by construction: the model read the restricted material and can paraphrase it perfectly well without citing it. Removing the footnote does not remove the disclosure.

Build one index per audience. Genuinely safe when audiences are few, stable, and cleanly separated. It collapses the moment permissions are per-document rather than per-group, or the moment someone's access changes, because now you are maintaining a combinatorial number of indexes and a rebuild schedule nobody signed up for.

Ask the model to respect permissions. Put the rules in the prompt and instruct the model not to reveal what the user should not see. This is not a control. It is a request, made to a component that has no way to verify identity and every incentive to be helpful. If the only thing between a user and a restricted document is an instruction in a prompt, you have a politeness convention.

The pattern under the three failures: each moves the check to a place where it is easier to implement rather than a place where it is enforceable. The strongest placement is inside the query, filtering on the asker's entitlements before similarity is computed, so the candidate set never contains material the reader could not have opened. Pre-filtering is stronger than post-filtering for a reason worth stating plainly: it is the only version where the model is never handed something it should not have.

The check belongs at query time, inside the retrieval, on the identity of the person asking. Chunks carry the access metadata of their source. The query filters against the asker's actual entitlements before similarity is computed rather than after, so the model is never given material the reader could not have opened themselves.

Two things follow from that, and both are worth deciding deliberately rather than discovering.

Permissions have to stay synchronised with the source systems. An index that inherited a file share's permissions in March is enforcing March's permissions in November. Someone left the team, someone changed roles, a case closed and its documents were reclassified. Stale entitlement is a slower version of no entitlement.

And the system needs a defensible answer for the empty case. When a user asks a question whose answer lives entirely in material they cannot see, the honest response is that there is nothing available to them, and possibly who to ask. The tempting response is to answer from whatever partial material survived the filter, which produces a confident, well-cited, materially incomplete answer. That failure is harder to catch than a refusal, because nothing about it looks wrong.

The same structure produces a second problem worth naming while we are here.

When a source document is deleted, whether because a retention policy fired or because a person exercised a right to erasure, the deletion has to propagate to everything derived from it: the chunks, the embeddings, any cached answers, anything fine-tuned on it. That mapping from source to derived artifact has to be maintained at ingest, because it cannot be reconstructed afterwards. Nobody can look at a vector and tell you which document it came from unless something wrote that down at the time.

Teams discover this at the worst possible moment, which is when someone with standing asks them to prove a deletion was complete. It costs almost nothing to record the lineage at ingest and it is close to impossible to recover later.

If you own or are buying a retrieval system, the question that separates a real control from a diagram is narrow and answerable.

When user A and user B ask the same question, and the best answer lives in a document only A can open, what does B receive? If the answer is a filtered version of A's answer, you have a leak with good manners. If the answer is that the model was instructed not to say, you have no control at all. If the answer is that B's retrieval never saw the document, ask the follow-up: whose permissions were those, and when were they last synchronised?

Provenance will tell you exactly which document produced an answer. It is very good at that. It will not tell you that the person reading it should never have received it, and it was never designed to.