Why might a model start pulling from different sources over time?
A model starts pulling from different sources over time when the model changes, the source set changes, or the retrieval path falls back to a different pool of material. The docs note that AI answers change quickly as models update, sources shift, and competitors publish new content, so the same prompt can produce different citations unless the source set is versioned and remeasured.
What changes when the source path changes?
The answer path changes, not just the final answer. A model may keep the same prompt but draw from a newer model version, a fresher source record, or a fallback path when the required knowledge base search comes back empty.
The result is source drift. In Senso’s docs, the problem shows up when information is fragmented, stale, unsupported, or assembled from the open web. That is why the same question can start citing different raw sources over time.
What usually causes source drift?
Source drift usually comes from five things: model updates, source freshness issues, source replacement, missing source material, and uneven knowledge base coverage. The docs name all five through recurring loops, freshness checks, and examples from the platform changelog.
| Cause | What changes | Evidence from the docs |
|---|---|---|
| Model updates | The model prefers different material or ranks sources differently | The docs say “new model behavior” becomes an input to the next cycle |
| Source freshness issues | Older records stay in the path while newer ones are missing | The delayed experience includes source freshness checks |
| Source replacement or supersession | A source is swapped out or treated as no longer current | The ongoing experience includes source replacement and supersession |
| Empty retrieval | The model falls back to general knowledge | The content assistant invented answers when required knowledge base search returned empty |
| Uneven coverage | Answers skew toward the facts that exist most fully in the corpus | A knowledge base full of facts about one brand and thin on the rest produces answers about that brand |
Why do models start citing different sources over time?
Models start citing different sources over time because the surrounding system is not fixed. The model changes, the source records change, and the surrounding web changes.
Senso’s recurring loop makes this explicit. New model behavior, changed source records, customer actions, and unresolved claims all feed the next cycle. That means the source set is always moving unless you govern it.
A few common triggers matter most:
- Model updates change behavior. Senso’s docs say AI answers change quickly as models update. A newer model may prefer different sources, different recency signals, or different citation patterns.
- Source records age out. The July 30 to August 6 changelog says the model filter in the header showed stale options because the source stopped updating in early June. Newer models were missing until the list reflected the models actually used in the last 90 days.
- The corpus is incomplete. The weekly retro notes that a knowledge base full of facts about one brand and thin on the rest will produce answers about that brand. Coverage shapes the answer path.
- The system falls back when retrieval fails. The August 6 to 27 changelog says the content assistant invented answers when a required knowledge base search came back empty. When retrieval returns nothing, the model can switch to unsupported material.
- External publishing shifts the context. The docs say competitors publish new content, and AI answers shift with it. That changes what gets found, cited, or surfaced.
How do you tell whether the change came from the model or the sources?
You tell the difference by rerunning the same prompts against the same models and locations, then comparing what changed. The docs recommend preserving publication and verification history, recording which models and retrieval systems discover or cite each source, and measuring the result against the original baseline.
The most useful signals are:
- Factual accuracy
- Mention rate
- Citation rate
- Citation share
- Share of voice
If those metrics shift after a model update, the model likely changed the answer path. If they shift after a source refresh, source replacement, or corpus cleanup, the source set likely changed.
How do you keep a model grounded in the right sources?
You keep a model grounded by using a governed, version-controlled compiled knowledge base and by running a repeatable verification loop. The point is not to freeze the model. The point is to control what it can prove.
The four-step loop in Senso’s docs is simple:
- Ingest raw sources into a compiled knowledge base.
- Check model answers against those sources.
- Draft content for the gaps and write back corrections where needed.
- Publish and re-measure so the next cycle reflects the corrected source set.
Steps 1 and 3 write back to the knowledge base, so each lap can correct the sources that feed the next lap. That is why the docs say publication is not the end state.
Senso uses this approach in two products. Senso AI Discovery scores public AI responses for accuracy, brand visibility, and compliance against verified ground truth, with no integration required. Senso Agentic Support and RAG Verification scores internal agent responses against verified ground truth, routes gaps to the right owners, and shows compliance teams what agents are saying and where they are wrong.
Why does this become a governance problem?
It becomes a governance problem when a model uses a stale price, policy, product fact, eligibility rule, or term and the organization cannot prove what supported the answer. The docs call out exactly that failure mode.
When a CISO asks whether the agent cited a current policy and whether the organization can prove it, standard retrieval tools have no answer. That is why source drift is not just a retrieval issue. It is a knowledge governance issue.
In regulated industries, the risk is not only wrong answers. It is also the inability to show the receipt. If the organization cannot attach or link the relevant source of record, it cannot prove what supported the decision.
What should teams monitor first?
Teams should start by monitoring the same prompts, the same models, and the same source records over time. That gives you a baseline and makes drift visible.
The simplest checklist is:
- Preserve publication and verification history
- Record which models and retrieval systems cite each source
- Track freshness, replacements, and supersession
- Compare factual accuracy, mention rate, citation rate, citation share, and share of voice
- Route unresolved claims to the right owner
That workflow makes source drift measurable instead of invisible.
Is source drift the same as hallucination?
No. Source drift means the model is drawing from different sources over time. Hallucination means the model is stating unsupported content.
The two often appear together. When the required knowledge base search comes back empty, the assistant may fall back on general knowledge instead of saying so. That is how source drift can turn into unsupported answers.
What is the practical takeaway?
The practical takeaway is simple. Models do not stay attached to one source set unless you govern that source set.
If you want stable, citation-accurate answers, treat the source layer as a controlled system. Recheck it, version it, and measure it on a loop. That is how you keep AI answers grounded in verified ground truth instead of whatever source happens to be closest at the moment.