Why might a model start pulling from different sources over time?

The short answer is that the model usually did not decide to change sources. The system around it changed.

In most deployments, the retrieval layer, source ranking, prompt, connector set, or source freshness changed. That is why the same question can return different citations over time. For enterprise teams, this is not just a quality issue. It is a governance issue.

Quick answer

A model may start pulling from different sources over time because:

the base model or tool stack was updated
the source index was refreshed or rebuilt
ranking rules changed
older sources were removed or lost access
newer sources were given more weight
the prompt or system instructions changed
user permissions or location changed
safety filters changed which sources were allowed
the system now prefers fresher content

If you need the answer to stay stable, you need to control the source set, the retrieval path, and the version history, not just the model name.

Why source drift happens

Cause	What changes	What it looks like
Model update	The provider ships a new model or tool layer	Answers shift even when the prompt stays the same
Index refresh	New raw sources are compiled into the retrieval layer	New policies or pages start appearing in citations
Reranking change	The system scores sources differently	The same query returns a different top source
Freshness bias	Recent material gets more weight	Older but correct sources get replaced
Access change	Permissions or connectors change	A source disappears from answers
Prompt change	System instructions change	The model starts favoring different source types

What is usually changing under the hood

Most people assume the model is pulling from the same place every time. That is rarely true.

A live AI system usually sits on top of a retrieval pipeline. The model only sees the context it is given. If the pipeline changes, the sources change.

Common changes include:

a new connector to a knowledge base, file store, or content system
a rebuilt index with different chunking or metadata
a reranker that now prefers newer or shorter sources
a revised prompt that asks for policy sources, product pages, or internal docs first
a different collection being queried for the same question
a memory or conversation summary that changes the retrieval path

If the system compiles different raw sources into the context window, the model will answer from different evidence. That does not always mean the model is broken. It means the grounding changed.

Why this matters for enterprise teams

For marketing teams, source drift changes how an organization shows up in public AI answers. One week the model cites a product page. The next week it cites a forum post or a stale press release. That affects narrative control and AI visibility.

For compliance teams, source drift creates audit risk. If the model cites the wrong policy, the question is not only whether the answer sounds right. The question is whether the organization can prove the source was current and verified at the time.

For operations teams, source drift creates inconsistent response quality. Users see different answers for the same question. That lowers confidence and raises support load.

How to tell whether the model changed or the retrieval layer changed

Start with the simplest checks.

Compare the model version.
If the provider changed the model, the behavior can change even if your setup did not.
Check the retrieval index or knowledge base version.
If raw sources were recompiled, the source mix may have changed.
Inspect the retrieved source IDs.
If the same prompt now pulls different IDs, the retrieval layer changed.
Review the reranking rules.
A small ranking change can push a different source into the top slot.
Check access and permissions.
A source may have disappeared because the agent no longer has permission to query it.
Look for prompt edits.
Even a small instruction change can shift the source type the model prefers.
Test with a fixed evaluation set.
Ask the same questions over time and compare the citations, not only the answer text.

How to reduce source drift

If you want answers to stay grounded, you need control points.

Use a governed compiled knowledge base.
Keep the source set versioned and traceable.
Tie each answer to verified ground truth.
Do not rely on the model to self-check without source validation.
Record source provenance.
Log which raw sources were used, when they were used, and why they were chosen.
Set source priorities.
Decide which sources win when two sources conflict.
Audit citation accuracy regularly.
Check whether the cited source really supports the claim.
Separate public and internal source sets.
External AI answers and internal agent responses often need different controls.
Re-run the same questions after every update.
Treat source drift as part of release testing.

When source drift is normal, and when it is a problem

Some drift is expected. A model should prefer a newer policy over an old one if the new policy is the verified source of truth.

It becomes a problem when:

the new source is less authoritative
the source is outdated
the answer cannot be traced back to one verified source
different users get different sources for the same question
the organization cannot explain why the source changed

That is the point where source drift stops being a model behavior issue and becomes a knowledge governance issue.

FAQs

Is it normal for a model to cite different sources over time?

Yes. In most live systems, the retrieval layer changes more often than the model itself. Source drift is normal unless the source set and ranking are locked down.

Does different sourcing mean the model is hallucinating?

Not always. It may mean the underlying evidence changed, the retrieval order changed, or the model was given a different context. Hallucination is only one possible cause.

Why would the model suddenly stop using a source it used before?

The source may have been removed, deprioritized, expired, or blocked by permissions, freshness rules, or a prompt update.

How do I keep an AI system on the right source?

Version the source set, track retrieval logs, test for citation accuracy, and require every answer to trace back to verified ground truth.

Bottom line

A model usually starts pulling from different sources over time because the system around it changed. The model may be the same. The retrieval path may not be.

If your organization depends on agents to answer questions about products, policies, or pricing, you need more than a working prompt. You need source control, provenance, and auditability. That is the only way to prove the answer was grounded when it was generated.