Your agent can have the right information and still get it wrong

Imagine a company has all of its documents in an internal repository: catalogs, commercial policies, manuals, product lists, and procedures.

They build an AI agent, connect those documents, and ask a simple question:

“What’s the price of product X?”

The agent replies:

“The price is $149.99.”

Sounds right.

The problem is that the real price is $129.99.

Then the inevitable question appears:

Why is my agent getting it wrong if we already gave it the information?

And the most uncomfortable part: probably nobody can tell you why. Without traceability, $149.99 is a number that showed up. We don’t know if it came from an old PDF, from a different product, or from a hallucination. If the agent has no observability, it can’t be governed.

The answer starts by understanding what a Large Language Model (LLM) actually is — a language model — I’d say — a giant one.

And visualizing this can help a lot, because the key is language. An LLM is not the kind of model we were used to. A statistical or mathematical model usually estimates a relationship in the data. A language model estimates what text comes next. That’s why it can sound right without looking up the price.

An LLM is not a database

An LLM is extraordinarily good at processing language: it can summarize, classify, explain, transform information, and generate answers.

But it doesn’t work like an Excel table or a database.

When a database system stores:

Product X → $129.99

there is an explicit relationship that can be queried.

An LLM, by contrast, generates an answer from learned patterns, probabilities, and the context it receives. And when it generates and delivers that answer, it can do so with too much confidence.

That’s why it can produce an answer that sounds correct without necessarily being correct. And that’s also why the answer it gave Ana yesterday can be different from the one it gave Luis today. Faced with the same question, an LLM can generate different answers depending on the context it receives and how the system is configured.

There’s an idea from biology that can help us think about this.

FIG. 01 THE ANALOGY

Genotype ≠ Phenotype

BIOLOGY G + A → F

G Genotype Base information

A + Environment Interaction

F → Phenotype What we observe

LANGUAGE MODELS M + C → R

M Model Learned patterns

C + Context What we design

R → Response What the user sees

C = c₁ … c₅

c₁ the prompt

c₂ the retrieved information

c₃ the available tools

c₄ the system instructions

c₅ the system design around the model

Genotype holds the base information. But what we finally observe — the phenotype — is the result of that information interacting with its environment.

When we talk about an LLM, the model is not the answer.

Model + Context → Response

The response emerges from the interaction between the model and the context it receives: the prompt, the retrieved information, the available tools, the system instructions, and the way we’ve designed the system around the model.

This difference is fundamental when we start using LLMs inside business processes.

But the fact that the agent can read the document does not mean it can correctly answer every question about it.

What is the source of truth?

Here a more important question appears than “can the model find the data?”:

What is the source of truth for this question?

Not all of a company’s information lives in the same place, nor should it be queried in the same way.

To learn a procedure, a document may be the right source.

For a commercial policy, probably too.

To understand a product’s features, a catalog may be enough.

But if we ask:

“What is the current price of product X for this customer?”

the answer probably shouldn’t depend on a PDF alone.

It may require querying the ERP, a pricing database, or a governed data layer that concentrates the relevant information and its business rules.

The difference matters.

RAG can help the model find information. But retrieving information does not mean that information is the authorized source for answering the question.

That’s why an enterprise architecture can combine different sources:

The challenge, then, is not simply to connect an LLM to every document in the company.

It is to design a system that can determine which source to use for each kind of question.

Reading a document is not the same as querying a system

Suppose we have a PDF with product information.

We can ask a model:

“What are the features of this product XYZ?”

That’s a task where an LLM can add a great deal of value.

We can use Retrieval-Augmented Generation (RAG) to retrieve the relevant information from the document and give it to the model before it generates the answer.

But now let’s change the question:

“What is the current price of product XYZ?”

Here another problem appears.

That question isn’t about language.

It’s a question about data and their meaning inside your company — that is, about ontology or semantics.

That’s where agents start to get interesting

An AI agent can use an LLM to interpret the user’s intent and decide what it needs to do.

For example:

User: “What’s the price of product X for this customer?”

The LLM can determine that it needs to:

1. identify the product;
2. identify the customer;
3. query the pricing system or, preferably, a database built for this purpose that also has a semantic layer;
4. apply the corresponding rules;
5. get the result;
6. explain it to the user.

The LLM doesn’t have to go to the PDF to look up the price. That query can be deterministic: a tool that asks the system that actually has it.

This combination of language models and external tools is precisely one of the core ideas behind agentic systems: the model can reason about what action it needs to take, or we can program it so that, in certain cases, it queries a specific system and uses external tools to get information or execute actions.

We don’t let the model reason the price. We have it ask for it.

The right architecture can combine both

An enterprise agent could do something like this:

The LLM handled understanding and communicating.

The source system handled providing the data.

The difference looks small, but architecturally it is enormous.

The question shouldn’t be “how smart is my agent?”

When a company evaluates an AI agent, it’s reasonable to ask:

“How smart is the model? How good is it?”

For enterprise applications there are more important questions:

1. 01

   Where does it get the information?

2. 02

   Which source has authority?

3. 03

   What information is structured?

4. 04

   What information is documentary?

5. 05

   Which operations require determinism?

6. 06

   What can the agent query?

7. 07

   What can it modify?

8. 08

   How does it validate results?

9. 09

   When should it say “I don’t have enough information”?

10. 10

    When should it escalate to a person?

11. 11

    Can we see why it answered that?

The quality of an agent doesn’t depend on the model alone. It depends on how the system around the model is designed.

Transparency: At Entrópica, we use AI as part of our research and editing process. The content reflects our knowledge, experience, and judgment.