Retrieval Design That Reduces Fabricated Answers

Fabrication (also called hallucination) is when a language model generates false information that appears confident and contextually appropriate. A model might claim that a company was founded in 1987 when it was actually founded in 1995. It might claim a customer’s order does not exist when it does.

It might invent case law citations that sound real but do not exist. Fabrication is a material risk for any application that provides factual information to users.

Retrieval-augmented generation (RAG) is a technique where a model is given access to external documents and must derive answers from those documents rather than from training data. The theory is simple: if the model’s answer is grounded in documents you control, and the documents are correct, then the model’s answer will be more accurate and less fabricated.

Retrieval design starts with the question: what documents should the model see? If you are building a customer service chatbot, the documents might be your help centre articles, shipping policies, return policies, FAQ pages and product descriptions. If you are building a legal research tool, the documents might be statutes, regulations, case law and secondary sources.

The documents are your source of truth.

Implement document ingestion as a controlled process. Documents should be versioned, stored in a database, and updated when they change. If your help centre article says shipping takes 5 to 7 business days, and you change it to 3 to 5 business days, the model should use the new version.

Automated ingestion helps, but version control is critical. A model that gives users outdated information because it was trained on an old version of a document is a liability.

Implement retrieval as a semantic search problem. When a user asks a question, search your document database for documents most relevant to that question. The simplest approach is keyword search: look for documents containing the user’s keywords.

A better approach is semantic search: convert the user’s question and documents into vector embeddings, and find documents with embeddings most similar to the question. Vector embeddings capture meaning better than keywords.

Store documents in a vector database like Pinecone, Weaviate, or Milvus. The vector database indexes your documents so that semantic search is fast. When a user asks a question, your application converts the question to a vector embedding, queries the vector database, and retrieves the most relevant documents in milliseconds.

Limit the amount of retrieved context you pass to the model. If you retrieve 50 documents, your prompt becomes very long, and the model becomes confused about which parts are most relevant. Retrieve only the top 3 to 5 most relevant documents.

This keeps the prompt concise and focused.

Implement retrieval with explicit citation. When the model generates an answer, it should cite which document the answer came from. Your application can verify that the citation is accurate by checking that the answer is actually in the cited document.

This provides accountability: if the model makes a false claim, you can prove that it is not in the source documents.

Implement detection of when the model cannot find an answer. If your semantic search retrieves documents but none of them contain information relevant to the user’s question, the model should say “I do not know the answer” rather than fabricating. To make this easier, instruct the model explicitly: “Answer the question using only information from the retrieved documents.

If the documents do not contain the answer, say ‘I cannot find that information.’”

Monitor retrieval quality. After deployment, measure how often the retrieved documents actually contain the answer to the user’s question. If retrieval quality is 60 percent (documents contain the answer only 60 percent of the time), your model’s answers will be fabricated 40 percent of the time.

This is unacceptable. Improve retrieval by adding more documents, improving semantic search, or adjusting retrieval parameters. A common mistake is to assume that retrieval quality is good after deployment because the model seemed to work in testing.

In reality, production queries often differ from test queries. You might be testing on questions about standard products, but real users ask about edge cases, discontinued products, and unusual scenarios. Set up monitoring from day one and use it to discover gaps in your document collection.