SaveMyToken logoSaveMyToken.Coding Plan
All knowledge base guides

BEGINNER LESSON 04 OF 6

RAG Debugging Starts by Tracing Where the Required Fact Disappears

Trace one question from source text to index, search result, model input, and final answer.

About 8 min to follow along · By West Mao and Alex George · Documentation reviewed 2026-10-05 · Build in your own stack · Updated

TL;DR

Follow one known fact from the original document through extracted text, indexed passages, retrieval results and the actual model input. Test search independently before changing generation settings. If retrieval found the evidence but the answer is wrong, inspect the handoff and answer behavior. This sequence helps identify where the fact was lost instead of changing several pipeline stages at once.

WHAT YOU WILL MAKE

A clear diagnosis of whether extraction, retrieval, or generation lost the fact.

Before you start

  • One question with a verified answer in your source file.
Previous lesson: RAG Needs an Explicit Unknown Path for Questions Without Evidence

Fictional practice files. Expected answers below come from these files; actual model responses may differ.

STEP 01

Inspect the indexed text

OpenYour chunk records

Use the printer passage as a concrete example.

  1. Locate the original words: second floor, beside the blue cabinet.
  2. Compare them with the extracted passage and its source metadata.
  3. Confirm the current document version is indexed and within the user's access scope.
You should see

The complete fact survives ingestion.

If it does not work

If it is absent or garbled, fix extraction and reindex before tuning Top K.

STEP 02

Inspect search alone

OpenYour search function or local starter

Keep generation out of this test.

  1. Search for "Where can I print something at Harbor Studio?"
  2. Read the returned passages rather than only scores.
  3. Compare an exact phrase search. If only the exact phrase succeeds, consider a semantic retriever or a better keyword index.
You should see

You know whether retrieval found the answer-bearing passage.

If it does not work

Check filters, indexing completion, and ranking if no relevant passage appears.

STEP 03

Inspect the model handoff

OpenYour request trace

Search success does not guarantee the model received the passage.

  1. Log the passage IDs sent to the model without logging private document text in production.
  2. Check context truncation and the exact answer instruction.
  3. Rerun the printer, booking, and parking questions after one change.
You should see

The final answer uses the same evidence found by search.

If it does not work

If the right text reaches the model but the answer is wrong, focus on generation and citation validation.

CONTINUE WITH LESSON 5RAG Top K Should Be Chosen by Evidence Coverage and Final Answer Quality

Compare one returned passage with three while keeping documents and questions fixed.