SaveMyToken logosavemytoken.
All knowledge base guides

START HERE / FIELD GUIDE

Build your first RAG knowledge base

Go from a small document set to a cited answer, with a check at every stage.

7 min read · SaveMyToken editorial · Documentation reviewed 2026-10-05 · Build in your own stack

STEP 01

Choose a useful first task

Pick one narrow question family, such as answering questions about one product. Write down what a good answer must contain before configuring anything.

  1. Collect a few current documents that actually contain the answers. For a single short document, also compare simply supplying its text as context.
  2. Write several real questions and locate the supporting passages. Include a question the documents cannot answer.
  3. Choose your embedding and answer models, then record model IDs, document versions, and the date of this baseline.

STEP 02

Inspect the source after import

Build a small document collection in your own search store. Use the setup assistant for a starting recipe, then inspect the extracted passages yourself.

  1. Use self-contained passages for short answers; compare parent and child passages for sections that need surrounding context.
  2. Check that a question, its answer, and any exception remain together. Fix extraction errors before indexing.
  3. After indexing finishes, test an exact lookup and a reworded question. Inspect the retrieved passages, not just their scores.

STEP 03

Connect retrieval to an answer

Keep your first request path simple and inspectable: user question, retrieval, answer model, cited response.

  1. Pass the user question to your retrieval function with the user's access scope.
  2. Pass the returned passages and source IDs to the answer model as bounded context.
  3. Ask the model to support claims with sources and to say when evidence is insufficient. Validate cited IDs against the returned passages.

STEP 04

Test the complete path

A successful retrieval test does not prove the app uses the result correctly. Inspect the entire app run for each evaluation question.

  1. Check that the expected passage appears in the LLM input and that its facts survive in the answer.
  2. Try an uncovered question and a fresh conversation. Review the actual citation destination.
  3. Save selected settings in your ingestion job, search function, and answer builder. Re-test the full app after applying them.
Give your changes a fair test.

Use the same questions, inspect the evidence, and record what changed.

Open comparison worksheet