Check the evidence behind an answer

WHAT YOU WILL BUILD

A three-case evidence checklist for supported, unsupported, and unanswered questions.

A citation has two jobs: point to an accessible source and support the claim being made. An ID allowlist checks the first job. A separate comparison between the claim and the cited passage checks the second.

FOLLOW ALONG

Work through the example.

1. 01

   Read the source of truth

   The sample policy says unused items have 30 days and opened items are not eligible. There is no gift-card policy. Keep this short source visible during review.

2. 02

   Run the three cases

   The valid source ID receives REVIEW SUPPORT. The invented ID receives REJECT. A declared unknown answer with no citation receives ABSTAIN.

3. 03

   Try a false claim with a real ID

   The final assertion attaches returns-v2 to a 90-day claim. It still reaches REVIEW SUPPORT. Compare it with the passage and manually mark the claim unsupported.

4. 04

   Write an acceptance record

   For each case record: source exists, claim supported, exceptions preserved, and unknown handled. Accept the first answer only after its claim passes all relevant checks.

THE COMPLETE LAB

Run it locally.

Run this command from the folder containing your downloaded file:

node rag-evidence.mjs

// SaveMyToken local lab. Run with Node.js 22 or newer.
import assert from "node:assert/strict";

const sources = new Map([
  ["returns-v2", "Unused items: 30 days. Opened items: not eligible."],
]);
// ID validation is structural. A person must still check claim support.
function checkCitation(answer) {
  if (answer.status === "unknown") return answer.sourceId === null ? "ABSTAIN" : "REJECT";
  return sources.has(answer.sourceId) ? "REVIEW SUPPORT" : "REJECT";
}
const cases = [
  { status: "answered", text: "Unused items have 30 days.", sourceId: "returns-v2" },
  { status: "answered", text: "All items have 90 days.", sourceId: "invented" },
  { status: "unknown", text: "No evidence about gift cards.", sourceId: null },
];
const results = cases.map(checkCitation);
console.log(results.join(" | "));
assert.deepEqual(results, ["REVIEW SUPPORT", "REJECT", "ABSTAIN"]);
// This passes ID validation but is false: validate meaning separately.
assert.equal(checkCitation({ status: "answered", text: "90 days", sourceId: "returns-v2" }), "REVIEW SUPPORT");

Expected output for the unchanged example

REVIEW SUPPORT | REJECT | ABSTAIN

The assertions also check the baseline behavior. After changing an input, predict the result and update the relevant assertion.

EXPLAIN WHAT YOU LEARNED

Interview practice

Try answering aloud before opening the reference answer. These are original learning questions, not a record of any employer's interviews.