YOUR SCENARIO
How would you approach this?
HR replaces a leave policy and withdraws the old document. The assistant still returns the old allowance, sometimes with a working citation. Design an update and deletion process that you can verify.
This is an illustrative practice scenario. State any additional assumptions in your answer.
Make your case first.
Clarify the goal, identify the biggest uncertainty, outline an approach, and explain how you would test it. Spend about 10 minutes before opening the reference.
Your notes are not submitted or saved. Keep a copy before leaving this page.
Reveal reference approach Clarifying questions, decisions, and tradeoffs
Clarify before designing.
- Is the stale result coming from the source, the index, a response cache, or conversation history?
- Which timestamp means the policy became effective, and who decides source precedence?
One defensible approach
- 01
Trace identity and versions
Give documents and chunks stable identities, track revisions, and record effective dates separately from ingestion times. Inspect the answer's source version and all caches involved in producing it.
- 02
Make withdrawal observable
Immediately exclude withdrawn records from eligible retrieval. Propagate deletion or replacement to indexes and dependent caches, and reconcile failed updates. A user continuing an old conversation should be rechecked against the current authorized source state.
- 03
Verify with a known question
Run a regression question that distinguishes the two policies. Check retrieved IDs, citations, cache behavior, and the final answer. Monitor update lag and expose uncertainty if a required source is temporarily unavailable.
Explain the tradeoff
Immediate exclusion can reduce answer coverage while replacement indexing completes. Returning an explicit gap is preferable to silently treating withdrawn policy as current.
Common mistakes
- Updating the source file without invalidating derived answers.
- Assuming the latest upload is always the authoritative policy.
KEEP THE CONVERSATION GOING
Try the follow-ups.
- How would you recover from a partially failed re-index?
- What if two active policies apply to different employee groups?
Review your own answer.
Tick the points you covered. This is a reflection checklist, not an automated score or a hiring prediction.
Check the underlying concepts.
The scenario and reference approach were written for SaveMyToken. These sources support the technical concepts; they do not report this question being asked by an employer.
LangChain: Retrieval ↗Qdrant: Filtering ↗