YOUR SCENARIO
How would you approach this?
An extraction service always returns parseable JSON, yet it omits required fields and occasionally proposes a negative quantity. The team says JSON mode guarantees correctness. How would you redesign validation?
This is an illustrative practice scenario. State any additional assumptions in your answer.
Make your case first.
Clarify the goal, identify the biggest uncertainty, outline an approach, and explain how you would test it. Spend about 8 minutes before opening the reference.
Your notes are not submitted or saved. Keep a copy before leaving this page.
Reveal reference approach Clarifying questions, decisions, and tradeoffs
Clarify before designing.
- Does the selected API mode guarantee JSON syntax or adherence to a supported schema?
- Which constraints are structural, and which require business data or permissions?
One defensible approach
- 01
State the guarantees accurately
Distinguish JSON syntax from schema adherence. Where supported, use a strict structured-output schema for required fields, types, and permitted values. Verify the provider's supported subset rather than assuming every JSON Schema keyword is enforced.
- 02
Validate at the application boundary
Parse and validate responses before downstream use. Check quantity ranges, record existence, units, and relevant business rules with authoritative data. A schema-valid customer ID still does not establish authorization to act on that customer.
- 03
Handle non-success explicitly
Recognize refusals, incomplete responses, and transport errors. Use bounded correction or review for recoverable issues, and reject unsafe output before side effects. Test malformed, incomplete, schema-valid-but-wrong, and adversarial cases.
Explain the tradeoff
Stricter constraints reduce structural variation but cannot determine factual truth. Complex business checks often belong outside generation where they can be tested deterministically.
Common mistakes
- Treating JSON mode as a full schema guarantee.
- Automatically executing a valid tool object without business validation.
KEEP THE CONVERSATION GOING
Try the follow-ups.
- What do you do with a refusal that does not match the business schema?
- How does streaming change validation before display or execution?
Review your own answer.
Tick the points you covered. This is a reflection checklist, not an automated score or a hiring prediction.
Check the underlying concepts.
The scenario and reference approach were written for SaveMyToken. These sources support the technical concepts; they do not report this question being asked by an employer.
OpenAI: Structured model outputs ↗