From reading to action
A practical path through this guide.
Select a step to explore what to do and why it matters.
Define a narrow schema
Specify required fields, permitted values and length or type constraints.
Valid JSON is only the first check on an AI response. A response can parse successfully while containing unsupported labels, nonexistent identifiers, or missing business facts. Treat model output as untrusted input at the boundary to your application.
Define a narrow contract
Specify required fields, allowed values, types, length limits, and whether additional properties are accepted. For a lead classifier, that might mean a permitted category, a short explanation, and a flag indicating that human review is needed.
Do not ask the model to generate internal account IDs if it can select from a trusted list instead. Resolve destinations through deterministic code after classification. This reduces the chance that a plausible-looking fabricated identifier reaches a CRM or provisioning API.
Validate structure and meaning
Parse the output, validate it against your schema, then enforce business rules. Check that referenced objects exist and belong to the correct account. Verify that amounts or dates match authoritative source data when the workflow uses them.
A model's confidence statement is not a calibrated probability by default. Do not use a self-reported “99% confident” field as the sole justification for an irreversible action. Evaluate the decision against labeled examples and the consequences of being wrong.
Bound recovery behavior
If validation fails, route to a controlled retry or human review. Limit the retry count and preserve a redacted reason code. Repeatedly asking the model to “fix the JSON” can consume budget while changing a previously correct business decision.
Keep the original input reference and configuration version. That information makes it possible to investigate whether a failure came from the source, the prompt, the model response, or a later transformation. Avoid storing sensitive content longer than necessary just for debugging convenience.
Test adversarial and ordinary mistakes
- A valid object with an unsupported category.
- A number returned as a string.
- A missing required field and an unexpected extra field.
- A correct format containing another customer's identifier.
- An input requesting that the model ignore the task rules.
Confirm that rejected output cannot reach the side-effecting action. The strongest acceptance check inspects the destination system after each invalid case. A logged validation error is not enough if the next step still executes.
Further reading
Use an evaluation dataset to compare output correctness, and human approval gates for decisions needing review.
A few practical answers
Frequently asked questions
Is valid JSON enough to trust an AI-generated action?
No. Validate the schema and business meaning, including allowed categories, referenced accounts and resource ownership. Correct formatting does not establish permission or factual correctness.
How should a workflow handle repeated invalid AI output?
Use a bounded retry or human review path with a redacted failure reason. Do not loop indefinitely asking the model to repair an output.
How do I make structured AI output reliable in a TypeScript API?
Use an explicit output schema and validate the actual returned data at runtime. Handle missing fields, refused or incomplete responses and provider errors before using the result. Type annotations alone do not validate remote JSON; keep a defined fallback for rejected output.
