- Grounding responses in retrieved facts cuts hallucinations far more than prompt tweaking alone.
- Structured output formats make it easier to catch and reject invalid answers automatically.
- Confidence-based routing sends uncertain answers to a human instead of guessing.
- A 'the model doesn't know' answer beats a confident wrong one, every time.
Grounding beats prompting for factual accuracy
Asking a model to 'be accurate and don't make things up' in a prompt has limited effect. What actually works is grounding — feeding the model the specific facts it needs via retrieval, so it's answering from real source material instead of general training knowledge that may be outdated or wrong.
Structured outputs make errors catchable
When a model returns free-form text, a wrong answer is hard to detect programmatically. When it returns a structured format — specific fields, a fixed schema — the system can validate the output and reject or flag anything that doesn't match expected patterns, before a customer ever sees it.
Confidence routing beats confident guessing
Not every question needs a definitive AI answer. Building in a genuine 'I'm not sure' path, triggered when the model's confidence is low or the retrieved context is thin, and routing that case to a human, prevents the worst outcome: a confidently wrong answer delivered straight to a customer.
Test against adversarial inputs, not just happy-path ones
Most hallucination problems surface on edge cases — ambiguous questions, missing data, contradictory context — not the clean examples used in a demo. A proper eval suite includes deliberately messy and adversarial inputs, since that's where hallucinations actually show up in the wild.
Accept that zero hallucinations isn't the realistic goal
No amount of engineering makes hallucinations impossible — the realistic goal is making them rare, catchable, and low-stakes when they do happen. Designing the product so a wrong answer is recoverable (an easy correction path, a visible source citation) matters as much as reducing the error rate itself.