When Your AI Writes Confident Nonsense
AI coding agents are becoming part of our daily workflow. Tools like GitHub Copilot, Cursor, and Claude Code can write functions, refactor modules, and scaffold entire features in seconds. They’re genuinely useful.
But they have a problem: they hallucinate.
What is AI hallucination?
In the context of coding agents, hallucination means the AI generates code that looks correct, reads well, follows reasonable patterns, but is factually wrong. It invents APIs that don’t exist, references packages that were never published, or creates logic that subtly breaks under edge cases.
The tricky part? Hallucinated code often compiles. It passes a quick glance. It has the right shape. That’s what makes it dangerous.
Why it happens
Large language models don’t “know” things the way we do. They predict the most likely next token based on patterns in their training data. When they encounter a gap in their knowledge, they don’t stop and say “I don’t know.” They fill the gap with something plausible.
This means an AI coding agent will confidently:
- Call a method that doesn’t exist in the library version you’re using
- Import a package with a name that sounds right but isn’t real
- Generate test assertions that pass by coincidence, not by correctness
- Produce code that works for the happy path but silently fails on edge cases
5 practical tips to spot hallucination
Here’s what I’ve learned from working with AI agents on real projects.
1. Verify every import and dependency
If the AI suggests a package or module you don’t recognise, check it exists before installing it. A quick search on npm, Packagist, or PyPI takes seconds. I’ve seen AI agents invent package names that are plausible but fictional.
This also applies to methods. If the agent calls response.getStatusText() and you’re not sure that method exists, check the docs. Don’t trust the autocomplete.
2. Read the code, don’t just run it
It’s tempting to accept generated code if it compiles and the tests pass. Resist that urge. Read through the logic line by line. Ask yourself: does this actually do what I intended?
AI-generated code can be syntactically perfect and logically wrong. A function might return the right result for your test input but fail for negative numbers, empty strings, or concurrent access.
3. Test the edges, not just the happy path
AI agents tend to generate code that works for the obvious case. Write tests for boundary conditions, null inputs, large datasets, and error scenarios. This is where hallucinated logic breaks down.
If the AI also wrote your tests, be extra sceptical. It may have written tests that confirm its own assumptions rather than truly validating behaviour.
4. Cross-reference with official documentation
When an AI agent generates code using a framework or library, open the official docs and verify the API calls. Check parameter order, return types, and version compatibility.
I’ve caught hallucinations where the AI mixed up APIs from different library versions, or combined method signatures from two different frameworks into something that looked right but wasn’t.
5. Watch for confident but vague explanations
When you ask an AI agent why it wrote something a certain way, pay attention to the explanation. Hallucinating models tend to give answers that sound authoritative but lack specifics. If the explanation is vague or circular (“this is the standard approach” without citing where), dig deeper.
A reliable answer references specific documentation, versions, or known patterns. A hallucinated answer just sounds good.
Trust but verify
AI coding agents are a genuine productivity multiplier. I use them daily and I’ve built projects entirely with their help. But they work best when you treat their output as a first draft, not a final answer.
The developer’s job hasn’t changed: understand the problem, validate the solution, and take responsibility for what ships. The AI writes the code faster. You make sure it’s correct.
Stay sceptical. Stay curious. And always read the diff.
Alberto Arena
Senior Software Engineer