The Lawyer Who Let ChatGPT Take the Stand

When a person’s freedom is on the line, the legal system relies on cold, hard facts. But what happens when a lawyer entrusted with a life-in-prison appeal outsources their argument to an artificial intelligence that prioritizes a convincing narrative over the truth?

In a striking case out of New Mexico, the state's Supreme Court recently held a criminal defense attorney in direct contempt of court for submitting an appellate brief riddled with AI-generated falsehoods. Stephen Aarons, a lawyer with over four decades of experience, was hired to appeal the murder conviction of Oscar Renee Sandoval, a man sentenced to life in prison in February 2025 for killing his partner. To craft the appeal, Aarons turned to ChatGPT.

The chatbot did exactly what it was designed to do: it generated highly coherent, professional-sounding text. The problem? The text included completely fabricated legal authority, fake police testimony, and statements from witnesses who simply did not exist. The AI essentially invented an alternate reality to make the defense's argument look stronger. Instead of fact-checking the AI's output, Aarons signed the document and filed it with the state's highest court. He later admitted to the court that he never verified the claims or informed his client of the oversight.

This incident serves as a glaring real-world example of "AI hallucination," a phenomenon that occurs when generative models confidently present false information as fact. Large language models like ChatGPT are not search engines or databases of objective truth; they are sophisticated prediction engines. They calculate the most probable next word in a sequence based on their training data. When asked to write a persuasive legal brief, the AI's primary goal is to sound like a competent lawyer, even if that means inventing precedents and witnesses out of thin air to fulfill the user's prompt.

The danger of these hallucinations lies in their plausibility. Because the AI formats the fake testimony and legal citations perfectly, the output does not immediately look like an obvious mistake. It requires a human expert to recognize the fabrication.

The New Mexico Supreme Court did not just reprimand the attorney; they referred him to a disciplinary board, noting a profound "lack of remorse and a lack of concern for his client." As AI tools become deeply integrated into everyday professional workflows, this case draws a hard line in the sand for accountability. Generative AI can be a powerful brainstorming partner or drafting assistant, but in high-stakes fields like law, medicine, and finance, where accuracy is paramount, human verification is not just recommended—it is an absolute ethical obligation. Technology may accelerate the writing process, but the ultimate liability always rests with the human at the keyboard.

Key Points

• A New Mexico attorney with 40 years of experience was held in contempt of court for filing a ChatGPT-generated brief.
• The AI fabricated police testimony, non-existent witnesses, and fake legal citations to support a murder appeal.
• The lawyer admitted to signing and submitting the document without verifying any of the AI's factual claims.
• The case highlights the dangers of 'AI hallucinations' and the critical need for human oversight in high-stakes professions.

Why It Matters

As generative AI becomes a common workplace tool, this incident underscores that professionals remain entirely responsible for the accuracy of their work, regardless of the software they use.

Sources:

• ChatGPT-using lawyer punished for citing fake testimony from made-up witnesses — Ars Technica AI