Attorneys balance use of powerful AI tools with risks — including legal hallucinations

The 14-page motion looked like any standard court filing in a civil lawsuit. Filed in April in San Diego Superior Court, attorneys defending a local company were making a routine request for a judge to authorize additional expert witnesses in a vehicle collision injury case.

But on the sixth page of the motion, the attorneys cited legal authority from a case that does not exist — it was hallucinated by artificial intelligence.

A San Diego judge wrote in October that she was “deeply troubled” by the conduct of the company’s civil defense attorneys. She found they had filed multiple documents containing AI hallucinations, including citations to non-existent cases, fake quotes from real cases and inaccurate citations to real but irrelevant legal authorities.

That is one of at least two known San Diego cases that are part of a troubling trend of attorneys abusing or misusing AI, especially generative AI, a powerful technology that can at times fabricate information with no basis in reality.

But even as legal watchdog groups have documented hundreds of AI hallucination cases in the U.S. and around the world, legal experts and attorneys say those cases are rare, arguing that AI is an important tool being put to good use throughout the legal profession, helping lawyers research case law, analyze e...