Anthropic's Claude Hallucinates Legal Citation - Lawyer Apology

AI-Generated Citation Error in Legal Case

Anthropic, through its legal representation, has acknowledged the use of an incorrect citation originating from its Claude AI chatbot within the context of its current legal proceedings against music publishers.

The filing, submitted to a Northern California court on Thursday, details that Claude produced a citation containing both an inaccurate title and incorrect author attributions.

Details of the Error

Anthropic states that a manual review of citations by their legal team failed to identify this error, alongside several others stemming from the chatbot’s tendency to hallucinate information.

The company has issued an apology, characterizing the incident as a genuine mistake in citation rather than a deliberate attempt to fabricate legal authority.

Allegations and Court Response

Previously, legal counsel for Universal Music Group and other music publishers alleged that Anthropic’s expert witness, Olivia Chen, had utilized Claude to include fabricated articles within her submitted testimony.

Following these accusations, Federal judge Susan van Keulen requested a response from Anthropic regarding the claims.

Broader Implications

This lawsuit represents one of numerous legal conflicts arising between copyright holders and technology companies concerning the potential misuse of copyrighted material in the development of generative AI technologies.

Growing Trend of AI Errors in Legal Settings

This incident is not isolated. A similar situation recently occurred where a California judge criticized law firms for submitting “bogus AI-generated research.”

Furthermore, an Australian lawyer faced repercussions in January after being discovered to have used ChatGPT to prepare court documents, resulting in the inclusion of flawed citations.

Continued Investment in AI Legal Tools

Despite these documented errors, significant investment continues to flow into startups focused on automating legal processes.

Harvey, a company leveraging generative AI to aid lawyers, is reportedly in negotiations to secure over $250 million in funding, potentially reaching a $5 billion valuation.

• Claude is an AI chatbot developed by Anthropic.
• Universal Music Group is a major player in the music publishing industry.
• ChatGPT is an AI chatbot developed by OpenAI.