Overview

AI is transforming legal practice, but with significant limitations—lawyers who effectively use AI will replace those who don't, though AI itself won't replace lawyers entirely.

"Legal hallucinations" represent a critical problem, with research showing alarming rates (58-88% in state-of-the-art models) where AI generates fictitious case citations and legal references, leading to potential sanctions for attorneys.

Current AI systems often undermine access to justice goals by providing incorrect information to underrepresented individuals and pro se litigants, while failing to understand that legal judgment involves more than just retrieving information.

AI tends to improve performance of less experienced legal professionals while offering minimal benefits to high-performers, creating an uneven impact across the profession.

Stanford Law School is pioneering responsible AI integration through educational initiatives that reveal both AI capabilities and limitations, focusing on controlled use cases like e-discovery and document review rather than core legal reasoning.

Content

AI's Impact on the Legal Profession

The podcast features experts Dan Ho (Stanford Law Professor, Computer Science Professor) and Mirach Suzga (J.D./Ph.D. student, Law and Technology expert) discussing AI's impact on legal practice.

A key perspective emphasized throughout: AI won't replace lawyers, but lawyers who effectively use AI will replace those who don't.

AI-assisted tools are being rapidly adopted for legal tasks like document review and research, similar to how e-discovery technology previously transformed legal work.

"Legal Hallucinations" and AI Limitations

"Legal Hallucinations" represent a significant problem in AI legal applications:

- Lawyers have submitted fictitious case citations generated by AI - A Wyoming judge threatened to sanction lawyers for using fabricated legal references - Chief Justice Roberts has highlighted hallucination as a serious concern

Initial research found alarming hallucination rates:

- Between 58% to 88% in state-of-the-art models - Even specialized legal research engines showed hallucination rates of 20-33%

Language models struggle with fundamental distinctions because they:

- Are trained on massive textual data including both fictional and non-fictional content - Learn statistical patterns of language use rather than developing true understanding - Cannot reliably distinguish between fictional and real information - Lack comprehension of legal hierarchies and context

Specific Examples of AI Legal Failures

The speakers discuss a fictional judge, Luther A. Willgarten Jr., who:

- Originated from an after-dinner speech - Was inserted into a law review article as a joke - Demonstrates how AI can generate false information about non-existent sources

AI models demonstrate concerning limitations:

- Treating fictional sources (like Harry Potter) with the same authority as legal documents - Lacking understanding of basic legal concepts taught to first-year law students - Producing absurd claims (e.g., suggesting Nebraska Supreme Court overruled U.S. Supreme Court) - Confusing different legal actors (e.g., mixing up Justice Ginsburg with her daughter)

In the Harvard vs. SFA case, AI models incorrectly stated that the decision explicitly overruled previous cases, demonstrating ongoing challenges in AI's legal comprehension.

Research Methodology and Findings

Researchers evaluated generative AI legal systems by:

- Designing legal research questions - Manually reading and grading AI-generated essay answers - Checking legal propositions for accuracy and grounding

Key challenges identified with AI legal models:

- Lacking humility and always attempting to provide an answer - Inability to acknowledge uncertainty or say "I don't know" - Inherent design to generate responses rather than assess accuracy - Lack of nuanced understanding of legal context and precision

Access to Justice Implications

AI systems currently underperform in critical areas:

- Struggle with trial court level decisions - Misfire in scenarios most needed by underrepresented individuals - Provide potentially incorrect information for pro se litigants - Risk undermining the promise of improving legal access

Legal judgment involves more than just retrieving statutes or cases:

- There's value in the process itself, beyond pure accuracy - Example: Veterans wanting to "be heard" even if they might lose their case - People feel better about outcomes if they feel they've been properly listened to

Retrieval-Augmented Context (RAC) Systems

RAC retrieves relevant documents before generating output, distinguishing it from traditional generation systems

Challenges with RAC include:

- Potentially favoring recent case law over authoritative case law - Not accounting for jurisdictional nuances - Lacking nuanced understanding of legal contexts

Fact-checking AI outputs can be time-consuming, potentially undermining efficiency gains

AI Assistant Performance Patterns

Research suggests AI tends to:

- Improve performance of less experienced professionals - Minimally boost performance of high-performing professionals

Concerns with AI legal tools include:

- Bar associations focusing on general AI training may miss tool-specific risks - Example: Westlaw system potentially generating passages from overruled cases while removing citations - Lack of transparency in AI-generated legal content

Stanford Law School's AI Integration

Professors are actively incorporating AI into legal education through:

- Using AI to generate legal fact patterns - Hosting AI and law workshops - Conducting exercises that reveal AI language model limitations - Planning future workshops on AI usage in legal contexts

Stanford is positioning itself as a leader in exploring AI's legal applications with emphasis on understanding AI's capabilities and limitations.

Responsible AI Integration in Legal Practice

More controlled use cases are recommended:

- E-discovery - Document review - Deposition summarization

Judge Newsom is noted as using AI tools (Claude and GPT-4) for statutory and contract interpretation, including analyzing specific legal language definitions.

Understanding AI technology's design and potential pitfalls is crucial for responsible legal application.

AI models demonstrate "model sycophancy" - tendency to respond to users regardless of premise accuracy.

The podcast was identified as "Stanford Legal" hosted by Pam Carlin.

AI, Liability, and Hallucinations in a Changing Tech and Law Environment