Why in News?
The Supreme Court recently set aside a National Company Law Tribunal (NCLT) order after finding that it relied on six AI-hallucinated legal citations. Three cited judgments did not exist, while others were either wrongly quoted or unrelated to the issue.
What are AI Hallucinations?
-
-
- AI hallucination refers to the generation of false, fabricated, or inaccurate information by AI systems while presenting it as authentic.
- In the legal field, AI hallucinations can lead to:
- Fake judgments and citations.
- Incorrect interpretation of laws.
- Misleading legal arguments.
- Faulty judicial decisions.
- Fake judgments and citations.
- AI hallucination refers to the generation of false, fabricated, or inaccurate information by AI systems while presenting it as authentic.
-
Background of the Case:
-
-
- Essel Infraprojects Ltd had provided a corporate guarantee for a ₹200 crore loan given by Jammu and Kashmir Bank to Pan India Utilities Distribution Company Ltd.
- After the borrower defaulted, the bank initiated insolvency proceedings against Essel under Section 7 of the Insolvency and Bankruptcy Code (IBC). Essel argued that its liability had shifted after a 2014 demerger and amalgamation approved by the Bombay High Court.
- The NCLT and NCLAT rejected Essel’s defence, but the Supreme Court later set aside the order, finding that it relied on fabricated legal precedents.
- Essel Infraprojects Ltd had provided a corporate guarantee for a ₹200 crore loan given by Jammu and Kashmir Bank to Pan India Utilities Distribution Company Ltd.
-
Supreme Court’s Observations:
-
-
- The NCLT’s order was based on six cited judgments, but the Supreme Court found serious irregularities in these references. Three judgments were non-existent, two authentic judgments did not support the propositions attributed to them, and one citation belonged to an entirely different case.
- The Court noted that these references were not placed before the tribunal by either party and appeared to have been generated through AI-assisted research.
- The Supreme Court observed that reliance on fabricated case law undermines the foundation of judicial decision-making. It held that even a small amount of unverified AI-generated material can compromise the validity of a judgment and directed the Bar Council of India to examine safeguards for the use of AI in legal proceedings.
- The NCLT’s order was based on six cited judgments, but the Supreme Court found serious irregularities in these references. Three judgments were non-existent, two authentic judgments did not support the propositions attributed to them, and one citation belonged to an entirely different case.
-
Key Challenges pose by AI in Judiciary:
-
-
- Accuracy Issues: AI may generate false citations and incorrect information.
- Bias: AI may reproduce biases present in historical legal data.
- Human Judgment Limitations: Judicial decisions require empathy, constitutional values, and contextual understanding.
- Regulatory Gaps: India lacks a comprehensive framework governing AI use in courts.
- Accuracy Issues: AI may generate false citations and incorrect information.
-
Way Forward:
-
-
- AI should remain an assistive tool, not a replacement for judges.
- Develop ethical guidelines and regulatory frameworks for AI use.
- Train judges, lawyers, and court staff in AI literacy.
- Ensure verification of AI-generated legal references.
- Conduct regular audits to prevent bias and errors.
- AI should remain an assistive tool, not a replacement for judges.
-
Conclusion:
AI has significant potential to improve India's judicial system by reducing delays, enhancing legal research, and improving accessibility. However, the issue of AI-hallucinated judgments shows that unchecked reliance on technology can threaten judicial credibility. A balanced approach based on human oversight, verification, transparency, and ethical regulation is essential for responsible AI integration in the justice system.
