Supreme Court flags the risks of using AI for drafting petitions

UPSC Study Note: Supreme Court Flags Risks of Using AI for Drafting Petitions

1. At a Glance

• The Supreme Court of India has publicly flagged the "alarming" trend of lawyers using Generative AI (GenAI) to draft legal petitions, resulting in citation of fictitious/non-existent case laws — a phenomenon called AI hallucination. [S1]
• This is a landmark intersection of technology governance, judicial ethics, and legal accountability — directly relevant to UPSC GS-II (Judiciary) and GS-IV (Ethics). [S1]
• The Court has moved from verbal warnings to formal regulatory drafting: a Draft Regulations for Use of AI in Courts, 2026 was published on 3 June 2026. [S2]
• UPSC aspirants must understand both the technical concept of hallucination and its constitutional/ethical implications for judicial integrity. [S1]

2. Why in the News

• 18 February 2026: A Bench comprising Chief Justice Surya Kant, Justice B.V. Nagarathna, and Justice Joymalya Bagchi described the use of AI for legal drafting as "absolutely uncalled for." [S5]
• Justice Nagarathna specifically recalled a petition that cited a fabricated case — "Mercy vs. Mankind" — which does not exist in any law reporter. [S5]
• The Bench also referenced another case before a Bench led by Justice Dipankar Datta where non-existing judicial precedents were quoted. [S5]
• December 2025: Chief Justice Surya Kant had earlier called the court "very conscious of risks" from "indiscriminate use of GenAI in legal work," while hearing a petition filed by Kartikeya Rawal on GenAI dangers. [S5]
• January 2026: Bombay High Court imposed costs of ₹50,000 on a party for submitting fabricated case laws in written submissions. [S2]
• September 2025: Delhi High Court witnessed a petition withdrawal after opposing counsel exposed entirely fabricated citations. [S2]
• November 2025: The Supreme Court's Centre for Research and Planning released a White Paper on Artificial Intelligence and the Judiciary. [S2]
• 3 June 2026: The AI Committee of the Supreme Court of India published the Draft Regulations for Use of Artificial Intelligence in Courts, 2026, open for stakeholder comments. [S2]

3. Background & Evolution

• Pre-2023: AI tools were minimally used in Indian legal practice; research relied on databases like SCC Online, Manupatra, and Indian Kanoon.
• 2023 onwards: Explosion of Large Language Models (LLMs) such as ChatGPT, Gemini, and Copilot made AI-assisted legal drafting accessible to any litigant or junior advocate.
• Global precedent — Mata v. Avianca (USA, 2023): A New York attorney submitted a brief with ChatGPT-fabricated citations; the federal judge sanctioned the lawyers — this was the first major global wake-up call. [S3]
• India: Since mid-2023, over 120 cases of AI-generated legal hallucinations were globally identified; 58 occurred in 2025 alone. [S3]
• Income Tax Appellate Tribunal (India): Recalled an order after discovering reliance on fictitious case law — one of the earliest Indian institutional acknowledgements. [S2]
• December 2025: CJI Surya Kant's remarks during the Kartikeya Rawal petition — first formal Supreme Court observation on GenAI risks. [S5]
• February 2026: Supreme Court escalates from concern to formal condemnation ("absolutely uncalled for"). [S5]
• June 2026: Draft Regulations published — India moves toward a formal regulatory framework for AI in courts. [S2]

4. Core Static Facts

5. Multi-Dimensional Analysis

Legal / Constitutional

• Article 19(1)(g) (right to practise any profession) read with Bar Council of India Rules impose a duty of accuracy and truthfulness on advocates — citing fabricated precedent may amount to contempt of court and professional misconduct. [S2]
• The Supreme Court's plenary powers under Article 142 and its inherent power to regulate its own procedure give it authority to frame AI-use regulations without waiting for Parliamentary legislation. [S2]
• The Draft Regulations, 2026 propose mandatory disclosure of AI tool identity, source tracing, and human oversight certification — creating a new layer of procedural obligation. [S2]
• Citing non-existent precedent has been categorised by the SC as a "grave form of judicial misconduct" — potentially actionable under the Contempt of Courts Act, 1971. [S2]

Ethical / Governance

• The core tension: efficiency vs. accuracy — the SC explicitly noted that "easy means to do legal research should not be at the cost of accuracy." [S5]
• Access to justice paradox: AI lowers the cost of petition drafting for under-resourced litigants, but hallucinations can undermine the credibility of genuine grievances. [S3]
• Asymmetric risk: A lay litigant using AI cannot verify citations; the burden of verification must rest on the enrolled advocate who signs the petition. [S2]
• Raises the question of algorithmic accountability — who is liable when an AI fabricates a citation: the user, the platform, or both?

Scientific / Technological

• LLM hallucination occurs because these models predict statistically probable token sequences, not retrieve verified facts — legal citation databases are poorly represented in training data, increasing error rates. [S3]
• Retrieval-Augmented Generation (RAG) architectures, which ground outputs in verified document stores (e.g., SCC Online), are a partial technical mitigation but not yet widely deployed in Indian legal AI tools. [S3]
• Global analogues: EU AI Act (2024) classifies AI in justice administration as high-risk, requiring conformity assessments before deployment. [S4]

Administrative

• Judiciary lacks a central verification mechanism — there is no automated system to cross-check cited case numbers against official law reporters in real time. [S2]
• The Draft Regulations propose audit trails for AI-assisted filings, but implementation requires integration with e-Courts infrastructure (eCourts Mission Mode Project, Phase III). [S2]
• Capacity gap: Many district courts and tribunals lack trained personnel to detect AI-hallucinated citations — the problem is most acute below the High Court level. [S2]

Social

• Democratisation of legal drafting via AI disproportionately benefits first-generation litigants and rural users who lack access to experienced lawyers — making the hallucination risk a matter of access-to-justice equity. [S3]
• Vulnerable litigants who rely on AI-drafted petitions without legal counsel face the greatest exposure to penalty (costs, dismissal) for inadvertent hallucinated citations. [S3]

6. Recent Developments (Last 12–18 Months)

• November 2025: SC's Centre for Research and Planning releases White Paper on AI and the Judiciary. [S2]
• December 2025: CJI Surya Kant, hearing the Kartikeya Rawal petition, publicly flags GenAI risks in legal work for the first time from the Bench. [S5]
• January 2026: Bombay High Court imposes ₹50,000 costs on a party for AI-generated fictitious citations in written submissions. [S2]
• 18 February 2026: Three-judge Supreme Court Bench terms AI use in legal drafting "absolutely uncalled for"; Justice Nagarathna cites "Mercy vs. Mankind" as an example of hallucination. [S5]
• February 2026: SC flags a separate case before Justice Dipankar Datta's Bench involving fabricated judicial precedents. [S5]
• 3 June 2026: AI Committee of the Supreme Court publishes Draft Regulations for Use of AI in Courts, 2026 — seeks stakeholder comments. [S2]

7. Prelims Hooks

1. The Supreme Court Bench that flagged AI drafting risks on 18 February 2026 comprised CJI Surya Kant, Justice B.V. Nagarathna, and Justice Joymalya Bagchi. [S5]
2. The fictitious case law cited before Justice Nagarathna's Bench was "Mercy vs. Mankind" — it does not exist in any Indian law reporter. [S5]
3. The Supreme Court's Centre for Research and Planning released a White Paper on AI and the Judiciary in November 2025. [S2]
4. The Draft Regulations for Use of AI in Courts, 2026 were published by the AI Committee of the Supreme Court of India on 3 June 2026. [S2]
5. The Bombay High Court imposed costs of ₹50,000 for submitting AI-generated fake case laws — January 2026. [S2]
6. The global landmark case on AI hallucination in legal filings is Mata v. Avianca (Southern District of New York, 2023). [S3]
7. The term for a GenAI model confidently generating false information (e.g., non-existent judgments) is "hallucination". [S3]
8. The EU AI Act (2024) classifies AI tools used in justice administration as high-risk AI systems. [S4]
9. The Draft Regulations require lawyers to disclose the AI tool used, original sources, and human review process in every AI-assisted court filing. [S2]
10. The petition filed by Kartikeya Rawal specifically highlighted the dangers of Generative AI creating fictitious judgments — heard by the Supreme Court in December 2025. [S5]
11. Over 120 global cases of AI-generated legal hallucinations had been identified since mid-2023; 58 occurred in 2025 alone. [S3]
12. The SC stated AI must NOT "overpower the justice administration process" — a direct articulation of the principle of human oversight in adjudication. [S5]
13. The Income Tax Appellate Tribunal recalled an order after discovering reliance on fictitious AI-generated case law — one of India's earliest institutional acknowledgements. [S2]

8. Mains Relevance

GS Paper Mapping:

Plausible Mains Question Stems:

1. "The Supreme Court's flagging of AI-generated hallucinations in legal petitions raises fundamental questions about the intersection of technology, professional ethics, and access to justice. Examine." (GS-II + GS-IV)

2. "Critically analyse the regulatory challenges posed by Generative AI in India's judicial system. What principles should guide the Draft Regulations for Use of AI in Courts, 2026?" (GS-II + GS-III)

3. "Artificial intelligence democratises legal drafting but simultaneously threatens the integrity of judicial proceedings. How should India balance these competing imperatives?" (GS-IV)

9. Related Topics to Study Next

10. Common Errors / Trap Areas

1. Confusing "hallucination" with "bias": Hallucination = AI fabricates non-existent information; bias = AI systematically skews outputs based on training data. Both are distinct failure modes — examiners may test this distinction.

2. Attributing the February 2026 warning to a two-judge Bench: It was a three-judge Bench (CJI Surya Kant + Justices Nagarathna and Bagchi) — not a division bench of two.

3. Mixing up the regulatory body: The Draft Regulations, 2026 come from the AI Committee of the Supreme Court, not MeitY or NITI Aayog. Do not confuse with the National AI Strategy or the IndiaAI Mission.

4. Overstating the legal bar: The SC said AI use in drafting is "absolutely uncalled for" — but this is a judicial observation/policy direction, not yet a statutory prohibition. The Draft Regulations are still in the public comment phase as of June 2026.

5. Ignoring the global dimension: UPSC questions on AI in judiciary often expect comparative awareness. The Mata v. Avianca case (USA, 2023) and the EU AI Act are the two key global reference points — not mentioning them in a Mains answer weakens the response.

11. Sources

• [S1] India's AI-Driven Legal Future: Opportunities and Emerging Trends in 2025 — https://indiaai.gov.in/article/india-s-ai-driven-legal-future-opportunities-and-emerging-trends-in-2025 — (tier: 1, IndiaAI/MeitY portal)
• [S2] SC Says Citing AI-Generated Fake Case Laws Is Misconduct — https://www.medianama.com/2026/03/223-supreme-court-ai-fake-case-laws-misconduct/ — (tier: 4, journalism)
• [S3] The Phantom Brief: AI Hallucinations Threaten Legal Integrity and Professional Responsibility — https://markets.financialcontent.com/pennwell.pennenergy/article/tokenring-2025-11-20-the-phantom-brief-ai-hallucinations-threaten-legal-integrity-and-professional-responsibility — (tier: 3/4, reference journalism)
• [S4] India's Supreme Court Proposes Landmark AI Framework for Courts — https://www.indialaw.in/blog/cyber-law/supreme-court-ai-framework-courts-india/ — (tier: 4, legal journalism)
• [S5] Supreme Court flags the risks of using AI for drafting petitions — The Hindu, 18 February 2026 (article excerpt provided as primary source) — https://www.thehindu.com/todays-paper/2026-02-18/ — (tier: 4, Tier 4 primary source for this note)