Governing AI Hallucinations: Evaluating the Supreme Court’s Zero-Tolerance Rule

An analysis of the Supreme Court’s decision in Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd., examining its zero-tolerance approach to AI-generated hallucinations in judicial decisions, the materiality-based alternative, verification burdens, professional accountability, and the impact on judicial integrity and finality.

On 2 July 2026, the Supreme Court passed its first substantive decision governing AI hallucinations in judicial decision-making. In Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd.¹, the Supreme Court set aside orders of the National Company Law Tribunal (NCLT) and the National Company Law Appellate Tribunal (NCLAT) solely on the ground that both tribunals had relied on non-existent, artificial intelligence (AI)-generated case law. In doing so, the Supreme Court adopted a “zero-tolerance” rule for decisions relying on hallucinated AI-generated material, holding that “such a decision is no decision in the eyes of the law”.

It is undeniable that courts and lawyers must not rely on unverified AI-generated material. As probabilistic tools, AI models function on statistical predictions of the next word in a sentence rather than producing verified facts. To that end, all AI tools are prone to hallucinating or making up false citations and quotes.² At the same time, AI tools have exponential capacity to transform court processes and expedite access to justice. The Supreme Court’s own Draft Regulations for Use of AI in Courts, 2026, promote responsible AI “as a catalyst for impactful innovation in the justice delivery system”.³ The Draft Regulations further recognise “legal research, precedent retrieval, citation verification and document summarisation” as permissible uses of AI systems.⁴

Given the widespread use and transformative potential of AI in legal workflows, courts must decide a fundamental question: How should we govern AI-generated errors or hallucinations in decision-making? In the case of Pooja Ramesh Singh, the Supreme Court answered that question with a bright-line rule — even an “iota of fake or hallucinated material” would automatically invalidate the underlying decision. This article, instead, argues for a materiality-based approach that asks whether the AI-generated error materially affected the reasoning or outcome of the case.

Factual background

The proceedings in the case of Pooja Ramesh Singh⁵ arose from certain credit facilities availed by Pan India Utilities Distribution Company Ltd. (PIUDCL) from Jammu and Kashmir Bank Ltd. These credit facilities were obtained on the corporate guarantee of Essel Infraprojects Ltd. (EIL). Subsequently, when PIUDCL defaulted on its payment obligations, the Bank filed an application under Section 7, Insolvency and Bankruptcy Code, 2016 (IBC) to initiate a corporate insolvency resolution process against EIL. On 28 August 2024, the NCLT admitted the insolvency application, relying on six purported Supreme Court decisions.⁶

On independent verification, the Supreme Court discovered that all these decisions were tainted by AI hallucinations. Two were valid cases with correct citations, but the NCLT quoted non-existent paragraphs from them — Canara Bank v. N.G. Subbaraya Setty⁷. One judgment existed but the quoted citation pertained to another case law- Everest Kento Cylinders Ltd. v. Union of India⁸. One decision had both a wrong case name and an invented passage — SBI v. Shree Ram Urban Infrastructure Ltd.⁹. Three other decisions were entirely fictitious — those case names and citations simply did not exist — ICICI Bank Ltd. v. Urban Infrastructure Real Estate Ltd.¹⁰; VS Dempo & Co. Ltd. v. Reliance Communications Ltd.¹¹; and Sarbjit Singh v. Union Bank of India¹².

Cumulatively viewed, this was not a clerical mistake but an unverified reliance on generative AI output. On appeal, the NCLAT compounded the error by simply reproducing all six decisions in a single paragraph without verifying them.¹³ The hallucinated cases escaped the scrutiny of both the advocates and the appellate forum.

Critically, however, the NCLAT’s decision did not rest solely on the fabricated citations. EIL had not disputed the underlying debt or default. Its primary substantive defence — that its liabilities had been transferred to another company by a scheme of demerger and a subsequent amalgamation — was rejected on an independent, fact-specific ground by the NCLAT. Clause 8 of the guarantee deed expressly provided that the guarantee would survive any absorption or amalgamation of the debtor with any other company. To that end, the decision had an independent basis, outside the fabricated citations. However, the Supreme Court did not concern itself with the merits of the fact-based adjudication. In the court’s own words, such a decision must be set aside “irrespective of whether such (AI-hallucinated) material had a direct or indirect bearing on the decision-making”.

Materiality-based standard

Compare this with the materiality-based standard adopted by the Andhra Pradesh High Court in Gummadi Usha Rani v. Sure Mallikarjuna Rao¹⁴. There, the High Court held that the mere presence of an AI-generated citation would not, by itself, vitiate a judicial order if the applicable law had otherwise been correctly applied.¹⁵ The High Court distinguished between the presence of an AI hallucination and its impact on the decision. An AI error is fatal only “if the principle of law applied is not the law of the land or its application in a given case is faulted because of relying on non-existent rulings generated by AI”.

(emphasis supplied)

A similar approach was adopted by the High Court of Jammu and Kashmir and Ladakh.¹⁶ While the High Court found that the impugned order relied on two AI-hallucinated precedents, it held that “this circumstance, by itself, does not persuade this court to interfere with the impugned order on merits…” Rather, the court cautioned that any precedent generated or suggested by an AI tool “must be independently verified from authentic and authoritative sources before being relied upon in a judicial order”.

This divergence between the Supreme Court’s bright-line rule and the High Courts’ materiality-based standard is not new. Law and economics scholarship has long debated the relative merits of rules and standards. The following section builds on that scholarship and also examines whether the Supreme Court’s underlying justifications for a bright-line rule — verification burdens, deterrence, and institutional integrity — actually hold up.

Rules versus standards

In his seminal paper, Louis Kaplow argues that rules (such as “any hallucinated citation voids the order”) are cheap to apply and highly predictable, because they require no case-specific enquiry at the point of application. In contrast, a standard (such as “vitiate only if the hallucination was material to the outcome of the case”) pushes the cost of adjudication downstream, requiring a fresh, resource-intensive enquiry every time it is invoked.¹⁷

While it is true that appellate courts must apply the materiality standard afresh in every case, such adjudication is neither novel nor exceptional for appellate courts. The courts routinely determine whether a particular authority was correctly applied to the facts before them, whether it can be distinguished or narrowed, and the extent to which it influenced the lower court’s reasoning and decision. A materiality-based standard, therefore, does not impose an onerous obligation on an ill-suited institution. Rather, it requires appellate courts to perform the very function they are institutionally designed to discharge. On the other hand, a bright-line rule fetters the discretion of appellate courts — they must set aside the impugned decision even if the AI-hallucination is trivial.

Compared with the Supreme Court’s zero-tolerance rule, a materiality-based standard better aligns with the principle of proportionality, ensuring that the remedy is commensurate with the seriousness of the error. It also preserves the finality of adjudication by preventing immaterial errors from unsettling otherwise sustainable decisions.

Verification burden

Nevertheless, the court in the case of Pooja Ramesh Singh¹⁸ emphasises the additional verification burden of a subjective materiality standard:

16. Today’s courts and Tribunals implicitly trust lawyers when referring to precedents cited before them. Imagine the hardship of a situation in which the court must verify the authenticity of each judgment cited by an advocate.

But a zero-tolerance rule does not absolve courts of their verification obligations. If anything, it raises the stakes of getting verification wrong, since even a single unverified, non-material citation now voids the entire decision. In that sense, the Supreme Court does not address the verification burden; it simply imposes a harsher penalty if courts and advocates fail to discharge it sufficiently.

Further, a better institutional response to the “hardship” of verification is to build infrastructure and capabilities that make detecting AI hallucinations easier and cheaper. This includes training lawyers and judges to recognise patterns of hallucination and building or adopting tools that cross-check AI-generated materials against authoritative sources and databases.¹⁹

Deterrence

Admittedly, a zero-tolerance rule serves as a strong deterrent. If advocates, law clerks, and judges know that failure to verify an AI-generated citation could invalidate an entire judgment, they are likely to exercise significantly greater caution. But this raises a distinct jurisprudential question: Should the law aim to deter the responsible actor, or should it invalidate the underlying decision itself?

In the landmark case of Mata v. Avianca, Inc.²⁰, a US Federal Court sanctioned the plaintiff’s attorneys, fining them $5,000 for acting in bad faith — for misleading the court with AI-generated fake citations. In R. (Ayinde) v. Haringey London Borough Council²¹, the UK High Court noted that placing false material before the court “with the intention that the court treats it as genuine may, depending on the person’s state of knowledge, amount to a contempt”. The focus in both cases is on professional accountability rather than automatic invalidation of the underlying proceedings.

Notably, the Indian Supreme Court itself appears to endorse an actor-specific model of deterrence. In the case of Pooja Ramesh Singh²², the court directed the Bar Council of India to prescribe guiding principles “to prevent such occurrences, along with the disciplinary action that will follow a violation of the norms”. Deterrence may justify imposing sanctions on the individual responsible for the lapse. It does not justify treating the presence of AI-hallucinated material as automatic grounds for nullifying a decision that would otherwise be sustainable.

More fundamentally, deterrence should not be the judiciary’s sole response to the increasing use of AI. As generative AI becomes deeply embedded in legal research, analysis, and drafting, the long-term objective must be to build institutional capacity for its responsible use. In this respect, the Supreme Court’s Draft Regulations represent a promising development, requiring judges, advocates, and court staff to undergo structured training in the technical, legal, and ethical aspects of AI-assisted legal work.²³ Where the risk arises as much from unfamiliarity with the technology as from poor professional ethics, the more effective response may be ex ante investment in institutional capability rather than ex post disciplinary sanctions.

Institutional integrity

Crucially, the strongest argument advanced by the Supreme Court in favour of a zero-tolerance rule is not the verification burden or deterrence, but the protection of judicial integrity. The court repeatedly emphasised that fabricated authorities undermine “the integrity of the adjudication and its processes”, making it necessary to set aside the underlying judgments.

Preserving the integrity of judicial proceedings is an undeniable good. But it is not a standalone good that overrides all competing considerations. In this case, maintaining institutional integrity comes at a concrete and measurable cost: further delay in the insolvency resolution process. Setting aside the NCLT and NCLAT orders reintroduces exactly the kind of delay the IBC was built to prevent, and it does so for a reason unconnected to the merits of the underlying claim.

The consequences are not purely theoretical. Nearly two years after the NCLT’s original order, the Jammu and Kashmir Bank’s Section 7 application must be decided afresh. According to the Bank’s affidavit, its counsel had never cited the six fabricated authorities; they appear to have been introduced through the Tribunal’s own research. Although the Supreme Court directed the NCLT to dispose of the application expeditiously, preferably within two weeks, that direction cannot undo the delay already occasioned by restarting the insolvency proceedings. The burden of the court’s zero-tolerance rule, therefore, falls not on the actor responsible for the hallucination but on the litigants whose dispute remains pending.

The challenge for courts is not whether AI-generated hallucinations should attract serious consequences (they undoubtedly should), but how those consequences should be allocated. That is a complex institutional question requiring courts to balance competing values of materiality, accountability, judicial integrity, and the finality of adjudication. A zero-tolerance rule, unfortunately, offers an over simplistic answer to that layered question.