In brief
The survey, conducted Oct 6‑16, revealed that 80 % of judges had seen nonexistent or false AI‑generated precedents cited in civil, family, and administrative cases. Nearly a decade of the judges themselves and many lawyers had used such citations, and 92 % urged measures such as fines and verification requirements. The main uncertainty remains how to effectively implement these safeguards to prevent AI hallucination in courtrooms.
A recent survey conducted by the Supreme Court of South Korea found that about 80 percent of judges have encountered AI‑generated fake legal precedents in court cases, highlighting the growing problem of artificial‑intelligence hallucination in the judicial process.
The survey, carried out over 11 days from October 6 to 16, covered 360 judges who preside over civil, family and administrative matters, providing a representative sample of the country’s judiciary under the auspices of the Supreme Court.
According to the results, 196 judges, or 80.7 percent, reported that they had witnessed cases in which nonexistent legal precedents generated by computers were cited as if they were real law.
In addition, 159 judges, equating to 65.4 percent, said they had come across cases involving the use of false AI‑created precedents that were not supported by actual court decisions.
A minority of respondents—41 judges, or 16.9 percent—stated they had seen instances where AI systems hallucinated laws and regulations that do not exist, while 10 judges, or 4.1 percent, reported encountering fabricated theses and academic publications used as evidence.
Among the judges surveyed, 164 signaled that they themselves had cited such false precedents in court, revealing that even the arbiters of law are not immune to the influence of misleading AI outputs.
Further, 136 judges, or 57.6 percent, acknowledged that attorneys representing parties in these cases had also introduced counterfeit legal references or other misinformation into proceedings.
An overwhelming 92 percent of the respondents expressed the belief that institutional measures are necessary to prevent the recurrence and spread of AI‑generated fake precedents.
Respondents outlined several potential safeguards, including the creation of legal grounds for imposing fines on offenders, the establishment of verification requirements for legal citations, and the development of disciplinary protocols against lawyers who repeatedly submit falsified legal material.
An official at the National Court Administration emphasized that the improper use of AI‑generated citations can significantly delay court proceedings, underscoring the practical urgency of designing effective countermeasures based upon the survey’s findings.
Despite the broad consensus on the problem’s severity, the survey does not provide a definitive blueprint for implementing the proposed safeguards, leaving questions about enforcement mechanisms, the scope of penalties, and the coordination between the judiciary and legal practice unresolved.
