Study says AI rollout in trial courts raised case disposals by over 6pc

ISLAMABAD: A large field study on the use of generative artificial intelligence in Pakistan’s trial courts has found that the technology was linked to an estimated 1,848 additional cases being resolved each year, equivalent to a 6.3 per cent increase over the average.

The findings were presented in a research paper published on Tuesday, titled Courts of Tomorrow: Evidence from a Nationwide Rollout of Generative AI. The paper was authored by researchers Sultan Mehmood, Christoph Goessmann and Elliott Ash. According to the study, the AI assistant used in the trial was a customised chatbot called JudgeGPT, built on OpenAI’s GPT-4 family of models and adapted for use in Pakistan. It was beta-tested intensively with the Federal Judicial Academy before being rolled out to 1,559 judges working across 118 courts.

The paper said the strongest results came when access to the tool was paired with focused instruction for judges. It found that judges who received both the AI tool and targeted training were more likely to adopt it, use it more often and continue using it over time. Their views of the technology also changed, with the study saying they expected the tool and the training to improve productivity.

“This increase does not appear to come at the expense of reduced quality,”

the research paper observed.

How the trial was conducted

The study said the trial covered roughly half of the country’s trial judges and 80pc of district courts. The 1,559 judges were randomly placed into three groups. One group received JudgeGPT along with targeted training on using it in judicial work. A second group was given the tool but only generic training on technology and law. The control group received generic training without access to the AI assistant.

Researchers measured outcomes through a baseline survey on judges’ attitudes toward generative AI and a follow-up survey conducted about three months after the rollout to assess post-treatment perceptions and expected gains in productivity. They also reviewed JudgeGPT platform records to track uptake and identify the types of tasks judges assigned to the tool.

In addition, district-level administrative court records were examined to assess whether assigning judges to the AI system increased local case resolution. Judicial opinions from before and after the trial were also analysed to evaluate effects on writing quality and to test whether AI changed written attitudes relating to gender or religion.

Use patterns and quality findings

According to the study, judges mainly used the assistant for legal research and writing support. It said targeted training appeared to direct usage toward tasks where language models were more useful, such as text improvement and summarisation, while reducing reliance on more open-ended legal questions whose answers were harder to verify.

The paper found that written opinions from judges who received the intervention contained more text identified as AI-generated than those in the control group, but it said there was little indication that this harmed writing quality.

“If anything, there is a positive effect of AI on the quality assessment.”

The study also reported little evidence of systematic shifts in pro-Muslim or gender bias in judicial language.

It added that targeted training moved use toward bounded support tasks rather than full-text generation, which the authors said was better suited to large language models and more likely to preserve judicial agency.

“We do not study AI as a replacement for judges,”

the paper stated.

“We study it as a tool that may change how judges carry out recurring parts of their work.”

In its conclusion, the paper said AI was not a cure-all for court backlogs, but could serve as a practical way to improve state capacity when designed around relevant legal materials and paired with training that directs judges toward appropriate uses.

In April, the National Judicial Policy Making Committee formally issued national guidelines on AI use in judicial institutions. According to the guidelines, AI is to support rather than replace judicial decision-making, with judges remaining the ultimate arbiters. The framework also emphasises ethical and transparent use, safeguards against bias, and requirements related to explainability and accountability.