Newest Version of ChatGPT Passes the Bar Exam

The latest version of the artificial intelligence program ChatGPT has passed the Uniform Bar Examination by earning a combined score of 297, a mark that surpasses even the highest threshold score set by any individual state.

GPT-4 took all sections of the July 2022 bar exam and earned a score so high that it approaches the 90th percentile of test-takers, according to researchers Daniel Martin Katz, a professor at the Illinois Institute of Technology’s Chicago-Kent College of Law, and Michael James Bommarito, a professor at the Michigan State University College of Law.

“Our analysis highlights that GPT-4 has indeed passed the bar and has done so by a significant margin,” they wrote in a paper posted on March 15th. The professors collaborated with legal AI company Casetext.

GPT-4 took all sections of the bar exam and did particularly well on the multiple-choice section known as the Multistate Bar Examination (MBE). GPT-4 got 75.7% of the questions correct on the multiple-choice MBE, compared to the human average of 68%.

GPT-4 got a passing grade in all seven subjects tested on the MBE, doing best in contracts (answering 88.1% of the questions correctly), followed by evidence (85.2%) and criminal law and procedure (81.1%).

Two of the researchers assigned scores to the essay questions on the Multistate Essay Examination and also got input from peers.

“While GPT-4 performs well on many questions, its output is not completely free of errors,” the researchers concluded.

Yet GPT-4 got a score of 4.2 out of 6 points. Most jurisdictions use the same scale, and a score of at least 4 on an essay is considered passing.

On the Multistate Performance Test, GPT-4 also received a score of 4.2 out of 6 points.

“We were somewhat surprised at the quality of the output which was generated,” the researchers wrote.

The previous version of ChatGPT had failed the multiple-choice section of the bar exam just last year, so this new improved result surprised many.

You can read more about this result here.

And for the humans that still have to pass the bar exam, if you just received your exam results and were unsuccessful, we will give you a completely free score evaluation from our bar exam experts to show you what went wrong and exactly what you need to do to pass on the next try. We are dedicated to helping retaker students pass and we urge you to contact us right away, as the sooner we speak to you, the sooner we can help you set up a gameplan for success on the exam. Just submit your score report to us here or email us at [email protected].