REPORT 2025-051 - California State Auditor - Summary

Sep 21, 2026

Uploaded by McGucket

Audit report on the California State Bar’s February 2025 bar exam administration and its implementation of exam changes. Finds that the State Bar poorly executed key steps in developing AI-generated multiple-choice questions, including inadequate specification of required legal topics and flawed content validation. Also identifies weak vendor management for Meazure Learning, insufficient performance evaluation criteria, and failure to address known technical issues from a prior pretest. Resulted in higher pass rates, $5.1 million costs, and at least $4 million in revenue losses.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages73

Reading Time3 h 4 min

AccessFree to download

Summary

Key Findings and Recommendations

Introduction

Audit Results

The State Bar’s Timeline and Poor Planning Created Challenges With Question Development

Question Issues Forced the State Bar to Remove More Questions From Scoring Than It Had Intended

The State Bar Contracted With Meazure Learning Without Fully Verifying Its Ability to Effectively Administer the February 2025 Bar Exam

The State Bar Fell Short of Preventing the Technical and Administrative Failures That Occurred in the February 2025 Bar Exam

The State Bar Experienced No Cost Savings on the February 2025 Bar Exam and at Least $4 Million in Revenue Losses

Other Areas We Reviewed

Recommendations

Appendix A

Appendix B

Response to the Audit

The February 2025 Bar Exam The State Bar’s Poor Implementation of Changes to the Exam Negatively Affected Test Takers July 2026 REPORT 2025-051 | | CALIFORNIA STATE AUDITOR
621 Capitol Mall, Suite 1200 | Sacramento | CA | 95814 | | --- | --- | | 916.445.0255 | TTY 711 | | | For complaints of state employee misconduct, contact us through the Whistleblower Hotline:
1.800.952.5665 | | Don’t want to miss any of our reports? Subscribe to our email list at [auditor.ca.go](auditor.ca.gov)[v](auditor.ca.gov) For questions regarding the contents of this report, please contact our Communications Office at 916.445.0255 This report [is also available online at www.auditor.ca.gov](is also available online at www.auditor.ca.gov | Alternative format)[ |](is also available online at www.auditor.ca.gov | Alternative format)[ Alternative format](is also available online at www.auditor.ca.gov | Alternative format) reports available upon request | Permission is granted to reproduce reports Grant Parks State Auditor Laura Kearney Chief Deputy July 9, 2026 2025-051 The Governor of California President pro Tempore of the Senate Speaker of the Assembly State Capitol Sacramento, CA 95814 Dear Governor and Legislative Leaders: As required by Business and Professions Code section 6145, subdivision (d), my office conducted an audit of the State Bar of California (State Bar) and its administration of the California Bar Examination administered in February 2025 (February 2025 bar exam). In general, we determined that the State Bar poorly implemented certain changes to the bar exam, which had a negative effect on test takers. In part to reduce costs and address the solvency of its Admissions Fund, the State Bar decided in 2024 to switch from administering the bar exam in person to offering test takers the option to take the bar exam remotely or in person. To make this switch, the State Bar contracted with Kaplan Exam Services, LLC (Kaplan) to develop multiple-choice questions for the bar exam. However, the State Bar did not specify to Kaplan certain legal topics required for the bar exam, and the State Bar asked another of its vendors, ACS Ventures, LLC (ACS Ventures) to develop additional exam questions, which it did using artificial intelligence. Although the State Bar engaged in a content validation process to review, edit, and validate all the newly developed multiple-choice questions, the State Bar’s execution of this process had flaws that hindered its effectiveness. The State Bar also contracted with ProctorU Inc., dba Meazure Learning (Meazure Learning), toadminister the February 2025 bar exam without first defining business requirements or establishing criteria to evaluate the vendor’s performance. It also contracted with Meazure Learning despite knowing that it lacked an important tool to administer the exam’s performance test portion. Further, the State Bar did not ensure that Meazure Learning addressed the causes of computer freezes, crashes, and error messages that test takers faced during a November 2024 pretest. The technical and administrative problems that test takers experienced caused the State Bar to raise its pass rate from about 36 percent of test takers to about 65 percent of test takers ofthe February 2025 bar exam. The State Bar did not achieve cost savings in its administration of the February 2025 bar exam, which has cost $5.1 million, not including the costs of pending legal matters, and it experienced at least $4 million in lost revenue from the remedies it provided to February 2025 test takers, which included exam refunds and fee waivers. Respectfully submitted, GRANT PARKS California State Auditor 621 Capitol Mall, Suite 1200 | Sacramento, CA [95814 | 916.445.0255 |](95814 | 916.445.0255 | www. auditor. ca . gov)[ www. auditor. ca . gov](95814 | 916.445.0255 | www. auditor. ca . gov) iv CALIFORNIA STATE AUDITOR July 2026 | Report 2025-051 Selected Abbreviations

FAQs

What was the audit focused on regarding the February 2025 bar exam?

It reviewed how the State Bar implemented changes to the exam and administered the February 2025 California Bar Examination, including question development processes and vendor administration.

How did the State Bar develop new multiple-choice questions for the February 2025 exam?

It contracted for Kaplan to develop multiple-choice questions and asked another vendor, ACS Ventures, to develop additional questions using artificial intelligence, then conducted a content validation process with flaws.

Why did test takers experience significant problems during the February 2025 exam?

The State Bar did not define requirements or evaluation criteria for Meazure Learning in advance and did not ensure the vendor addressed causes of technical issues seen in a November 2024 pretest, contributing to computer freezes, crashes, and error messages.

Sep 21, 2026

Uploaded by McGucket