Academic

LegalBench

Updated 10/13/2026

Evaluating language models on a wide range of open source legal reasoning tasks.

As of October 13, 2026, Claude Fable 5 ranks first on LegalBench with 88.56%, followed by Claude Fable 5.1 (88.51%) and Gemini 4 Argon (88.30%).

LegalBenchOpen-source legal reasoning tasks
ACCURACY

LegalBench leaderboard

Rank Model Accuracy Cost In / Out Latency
1 Claude Fable 5 88.56% $10 / $50 8.96s
2 Claude Fable 5.1 88.51% $10 / $50 12.40s
3 Gemini 4 Argon 88.30% $4 / $20 12.62s
4 Wonder Full 87.42% $4 / $20 38.27s
5 Gemini 3.1 Pro Preview (02/26) 87.40% $2 / $12 10.06s
6 Gemini 3.7 Flash 87.26% $1.5 / $7.5 2.23s
7 Gemini 3 Pro (11/25) 87.03% $2 / $12 8.33s
8 Gemini 3.8 Flash 86.99% $1.5 / $7.5 3.32s
9 Claude Opus 5 86.97% $5 / $25 4.42s
10 GPT-5.6 Sol 86.97% $4 / $20 6.20s
11 Gemini 3 Flash (12/25) 86.86% $0.5 / $3 4.67s
12 Gemini 3.6 Flash 86.70% $1.5 / $7.5 2.96s
13 GPT 5.5 86.52% $5 / $30 18.14s
14 Grok 4.6 86.31% $2 / $6 27.42s
15 GLM Legal 86.23% $0.075 / $0.25 6.00s
16 Kimi K3 86.21% $3 / $15 7.11s
17 GPT 5.4 (xhigh) 86.04% $2.5 / $15 27.79s
18 GPT 5 86.02% $1.25 / $10 18.59s
19 Grok 4.5 85.97% $2 / $6 67.88s
20 GPT 5.1 85.68% $1.25 / $10 7.05s
21 MiniMax-M3 85.42% $0.6 / $2.4 8.21s
22 Claude Opus 4.6 (Thinking) 85.30% $5 / $25 3.75s
23 Muse Spark 1.2 85.26% $1.25 / $4.25 30.10s
24 Claude Opus 4.7 85.25% $5 / $25 26.62s
25 GPT-5.6 Terra 85.11% $2 / $12 2.78s
26 Qwen 3.5 Plus 85.10% $0.4 / $2.4 30.42s
27 Muse Spark 1.1 84.98% $1.25 / $4.25 17.15s
28 Qwen 3.7 Max 84.91% $2.5 / $7.5 11.85s
29 GLM 5.3 84.84% $1.4 / $4.4 22.48s
30 Kimi K2.6 84.74% $0.95 / $4 50.07s
31 Claude Opus 4.5 (Thinking) 84.60% $5 / $25 7.64s
32 Grok 4.3 84.46% $1.25 / $2.5 5m48s
33 GLM 5.1 84.39% $1 / $3.2 13.75s
34 Grok 4.7 84.39% $2 / $6 26.02s
35 Gemini 2.5 Pro Exp 84.32% $1.25 / $10 4.02s
36 Qwen 3.5 Flash 84.28% $0.1 / $0.4 16.73s
37 Qwen 3.6 Plus 84.23% $0.5 / $3 19.07s
38 Muse Spark 84.22% N/A 34.20s
39 Claude Sonnet 4.5 (Thinking) 84.08% $3 / $15 13.29s
40 Gemini 3.5 Flash Lite 84.07% $0.3 / $2.5 1.67s
41 GLM 5.2 84.07% $1.4 / $4.4 5.77s
42 GLM 5 84.06% $1 / $3.2 16.06s
43 GPT-5.6 Luna 84.03% $0.2 / $1.2 5.89s
44 MiniMax-M2.7 83.98% $0.3 / $1.2 8.92s
45 GLM 5.3 Flash 83.93% $0.075 / $0.25 12.52s
46 Claude Sonnet 5 83.92% $2 / $10 4.90s
47 Gemini 2.5 Flash Preview 4/17 (Nonthinking) 83.80% $0.3 / $2.5 0.55s
48 Gemini 3.1 Flash Lite Preview 83.76% $0.25 / $1.5 2.70s
49 o3 83.76% $2 / $8 5.98s
50 Hy4 Preview 83.76% $0.834 / $2.501 90.56s
51 Qwen 3.8 Max 83.61% $2 / $6 36.37s
52 Gemini 3.5 Flash 83.60% $1.5 / $9 34.58s
53 Claude Opus 4.8 83.57% $5 / $25 3.11s
54 Claude Opus 4.1 (Nonthinking) 83.46% $15 / $75 2.79s
55 GLM 4.7 83.36% $0.6 / $2.2 30.68s
56 DeepSeek V4.1 Flash 83.28% $0.3 / $1.2 6.53s
57 Grok 4 83.19% $3 / $15 27.35s
58 Grok 3 Mini Reasoning 83.14% $0.3 / $0.5 5.09s
59 GPT 4.1 83.10% $2 / $8 0.51s
60 Claude Opus 4 (Nonthinking) 83.07% $15 / $75 3.73s
61 Mercury 2.5 83.06% $0.2 / $0.75 3.91s
62 Inkling Small 82.99% $0.3 / $1.2 17.66s
63 Claude Sonnet 4 (Nonthinking) 82.95% $3 / $15 4.85s
64 Inkling 82.95% $1 / $4.05 23.14s
65 Claude Opus 4.5 (Nonthinking) 82.84% $5 / $25 2.38s
66 GPT 5.2 82.76% $1.75 / $14 17.14s
67 Gemini 2.5 Flash Preview (9/25) (Thinking) 82.63% $0.3 / $2.5 1.97s
68 Grok 3 82.59% $3 / $15 0.74s
69 Claude 3.7 Sonnet (Thinking) 82.52% $3 / $15 6.54s
70 Gemini 2.5 Flash Preview (9/25) (Nonthinking) 82.49% $0.3 / $2.5 1.15s
71 Grok 4.1 Fast (Reasoning) 82.45% $0.2 / $0.5 8.58s
72 Qwen 3.8 27B 82.43% $0.5 / $3 47.81s
73 DeepSeek V4 Pro 0813 82.36% $1.32 / $3.96 20.05s
74 GPT 4o (2024-11-20) 82.21% $2.5 / $10 0.45s
75 Claude Sonnet 4.6 82.12% $3 / $15 21.63s
76 Nemotron 3 Ultra 82.07% N/A 5.18s
77 Claude Sonnet 4 (Thinking) 82.06% $3 / $15 10.61s
78 Gemini 2.5 Flash Lite (9/25) (Thinking) 82.04% $0.1 / $0.4 3.14s
79 Grok 3 Mini Reasoning 81.92% $0.3 / $0.5 3.44s
80 Qwen 3 Max 81.86% $1.2 / $6 N/A
81 GPT 5 Mini 81.77% $0.25 / $2 9.43s
82 Kimi K2 Instruct 81.45% $1 / $3 1.17s
83 Claude Haiku 4.5 (Thinking) 81.24% $1 / $5 10.37s
84 DeepSeek V3 80.76% $0.9 / $0.9 4.39s
85 Grok 4 Fast (Reasoning) 80.60% $0.2 / $0.5 3.80s
86 GPT 4 Turbo 80.46% $10 / $30 1.41s
87 o1 80.39% $15 / $60 8.34s
88 Qwen 3 Max Preview 80.33% $1.2 / $6 1.89s
89 DeepSeek V4 80.32% $1.32 / $3.96 3m04s
90 Kimi K2 Thinking 80.20% $0.6 / $2.5 43.99s
91 Qwen 3 (235B) 80.18% $0.22 / $0.88 12.37s
92 GPT 4o (2024-08-06) 80.12% $2.5 / $10 0.53s
93 Claude 3.7 Sonnet (Nonthinking) 80.00% $3 / $15 1.20s
94 MiniMax-M2.5 79.96% $0.3 / $1.2 15.82s
95 Command A 79.70% $2.5 / $10 0.80s
96 Ling 3.0 Flash 79.69% $0.075 / $0.22 3.90s
97 GLM 4.6 79.61% $0.6 / $2.2 29.82s
98 Qwen 2.5 Instruct Turbo (72B) 79.40% $1.2 / $1.2 0.83s
99 o4 Mini 79.19% $1.1 / $4.4 4.03s
100 Mistral Large 3 79.14% $0.5 / $1.5 1.73s
101 Grok 4.1 Fast Non-Reasoning 79.11% $0.2 / $0.5 1.76s
102 Gemini 2.5 Flash Lite (9/25) (Nonthinking) 79.01% $0.1 / $0.4 0.46s
103 MiMo V2.5 78.89% $0.14 / $0.28 11.18s
104 Grok 4 Fast (Non-Reasoning) 78.40% $0.2 / $0.5 0.93s
105 Gemini 2.0 Flash (001) 78.36% $0.1 / $0.4 0.45s
106 GPT 4.1 Mini 78.04% $0.4 / $1.6 0.47s
107 GPT 5.4 Nano 77.92% $0.2 / $1.25 2.70s
108 Llama 4 Maverick 77.81% $0.22 / $0.88 0.47s
109 Grok 4.20 (Reasoning) 77.74% $2 / $6 7.57s
110 DeepSeek V3 (03/24/2025) 77.73% $0.9 / $0.9 7.71s
111 DeepSeek V4 Flash 0731 77.71% $0.44 / $1.32 3.95s
112 Llama 3.3 Instruct Turbo (70B) 77.18% $0.88 / $0.88 0.55s
113 MiMo V2.5 Pro 77.13% $0.435 / $0.87 27.14s
114 DeepSeek V3.2 (Thinking) 76.08% $0.56 / $1.68 31.47s
115 GPT OSS 120B 75.94% $0.15 / $0.6 7.85s
116 GLM 4.5 75.63% $0.6 / $2.2 16.40s
117 MiniMax-M2.1 75.45% $0.3 / $1.2 11.49s
118 Laguna M.1 75.14% N/A 42.36s
119 Jamba 1.5 Large 74.16% $2 / $8 0.81s
120 Llama 4 Scout 72.04% $0.18 / $0.59 0.39s
121 Jamba 1.6 Large 71.96% $2 / $8 0.81s
122 o3 Mini 71.54% $1.1 / $4.4 12.01s
123 Laguna XS.2 71.03% N/A 11.92s
124 GPT OSS 20B 70.85% $0.07 / $0.3 7.19s
125 Claude 3.5 Haiku Latest 70.33% $0.8 / $4 1.07s
126 Gemini 1.0 Pro 002 70.25% $0.5 / $1.5 0.47s
127 Jamba 1.6 Mini 69.73% $0.2 / $0.4 0.30s
128 Qwen 2.5 Instruct Turbo (7B) 69.56% $0.3 / $0.3 0.40s
129 Mistral Small 3.1 (03/2025) 69.16% $0.075 / $0.3 0.58s
130 Gemini 1.5 Pro (002) 69.08% $1.25 / $5 0.90s
131 Ling 3.0 Flash Fin 68.74% $0.06 / $0.18 6.35s
132 Command R+ 68.29% $2.5 / $10 0.40s
133 Gemma 2 (9B) 68.14% $0.2 / $0.2 0.58s
134 Gemma 2 (27B) 67.98% $0.5 / $0.5 0.89s
135 DeepSeek R1 67.32% $1.35 / $5.4 18.55s
136 Jamba 1.5 Mini 66.62% $0.2 / $0.4 0.28s
137 DeepSeek V3.2 (Nonthinking) 65.96% $0.56 / $1.68 2.19s
138 GPT 3.5 64.37% $0.5 / $1.5 0.54s
139 Gemini 1.5 Flash (002) 63.24% $0.075 / $0.3 0.48s
140 Llama 2 (70B) 62.43% $0.9 / $0.9 14.20s
141 Mistral Medium 3.1 (05/2025) 61.98% $0.4 / $2 0.68s
142 Command A+ 61.42% N/A 22.65s
143 GPT 4.1 Nano 61.06% $0.1 / $0.4 0.36s
144 Mixtral (8x7B) 55.84% $0.6 / $0.6 14.14s
145 Magistral Medium 1.2 (09/2025) 54.77% $2 / $5 11.73s
146 Mistral (7B) 53.77% $0.18 / $0.18 1.72s
147 Llama 2 (13B) 53.70% $0.2 / $0.2 1.67s
148 Llama 2 (7B) 51.87% $0.2 / $0.2 0.98s
149 GPT 5 Nano 50.13% $0.05 / $0.4 22.50s
150 Magistral Small 1.2 (09/2025) 40.02% $0.5 / $1.5 6.67s
151 Command R 32.97% $0.15 / $0.6 0.24s

Partners in Evaluation

codex
lawbeta
tai

Results


The results per question type are summarized in the graph below.

LegalBench

Model Output Example

We show some examples from one of the LegalBench free response tasks (Rule QA) here. The models were asked “Where in the Federal Rules of Civil Procedure are notice requirements described?”

This question is tricky because both Rules 4 and 5 talk about the service process and notice requirements. However, Rule 4 is the main rule that defines the first notice in a proceeding, whereas Rule 5 gives the procedures for filing pleadings, etc.

This question is also difficult because implicit in the question is that it’s “notice” of a suit. GPT correctly determines that the question is talking about the initial notice, whereas Opus does not. Interestingly, Gemini gets the answer wrong, but it is much more terse than the other two models. Llama 3.1 makes the same mistake as Opus, citing Rule 5 instead of Rule 4.

It’s important this is one question from a very large evaluation data set and is not representative of overall performance.

Question

Where in the Federal Rules of Civil Procedure are notice requirements described?

The notice requirements in federal civil litigation are primarily described in Rule 5 of the Federal Rules of Civil Procedure (FRCP). Specifically: 1. Rule 5(a) outlines the service requirements for pleadings and other papers, stating that unless the rules provide otherwise, each paper must be served on every party. 2. Rule 5(b) describes how service can be made, including the methods of service and the permissible means of electronic service. 3. Rule 5(c) states that all papers after the complaint required to be served must be filed with the court within a reasonable time after service. 4. Rule 5(d) covers the filing requirements, including electronic filing and the filing of discovery materials. Additionally, other rules in the FRCP may contain specific notice requirements for certain situations, such as Rule 4 for serving a summons and complaint, Rule 6 for computing and extending time, and Rule 55 for default judgments.

INCORRECT

Dataset and Context

There has been a considerable effort to measure language model performance in academic tasks and chatbot settings but these high-level benchmarks are not applicable to specific industry use cases. Here we start to remedy this by reporting our application-specific findings and live leaderboard results on LegalBench, a large crowd-sourced collection of legal reasoning tasks. The data set is quite comprehensive, covering six major categories.

  • Issue-spotting, where a model must determine if a fact has relevance to a particular law or legal entity.
  • Rule-recall, where a model must identify a relevant rule or state its characteristics.
  • Rule-conclusion, where a model must predict a legal outcome.
  • Rule-application, where a model must analyze how a rule was applied to reach a conclusion.
  • Interpretation, where a model must parse and understand legal text.
  • Rhetorical understanding, where a model must determine whether a legal argument performs a certain function.