Academic

LegalBench

Updated 10/13/2026

Evaluating language models on a wide range of open source legal reasoning tasks.

As of October 13, 2026, Claude Fable 5 ranks first on LegalBench with 88.56%, followed by Claude Fable 5.1 (88.51%) and Gemini 4 Argon (88.30%).

LegalBenchOpen-source legal reasoning tasks
LegalBench•Oct 13, 2026
ACCURACY
Lab:
Pinch or Ctrl/Cmd + scroll to zoom. Once zoomed, scroll or swipe with two fingers to pan, or drag to pan. Shift + drag draws a box to zoom into; turn on box zoom to draw one without Shift. Plus and minus keys zoom, arrow keys pan, 0 resets.

LegalBench leaderboard

Rank Model Accuracy Cost In / Out Tokens / Test Latency
1 Claude Fable 5 88.56% $10 / $50 1.52k 8.96s
2 Claude Fable 5.1 88.51% $10 / $50 1.88k 12.40s
3 Gemini 4 Argon 88.30% $4 / $20 - 12.62s
4 Wonder Full 87.42% $4 / $20 - 38.27s
5 Gemini 3.1 Pro Preview (02/26) 87.40% $2 / $12 1.20k 10.06s
6 Gemini 3.7 Flash 87.26% $1.5 / $7.5 1.16k 2.23s
7 Gemini 3 Pro (11/25) 87.03% $2 / $12 1.51k 8.33s
8 Gemini 3.8 Flash 86.99% $0.75 / $3.75 1.69k 3.32s
9 Claude Opus 5 86.97% $5 / $25 1.33k 4.42s
10 GPT-5.6 Sol 86.97% $4 / $20 1.05k 6.20s
11 Gemini 3 Flash (12/25) 86.86% $0.5 / $3 1.33k 4.67s
12 Gemini 3.6 Flash 86.70% $0.75 / $3.75 1.25k 2.96s
13 GPT 5.5 86.52% $5 / $30 1.50k 18.14s
14 Grok 4.6 86.31% $2 / $6 1.64k 27.42s
15 GLM Legal 86.23% $0.075 / $0.25 - 6.00s
16 Kimi K3 86.21% $3 / $15 1.30k 7.11s
17 GPT 5.4 (xhigh) 86.04% $2.5 / $15 2.31k 27.79s
18 GPT 5 86.02% $1.25 / $10 1.93k 18.59s
19 Grok 4.5 85.97% $2 / $6 1.39k 67.88s
20 GPT 5.1 85.68% $1.25 / $10 1.14k 7.05s
21 MiniMax-M3 85.42% $0.3 / $1.2 1.17k 8.21s
22 Claude Opus 4.6 (Thinking) 85.30% $5 / $25 925 3.75s
23 Muse Spark 1.2 85.26% $1.25 / $4.25 2.16k 30.10s
24 Claude Opus 4.7 85.25% $5 / $25 1.22k 26.62s
25 GPT-5.6 Terra 85.11% $2 / $12 854 2.78s
26 Qwen 3.5 Plus 85.10% $0.4 / $2.4 2.18k 30.42s
27 Muse Spark 1.1 84.98% $1.25 / $4.25 2.77k 17.15s
28 Qwen 3.7 Max 84.91% $2.5 / $7.5 1.31k 11.85s
29 GLM 5.3 84.84% $1.4 / $4.4 2.02k 22.48s
30 Kimi K2.6 84.74% $0.95 / $4 2.28k 50.07s
31 Claude Opus 4.5 (Thinking) 84.60% $5 / $25 1.16k 7.64s
32 Grok 4.3 84.46% $1.25 / $2.5 1.80k 5m48s
33 GLM 5.1 84.39% $1 / $3.2 1.32k 13.75s
34 Grok 4.7 84.39% $2 / $6 3.46k 26.02s
35 Gemini 2.5 Pro Exp 84.32% $1.25 / $10 789 4.02s
36 Qwen 3.5 Flash 84.28% $0.1 / $0.4 2.48k 16.73s
37 Qwen 3.6 Plus 84.23% $0.5 / $3 1.66k 19.07s
38 Muse Spark 84.22% - 2.56k 34.20s
39 Claude Sonnet 4.5 (Thinking) 84.08% $3 / $15 1.40k 13.29s
40 Gemini 3.5 Flash Lite 84.07% $0.3 / $2.5 1.19k 1.67s
41 GLM 5.2 84.07% $1.4 / $4.4 924 5.77s
42 GLM 5 84.06% $1 / $3.2 1.34k 16.06s
43 GPT-5.6 Luna 84.03% $0.2 / $1.2 1.40k 5.89s
44 MiniMax-M2.7 83.98% $0.3 / $1.2 1.04k 8.92s
45 GLM 5.3 Flash 83.93% $0.075 / $0.25 1.69k 12.52s
46 Claude Sonnet 5 83.92% $2 / $10 1.44k 4.90s
47 Gemini 2.5 Flash Preview 4/17 (Nonthinking) 83.80% $0.3 / $2.5 786 0.55s
48 Gemini 3.1 Flash Lite Preview 83.76% $0.25 / $1.5 1.31k 2.70s
49 o3 83.76% $2 / $8 1.07k 5.98s
50 Hy4 Preview 83.76% $0.834 / $2.501 5.82k 90.56s
51 Qwen 3.8 Max 83.61% $2 / $6 2.32k 36.37s
52 Gemini 3.5 Flash 83.60% $1.5 / $9 1.14k 34.58s
53 Claude Opus 4.8 83.57% $5 / $25 1.24k 3.11s
54 Claude Opus 4.1 (Nonthinking) 83.46% $15 / $75 822 2.79s
55 GLM 4.7 83.36% $0.6 / $2.2 1.71k 30.68s
56 DeepSeek V4.1 Flash 83.28% $0.3 / $1.2 1.87k 6.53s
57 Grok 4 83.19% $3 / $15 1.78k 27.35s
58 Grok 3 Mini Reasoning 83.14% $0.3 / $0.5 1.57k 5.09s
59 GPT 4.1 83.10% $2 / $8 755 0.51s
60 Claude Opus 4 (Nonthinking) 83.07% $15 / $75 822 3.73s
61 Mercury 2.5 83.06% $0.2 / $0.75 1.61k 3.91s
62 Inkling Small 82.99% $0.3 / $1.2 1.78k 17.66s
63 Claude Sonnet 4 (Nonthinking) 82.95% $3 / $15 822 4.85s
64 Inkling 82.95% $1 / $4.05 2.05k 23.14s
65 Claude Opus 4.5 (Nonthinking) 82.84% $5 / $25 818 2.38s
66 GPT 5.2 82.76% $1.75 / $14 2.39k 17.14s
67 Gemini 2.5 Flash Preview (9/25) (Thinking) 82.63% $0.3 / $2.5 946 1.97s
68 Grok 3 82.59% $3 / $15 756 0.74s
69 Claude 3.7 Sonnet (Thinking) 82.52% $3 / $15 1.17k 6.54s
70 Gemini 2.5 Flash Preview (9/25) (Nonthinking) 82.49% $0.3 / $2.5 947 1.15s
71 Grok 4.1 Fast (Reasoning) 82.45% $0.2 / $0.5 1.49k 8.58s
72 Qwen 3.8 27B 82.43% $0.5 / $3 2.23k 47.81s
73 DeepSeek V4 Pro 0813 82.36% $1.32 / $3.96 1.75k 20.05s
74 GPT 4o (2024-11-20) 82.21% $2.5 / $10 754 0.45s
75 Claude Sonnet 4.6 82.12% $3 / $15 932 21.63s
76 Nemotron 3 Ultra 82.07% - 1.19k 5.18s
77 Claude Sonnet 4 (Thinking) 82.06% $3 / $15 1.31k 10.61s
78 Gemini 2.5 Flash Lite (9/25) (Thinking) 82.04% $0.1 / $0.4 1.20k 3.14s
79 Grok 3 Mini Reasoning 81.92% $0.3 / $0.5 1.31k 3.44s
80 Qwen 3 Max 81.86% $1.2 / $6 780 -
81 GPT 5 Mini 81.77% $0.25 / $2 1.73k 9.43s
82 Kimi K2 Instruct 81.45% $1 / $3 754 1.17s
83 Claude Haiku 4.5 (Thinking) 81.24% $1 / $5 1.76k 10.37s
84 DeepSeek V3 80.76% $0.9 / $0.9 759 4.39s
85 Grok 4 Fast (Reasoning) 80.60% $0.2 / $0.5 2.12k 3.80s
86 GPT 4 Turbo 80.46% $10 / $30 834 1.41s
87 o1 80.39% $15 / $60 1.46k 8.34s
88 Qwen 3 Max Preview 80.33% $1.2 / $6 790 1.89s
89 DeepSeek V4 80.32% $1.32 / $3.96 1.89k 3m04s
90 Kimi K2 Thinking 80.20% $0.6 / $2.5 1.44k 43.99s
91 Qwen 3 (235B) 80.18% $0.22 / $0.88 1.30k 12.37s
92 GPT 4o (2024-08-06) 80.12% $2.5 / $10 749 0.53s
93 Claude 3.7 Sonnet (Nonthinking) 80.00% $3 / $15 816 1.20s
94 MiniMax-M2.5 79.96% $0.3 / $1.2 1.16k 15.82s
95 Command A 79.70% $2.5 / $10 782 0.80s
96 Ling 3.0 Flash 79.69% $0.075 / $0.22 1.43k 3.90s
97 GLM 4.6 79.61% $0.6 / $2.2 1.66k 29.82s
98 Qwen 2.5 Instruct Turbo (72B) 79.40% $1.2 / $1.2 787 0.83s
99 o4 Mini 79.19% $1.1 / $4.4 1.04k 4.03s
100 Mistral Large 3 79.14% $0.5 / $1.5 811 1.73s
101 Grok 4.1 Fast Non-Reasoning 79.11% $0.2 / $0.5 910 1.76s
102 Gemini 2.5 Flash Lite (9/25) (Nonthinking) 79.01% $0.1 / $0.4 792 0.46s
103 MiMo V2.5 78.89% $0.14 / $0.28 1.53k 11.18s
104 Grok 4 Fast (Non-Reasoning) 78.40% $0.2 / $0.5 1.72k 0.93s
105 Gemini 2.0 Flash (001) 78.36% $0.1 / $0.4 764 0.45s
106 GPT 4.1 Mini 78.04% $0.4 / $1.6 753 0.47s
107 GPT 5.4 Nano 77.92% $0.2 / $1.25 939 2.70s
108 Llama 4 Maverick 77.81% $0.22 / $0.88 765 0.47s
109 Grok 4.20 (Reasoning) 77.74% $2 / $6 1.74k 7.57s
110 DeepSeek V3 (03/24/2025) 77.73% $0.9 / $0.9 760 7.71s
111 DeepSeek V4 Flash 0731 77.71% $0.3 / $1.2 1.11k 3.95s
112 Llama 3.3 Instruct Turbo (70B) 77.18% $0.88 / $0.88 779 0.55s
113 MiMo V2.5 Pro 77.13% $0.435 / $0.87 1.55k 27.14s
114 DeepSeek V3.2 (Thinking) 76.08% $0.56 / $1.68 1.75k 31.47s
115 GPT OSS 120B 75.94% $0.15 / $0.6 2.10k 7.85s
116 GLM 4.5 75.63% $0.6 / $2.2 1.09k 16.40s
117 MiniMax-M2.1 75.45% $0.3 / $1.2 1.30k 11.49s
118 Laguna M.1 75.14% - 1.35k 42.36s
119 Jamba 1.5 Large 74.16% $2 / $8 802 0.81s
120 Llama 4 Scout 72.04% $0.18 / $0.59 767 0.39s
121 Jamba 1.6 Large 71.96% $2 / $8 809 0.81s
122 o3 Mini 71.54% $1.1 / $4.4 2.26k 12.01s
123 Laguna XS.2 71.03% - 1.26k 11.92s
124 GPT OSS 20B 70.85% $0.07 / $0.3 2.76k 7.19s
125 Claude 3.5 Haiku Latest 70.33% $0.8 / $4 818 1.07s
126 Gemini 1.0 Pro 002 70.25% $0.5 / $1.5 757 0.47s
127 Jamba 1.6 Mini 69.73% $0.2 / $0.4 804 0.30s
128 Qwen 2.5 Instruct Turbo (7B) 69.56% $0.3 / $0.3 785 0.40s
129 Mistral Small 3.1 (03/2025) 69.16% $0.075 / $0.3 804 0.58s
130 Gemini 1.5 Pro (002) 69.08% $1.25 / $5 787 0.90s
131 Ling 3.0 Flash Fin 68.74% $0.06 / $0.18 1.65k 6.35s
132 Command R+ 68.29% $2.5 / $10 - 0.40s
133 Gemma 2 (9B) 68.14% $0.2 / $0.2 794 0.58s
134 Gemma 2 (27B) 67.98% $0.5 / $0.5 793 0.89s
135 DeepSeek R1 67.32% $1.35 / $5.4 1.13k 18.55s
136 Jamba 1.5 Mini 66.62% $0.2 / $0.4 800 0.28s
137 DeepSeek V3.2 (Nonthinking) 65.96% $0.56 / $1.68 768 2.19s
138 GPT 3.5 64.37% $0.5 / $1.5 827 0.54s
139 Gemini 1.5 Flash (002) 63.24% $0.075 / $0.3 787 0.48s
140 Llama 2 (70B) 62.43% $0.9 / $0.9 - 14.20s
141 Mistral Medium 3.1 (05/2025) 61.98% $0.4 / $2 798 0.68s
142 Command A+ 61.42% - 3.84k 22.65s
143 GPT 4.1 Nano 61.06% $0.1 / $0.4 751 0.36s
144 Mixtral (8x7B) 55.84% $0.6 / $0.6 - 14.14s
145 Magistral Medium 1.2 (09/2025) 54.77% $2 / $5 1.74k 11.73s
146 Mistral (7B) 53.77% $0.18 / $0.18 - 1.72s
147 Llama 2 (13B) 53.70% $0.2 / $0.2 - 1.67s
148 Llama 2 (7B) 51.87% $0.2 / $0.2 - 0.98s
149 GPT 5 Nano 50.13% $0.05 / $0.4 4.12k 22.50s
150 Magistral Small 1.2 (09/2025) 40.02% $0.5 / $1.5 1.40k 6.67s
151 Command R 32.97% $0.15 / $0.6 - 0.24s

Partners in Evaluation

codex
lawbeta
tai

Results


The results per question type are summarized in the graph below.

LegalBench

Model Output Example

We show some examples from one of the LegalBench free response tasks (Rule QA) here. The models were asked “Where in the Federal Rules of Civil Procedure are notice requirements described?”

This question is tricky because both Rules 4 and 5 talk about the service process and notice requirements. However, Rule 4 is the main rule that defines the first notice in a proceeding, whereas Rule 5 gives the procedures for filing pleadings, etc.

This question is also difficult because implicit in the question is that it’s “notice” of a suit. GPT correctly determines that the question is talking about the initial notice, whereas Opus does not. Interestingly, Gemini gets the answer wrong, but it is much more terse than the other two models. Llama 3.1 makes the same mistake as Opus, citing Rule 5 instead of Rule 4.

It’s important this is one question from a very large evaluation data set and is not representative of overall performance.

Question

Where in the Federal Rules of Civil Procedure are notice requirements described?

The notice requirements in federal civil litigation are primarily described in Rule 5 of the Federal Rules of Civil Procedure (FRCP). Specifically: 1. Rule 5(a) outlines the service requirements for pleadings and other papers, stating that unless the rules provide otherwise, each paper must be served on every party. 2. Rule 5(b) describes how service can be made, including the methods of service and the permissible means of electronic service. 3. Rule 5(c) states that all papers after the complaint required to be served must be filed with the court within a reasonable time after service. 4. Rule 5(d) covers the filing requirements, including electronic filing and the filing of discovery materials. Additionally, other rules in the FRCP may contain specific notice requirements for certain situations, such as Rule 4 for serving a summons and complaint, Rule 6 for computing and extending time, and Rule 55 for default judgments.

INCORRECT

Dataset and Context

There has been a considerable effort to measure language model performance in academic tasks and chatbot settings but these high-level benchmarks are not applicable to specific industry use cases. Here we start to remedy this by reporting our application-specific findings and live leaderboard results on LegalBench, a large crowd-sourced collection of legal reasoning tasks. The data set is quite comprehensive, covering six major categories.

  • Issue-spotting, where a model must determine if a fact has relevance to a particular law or legal entity.
  • Rule-recall, where a model must identify a relevant rule or state its characteristics.
  • Rule-conclusion, where a model must predict a legal outcome.
  • Rule-application, where a model must analyze how a rule was applied to reach a conclusion.
  • Interpretation, where a model must parse and understand legal text.
  • Rhetorical understanding, where a model must determine whether a legal argument performs a certain function.