Industry Partner

HLAB-Vals-1.1

Updated 10/30/2026

Tests an agent's ability to complete legal work using documents, spreadsheets, presentations, and file-system tools.

As of October 30, 2026, Gemini 4 Argon ranks first on HLAB-Vals-1.1 with 27.08%, followed by Muse Spark Open (25.00%) and Wonder (23.75%).

HLAB-Vals-1.1Agentic legal work with files
HLAB-Vals-1.1•Oct 30, 2026
ACCURACY
Lab:
Pinch or Ctrl/Cmd + scroll to zoom. Once zoomed, scroll or swipe with two fingers to pan, or drag to pan. Shift + drag draws a box to zoom into; turn on box zoom to draw one without Shift. Plus and minus keys zoom, arrow keys pan, 0 resets.

HLAB-Vals-1.1 leaderboard

Rank Model Task Pass Rate Cost / Test Tokens / Test Duration / Test
1 Gemini 4 Argon 27.08% $10.55 6.92m 29m15s
2 Muse Spark Open 25.00% $13.63 22.26m 3h59m
3 Wonder 23.75% $7.08 11.89m 15m06s
4 Wonder Full 23.33% $5.41 11.36m 44m09s
5 Grok 4.7 18.33% $10.75 9.24m 35m48s
6 Gemini 4 Carbon 17.50% $6.52 7.48m 28m41s
7 Mistral Large 4 17.50% $2.55 6.88m 58m04s
8 Mistral Large 4 OSS 15.83% $0.73 3.98m 11m29s
9 Qwen 3.8 Max 0902 15.42% $3.36 7.91m 49m11s
10 GLM 5.3 14.58% $3.99 11.60m 31m51s
11 DeepSeek V4.1 Flash 1017 14.17% $0.61 46.25m 22m08s
12 MiMo V2.6 Pro 13.75% $0.22 5.35m 31m02s
13 MiMo V2.6 Flash 13.33% $0.09 8.09m 22m23s
14 Step 5 Preview 12.92% $1.13 12.90m 16m33s
15 Kimi K3 12.92% $4.14 7.48m 42m36s
16 Gemini 3.8 Flash 12.08% $3.66 10.87m 19m22s
17 Kimi K3.1 Pro 11.67% $9.00 32.06m 1h13m
18 DeepSeek V4 Pro 0813 10.83% $0.80 6.30m 10m55s
19 GLM 5.3 Flash 10.42% $0.32 8.47m 12m25s
20 Claude Sonnet 5.5 10.42% $16.61 63.93m 1h13m
21 Claude Opus 5.5 9.17% $21.38 31.62m 58m27s
22 Hy4 Preview 8.75% $0.98 11.55m 33m09s
23 MiniMax-M3.1-Flash 7.50% $0.47 22.92m 24m58s
24 GPT-6 Astra 7.08% $25.46 11.01m 32m33s
25 GPT-6.1 Sol 6.25% $3.54 10.32m 40m26s
26 Mercury 3 5.00% $0.19 5.54m 14m47s
27 Beam 1.0 4.58% - 4.54m 116.88s
28 GPT-6 Luna 4.17% $0.30 11.12m 21m45s
29 Claude Haiku 5.5 4.17% $4.26 41.45m 43m41s
30 Nemotron 3.5 Super VL 3.75% $0.37 5.03m 2m52s
31 Inkling 3.75% $1.25 4.55m 8m50s
32 Inkling Small 2.92% $0.18 1.72m 7m35s
33 GPT-6.1 Terra 1.67% $1.48 6.14m 13m42s
Reasoning EffortAccuracy at each model's lowest, middle and default reasoning effort
HLAB-Vals-1.1•Oct 30, 2026
Accuracy
Model:
Effort Level:
Error bars show ±1 standard error.

Partners in Evaluation

harvey

Benchmark

The Legal Agent Benchmark is a benchmark released by Harvey to test the ability of models to support legal work in an agentic setting. It has a public set and a held-out test set. The results here are from the held-out set.

Each task asks an agent to produce legal work against a set of task-specific criteria. The agent is provided with six tools: Read File, Edit File, Write File, Glob, Bash, and Grep. It also has three skills: docx, pptx, and xlsx.

We run Harvey’s harness, tools, skills and grading protocol, plus some improvements of our own (see What changed in v1.1). Results are not comparable with the archived v1 board or with Artificial Analysis’s independent Harvey LAB-AA. Criteria pass rate is included to show how often models satisfy individual requirements, even when they do not fully resolve the task.

Results

Among models with a nonzero final score, MiMo V2.6 Flash, MiMo V2.6 Pro, DeepSeek V4.1 Flash 1017, Mistral Large 4 OSS, Mistral Large 4, Wonder Full, Wonder, and Gemini 4 Argon form the cost/performance frontier, from cheapest to strongest.

Gemini 4 Argon leads on the overall Harvey final score at 27.08%, with Muse Spark Open second at 25.00%, Wonder third at 23.75%, Wonder Full fourth at 23.33%, and Grok 4.7 fifth at 18.33%. The criteria pass rates are much higher: Gemini 4 Argon reaches 96.30%, Wonder Full 96.13%, Wonder 96.01%, and Gemini 4 Carbon 94.78%.

Criteria Pass Rate by Task TypePercent of criteria passed per task type
Task type
Gemini 4 Argon96.30% avg
Wonder Full96.13% avg
Wonder96.01% avg
Gemini 4 Carbon94.78% avg
Qwen 3.8 Max 090293.85% avg
Corporate M&A97.3%97.6%97.9%96.5%96.9%
Intellectual Property98.4%97.9%97.2%96.6%97.3%
Data Privacy/Cybersecurity98.4%98.1%97.7%97.7%97.7%
Capital Markets96.7%96.1%96.3%96.3%94.9%
Energy/Natural Resources98.5%98.2%98.5%97.4%97.4%
Real Estate97.1%98.4%97.2%97.7%94.9%
Trusts & Estates/Private Client96.0%96.6%96.2%94.1%95.4%
Corporate Governance97.9%94.7%97.6%95.0%93.5%
Healthcare/Life Sciences97.8%96.6%95.1%95.8%96.8%
Litigation Dispute Resolution96.2%95.1%95.1%97.2%93.8%
0%99% criteria passed

The leaderboard can be filtered by task type and switched between task pass rate and criteria pass rate. Averaged across all models, task resolution is highest in Energy/Natural Resources (30.6%) and Healthcare/Life Sciences (23.0%), while criteria pass rates are highest in Corporate M&A (94.9%), Intellectual Property (94.4%), and Data Privacy/Cybersecurity (93.7%).

Criteria Pass Rate vs. Task Resolution
CRITERIA PASS RATETASK RESOLUTION
Gemini 4 Argon
96.3%/27.1%
Wonder Full
96.1%/23.3%
Wonder
96.0%/23.8%
Gemini 4 Carbon
94.8%/17.5%
Qwen 3.8 Max 0902
93.8%/15.4%
GLM 5.3
93.7%/14.6%
Grok 4.7
93.7%/18.3%
Kimi K3
93.5%/12.9%

Harvey grades a task as resolved only if every criterion passes. A model can satisfy most individual criteria and still miss task resolution credit.

There is a clear trend: strong models and agents satisfy most criteria, and the top five by criteria pass rate all clear 93.85%. The remaining gaps are large enough that task resolution stays low even when criterion-level performance looks strong.

Tool Calling Statistics5/33 models

Average tool count per item, across the six tools in the benchmark harness.

Models heavily prefer Bash and Read File, which make up 92% of tool calls. Edit File (4.4%), Write File (2.7%), Glob (1.0%), and Grep (0.3%) follow.

The available skills rely on shell commands in their instructions and to run their scripts. The tools also often encourage models to read files through the harness.

The skills do not directly emphasize Edit File, Write File, Glob, or Grep. Grep is not well-suited to the binary format of .docx, .pptx, and .xlsx files. Likewise, Edit is useful for text-based files such as .md files, not for these filetypes.

Skill Invocation Statistics5/33 models

Average skill invocations per item, across the three skills available to the agent.

Skill calls split docx 49%, xlsx 44%, and pptx 7%.

Methodology

We use Harvey’s generation and grading protocol in the same environment, with internet access disabled.

Harvey grades each submission with two LLM judges. Each judge computes a task pass rate. A task passes only if 100% of its criteria pass, and Harvey’s final score is the average of the two judge task pass rates.

The two judges were GPT 5.5 and Claude Sonnet 4.6. GPT 5.5 used medium reasoning and Claude Sonnet 4.6 was not modified.

While running the benchmark, we found that redline criteria needed DOCX tracked changes preserved when reading submitted files so judges could see inserted and deleted text. We fixed that bug and merged it upstream in harveyai/harvey-labs#76. The scoring rubric is unchanged.

The benchmark was modified to use our model library, an abstraction over various LLM provider APIs, and to run on Valkyrie, our framework for running agentic benchmarks. These are infrastructure changes.

What changed in v1.1

  • Data. Harvey fixed the private holdout: 17 unsolvable criteria were removed and one mismatched source document was replaced. Still 120 tasks.
  • Harvey’s harness updates. Re-synced to Harvey’s current upstream: an explicit finish tool, a reasoning-first judge schema, judge-error retries, Word comments visible to judges, and pandoc 3.11.
  • Our improvements. Fixes we made to the harness on top of Harvey’s:
    • Context compaction instead of failing on context overflow.
    • A nudge when a model stops without calling finish, instead of ending the run.
    • A cap on the size of a single tool result.
    • Images opened with read are shown as images, not as binary text.
    • File tools expand $WORKSPACE_DIR, $DOCUMENTS_DIR and $OUTPUT_DIR, so deliverables written to those paths reach the judges.
  • Scoring. Criteria pass rate now follows Harvey’s definition (each judge’s pass fraction, averaged) instead of requiring both judges to pass, so it reads higher. The headline score and judges are unchanged.
  • Runs. Every model was run from scratch.