Industry Partner

HLAB-Vals-1.1

Updated 10/22/2026

Tests an agent's ability to complete legal work using documents, spreadsheets, presentations, and file-system tools.

As of October 22, 2026, Gemini 4 Argon ranks first on HLAB-Vals-1.1 with 27.08%, followed by Muse Spark Open (25.00%) and Wonder (23.75%).

HLAB-Vals-1.1Agentic legal work with files
ACCURACY
Pinch or Ctrl/Cmd + scroll to zoom. Once zoomed, scroll or swipe with two fingers to pan, or drag to pan. Shift + drag draws a box to zoom into; turn on box zoom to draw one without Shift. Plus and minus keys zoom, arrow keys pan, 0 resets.

HLAB-Vals-1.1 leaderboard

Rank Model Task Pass Rate Cost / Test Input / Output Cost Duration
1 Gemini 4 Argon 27.08% $10.55 $4 / $20 29m15s
2 Muse Spark Open 25.00% $13.63 $1.25 / $4.25 3h59m
3 Wonder 23.75% $7.08 $4 / $20 15m06s
4 Wonder Full 23.33% $5.41 $4 / $20 44m09s
5 Grok 4.7 18.33% $10.75 $2 / $6 35m48s
6 Mistral Large 4 17.50% $4.18 $1.36 / $4.18 58m04s
7 Qwen 3.8 Max 0902 15.42% $3.36 $2 / $6 49m11s
8 GLM 5.3 14.58% $3.99 $1.4 / $4.4 31m51s
9 DeepSeek V4.1 Flash 1017 14.17% $0.61 $0.3 / $1.2 22m08s
10 MiMo V2.6 Pro 13.75% $0.22 $0.435 / $0.87 31m02s
11 MiMo V2.6 Flash 13.33% $0.09 $0.14 / $0.28 22m23s
12 Kimi K3 12.92% $4.14 $3 / $15 42m36s
13 Gemini 3.8 Flash 12.08% $3.66 $1.5 / $7.5 19m22s
14 DeepSeek V4 Pro 0813 10.83% $0.80 $1.32 / $3.96 10m55s
15 GLM 5.3 Flash 10.42% $0.32 $0.075 / $0.25 12m25s
16 Claude Sonnet 5.5 10.42% $16.61 $2 / $10 1h13m
17 Claude Opus 5.5 9.17% $21.38 $4 / $20 58m27s
18 Hy4 Preview 8.75% $0.98 $0.834 / $2.501 33m09s
19 MiniMax-M3.1-Flash 7.50% $0.47 $0.15 / $0.6 24m58s
20 GPT-6 Astra 7.08% $25.46 $10 / $50 32m33s
21 GPT-6.1 Sol 6.25% $3.54 $2 / $10 40m26s
22 Mercury 3 5.00% $0.19 $0.1 / $0.35 14m47s
23 Beam 1.0 4.58% N/A N/A 116.88s
24 GPT-6 Luna 4.17% $0.30 $0.1 / $0.5 21m45s
25 Claude Haiku 5.5 4.17% $4.26 $0.1 / $0.5 43m41s
26 Inkling 3.75% $1.25 $1 / $4.05 8m50s
27 Inkling Small 2.92% $0.18 $0.3 / $1.2 7m35s

Partners in Evaluation

harvey

Benchmark

The Legal Agent Benchmark is a benchmark released by Harvey to test the ability of models to support legal work in an agentic setting. It has a public set and a held-out test set. The results here are from the held-out set.

Each task asks an agent to produce legal work against a set of task-specific criteria. The agent is provided with six tools: Read File, Edit File, Write File, Glob, Bash, and Grep. It also has three skills: docx, pptx, and xlsx.

We run Harvey’s harness, tools, skills and grading protocol, plus some improvements of our own (see What changed in v1.1). Results are not comparable with the archived v1 board or with Artificial Analysis’s independent Harvey LAB-AA. Criteria pass rate is included to show how often models satisfy individual requirements, even when they do not fully resolve the task.

Results

Among models with a nonzero final score, MiMo V2.6 Flash, MiMo V2.6 Pro, DeepSeek V4.1 Flash 1017, Qwen 3.8 Max 0902, Mistral Large 4, Wonder Full, Wonder, and Gemini 4 Argon form the cost/performance frontier, from cheapest to strongest.

Gemini 4 Argon leads on the overall Harvey final score at 27.08%, with Muse Spark Open second at 25.00%, Wonder third at 23.75%, Wonder Full fourth at 23.33%, and Grok 4.7 fifth at 18.33%. The criteria pass rates are much higher: Gemini 4 Argon reaches 96.30%, Wonder Full 96.13%, Wonder 96.01%, and Qwen 3.8 Max 0902 93.85%.

Criteria Pass Rate by Task TypePercent of criteria passed per task type
Task type
Gemini 4 Argon96.30% avg
Wonder Full96.13% avg
Wonder96.01% avg
Qwen 3.8 Max 090293.85% avg
GLM 5.393.72% avg
Corporate M&A97.3%97.6%97.9%96.9%96.3%
Intellectual Property98.4%97.9%97.2%97.3%97.3%
Data Privacy/Cybersecurity98.4%98.1%97.7%97.7%97.9%
Capital Markets96.7%96.1%96.3%94.9%94.3%
Real Estate97.1%98.4%97.2%94.9%94.6%
Trusts & Estates/Private Client96.0%96.6%96.2%95.4%94.5%
Energy/Natural Resources98.5%98.2%98.5%97.4%97.9%
Corporate Governance97.9%94.7%97.6%93.5%94.3%
International Trade Sanctions96.2%97.2%96.4%94.0%93.6%
Litigation Dispute Resolution96.2%95.1%95.1%93.8%93.4%
0%99% criteria passed

The leaderboard can be filtered by task type and switched between task pass rate and criteria pass rate. Averaged across all models, task resolution is highest in Energy/Natural Resources (31.1%) and Healthcare/Life Sciences (24.1%), while criteria pass rates are highest in Corporate M&A (94.9%), Intellectual Property (94.3%), and Data Privacy/Cybersecurity (93.4%).

Criteria Pass Rate vs. Task Resolution
CRITERIA PASS RATETASK RESOLUTION
Gemini 4 Argon
96.3%/27.1%
Wonder Full
96.1%/23.3%
Wonder
96.0%/23.8%
Qwen 3.8 Max 0902
93.8%/15.4%
GLM 5.3
93.7%/14.6%
Grok 4.7
93.7%/18.3%
Kimi K3
93.5%/12.9%
Mistral Large 4
93.0%/17.5%

Harvey grades a task as resolved only if every criterion passes. A model can satisfy most individual criteria and still miss task resolution credit.

There is a clear trend: strong models and agents satisfy most criteria, and the top five by criteria pass rate all clear 93.72%. The remaining gaps are large enough that task resolution stays low even when criterion-level performance looks strong.

Tool Calling Statistics5/27 models

Average tool count per item, across the six tools in the benchmark harness.

Models heavily prefer Bash and Read File, which make up 92% of tool calls. Edit File (4.0%), Write File (2.9%), Glob (1.0%), and Grep (0.3%) follow.

The available skills rely on shell commands in their instructions and to run their scripts. The tools also often encourage models to read files through the harness.

The skills do not directly emphasize Edit File, Write File, Glob, or Grep. Grep is not well-suited to the binary format of .docx, .pptx, and .xlsx files. Likewise, Edit is useful for text-based files such as .md files, not for these filetypes.

Skill Invocation Statistics5/27 models

Average skill invocations per item, across the three skills available to the agent.

Skill calls split docx 49%, xlsx 44%, and pptx 7%.

Methodology

We use Harvey’s generation and grading protocol in the same environment, with internet access disabled.

Harvey grades each submission with two LLM judges. Each judge computes a task pass rate. A task passes only if 100% of its criteria pass, and Harvey’s final score is the average of the two judge task pass rates.

The two judges were GPT 5.5 and Claude Sonnet 4.6. GPT 5.5 used medium reasoning and Claude Sonnet 4.6 was not modified.

While running the benchmark, we found that redline criteria needed DOCX tracked changes preserved when reading submitted files so judges could see inserted and deleted text. We fixed that bug and merged it upstream in harveyai/harvey-labs#76. The scoring rubric is unchanged.

The benchmark was modified to use our model library, an abstraction over various LLM provider APIs, and to run on Valkyrie, our framework for running agentic benchmarks. These are infrastructure changes.

What changed in v1.1

  • Data. Harvey fixed the private holdout: 17 unsolvable criteria were removed and one mismatched source document was replaced. Still 120 tasks.
  • Harvey’s harness updates. Re-synced to Harvey’s current upstream: an explicit finish tool, a reasoning-first judge schema, judge-error retries, Word comments visible to judges, and pandoc 3.11.
  • Our improvements. Fixes we made to the harness on top of Harvey’s:
    • Context compaction instead of failing on context overflow.
    • A nudge when a model stops without calling finish, instead of ending the run.
    • A cap on the size of a single tool result.
    • Images opened with read are shown as images, not as binary text.
    • File tools expand $WORKSPACE_DIR, $DOCUMENTS_DIR and $OUTPUT_DIR, so deliverables written to those paths reach the judges.
  • Scoring. Criteria pass rate now follows Harvey’s definition (each judge’s pass fraction, averaged) instead of requiring both judges to pass, so it reads higher. The headline score and judges are unchanged.
  • Runs. Every model was run from scratch.