New Finance Agent Benchmark Released

Anthropic's fastest model.

Released Date: 10/22/2024

Avg. Accuracy:

56.0%

Latency:

8.28s

Performance by Benchmark

Benchmarks

Accuracy

Rankings

FinanceAgent

14.3%

( 13 / 22 )

CorpFin

58.2%

( 14 / 35 )

CaseLaw

83.8%

( 10 / 55 )

ContractLaw

68.7%

( 16 / 62 )

TaxEval

63.0%

( 33 / 42 )

Math500

64.2%

( 35 / 38 )

AIME

3.3%

( 32 / 34 )

MGSM

85.9%

( 29 / 36 )

LegalBench

73.0%

( 34 / 60 )

GPQA

37.9%

( 30 / 35 )

MMLU Pro

64.1%

( 30 / 35 )

Academic Benchmarks
Proprietary Benchmarks (contact us to get access)

Cost Analysis

Input Cost

$1.00 / M Tokens

Output Cost

$5.00 / M Tokens

Input Cost (per char)

$0.35 / M chars

Output Cost (per char)

$1.48 / M chars

Join our mailing list to receive benchmark updates on

Stay up to date as new benchmarks and models are released.