New Finance Agent Benchmark Released

Released date

Model

Last Updated 10/22/2024

anthropic/claude-3-5-haiku-20241022

Claude 3.5 Haiku Latest

Anthropic's fastest model.

Released Date: 10/22/2024

Avg. Accuracy:

50.5%

Latency:

9.34s

Performance by Benchmark

Benchmarks

Accuracy

Rankings

14.3%

( 23 / 34 )

14.3%

23 / 34

58.2%

( 22 / 45 )

58.2%

22 / 45

63.0%

( 53 / 62 )

63.0%

53 / 62

64.2%

( 51 / 54 )

64.2%

51 / 54

3.3%

( 50 / 52 )

3.3%

50 / 52

84.6%

( 48 / 55 )

84.6%

48 / 55

73.0%

( 48 / 77 )

73.0%

48 / 77

37.9%

( 49 / 54 )

37.9%

49 / 54

64.1%

( 47 / 52 )

64.1%

47 / 52

41.9%

( 40 / 54 )

41.9%

40 / 54

Academic Benchmarks

Proprietary Benchmarks (contact us to get access)

Cost Analysis

Input Cost

$1.00 / M Tokens

Output Cost

$5.00 / M Tokens

Input Cost (per char)

N/A

Output Cost (per char)

$1.65 / M chars

Join our mailing list to receive benchmark updates on

Stay up to date as new benchmarks and models are released.