New Finance Agent Benchmark Released

Released date

Model

Last Updated 10/22/2024

anthropic/claude-3-5-haiku-20241022

Claude 3.5 Haiku Latest

Anthropic's fastest model.

Released Date: 10/22/2024

Avg. Accuracy:

54.8%

Latency:

8.49s

Performance by Benchmark

Benchmarks

Accuracy

Rankings

14.3%

( 22 / 33 )

14.3%

22 / 33

58.2%

( 22 / 44 )

58.2%

22 / 44

83.8%

( 15 / 73 )

83.8%

15 / 73

68.7%

( 18 / 76 )

68.7%

18 / 76

63.0%

( 51 / 60 )

63.0%

51 / 60

64.2%

( 51 / 54 )

64.2%

51 / 54

3.3%

( 48 / 50 )

3.3%

48 / 50

84.6%

( 46 / 53 )

84.6%

46 / 53

73.0%

( 46 / 75 )

73.0%

46 / 75

37.9%

( 47 / 52 )

37.9%

47 / 52

64.1%

( 45 / 50 )

64.1%

45 / 50

41.9%

( 37 / 51 )

41.9%

37 / 51

Academic Benchmarks

Proprietary Benchmarks (contact us to get access)

Cost Analysis

Input Cost

$1.00 / M Tokens

Output Cost

$5.00 / M Tokens

Input Cost (per char)

N/A

Output Cost (per char)

$1.57 / M chars

Join our mailing list to receive benchmark updates on

Stay up to date as new benchmarks and models are released.