We found that Mistral Large 3 struggles across domains:

• We found a variety of difficulties with agentic use-cases involving tool-calling. We’re working with the Mistral team to address one such issue.
• The model places second-to-last on our Vals Multimodal Index, ahead of Llama 4 Maverick.

However, there are a few caveats:

• the model represents a meaningful improvement over predecessors like Magistral Medium 1.2 (09/2025) and Mistral Medium 3.1 (05/2025).
• The model is open-weight and performs comparably to other open-weight models like GPT OSS 120B.

We look forward to further improvements from Mistral to come!

Mistral Large 3 is #20 on our Vals Index (out of 24).

A main source of failure was the model’s difficulty in calling tools - a common error mode was the model making a function call with a malformed tool name (e.g. inserting the function arguments into the tool name). (2/3)

However, it achieves a six percentage point increase over Magistral Medium 1.2 (09/2025), with comparable performance on the Vals Index to GPT OSS 120B.

Full results will be posted soon.