Harvey Assistant leverages multiple LLMs and custom fine-tuned models trained on legal processes and data in partnership with OpenAI, with each query involving between 30 and 1,500 model calls. The platform achieved the highest overall performance in the VLAIR study, ranking first in five out of six evaluated tasks and surpassing the lawyer baseline in all but redlining.

Performance Summary

Harvey Assistant was evaluated across 6 tasks in the VLAIR benchmark, with scores ranging from 65.0% to 94.8%.

Harvey Assistant demonstrated strong performance across all evaluated tasks, achieving particularly high accuracy in Document Q&A (94.8%) and surpassing the lawyer baseline in 5 of 6 tasks with consistently fast sub-60 second response times.

Key Strengths

Based on VLAIR evaluation results, Harvey Assistant demonstrates several notable capabilities:

1. Document Q&A excellence: Achieved 94.8% on document question-answering with a 24.7pp margin above baseline, demonstrating strong capability in answering questions about legal documents with comprehensive detail capture.

2. Speed advantage: Sub-minute response times across all six tasks enable immediate productivity gains and support iterative workflows where outputs serve as starting points for refinement.

3. Transcript processing: Strong performance at 77.8% (24.1pp above baseline) on scanned, messily-formatted court transcripts showcases sophisticated document processing that handles real-world challenges.

4. Comprehensive task coverage: Participation in 6 tasks demonstrates broad capabilities across the legal AI landscape, from extraction to analysis to summarization.

5. Advanced technical architecture: Sophisticated multi-model system with 30-1,500 calls per query delivers consistently strong results through custom fine-tuned models developed with OpenAI.

6. Integrated product suite: Unified platform includes drafting suite, Word Add-in, Document Extraction at scale, and agentic workflows without requiring separate licenses for different tools.

7. Consistent information delivery: The AI delivers comprehensive, precise responses reliably, particularly evident in document Q&A where granular details are captured consistently.

Based on the Vals Legal AI Report (VLAIR), February 2025 - First independent benchmarking study of legal AI tools using real legal work from Am Law 100 law firms.