# Contract Review Comparison An independent evaluation of AI-powered contract review tools ## Introduction Contract review makes up the majority of work done by in-house legal departments; enterprise teams review thousands of contracts per year. Legal teams recognize that generative AI could create efficiencies by automating slow, laborious manual review. A recent study found that, thanks to the increasing capabilities of AI, lawyers’ use of AI tools has doubled year over year. In 2026, [87% of general counsels](https://www.fticonsulting.com/about/newsroom/press-releases/ai-adoption-in-corporate-legal-departments-doubles-according-to-the-general-counsel-report#:~:text=FTI%20Consulting%20and%20Relativity%20report%20AI%20use,using%20AI%2C%20up%20from%2044%25%20in%202025.) reported their teams were using AI, primarily for legal research, e-discovery, document review, contract drafting, and contract analysis. As a result the market has responded. The AI-powered contract analysis tools market is worth $4.3 billion in 2026 and is projected to reach $12.06 billion by 2030, growing at a [29.4% CAGR](https://www.researchandmarkets.com/reports/6226104/ai-powered-contract-analysis-tools-market-report). The category is so crowded that many legal professionals find it difficult to evaluate how capable these tools really are. Our own research found that over 50% of respondents struggled to assess AI tools, citing difficulty objectively comparing performance across vendors. We created this independent benchmarking project to help in-house legal professionals determine which tool will work best for their needs. The study addresses three questions: - What is the right way to objectively measure AI tool output? - Can AI tools' claims be independently verified? - Lawyers are already using general AI tools like Claude or Microsoft Copilot. Why invest in a purpose-built product like Ivo if Claude is available? Can in-house lawyers rely on Claude to perform the same work at the same level? To answer these questions we assembled a panel of three senior attorneys to judge the contract review output of Ivo, Claude Opus 4.6 for Word, and an experienced human attorney. Ivo outperformed Claude by a significant margin. In addition, its performance was comparable to a senior practicing commercial attorney at an AmLaw 25 law firm. ## Methodology Ivo conducted a benchmarking study comparing a purpose-built legal AI tool to a general AI tool and a human for redlining tasks ### The Participants - Ivo's latest release - Claude for Word (Opus 4.6) - A practicing Special Counsel at an Am Law 25 firm with 8 years experience ### The Contracts 19 real, anonymized contracts were reviewed, spanning NDAs, MSAs, and DPAs. ### The Judges The outputs were judged by three attorneys with recent experience either working with an Am Law 100 firm or serving as in-house counsel at technology companies. Every output was stripped of identifying information and scored blind across five different criteria for contract review. ### The Scoring Outputs were judged on a scale of 1-10. Final scores represent the mean across all three judges. ## Overall scores Ivo’s performance was nearly indistinguishable from a human lawyer and significantly outperformed Claude on the five judging criteria. - 4.52 - Ivo - 3.50 - Claude for Word - 4.56 - Human attorney ## Key learnings ### Purpose-built legal AI cannot be replicated by general AI. Specialist teams have spent years crafting the prompts, logic, and outputs that create redlines comparable to top performing senior lawyers. General AI tools cannot yet compare to the performance. ### The largest score delta was in surgical redlining and judgment. Ivo’s team has spent a great deal of time perfecting its surgical redlining abilities. In addition, Ivo excelled at selecting which position is best for the client. Ivo also excelled at analyzing complex contracts with complicated transactions. ### Ivo’s output is broadly comparable to a senior practicing attorney at a highly regarded law firm. The human attorney and Ivo had very similar scores, suggesting Ivo’s output was comparable to a high-performing senior lawyer. However, the attorney completed their redlining tasks in 10 hours, whereas Ivo’s average performance was 2 minutes and 45 seconds and Claude for Word’s was 4 minutes and 53 seconds. ## Average time for each participant to complete document review - 2:45 - Ivo - 4:52 - Claude for Word - 10:00:00 - Human attorney ## Issue spotting analysis Did the author spot all the issues within the scope of the playbook? Did the author over-spot issues that may not apply to this contract? Playbook position The playbook prescribes California as the preferred governing law, with Delaware and New York as acceptable alternatives. Binding arbitration (JAMS or AAA) is the preferred dispute mechanism, preceded by a 15–30 day good-faith escalation step. Governing law should never be silent, and litigation in the counterparty's home jurisdiction should be avoided. - Ivo: 8 - Claude: 6 - Human attorney: 5 ### Evaluation This redline kept Delaware law but replaced litigation with JAMS arbitration in Wilmington, Delaware, added a 15-day good-faith negotiation step, and preserved an equitable relief carveout. This most precisely matches the playbook's preferred structure of binding arbitration with a senior-leadership escalation step before arbitration. ### Surgical redlining analysis Did the author make minimal, precise changes to fix the issue, or did they aggressively rewrite the whole paragraph? Playbook position Neither party should assign without consent, except to affiliates or in connection with M&A. Require 30-day advance written notice. Restrict assignment to direct competitors. - Ivo: 7 - Claude: 2 - Human attorney: 5 ### Formatting analysis When the author makes changes, did they maintain font, spacing, paragraph numbering, and cross-references flawlessly in Word? Did they respect and correctly capitalize defined terms specific to this document? - Ivo: 5 - Claude: 4 - Human attorney: 10 ### Commenting test analysis Did the author adhere to the playbook rules and added all the approved comments necessary? - Ivo: 7 - Claude: 2 - Human attorney: 8 ### Judgment test analysis Did the author pick the right position for each issue when there are different fallback options? When some playbook rules have ambiguity and a strict compliance may harm the party’s interest, did they make judgments that are the best for the party? - Ivo: 8 - Claude: 3 - Human attorney: 1 ## Conclusion This independent study offers legal professionals a clear, objective way to measure the performance and verify the claims of legal AI tools, and answers the question: is it necessary to invest in a purpose-built legal AI tool to perform legal tasks or is a general AI tool like Claude for Word good enough? The clear score delta between Ivo and Claude for Word (Opus 4.6) illustrates that Claude’s model is not yet fit for purpose for legal work. Its edits are too imprecise and its formatting too inconsistent to be credibly used by most in-house legal teams. In addition, the fact that these tools hold no organizational memory reduced the scores against specialized legal AI tools. For most in-house legal teams, specialized tools perform better against typical use cases and scenarios. ## FAQs ### Why did Ivo conduct this study? Objective performance data for AI contract review tools is rare. We wanted to create a rigorous, transparent benchmark that gives legal teams a real basis for evaluating tools, including ours. ### Who judged the outputs? Three attorneys with recent experience either working with an Am Law 100 firm or serving as in-house counsel at technology companies. All outputs were stripped of identifying information before scoring, so judges evaluated the work blind across five criteria. ### How were the tools tested? All three participants: Ivo, Claude for Word (Opus 4.6), and a practicing Special Counsel at an AmLaw 25 firm reviewed the same 20 real, anonymized contracts. These spanned NDAs, MSAs, DPAs, and other commercial agreements. ### How were outputs scored? Judges scored each output on a scale of 1–10 across five criteria for contract review: Issue spotting, Surgical redlining, Formatting retention, Judgment, and Comments. Final scores represent the mean across all five judges. ### What were the results? Ivo scored 4.38. The human attorney scored 4.37. Claude for Word scored 3.59. Ivo and the human attorney were nearly indistinguishable; both outperformed Claude by a significant margin. ### Where did Ivo succeed? The largest gap was in surgical redlining and legal judgment; specifically, selecting the right position for the client in context. This is an area where Ivo's team has invested years of product development. ### Does this mean Ivo is as good as a lawyer? On these tasks, Ivo's output was comparable to a senior practicing attorney at a highly regarded law firm. The attorney completed the same work in roughly 10 hours; Ivo took about 10 minutes. ### What are the limitations of this study? This benchmark reflects a single point in time (April 2026) with one human baseline, one prompt configuration for Claude for Word, and 20 contracts across six commercial types. The playbook was provided as system configuration for Ivo and as a user prompt for Claude — which reflects realistic deployment conditions, not a controlled model-to-model comparison.