AI Contract Review: How It Works and Where It Falls Short

Contract review is one of the most obvious places to try AI, because the work is repetitive, text heavy and expensive when done late at night by a senior person. The Solicitors Regulation Authority reports that three quarters of the largest solicitors' firms were using AI by the end of 2022, and that more than 60% of large firms were at least exploring the new generative systems.

That interest is justified, but the marketing around AI contract review often skips the awkward parts: where the tools get things wrong, what happens to the confidential contract you upload, and who is responsible when a missed clause costs money. This guide covers how AI contract review works in practice, what it does well, where it falls short, and how to set it up so a person stays accountable for the result.

Quick answer

AI contract review software reads a contract, extracts key clauses, compares them with your preferred positions and flags risks for a person to check. It is fast and consistent on routine agreements, but it can miss clauses or invent detail. Treat it as a first-pass assistant, keep a qualified reviewer in charge, and check the data handling terms before uploading anything.

How AI contract review actually works

Most tools follow the same basic sequence. The contract is converted to text, split into sections, and passed to a language model that has been instructed to find specific clause types, such as termination, liability caps, indemnities, governing law, renewal and payment terms. The output is a structured summary: what each clause says, where it sits in the document, and whether it matches or departs from a reference position.

The reference position is the important part. A playbook is simply your organisation's written view of what is acceptable, preferred and unacceptable for each clause. A good setup compares the incoming contract against that playbook and highlights deviations, rather than asking the model for a vague opinion on whether the contract is good.

There are three common layers in a working system:

Extraction is the most mature of the three. Research on the Contract Understanding Atticus Dataset, a benchmark built with dozens of legal experts and more than 13,000 annotations, gave the field a shared way to measure how well models find clauses. The dataset's authors described early transformer performance as nascent and said significant room for improvement remained, which is a useful reminder that clause finding was never a solved problem.

What AI does well on contracts

The strongest use cases share a pattern: high volume, similar documents, and a clear definition of what good looks like. Examples include non-disclosure agreements, supplier terms, standard customer agreements, data processing addendums and renewals that need a quick check for auto-renewal dates and notice periods.

Speed is the obvious gain, but consistency matters more in practice. A tired human reviewer on the fortieth agreement of the week may skim a liability clause. A system applies the same checklist to every document, every time, and records what it checked. That audit trail is valuable when a client or a partner asks how a contract was reviewed.

AI also helps with triage. Instead of every contract landing in the same queue, the system can sort agreements into three piles: matches the playbook and can be approved quickly, has minor deviations, and needs senior attention. The people then spend their time on the third pile.

Contract data extraction is a second benefit that is easy to overlook. Once clauses and key dates are pulled into structured fields, you can search across your whole portfolio, track renewals and report on obligations.

Where AI contract review falls short

The honest limitations are the part vendors tend to downplay, so they deserve space here.

Accuracy is good, not perfect

Stanford researchers tested the AI research tools sold by LexisNexis and Thomson Reuters, which are built specifically for legal work. They found that each hallucinated between 17% and 33% of the time on legal research queries. That study looked at research rather than contract review, so the figures should not be transferred directly to clause extraction, but the lesson carries over: a legal-specific product is not the same as a reliable one, and you need to test any tool on your own documents.

Contract-specific testing points the same way. The ContractEval benchmark compared 4 proprietary and 15 open-source models on clause-level risk identification, and found that open-source models often answered 'no related clause' even when a relevant clause was present. The authors concluded that most models performed at roughly the level of a junior legal assistant. That is useful for a first pass and not good enough to sign off unsupervised.

Missed clauses are the costly error

A false alarm costs a reviewer a few seconds. A missed clause can cost a business a renewal it did not intend, an uncapped liability or an exclusivity obligation nobody noticed. Because the second kind of error is silent, the system should be designed so that a reviewer can see what was not found, not only what was flagged.

Context sits outside the document

A contract does not exist alone. Whether a clause is acceptable depends on the commercial relationship, the value of the deal, your negotiating position, and other agreements already in force. A model that sees only one PDF cannot weigh those things unless someone supplies them, and some judgements, such as whether to accept a risk to win a customer, are business decisions rather than document analysis.

Jurisdiction and drafting style

The same words carry different weight under English law, New York law or the law of a civil law country. Playbooks should be written per jurisdiction, and a reviewer with local knowledge should check anything unusual. Heavily negotiated, bespoke or poorly scanned agreements are also harder for tools than clean standard forms.

Confidentiality: what happens to the contract you upload

Contracts contain pricing, personal data, intellectual property and sometimes privileged advice. Before choosing a tool, find out where documents are processed and stored, who can access them, whether they are retained, and whether they are used to train models.

Professional bodies are direct about this. The Law Society of England and Wales says that it is generally advisable not to feed confidential information into generative AI tools where you lack control over how the tool is developed and deployed, and that you should check where data is processed and stored before sharing client data with a vendor. The same guidance recommends using fictional data when testing or building templates.

The gap between a free consumer chatbot and an enterprise product matters here. Enterprise products usually come with contractual commitments. Microsoft, for example, states that prompts, responses and Graph data in its Copilot offering are not used to train foundation models, and that use by organisations is covered by its Data Protection Addendum with Microsoft acting as a data processor. Whatever vendor you choose, read the equivalent terms rather than relying on a sales claim, and confirm who counts as controller and who as processor.

Personal data in contracts also brings data protection duties. If you operate in the UK or EU, read our guide to GDPR and custom AI agents before routing contracts that contain employee or customer details through any external model.

Professional responsibility: the reviewer stays accountable

Regulators have been consistent on one point: using AI does not move responsibility to the tool. The SRA states that firms remain responsible and accountable for the outputs from AI they use, even when a third party provides it, and warns against trusting an AI system to judge its own accuracy.

The Law Society adds that using AI does not absolve you of legal responsibility or liability if the results are incorrect, and recommends documenting inputs, outputs and any errors where the tool does not keep its own history.

In the United States, a law firm summary of the American Bar Association's Formal Opinion 512, issued in July 2024, says it requires lawyers to keep a reasonable understanding of the technology, to independently verify generative AI output, to protect client information, and to obtain informed consent before using the tools in a client's matter, with boilerplate in an engagement letter not being enough. Law Society of Ireland guidance, reported in its Gazette, similarly puts the onus on firms and solicitors to maintain ethical and professional standards.

Not every reader is a lawyer. If you are a business reviewing supplier contracts in house, those professional rules may not bind you, but the principle is sound for any organisation: someone with authority must own the decision to sign, and an AI summary cannot be that person.

A practical workflow for AI contract review

Teams that get reliable results tend to follow a similar process. The detail varies, but the structure is consistent.

Build, buy or combine

Off-the-shelf contract tools are quick to adopt and suit teams with a standard workflow. Their limits tend to appear when your playbook is unusual, when you need the review to connect to your document store, CRM or approval process, or when data residency rules restrict where documents can go.

A custom approach lets you encode your own playbook, choose where data is processed, and wire the review into the systems your team already uses. It costs more design effort up front and needs ongoing testing. Many firms end up combining the two: a packaged tool for standard documents and a tailored workflow for the agreements that define their business.

Law firms and legal teams have particular requirements around privilege and client consent, which we cover in AI for solicitors and law firms.

How Klevere approaches AI contract review

Klevere designs contract review as a supervised workflow rather than a black box. With 500+ AI agents deployed across 50+ projects and 12 industries, we have seen that the projects that last are the ones where the human review step is designed in from the start.

In practice that means we begin with your playbook and a sample of past contracts, and define with your team what a correct answer looks like. We build the extraction and comparison steps so that every flag quotes the source text, and so that the system reports which clauses it could not find. We test against your own reviewed documents before anything goes live, and we set the escalation rules with the person who will be accountable for sign-off.

We also treat data handling as a design decision rather than an afterthought. Where documents are processed, how long they are kept and which model provider is used are agreed with you up front, and documented so your own compliance lead can review them. The review agent connects to the tools you already use through our AI automation work, and where a contract workflow needs bespoke logic we build it as part of custom AI agent development.

We do not promise that AI will replace your reviewers. The realistic aim is that routine agreements move faster, consistent checks are applied every time, and your experienced people spend their hours on the contracts that need judgement.

Frequently asked questions

Can AI review a contract accurately?

It can find and summarise standard clauses quickly, but it is not error free. Benchmarks show models sometimes report that a clause is absent when it is present. Accuracy depends on the contract type, the tool and the playbook, so test on your own documents and keep a qualified person responsible for the final decision.

Will AI replace lawyers for contract review?

Not for work that needs judgement. Regulators such as the SRA say firms stay accountable for AI outputs, and professional guidance expects verification. AI is better seen as a first-pass assistant that handles volume, while lawyers handle negotiation, risk decisions, jurisdiction-specific points and anything unusual.

Is it safe to upload contracts to an AI tool?

It depends on the tool and its terms. Free consumer chatbots are a poor choice for confidential contracts. Enterprise products can offer contractual commitments on data use, storage and training. Check where data is processed, who can access it, how long it is kept, and whether it trains models.

What types of contract suit AI review best?

High volume, standardised agreements suit it best: NDAs, supplier terms, data processing addendums, standard customer agreements and renewals. Heavily negotiated, bespoke or high value contracts still need close human reading, although AI can help by extracting key dates and obligations for the reviewer.

Do I have to tell clients I use AI on their contracts?

It depends on your jurisdiction and your professional rules. The SRA says firms should tell clients when AI will be used on their matters, and a summary of the ABA opinion says informed consent is needed in the United States. Check your own regulator's guidance and your client agreements.

How do I test an AI contract review tool?

Run 30 to 50 contracts that your team has already reviewed, then compare results clause by clause. Record missed clauses and false alarms separately, because missed clauses are the costly error. Repeat the test whenever the tool, the model or your playbook changes.

If you are weighing up AI contract review and want an independent view of where it would help and where it would not, book a free AI audit and we will look at your document workflows with you.