Pulling a payment term out of a contract PDF with a language model is a twenty-minute exercise. Building something a legal or procurement team will rely on for a commercial decision is a genuinely different engineering problem, and most teams discover the gap after the demo has already been praised.
We hit this building supplier intelligence platforms, where a portfolio of 1,000+ contracts has to yield structured commercial terms — payment windows, liability caps, service credits, renewal and termination provisions, indemnities. Here is what separated the pipelines that got adopted from the ones that got politely shelved.
The three decisions that determine adoption
1. Extract to a strict schema, never to prose
The naive version asks the model to summarise key terms. The output is readable and unusable, because nothing downstream can consume it and nothing can be validated.
Define a typed schema instead. Every field gets a name, a type, and constraints:
payment_terms_days: integer, 0-365
liability_cap: { amount: decimal, currency: ISO4217, basis: enum }
auto_renewal: boolean
notice_period_days: integer, 0-730
governing_law: string (jurisdiction)
Constrain the output format so the model returns structured data conforming to that schema, and validate on receipt. A value outside its constraint is a signal, not a rounding error — a payment_terms_days of 3650 means something went wrong and you want that surfaced rather than stored.
2. Every field carries a clause citation
This is the decision that actually determines whether legal will use the system.
A payment term of 60 days with no provenance is a claim. A payment term of 60 days pointing to document MSA-2024-117 , page 12, clause 7.2, with the source text retained, is a fact someone can verify in eight seconds.
When a dispute arises — and with contracts, disputes are the entire point of the data — the reviewer needs to get from the extracted value back to the source language immediately. A pipeline without citations forces them to re-read the contract, which means the pipeline saved nobody anything.
Store the citation as structured metadata alongside the value: document ID, page, clause reference, and the verbatim source span.
3. Confidence scores that route, not decorate
Ask for a confidence per field and use it to make a decision.
Above threshold, auto-accept into the governed store. Below threshold, queue for human review with the citation and source text displayed. The reviewer confirms or corrects, and the correction is recorded as a reviewed fact with an attribution.
The critical rule: never silently default a missing or low-confidence value. A contract with no explicit liability cap is not a contract with an unlimited cap or a zero cap — it is a contract requiring a human to look. Defaulting here is how you end up with a risk register that is confidently wrong.
Set thresholds per field rather than globally. An incorrect payment term is an inconvenience. An incorrect liability cap is a legal exposure. Their tolerances should not match.
Pipeline shape
PDF → text + layout extraction
→ clause segmentation
→ candidate retrieval per target field
→ structured extraction with citation + confidence
→ schema validation
→ confidence routing (auto-accept | review queue)
→ governed store (value + citation + provenance + reviewer)
Two notes on the middle stages.
Layout matters. Contracts carry meaning in structure — numbered clauses, schedules, annexes, defined terms. Flattening a PDF to raw text destroys the clause hierarchy you need for citations. Use extraction that preserves layout and section structure.
Retrieve before extracting. Do not feed a 60-page agreement into a single call and ask for twelve fields. Segment into clauses, retrieve the clauses likely to contain each target field, and extract from that focused context. Accuracy improves, cost drops, and the citation is precise by construction rather than by inference.
The part that stays human
Some things will not be reliably extractable, and pretending otherwise is where trust is lost.
Non-standard drafting, heavily negotiated bespoke clauses, terms defined by reference across multiple documents, and anything where the commercial meaning depends on an amendment history — these route to humans. The pipeline's value is handling the 80% that is standard so specialists spend their time on the 20% that is not.
A pipeline that claims 100% coverage will be wrong in ways nobody catches. A pipeline that explicitly flags what it could not determine gets trusted, because its limits are legible.
Why the boundary matters downstream
Extracted terms feed calculations — is this delivery late against the contractual window, did this incident breach the service credit threshold, what is our aggregate liability exposure by category.
Those calculations must run in deterministic code against stored, reviewed values. Never call the model at read time to answer them. Once an extracted value is accepted, it is a row with provenance; downstream logic reads the row. That is what makes the numbers reproducible when someone asks why an answer changed.
Realistic expectations
A portfolio of over a thousand contracts is processable in days rather than the year a manual review would take. Expect a meaningful minority to route to human review, and treat that as the system working correctly rather than as a shortfall.
The outputs that tend to deliver value fastest are not the sophisticated ones. A renewal calendar showing which agreements auto-renew within ninety days, and an aggregate liability exposure view by category, both fall out almost for free and are immediately useful to legal and finance.
Frequently Asked Questions
Why not just summarise contracts with an LLM?
A summary cannot be validated, queried, or joined to other data, and it carries no provenance. Structured extraction to a typed schema produces values that downstream systems can consume and that reviewers can verify against a cited clause.
How important are clause-level citations really?
They are the difference between adoption and rejection. Without a citation, verifying an extracted value means re-reading the contract, which eliminates the time saving the pipeline was supposed to deliver. Legal teams reasonably decline to rely on unverifiable claims.
What should happen when confidence is low?
Route to human review with the citation and source text visible. Never default the value. Set thresholds per field according to the cost of an error in that field — a wrong liability cap is far more damaging than a wrong payment term.
Should the whole contract go into one model call?
No. Segment into clauses, retrieve the segments relevant to each target field, and extract from that focused context. This improves accuracy, reduces cost, and makes the citation precise rather than inferred.
What proportion of a portfolio can be automated?
Expect the standard majority to extract cleanly and a meaningful minority — bespoke drafting, cross-referenced definitions, amendment-dependent terms — to require human review. A pipeline that flags its own gaps earns more trust than one claiming complete coverage.
This is one component of a larger architecture. The full piece covers the complete supplier intelligence stack, the build-versus-buy decision, and the cost model: SRM Software in 2026: A Build vs Buy Guide for CTOs .
More on our approach to LLM integration and AI development services .