Start typing to search this publication.
techcirkle
Open menu

Subscribe to techcirkle

Get new posts delivered straight to your inbox.

A Test Plan for Evaluating Generative AI Tools on Your Own Documents

Turn six evaluation criteria into a test plan for generative AI tools: data handling, context, workflow fit, accuracy, cost and exit.

techcirkle
5 min read
Cover image for A Test Plan for Evaluating Generative AI Tools on Your Own Documents

Model leaderboards move every few weeks, and the leading models are close enough that the ranking rarely decides business value. What decides it is how a specific tool behaves with your data, your permissions and your volume.

That can be tested. This post takes six evaluation criteria and turns each into a check you can run, with a pass condition.

A hand holding a tablet that projects a glowing AI head icon with gears and chat bubbles, representing the evaluation of generative AI tools
A hand holding a tablet that projects a glowing AI head icon with gears and chat bubbles, representing the evaluation of generative AI tools

Structure of the plan

The six criteria fall into two groups.

  • Gates. Data handling and context access. A tool that fails either should not reach a pilot, however good the demonstration.

  • Scored tests. Workflow fit, accuracy, cost at ten times volume, and exit cost. These are measured during a pilot and compared across candidates.

Gate 1: data handling

Question. What happens to your data once it enters the tool?

How to test. This is a paper exercise. Get written answers to the following for the exact plan you intend to buy.

  • Is customer data excluded from model training by contract, not only by a default setting?

  • Where is data stored, and do residency options meet your obligations, including GDPR for EU and UK data?

  • Are retention periods configurable, and can deletion be verified?

  • Are single sign-on, role-based access and audit logs included on this plan?

  • Does the vendor hold current SOC 2 or ISO 27001 certification, and will it sign a data processing agreement?

  • What is the position on intellectual property and indemnity for generated output?

Pass condition. Every answer is in writing and acceptable to whoever owns security and legal review. A verbal assurance does not count.

Gate 2: context access

Question. Can the tool reach the documents, tickets, code or records it needs, with the same permissions the user has?

A capable model with no context gives generic answers, so this gate has two halves: reach and permissions.

How to test.

  1. List the systems the use case depends on. Confirm a connector exists for each.

  2. Create two test users with different access rights in the source system.

  3. Ask both users the same question about a document only one of them is allowed to see.

Pass condition. The connector respects source-system permissions. The restricted user gets no answer drawn from the restricted document. If the tool indexes everything under one service account, it fails, because loose permissions mean it will surface a sensitive file to anyone who asks.

Test 3: workflow fit

Question. Does the tool work where people already work, or does it need a new tab and a new habit?

How to test. Walk through one real task from start to finish with a person who does that job. Note each time they leave their main application, and each time they copy output from the tool into another system by hand.

What to record. The number of context switches and manual copy steps per task. Adoption follows convenience, so a tool that needs many of either will see low use regardless of model quality.

Test 4: accuracy on your material

Question. How often is the tool wrong on your documents, and how obvious are the errors?

How to test.

  1. Assemble a set of real questions from the people who will use the tool, together with the source documents that answer them.

  2. Have someone who knows the material write the expected answer for each.

  3. Run every question through each candidate tool under the same conditions.

  4. Mark each response as correct, partly correct or wrong.

  5. For each wrong answer, add a second mark: would a typical user have noticed the error?

What to record. Two numbers per tool: the error rate, and the share of errors that were hard to spot. The second number matters as much as the first. An obvious mistake gets caught. A confident, plausible mistake reaches a customer.

Keep the question set. It becomes your regression suite when the vendor changes its model or you consider switching.

Test 5: cost at ten times pilot volume

Question. What does this cost at real usage?

How to test. Take the usage you measured in the pilot, such as requests, resolutions or active users, and multiply by ten. Price that volume under the vendor's actual pricing model. Do the same for each candidate.

What to record. Projected monthly cost at ten times volume, and how the cost behaves: flat per seat, or rising with each resolution or request. For high-volume, repetitive work, compare the result with token-based API pricing as well. A per-resolution fee can end up several times the cost of running the same workload on your own integration.

Test 6: exit cost

Question. If the vendor raises prices or falls behind, how hard is it to move?

How to test. Try an export during the pilot instead of asking whether one exists. Attempt to get out the three things you would need to rebuild elsewhere:

  • Prompts and any saved instructions

  • Workflows and automations

  • Stored knowledge, such as uploaded reference material

Pass condition. All three can be exported in a usable form. Also note the contract length. Leaders in this market change every few months, which makes multi-year commitments risky.

Scoring the results

Put the outcomes in one sheet with a row per candidate and a column per criterion. The gates are pass or fail. The four tests carry your recorded numbers.

Some outcomes say more about the use case than about the tool. If every candidate fails the context gate because the data sits in an internal system no vendor connects to, the answer may be a custom build, not a different product. That is the point where LLM integration with your own systems becomes the better investment.

Where this fits

This plan is the evaluation step from a broader buyer's guide, which also names the leading tools in each category and covers rollout and cost. You can read it here: best generative AI tools for business.

Frequently Asked Questions

Why not choose an AI tool based on model benchmarks?

Leaderboards change every few weeks and the leading models are close in capability. The ranking rarely decides business value. Data handling, access to your context, workflow fit and accuracy on your own material have far more effect on results.

What should disqualify a generative AI tool before a pilot?

Failing on data handling or context access. If the vendor cannot confirm in writing how your data is used, stored and deleted, or if the tool cannot reach the information it needs with correct permissions, a pilot would waste time.

How do we test the accuracy of an AI tool?

Use your own documents and questions, not the vendor's demo. Prepare expected answers in advance, run the same questions through each candidate, and record how often the tool is wrong and how obvious the errors are.

Why model cost at ten times the pilot volume?

Because pilot usage is small and usage-based or per-resolution pricing grows with volume. A tool that is cheap for a pilot group can become expensive at full rollout. Pricing ten times the pilot volume shows how the cost behaves before you sign.

What does exit cost mean for an AI tool?

It is the effort required to leave the vendor. If prompts, workflows and stored knowledge can be exported, moving is manageable. If they cannot, a price rise or a decline in quality leaves you with few options.

Subscribe to techcirkle