Start with a repeatable task whose output you can inspect. A weekly status report is easier to evaluate than a vague goal such as “make our team more productive”. Define what a correct result looks like before using an AI tool.
1. Record a useful baseline
Choose a set of comparable tasks and record the time spent collecting inputs, doing the work, reviewing it and correcting mistakes. Keep the original inputs and expected outputs. Use data that you have permission to process.
2. Keep the comparison fair
Run the same task with assistance, on similar inputs and with the same quality standard. A shorter draft is not an improvement if a reviewer must reconstruct missing information. Avoid comparing a difficult manual example with an easy assisted example.
3. Count the hidden work
| Measure | What to record |
|---|---|
| Preparation | Cleaning data and writing the brief |
| Execution | Generation and tool use |
| Review | Checking facts, calculations and completeness |
| Rework | Fixing errors and handling exceptions |
4. Make a decision from the evidence
Use the median and range of your observations rather than a single best result. Identify which task types worked and which did not. A small pilot may justify a larger experiment; it does not establish organization-wide savings.
Example decision rule
Before testing, agree that the assisted workflow must meet the existing quality standard and reduce total effort after review. Treat this as a decision rule you choose, not a universal benchmark. If quality slips, improve the process before expanding it.
Explore the review and rework trade-off in the simulator →
Keep a pilot record you can audit
For each case, record a case ID, task category, manual baseline, assisted execution time, review time, correction time and final acceptance decision. Keep prompt and tool versions with the record. A reviewer should be able to reconstruct what was tested without relying on a screenshot of the best output.
Use a table with one row per attempt, including failed attempts. Excluding failures understates the effort required to obtain a usable result. If an output is abandoned and the task is completed manually, count that replacement effort as part of the assisted attempt.
Make the next step proportionate
Expand only to work that resembles the evaluated cases. A successful internal summary pilot does not establish that a system can issue customer-facing advice. Recheck the controls when input sensitivity, audience or consequences change.