How to run a controlled AI-agent pilot

Most agent pilots are designed to produce activity.

Connect a system. Feed it examples. Let a small team experiment. Collect positive anecdotes. Present a demonstration to leadership.

That process may create enthusiasm. It rarely produces a defensible deployment decision.

A controlled pilot has a harder objective:

Determine whether a defined agent workflow creates sufficient value, reliability and control to justify bounded production use.

The pilot must be capable of reaching “no”.

Before the pilot: write the decision

Complete this sentence:

We will deploy this agent for **[workflow]** if it produces **[primary outcome]** at **[required threshold]**, remains inside **[guardrails]**, requires no more than **[human effort]**, and creates no **[disqualifying incidents]** during **[test period]**.

Example:

We will deploy the inbound qualification agent for UK mid-market website enquiries if it routes at least 90% of gold-set cases correctly, produces no unapproved product claims, records mandatory CRM fields with 98% accuracy, escalates every high-risk case and requires less than three hours of operator review per 100 conversations during a four-week constrained live test.

The numbers must reflect the real workflow. The structure is what matters.

Stage 1: Define the workflow boundary

Describe the job

Record:

• trigger;
• intended outcome;
• inputs;
• sequence of actions;
• systems touched;
• people involved;
• normal completion;
• common exceptions;
• cost of error;
• downstream consequence.

Choose a narrow first boundary

Prefer:

• one region;
• one customer or account segment;
• one enquiry or transaction type;
• one data classification;
• one bounded value range;
• one clear hand-off.

Avoid beginning with “all inbound”, “all suppliers” or “the full sales process”.

Name the owner

Assign:

• business owner;
• technical owner;
• security/privacy reviewer where relevant;
• operators handling exceptions;
• person authorised to pause the test;
• executive accepting the final decision.

Stage-one exit criterion: The team agrees exactly what is in and out of scope.

Stage 2: Build the evidence set

An agent should be tested against the reality of the workflow, not only ideal examples.

Create a gold set

Include representative historical cases with an agreed expected result.

Cover:

• normal successes;
• normal rejections;
• incomplete information;
• conflicting information;
• duplicate records;
• high-value cases;
• unusual but valid cases;
• cases outside authority;
• malicious or manipulative input;
• requests requiring human judgement;
• scenarios where the correct action is to stop.

Record the expected trajectory

Do not score only the final answer. Where material, record:

• which sources should be consulted;
• which tools should be used;
• which policy applies;
• which action is permitted;
• whether approval is required;
• which record should be created;
• which person receives the exception.

Establish a baseline

Measure the current human or automated process:

• completion rate;
• accuracy;
• cycle time;
• cost;
• manual effort;
• exception rate;
• business outcome.

Without a baseline, “the agent saved time” is an impression rather than evidence.

Stage-two exit criterion: The test contains enough representative cases to reveal failure, not only confirm capability.

Stage 3: Replay without authority

Run the agent against historical or synthetic cases in an isolated environment.

The agent may propose actions. It should not communicate externally, spend money or alter production records.

Evaluate:

1. 1.Task completion
2. 2.Factual accuracy
3. 3.Correct source use
4. 4.Tool selection
5. 5.Policy compliance
6. 6.Escalation judgement
7. 7.Output usability
8. 8.Cost and latency
9. 9.Repeatability
10. 10.Failure recovery

Repeat the test

Run important cases more than once where the system is non-deterministic. A single successful run does not establish reliability.

Change one thing at a time

Record changes to prompts, models, tools, retrieval, policy and workflow. Otherwise the team cannot identify which change improved or damaged performance.

Inspect false confidence

Pay particular attention to outputs that are well-written and wrong. Fluent failure is more dangerous than obvious failure because it is more likely to be accepted.

Stage-three decision: Reject, remediate or advance to shadow operation.

Stage 4: Shadow live work

The agent now observes live inputs and produces a recommendation. A person remains responsible for every external or consequential action.

Compare:

• agent recommendation;
• human decision;
• final outcome;
• time used by each;
• reason for disagreement.

Disagreement analysis is especially valuable.

Classify each disagreement:

• agent found something the person missed;
• person used context unavailable to the agent;
• policy was unclear;
• data was wrong or incomplete;
• agent reasoning failed;
• human decision was inconsistent;
• expected result was itself incorrect.

The pilot may reveal process problems that exist independently of the agent.

Measure hidden labour

Track the time people spend:

• preparing inputs;
• checking outputs;
• correcting records;
• handling exceptions;
• maintaining instructions and knowledge;
• explaining the system to users;
• investigating failures.

“Autonomous” is not a useful economic claim if the deployment hides substantial operational work.

Stage-four exit criterion: The exception pattern and supervision burden are understood well enough to design bounded authority.

Stage 5: Release constrained authority

Permit the agent to complete only the actions supported by evidence.

Define the mandate

Specify:

• permitted action;
• resources in scope;
• financial or commercial limit;
• approved counterparties or audiences;
• valid period;
• required evidence;
• approval triggers;
• rate or volume limits;
• excluded data and actions;
• revocation method.

Separate soft and hard controls

Soft control: Instruction in the prompt or workflow.

Hard control: Technical enforcement in the identity, policy, tool, payment or target system.

Use hard controls for material limits whenever possible.

Prepare the exception queue

Every exception should contain:

• proposed action;
• reason for escalation;
• supporting evidence;
• applicable policy;
• time sensitivity;
• consequence of approval or rejection;
• clear owner.

Set stop conditions

Pause the pilot immediately for defined events such as:

• access to excluded data;
• unapproved external commitment;
• financial action beyond mandate;
• material security incident;
• repeated failure to honour stop or unsubscribe instructions;
• corruption of a system of record;
• inability to reconstruct an important action;
• behaviour outside the tested workflow.

Stage-five exit criterion: Constrained live work meets the outcome and guardrail thresholds for the agreed period.

Stage 6: Decide and operationalise

The final review must present evidence for three possible decisions.

Deploy

Use when the agent meets outcome, control and cost requirements.

Deployment conditions should include:

• retained authority boundary;
• monitoring and sampling rate;
• exception owner;
• incident process;
• version/change controls;
• scheduled re-evaluation;
• criteria for expanding authority.

Remediate

Use when the job remains attractive but defined failures prevent production.

The remediation decision should state:

• failure to correct;
• owner;
• planned change;
• evidence required;
• cases to rerun;
• deadline;
• whether live authority remains suspended.

Reject

Use when value is insufficient, control is inadequate or the workflow is not ready.

Record the reason. A rejected pilot can prevent a costly contract, expose a broken process or clarify which infrastructure is missing.

The pilot scorecard

Create a downloadable and on-page table with these dimensions.

Do not create a universal weighted total by default. A single disqualifying failure may outweigh strong performance elsewhere.

The production review cadence

After deployment:

Daily or continuous

• incidents;
• policy violations;
• security alerts;
• failed actions;
• high-risk exceptions;
• unusual cost or volume.

Weekly

• sampled trajectories;
• outcome and acceptance;
• human-review burden;
• repeated failure themes;
• data or integration issues.

Monthly

• commercial outcome;
• total cost;
• authority use;
• model/tool changes;
• drift;
• user and counterparty feedback.

After every material change

• rerun the relevant gold set;
• inspect changed permissions;
• test escalation and stop paths;
• document approval for production.

Final principle

A pilot should not ask whether the agent is impressive.

It should ask whether the organisation has earned enough evidence to delegate this work under these conditions.

Autonomy is not the starting assumption.

It is an authority level justified by observed performance.