A business can switch on AI agents in one of two ways. It can pick a Monday, turn off the old process and trust that the build was right. Or it can run the agents on real work, beside the people who do the job now, until the record shows which way is better.
The first way feels decisive. It also removes the one thing that tells you whether the new way works: something to compare it with. Once the old process is gone, every mistake turns into an argument about whether it would have happened anyway.
A parallel run keeps the comparison. For a few weeks, the business does the same work twice, once the usual way and once with AI agents, and records every difference. It also does something nobody planned for: it measures the old way, often for the first time.
This piece is for the chief operating officer (COO) whose AI agents are built for a job and who has to decide when the business relies on them.
Four stages of a parallel run
- Shadow. People do the job exactly as before. The agents work the same cases, but nothing they produce reaches a customer or changes a live record. Each day, someone compares the two.
- Compare. For each case, note whether the agents' result matched the person's, differed for a good reason, or was wrong. The differences teach more than the matches. Some show where the agents go wrong. Others show where people do the job in different ways.
- Switch one slice. When a narrow slice of the work passes the test, the agents take that slice live, with a person approving their work. Everything else stays as it was. "Orders from existing customers in the standard format" is a slice. "Orders" is not. How much the agents may later do without approval is a separate decision, earned on the live record.
- Retire the old way last. The old process stays written down and ready for an agreed period after the switch. It is switched off only when that period passes with no reason to go back.
Write the switch-over test before the first day
The job's owner writes the test before the shadow run starts. A test written after the results arrive tends to fit them. Five lines are enough.
- How many cases, over what period. Enough to include the awkward ones: a month-end, a busy week, the customer who never uses the order form. Not two quiet days.
- What counts as a match. Decide what "the same result" means for this job, and who settles a disagreement.
- The mistakes that rule it out. Name the kinds of mistake that stop the switch, however good the rest of the record. One of those during the run, and the slice waits.
- Better, or as good? Say whether the agents must beat the current process or only match it. Matching can be enough when the point is to give people's time back to work that needs them.
- The way back. Write down what sends the work back to the old way after the switch, who decides, and how fast people can pick it up again.
An example: orders that arrive by email
Here is how a parallel run might go. It illustrates the method; it is not taken from a client.
At a mid-sized distributor, sales administrators type customer orders from emails into the order system. The operations manager owns the job. AI agents are built to read each email and prepare the order.
For four weeks, including a month-end, the administrators carry on as before. The agents prepare the same orders in a copy of the system that touches nothing live. Each afternoon, one administrator compares the two sets and notes every difference.
The differences fall into three groups. Some are the agents' mistakes, mostly with customers who write product names their own way. Some are the administrators' own mistakes, which nobody had been counting. And some show two administrators handling the same kind of request differently, a gap nobody had noticed until the run put the two side by side.
The operations manager agrees one way to handle those requests, and the agents' rules for product names are corrected. Orders from existing customers in the standard format then meet the test.
That slice goes live, with an administrator approving each order before it is confirmed. New customers, changed orders and anything that mentions a price stay with people. The typing process stays written down, and the team knows how to return to it.
Where the method breaks
Doing the work twice wears people out. A parallel run adds comparison to a full day. Keep the slice small enough to compare properly, and say when the run ends.
The old way is treated as the right answer. When the agents and a person disagree, it is tempting to assume the person is right. Sometimes the agent caught the mistake. Settle each disagreement against the rule, not against who made it.
Some work can only happen once. A reply to a customer or a payment cannot be made twice. For those, the agents prepare what they would have sent, and the comparison is made on the draft.
The run never ends. Without an end date, a parallel run becomes a second process that nobody trusts and everybody maintains. Set the date when you write the test, and decide on that date.
What stays with people
The job's owner writes the test, settles the disagreements and decides the switch. The record informs that decision; it never makes it. Anything that commits the business to a price, a payment or a promise to a customer stays with a person, after the switch as before.
Switch on evidence, one slice at a time
A parallel run answers two questions at once: are the AI agents ready, and how well does the current process work? Both answers are worth having before the business relies on the new way. Switch one slice at a time, on a test you wrote first, and keep the way back open until you are sure you will not need it.
Futuvara is a creative technology agency. We build, implement and run AI platforms for our partners. If your AI agents are built for a job and you need to decide when the business relies on them, start a conversation.
