A/B test a sequence
Run two variations of a Victoria AI sequence, split new leads between them, read the verdict, and end the test.
Last updated
An A/B test runs two versions of a campaign's sequence, Variation A and Variation B, on the same leads list. Each new lead is assigned to one variation and stays on it.
Turn on A/B testing for a campaign
- Open Campaigns → your campaign → Sequence.
- Select Enable A/B Testing. Variation B starts as a copy of A and the editor switches to B.
- Change B. Use the A and B buttons to switch between them.
Make the variations differ in angle, not just wording: a different opening, offer or call to action. Two near-identical versions rarely show a difference. If B is empty, Copy from Variation A fills it.
The readiness check adds two checks while A/B is on: Variation B has steps and Variation B steps have content.
Set the A/B traffic split
Split next to the variation buttons sets the share of new leads that get A: A 50 / B 50 through A 90 / B 10. It applies to leads added from then on. Leads already in the campaign keep their variation.
Leads that were in the campaign before you turned A/B on are Unassigned. They run Variation A, count in All, and count in neither arm.
Read the A/B verdict
Open Campaigns → your campaign → Analytics. With A/B on, the tab shows an A/B verdict card and a funnel for each variation, and a filter at the top (All, A, B, Unassigned) narrows every number on the page to one arm.
The verdict compares each arm's positive reply rate (leads with a positive reply ÷ leads contacted) with a two-proportion z-test:
| Verdict | Meaning |
|---|---|
| Not enough data yet | An arm has fewer than 50 contacted leads. |
| No meaningful difference | The arms are too close to call. |
| Leaning A or Leaning B | One arm is ahead, with p below 0.2. Keep it running. |
| A wins or B wins | One arm is ahead, with p below 0.05. |
The arm a lead belongs to is the variation it was assigned when enrolled. A + B + Unassigned always equals All.
Promote the winning variation
The Sequence tab has no promote button. Ask the AI Sales Copilot, for example "promote variation B on this campaign", and approve the card. A connected assistant can do the same with the promote_ab_winner tool.
- Promoting A sends every new lead to A. Leads already on B finish B.
- Promoting B copies B over A and sends every new lead to it. Leads still in progress on A continue on B's copy from their current step number.
A/B testing stays on after a promotion, so leads mid-way through B finish its copy.
Turn off A/B testing
Select Disable A/B, then Delete Variation B. Turning A/B off deletes Variation B's steps. Leads that were on B continue on Variation A from the same step number, which can skip or repeat a step if the two variations have different lengths. To keep B's copy, promote B first instead.