A/B testing your emails

An A/B test compares several versions (variants) of the same email to find out which one gets the most replies.

1. Open the campaign, Sequence tab, and choose an email step.
2. Click "Add variant" (A, B, C…). With several variants, leads are split according to each variant's weight.
3. Under A/B test, choose the metric (reply rate or positive reply rate), the minimum number of sends per variant and the confidence level.
4. Optional: turn on "Pick the winning variant automatically".
5. Follow the results in the campaign's Analytics tab.

How the winner is chosen

• Until a variant has reached the minimum number of sends, the screen shows how many sends are missing.
• A variant wins only if it beats all the others with statistical significance. Otherwise there is no winner.
• Default settings: 100 sends minimum per variant (adjustable from 10 to 10,000) and 95% confidence. With very few sends, results are unreliable.
• With automatic selection on, the winning variant is kept and all remaining sends use it. A variant you pick yourself is never replaced, and you can clear the winner to split the variants again.