Experimentation Best Practices

Tips

Understand the benchmarks.

What do past results look like? This is important to understanding the results you obtain from A/B testing. This will also help you set a practical goal.

Have a clear goal and hypothesis

What results are you aiming to achieve? What do you think is going to happen? These guiding questions will allow you to build an effective A/B test.

Control your experiment

Only change one variable at a time to get clear and helpful insights. To do this, finalize the first variant and create all the other variants. Then go into each variant to change that one variable.

Have a control version

This is the version you would have sent before testing any variables. The results with your control are the baseline to measure the results from your variants.

Test the variants at the same time.

Timing matters so you want to ensure users see the message at the same time and day.

Continuously experiment to optimize your strategy.

There are many variables to test, such as:

• Subject lines
• Email layouts (e.g., Image vs. image with text vs. image and text)
• Different CTA types (e.g., “Start Trial” vs “Claim My Free Trial”)
• Email copy lengths
• Offers/promotions
• Landing pages