CiviMail A/B Testing

CiviMail A/B Testing allows you to send two test e-mails to two random samples of your recipients. After the test mailings are complete, you can choose to send the more successful mailing, based on the metrics available, to your remaining recipients. This chapter looks at the type of testing available and how to configure a new test.

Before conducting a new A/B Test, be sure you put your recipients in a Mailing List. To learn more about how to configure Mailing List see the chapters on Groups and Tags.

Type of Tests

Creating a New A/B Test

To create a New A/B Test, navigate to Mailings > New A/B Test. At the Setup screen you can define the campaign that corresponds with the mailing and select the type of testing you want to conduct.

There are three different types of test you can perform, these are:

• Different Subject lines
• Different From lines
• Testing entirely different emails

At the Target screen you can define your recipients and the distribution percentage. You can modify the percentage of test mailing by dragging the blue percentage bar, which in turn controls the percentage of final mailings that will be sent to the remaining participants. The total of the test mailings and the final mailing will sum up to 100%.

The Compose screen will change dependent on the type of A/B Test you are performing. For example, if you chose to conduct a test on "different From lines," the compose screen will contain two From fields.

When you test using the From field, you will have the option to enter From (A) and From(B)

When you test using the Subject field, you will have the option to enter Subject(A) and Subject(B)

When you test using entirely different emails, you will add a From and Subject for each mailing:

Choose an email template that you have previously composed. Once you have selected your email template and specified the From and Subject field, make sure you send a test email before scheduling or sending your final mailing.

Metrics

Once you have completed your test mailings, you will have access to your results by navigating to Mailings > Manage A/B Tests, then selecting Results. On the A/B Test Results screen you can monitor your test results and select which mailings should be submitted as the Final mailing.