Testing your communication can be so much fun, but do you know what and why you are testing? Dig into a li'l walk though to learn more about testing your sendouts.
To create a successful AB, ABC or ABCD-test, first you should make sure that you have recipient lists that are big enough to generate a statistically adequate result.
If your have recipients with more than 10.000 per list it will be worth while. But if you have a lower amount of recipients you will, most probably, not be helped by an AB-test. Simply because the test groups selected will never be big enough to give a winning result that you can rely on.
PLEASE NOTE: There is not possible to have a send out speed limit when you do an AB(CD)-test.
How to set up a AB-test in Symplify?
- Decide the type of your test.
- Split test: Splits the target group in two, fifty/fifty. No winning version is generated.
- A/B/C/D test: splits the target group according to your adjustments but saves a remaining group to send the winning version to.
- Choose what part you want to test with your different groups. With Symplify ABCD-test you can choose to test on Subject Line or Email (content).
- Adjust the size of your different test groups. To the right you can see how many procent of the recipients that will receive the winning version.
Evaluation of winning version
Depending on the type of your test, you should select a proper evaluation type. If you selected to test on Subject line, the suitable evaluation type is Open rate. If you selected Content, then clicks or conversions is more appropriate.
The winning version is the A or B (or C or D) that generated the best result when the time for your test is up. In below example, open rate will evaluate the winning version after three hours.
PLEASE NOTE: Remember that the winning version is based on the result at the time you have set. So the result may actually change after the winning version has been sent. Which may indicate that you should set a longer evaluation period next time.
Statistics below are showing that the B-split was the most successful version at the time for when then winning version was sent out.
But, at the same time, the A-split was actually more successful over time.
Create AB(CD)-test on content
When you choose to create an AB(CD)-test based on different content you will get the possibility to create generic content that should be the same for all your different AB(CD)-tests. You can also choose to make adjustments for each version (A,B,C and D).
1. You start with the Shared Content, just hit the Content button and start to edit:
2. Then you can proceed with A and/or B:
You can copy specific slice content, start by clicking on this icon in the block you want to copy, then choose where you want to copy it: