Intercom makes it super-easy for you to A/B test your messages so that you can fine tune them to be as effective as possible. All A/B tests should start with a question; i.e., 'will A or B be better at achieving X?'.

Here are some examples:

  • Which content layout will result in more replies?

  • Which content will result in more users upgrading?

  • Which content layout will help me capture more visitor emails?

  • Which subject line will get more users to open my email?

  • Will a button generate more click-throughs than a text link?

There's lots more to why you should A/B test. We wrote a blog post all about it here.

Test, test, and test again

You should always be testing, and learning. For example, if you test a button against a link, and the button wins, next you should be testing the button colour. Then when you have a winner from that, test the button copy. The key point is that the more you test, the more effective your messages will become. Always be testing.

How to set up your A/B test

First, create a new message. Go to Outbound and click on 'New':

Before putting it live, make sure that you have at least:

  1. Given your message a name

  2. Selected your audience.

  3. Selected your channel.

  4. Added the content of your message. This message will be message "A" in your test.

Note: You can start an A/B test against a message that's already live. It doesn't have to be when you create your message for the first time.

Now, start your A/B test

Once you have your message A set up, click 'Run a new test' and choose the option for A/B test.

This will create a draft of your B message. Initially, this is just a duplicate of your existing message. You can now edit the duplicate with whatever it is you're going to test (e.g., different subject lines, different text, different button colours, or different images, whatever you want).

Best practice

We recommend that you only compare one difference at a time. This way, if there is a clear winner, you'll know the reason, and you won't be left guessing about which variant caused the difference.

Finally, put it live

Once you're happy with the two messages, put them live.

Monitor progress

You can check the performance on the auto message list page, or on the message page. On the message page just toggle between version A and version B to see the stats for each. As time goes by, you'll be able to see which variant is performing better in the metrics you care about most. For example, if you were A/B testing subject lines, you would be likely concerned with open rates.

Choose a winner once you have a result

Once you're happy that one message is outperforming the other (for example when one message's goal % is higher than the other) select the 'Pause' button on the message you'd like to stop. 100% of recipients will now get the winning message. This will automatically pause the other message and people will no longer receive it (unless you decide to put it live again).

What's next?

Did this answer your question?