The 8 Dos and Don’ts of Email Marketing A/B Testing

No Image
November 08 2018

You’ve probably heard time and time again that email marketing consistently offers the highest ROIs across any channel, especially among B2B industries where other options are often limited.

Yet, if your marketing team is like many, you may be underwhelmed by what you’ve seen from your efforts thus far. Your emails are delivering some results, but you were hoping you’d be seeing much more from a channel with such an impressive reputation.

If this sounds familiar, chances are your solution is email marketing A/B testing.

However, before you begin, it’s important that you know what to do–and not to do. Otherwise, this powerful tactic may continue to disappoint–or even produce unintended consequences.

That’s why today we’ll be talking about the dos and don’ts of email marketing A/B testing.

4 Dos of email marketing A/B testing

Testing your emails to see which elements hold the most promise is an enticing prospect, but it’s not one you should rush into.

At the very least, you’ll waste a lot of time if you don’t know the basic best practices of email marketing A/B testing.

Here are the most important four practices to keep in mind:

1. Do: Focus on general concepts first

The biggest rookie mistake most marketers make when testing emails? They home in on specific features long before they have a general concept of what works. It’s important to have a plan that produces some degree of returns.

In short, you need a basic control variable before you can begin email marketing A/B testing, and it should be an actual framework that has seen results. Even producing a significant amount of opens is enough when you’re just beginning.

Take some time to create your first marketing email based on what you know about your company’s audience. Modify it as necessary until you see results. Then–and only then–should you begin email marketing A/B testing.

2. Do: Segment your list as soon as possible

You probably already know about your B2B company’s various client segments. You know that your market is made up of different buyers and that each requires a different approach to turn them into clients.

In that case, be sure you segment your list before you begin email marketing A/B testing.

For example, let’s say you have two main segments: small and medium-sized businesses and enterprises/corporations. Chances are probably good that these two segments require very different types of marketing. Even something as simple as subject lines will probably differ greatly between these two segments.

So, it wouldn’t make any sense to test a new subject line by sending them to your entire list at random. The results won’t be as significant. In fact, you’d most likely see wildly different results every time you sent the same message to a new, random group.

Instead, separate these two segments. Test a new subject line on one and another new line on the other. Only then will the results be truly usable.

3. Do: Test with your ultimate email marketing goal in line

There is absolutely no end to the number of different tests you could run to improve your emails. Here are just some of the most common examples of the many components you can test:

  • Body text
  • Call to action
  • Customer reviews
  • Image placement
  • Imaged
  • Layout
  • Personalization
  • Rich media vs. Plain text
  • Subject line
  • Template
  • Testimonials
  • Time and Day the email is sent

All of these are probably worth testing at some point, too.

However, you want to make sure you test them with your email marketing campaign’s ultimate goal in mind.

Generally, this ultimate goal is conversions.

So, for example, people usually think the goal of a subject line is simply to get the email opened, but that wouldn’t be the case if your ultimate goal was to convert more leads. In that case, the subject line’s objective is to get the email opened and prime the recipient to convert.

If you don’t make this distinction, you might be very happy with a subject line that gets more opens and assume your next task is to work on your email copy to get more conversions. However, the problem might be that the subject line over promises, which gets the message opens but ultimately turns people off.

4. Do: Control for active subscribers only

Finally, when A/B testing, only send your emails to active subscribers. Otherwise, you may group a disproportionate amount of inactive subscribers in one segment, which will hurt the validity of your results.

You can still send test emails to inactive subscribers. Who knows? Maybe you’ll accidentally discover a new means of grabbing their attention. However, it will distort your results if you include results from inactive subscribers with those from active recipients.

Re-engagement emails would be another exception. If you’re looking to breathe new life into an unresponsive segment, A/B testing these emails could be very helpful.

4 Don’ts of email marketing A/B testing

There are also certain things you should never do when it comes to email marketing A/B testing. These are the kinds of mistakes that can either render your test useless or cause misleading results that convince you to make costly changes to your email campaigns.

Here are the four biggest email mistakes that you must avoid at all costs if your email marketing A/B testing is going to be effective.

1. Don’t: Test more than one feature at the same time

Getting caught up in the excitement of email marketing A/B testing is easy, especially as results come in and your efforts become better.

Still, you need to be patient enough to only test one feature at a time. Otherwise, you might see promising results, but you won’t know what’s responsible.

For example, say you test out a new subject line and add a new call-to-action to an email and test it against your control for a certain segment.

It turns out, this new email sees 10% more conversions, but, unfortunately, you don’t know why.

While your subject line obviously deserves the credit for increased opens, you don’t know if it was also responsible for the increased conversions or if that was because of your CTA.

In some rare situations, multivariate testing may make sense, but when you’re first starting email marketing A/B testing, there’s no reason to move past one element at a time.

2. Don’t: Act on results too quickly

Speaking of excitement, when results first start coming in–or a lack thereof becomes apparent–it can be very tempting to start making changes to the rest of your funnel.

For example, out of 1,000 emails, say you’ve seen 50% more conversions from the first 100 that have been opened. It would be hard to resist making changes right away to keep seeing that trend.

Of course, the risk is that the results from those first 100 emails are just outliers. The impressive results were just random chance. So if you begin applying them too quickly, you’re subjecting the rest of your campaign to the same volatility.

3. Don’t: Focus on minor changes

In the world of A/B testing the button-color test has become famous–or infamous, depending on how you look at it.

That’s because, while changing the color of a CTA button can produce dramatic results, it doesn’t happen nearly as often as the popularity of this test might convey.

Consequently, countless marketers have become obsessed with the most nuanced features of their emails, often focusing on components that recipients probably don’t even notice.

Perhaps the day will eventually come when you have finally fine-tuned your email campaigns to the point that a feature this minor actually matters. Until then, keep testing subject lines, body text, CTAs, and other components that actually make a difference.

4. Don’t: Ignore your automated and transactional emails

Finally, don’t ignore your B2B company’s automated and transactional emails, the one that gets sent to your client after they have made a purchase.

These emails don’t often receive the same love from marketers as promotional campaigns. In fact, they’re almost an afterthought.

But, as the saying goes, if it’s worth doing, it’s worth doing right.

Your company clearly sees some value in them or you wouldn’t bother sending them in the first place, so don’t forget them when you’re doing your A/B testing.

If nothing else, these emails can be fantastic opportunities for upselling and cross-selling. Instead of just sending automated order confirmations and follow-ups, why not use A/B testing to see if you can actually add more value to each and every sale?

Wrap up: Applying the dos and don’ts of email marketing A/B testing

Now you know some of the classic dos and don’ts of email marketing A/B testing. Without a doubt, the most important tip from our entire list is simply testing regularly.

While it might take some time to set up your first few A/B testing, the more you do it, the easier it becomes. Over time, it should become a normal part of your marketing team’s operation to think of new, testable elements. Test these elements (one at a time) to continually improve your efforts.

When this becomes the norm, your company will enjoy the high ROIs that email marketing has become known for across every B2B industry.