A Primer to A/B Testing Email Content in Your Campaigns


A/B testing email content is strangely overlooked, and it really bugs us.

I mean, why would you not want to test the actual words your subscribers read?

Granted, a lot plays into the success of an email marketing campaign.

Timing, audience segmentation and maintaining an up-to-date subscriber list all play a big part.

But there’s one item left out of that list:

Your email content.

Strangely, it’s one of the most important parts to any email.

And one of the easiest things to set up.

But how do you know whether your content really resonates with your subscribers?

Before we dive into that, let’s back up for a second.

What is email A/B testing?

Let’s pretend you own your own restaurant—Nikki’s Deli.

Things have been pretty flat in terms of customers and revenue recently.

You’ve decided to change the menu to determine whether or not it has an impact on sales.

Gosh, that sounds like an A/B test waiting to happen!

You have your existing menu with just text, no images of the meals on one side of the restaurant.

And here’s where A/B testing emails comes in:

On the other side of the restaurant, you create the exact same menu, with the exception of adding images of each of your specials at the top of the menu.

Now half of your patrons have the old version of your menu, and the other half are using the new menu.

A week goes by, and you compare revenue from the two different menus.

You’re happy to discover the menu with the images of the specials increased the average table order prices by 20 percent.

Congratulations, you have a winner!

winner

This is a prime example of how email A/B testing works.

You take an existing version of something (Variant A or your control) and make a change to another version (Variant B) and track the results of how each performs.

The same principle can be translated to your email marketing content as well.

Now let’s set our sights on email content:

Content: More than just words

Before you begin testing your content, make sure you’re thinking broadly about what the word content actually means.

It’s not just the body copy in your email message.

It’s everything!

It’s also your subject line, headline, images, layout, length, font…

Anything that is part of what your subscribers see is your content.

But when it comes to A/B testing email content, you shouldn’t test multiple variants at once.

Here’s why:

Narrow your focus

In order to get useful information from the results of your A/B testing, you need to test only one feature at a time and have a specific question you’re trying to answer.

Like whether a personalized subject line results in higher open rates.

In that case, you would send your A/B groups the same content, with the only difference is the inclusion of the recipient’s name in the subject line for one test group.

Don’t try to draw conclusions about metrics you weren’t testing.

For example, if you find people who read a personalized subject line are more likely to open your email, and a certain percentage of them actually made a purchase, don’t infer that personalization leads to more sales.

You will need to devise a method for testing that assumption – say, by looking at which segment of your audience is most likely to make a purchase after receiving a personalized email.

Some email software, like Delivra, will send a small sample the emails and then automatically choose an email winner to send to the remaining group.

Build upon lessons learned

As email A/B testing reveals which basic content strategies are most successful for your objectives, you can begin to experiment with calls to action.

Amazon.com employed split testing when asking readers to review a book.

ab testing email content amazon greg example

One approach asked the reader to click on a link to submit a review; the other approach asked the reader to click on a rating within the body of the email.

When you know your audience is pressed for time, condensing steps as Amazon did may be more successful at encouraging a specific action.

Ready to start A/B testing your email content?

Great! Here’s a list of testing ideas to get you started…

A/B Testing Email Content Examples

Subject Line

  • Take a shorter subject line and pin it up against a longer subject line.
  • An answer subject line vs. a question subject line
  • A negative subject line vs. a positive one
  • Using emojis vs. just text
  • Using an exclamation point vs. no !

Message Preview/Preheader

  • Use the first line of copy from your email
  • A short summary of what the email entails
  • Incorporate the call-to-action in the preheader

Personalization

  • Use contact’s first name in the subject line
  • Use contact’s first name in the salutation
  • Use contact’s first and last name in any of the tests mentioned above
  • Use the contact’s persona “marketer, sales reps, web developers, etc.”

Body Copy

  • Shorter copy vs. long form copy
  • Numbered list
  • Bullet points
  • Q&A style

Images

  • One image vs. no images
  • One image vs. multiple images
  • Animated GIF
  • Screenshot of a video
  • Color image vs. black and white

Calls-to-Action

  • Button color
  • Short CTA copy vs. longer CTA copy
  • ALL CAPS vs. regular case
  • Size of font or button
  • Exclamation point or no !
  • Bold copy or regular

While data can tell you a lot about your audience’s behaviors, sometimes it’s a challenge to know how to employ A/B testing to understand the performance of your content.

Delivra can help you with that!

Our experienced Client Success Managers can work with you to develop A/B testing strategies that give you the right information to fine-tune your digital marketing messages.

Check out our video overview or our grab a free copy of our Tried & Trusted Guide to Email Marketing.

It’s full of great tips on optimizing your email marketing efforts, including A/B testing email content!


EVENTS


Join the email marketing community at an upcoming event near you.

VIEW EVENTS