Email Marketing ROI Outperforms

The Challenge

Venezia’s is a fast growing, multi-location, family-owned and operated pizza chain that was made famous by the characters and crew of AMC’s award-winning TV show, Breaking Bad. You may remember the scene in which Walter White (played by Bryan Cranston) infamously threw a Venezia’s 24” party pizza onto the roof.

Venezia’s approached Zion & Zion to develop the company’s email marketing strategy and to implement the new strategy. A specific goal was for us to transform Venezia’s email marketing into something that would have demonstrably high ROI compared to the other forms of advertising that Venezia’s was also already engaged in. Over the years, Venezia’s had already whittled down the company’s list of media tactics to those that generated the highest ROI, which meant that the ROI bar was already quite high at the time we took on the challenge.

The Strategy

Our strategy encompassed multiple themes, including focusing not only strategy-driven testing, but also on timing.

Single Hypothesis vs. Competing Hypotheses

Note that it has been our agency’s experience that while companies sometimes engage in subject line testing, those companies generally do not use a hypothesis-driven approach. However, for our agency’s clients, we use a combination of single hypothesis and competing hypotheses approaches for driving email metric improvements.

For those not familiar with the difference, a single hypothesis approach involves us developing reasoning that version B is expected to be better than version A. While a competing hypothesis approach involves us developing dual lines of reasoning, where one line of reasoning argues that version B is expected to be better than version A, and the other line of reasoning argues that version A is expected to be better than version B. The statistical tests for single hypothesis and competing hypotheses approaches are different (one-tailed test vs. two-tailed test respectively). And while we won’t go into the subject any further here, anyone interested in the subject is welcome to contact us for more information.

By introducing a disciplined program of hypothesis-driven subject line testing, we achieved dramatic, statistically significant improvements for the client. Specifically, email click throughs increased 19%; web traffic from email increased 117%; and revenue from email increased 24%.

Competing Hypotheses: A Specific Example

When it comes to event-related email campaigns, such as an email offer campaign timed to be related to the Super Bowl, a natural debate arises. One might argue that we should send the email early for the benefit of customers that like to plan. Alternatively, one might argue that we should send the email the day-of for the last-minute folks.

While we could have simply used a competing hypotheses approach to evaluate which option was better, we also tested multiple other options based on reasoned arguments as to why each option had a shot at being the best option.

The winner? Sending both four days before and the morning of the Super Bowl, with exactly the same offers, but with a subject line that appealed to people that plan in the first email and with a subject line that appealed to last-minute people in the second email.

Using this strategy, we broke the record for highest Super Bowl-related email revenue—a whopping 245% increase in revenue to be exact.

Day-of-Week and Time-of-Day Optimization

Before we developed our strategy for Venezia’s, the client’s approach for when to send an email was based on having emails spaced out and guessing at what the best send-day and send-time would be.

Our approach, however, was to regularly review the email stats to see which days of the week and times of the day not only have the highest open rates and click through rates, but more importantly which days and times have the highest ultimate ROI. It’s important to monitor these stats continuously over time because the data does shift. Day-of-Week and Time-of-Day optimization led to ROI improvements of up to 40+%.

Number of Offers per Email

Even after we launched Venezia’s new email marketing strategy, we continued to make substantial additional improvements. For example, we noticed that over time we were strategically selecting and rotating the best offers in an attempt to increase ROI, however once we started tracking average ROI for emails with different numbers of coupons, we were anxious to start adjusting the number of coupons in email sends as we suspected that this would impact results. Previously, the strategy was to give people a handful of options, so that people could select the offer that best aligned with their own needs.

We therefore began testing competing hypotheses of whether more offers would be better due to the increased level of choice or whether fewer offers would be better due to reducing complexity. Our stats show that emails with two coupons averaged 51.3% more revenue than emails with one coupon; 54.8% more revenue than emails with three coupons; 85.6% more revenue than emails with four coupons; and a whopping 604% increase over emails with five coupons. With this information, we limited the number of coupons per email to range from one to three depending upon a variety of other factors.

The Results

The results referenced in the sections above clearly demonstrate dramatic improvements in both ROI and other, intermediate email metrics. However, as you may recall, at the beginning of this case study, we referenced that the client had set out an overarching challenge for us—namely, to transform Venezia’s email marketing into something that would have demonstrably high ROI compared to the other forms of advertising that Venezia’s was also already engaged in. The data do indeed reflect that we did just that as you can see from the table below that shows the relative ROI of our email marketing compared to the 14 other forms of media used by the client.