We’ve been told that every Facebook video, promotional or nonpromotional, should include subtitles.
And the numbers confirm: Facebook reports that captioning your videos increases the average view time by 12%.
Facebook has also found that up to 47% of the value in a video campaign is delivered in the first three seconds, while up to 74% of the value is delivered in the first ten seconds.
So, the two best practices for Facebook video ads are: 1) Make the first 10 seconds of your video highly engaging and 2) Add captions to your videos to increase the view time
Easy peasy… isn’t it? But the problem with best practices is that they’re not 100% perfect. They only work most of the time. But that is not enough for AdEspresso.
We set out to test whether adding subtitles to video ads makes sense in the B2B world. Here’s what we found…
Hypothesis and Experiment Set Up
We hypothesized that adding subtitles to our video ads would result in a significant increase in conversions and lower ad costs compared to non-caption video ads.
We decided to run the test with our own video ads that give an overview of AdEspresso’s product and features. The experiment included two videos: the original one without subtitles and another one with captions. The goal was to get warm leads to sign up for an AdEspresso free trial and later on convert them to users.
Campaign Set Up
We ran a 14-day A/B test campaign using AdEspresso to see which variation works best: video ads with or without captions.
To get statistically significant A/B test results, we only tested two variations and kept the ad copy and other ad elements the same for both ads. Each ad had a “Sign Up” call-to-action and the headline “Try AdEspresso Now – 14 Day Free Trial”.
As someone clicked on our ad, they were guided to our sign-up page with a single action – filling in the sign-up form.
Here’s what the video ad with no captions looked like:
Here’s an example of the video ad with subtitles:
Here’s what the campaign landing page looked like:
To set up the campaigns, we used the best Facebook ads A/B testing tool in the world – AdEspresso. It took about 15 minutes to have both campaigns up and running.
We set up each ad variation as an independent campaign for unmixed campaign results and accurate reporting.
We used two placements: Facebook Desktop and Mobile News Feed for both ad sets.
Both campaigns had the daily budget of $20 and ran for 14 days, the total budget being around $600.
We used automatic bidding and optimized our ads for video views, which meant paying for impressions. We decided against custom bidding as we didn’t want the bids to be affecting our A/B test’s delivery or results.
We’re a B2B company, so we decided to target warm leads with a high potential of becoming a customer. We created Facebook Custom Audiences to deliver our ads to people who had shown interest in our product but hadn’t yet signed up for a free trial.
We targeted the custom audience of people who have never had an AdEspresso free trial but had downloaded one of our eBooks.
As we wanted to be sure that our current customers and previous trial users won’t be delivered the ads, we excluded them as custom audiences. We didn’t add any other specific targeting options to avoid narrowing the audience down too much.
As a result, our campaigns had the potential reach of 64,000 people.
Note: We’ve noticed that warm audiences have a higher respond rate when it comes to Facebook ad campaigns. We urge you to consider various retargeting options when setting up your Facebook ads.
As mentioned before, we only used a single ad variation for each campaign and resisted testing other ad elements such as audience interests or ad placement.
After we had chosen all the ad elements, it was time to click the “Publish” button and start waiting for the results to come in.
To add some temperament to this A/B test, we have to admit that the results didn’t align with our initial hypothesis.
So what happened?
We found out that the video ads with subtitles vs. non-captioned video ads performed on a similar level.
This is the place to remind ourselves of the fact that the ad campaigns were targeted on a warm B2B audience. The same test in eCommerce, for example, could show entirely different results.
Here’s the comparison of the two campaigns:
Regarding cost-per-click, video ads with captions performed 3.2% better than video ads with no captions ($0.857/click vs. $0.884/click). When comparing the cost-per-conversion, the difference between the two ad groups was 3.1% for video ads with subtitles.
The B2B video ads with subtitles outperformed the video ads with no captions by 3.1%. While it’s a small difference, it supports the wide trend that we also noticed in other similar tests (keep reading to find out).
There are several reasons why the difference between the two campaigns is so small.
- We targeted a warm audience who’s already aware of AdEspresso’s benefits and features.
- We’re in the B2B marketplace, and the ad we showed was quite long (1:51 minutes), so whoever was interested in the offer, would click on the ad to play it with sound anyways.
This A/B test also gave us lots of fascinating insight concerning ad performance over time and the relevance score.
Here are all our findings:
Takeaway 1: Ad campaigns’ cost-per-click increased over time
When looking at the A/B test results, it’s easy to notice a trend: the cost-per-click significantly increases over time.
Cost-per-conversion stopped increasing at a certain point.
The same rule also applies to cost-per-conversion but on a smaller scale. While the cost-per-click of our ad campaigns increased at a steady pace, the cost-per-conversion hit the ceiling and then stopped growing (check the graph below).
Cost-per-click grew while the number of clicks dropped.
Here’s another graph that shows that while the cost-per-click kept growing, the number of clicks was actually plummeting.
The key message we’d like to deliver with these examples is that Facebook ad fatigue is quick to come. This could be avoided by keeping your eye on Facebook campaign metrics and optimizing your ad sets accordingly.
Takeaway 2: Ads’ relevance score decreased over time
When reviewing our campaign results, we noticed another trend: The relevance score of our video ads was highest at the beginning of the campaign and started to decrease about five days into the campaign.
The main reason behind the increase of Facebook campaigns’ cost-per-click and a drop in relevance score is high ad frequency.
We noticed that in some cases, Facebook delivered our video ads to people up to four times or more (4.44 is the average).
This could have been easily avoided by adding a simple optimization rule in AdEspresso. We’ll keep it in mind the next time and recommend that you do the same.
Takeaway 3: Warm audiences outperform cold audiences
In the section Audience set up of this article, we mentioned that we targeted people who had shown interest in our content but hadn’t yet started a free trial. This means that we targeted a warm audience compared to a cold audience of people who might not have even heard of AdEspresso.
We were curious to find out whether a cold audience reacts to video ads in a different way.
So, we created two more ad sets targeting a cold audience: one featuring the same video ads with subtitles and the other with non-captioned ads.
Guess what we found…
Regarding the cost-per-click, the warm audience outperformed the cold audience by 620% ($0.857/click vs. $6.167/click).
When comparing the effect of video subtitles on a cold audience, the difference is more notable. Video ads with subtitles outperformed the ones with no captions by 26%, in terms of click-through-rate and by 30% regarding the cost-per-click ($6.167/click vs. $7.992/click).
The final verdict:
While the margins were small in some cases, the results were unanimous: Video ads with subtitles outperform the same video ads without captions.
When targeting a warm audience with existing knowledge abut AdEspresso’s product offer, the B2B video ads with subtitles outperformed the video ads with no caption by 3.1%.
As we created another two ad campaigns to test the effect of captioned video ads on a cold audience, the results showed that the video ads with subtitles outperformed ads with no captions by 26% in terms of click-through-rate, and by 30% regarding the cost-per-click ($6.167/click vs. $7.992/click).
Other insights to take with you:
- Ad campaigns’ cost-per-click increases with time
- Ads’ relevance score decreased over time as the ad frequency augmented
- Warm audiences are A LOT more receptive to your offers than cold audiences
Alright, it’s time to wrap up the case study and add some captions to our video ads.
We’d love to hear about your experience with captioned video ads. Leave a comment to share your ideas and learnings with the community! We might even include your story in one of our upcoming blog articles!