There were two keys to this success, Yee said. Subtitles enabled viewers to watch videos that autoplayed without sound. The other is adding a trailer, which let audiences preview longer-form content.
But Refinery29 needed to figure out how these different tactics affected engagement rates versus completion. In doing so, the publication could impact results for advertisers who want to create branded video content with Refinery29.
It developed with Adaptly an A/B test comparing engagement rates between long-form video with a 22-second-long preview trailer versus long-form video without.
There were four test groups that represented 469,900 people in total: videos with no trailer nor subtitles; videos with no trailer, but with subtitles; videos with a trailer and no subtitles; and videos with a trailer as well as subtitles.
Each test group was shown the same long-form video content concurrently throughout a four-day period. During the process, two groups received long-form video with subtitles and two groups received the content without subtitles and measured the results.
Refinery29 learned a couple of things. First, teasing long-form content as a trailer increased the view rate by 6%. However, completion rates decreased because users had already gotten their fill of the content.
“It was really interesting that a trailer actually drives more viewership initially, but that viewership is actually less engaged because you’ve already given them a taste of what it is they’ll actually see,” Yee said.
Refinery29 also studied subtitles, which it already used because 90% of its video views on Facebook happened through muted autoplay.
Refinery29 found subtitles improved the number of likes the video got by 10% and increased the rate of comments on the video by 29%.
Ultimately, Refinery29 and Adaptly concluded that a combination of no trailers with subtitles increased completion rates and engagement the most.
In other words, trailers drove up total user view counts, but if you want to drive engagement as measured by likes, comments and shares, you might want to toss in some subtitles.
“Most of the industry, early on, was so focused on top-of-the-funnel and looking at total views and not really understanding what that qualification or definition of a view actually was,” Yee said. “Social media, especially Facebook, in many ways has commoditized reach. Marketers want to go deeper now around brand engagement and determine if that reach equated to impact or lift in a reader or customer’s sentiment.”