Analysis: NPR’s use of preferred audience tags on Facebook
By Dan Frohlich
Earlier this year Facebook launched Audience Optimization with the idea that publishers would be able to better engage with niche communities. Here at NPR we jumped on the idea almost immediately and started using preferred audience tags on the vast majority of our Facebook posts. (I also refer to this system as audience targeting). We recently reviewed the metrics on our link posts to see how adding preferred audiences impacted both reach and engagement.
For this analysis, we looked at 2,573 total link posts from March – May 2016. We used a difference-of-medians test to measure significance (noted on graphs with green/red arrows). Over this period, nearly 70% of our posts had at least one audience target applied to the post and a typical post with audience targeting had 5 different preferred audience tags applied.
First we looked at reach (the number of people who saw the post) and impressions (the total number of times a post was seen). Based on things Facebook has said publicly about their own internal testing, we didn’t expect these posts to reach more (or fewer) people. Our analysis, however, found that significantly more people were reached (and more impressions were received) for posts that had at least one preferred audience specified. We also looked at posts with more preferred audiences tags and found some significant differences there as well.
Looking at click-through rate we found that posts with at least one audience target performed worse than posts with no audience targets. This was particularly true for posts that had four to six preferred audiences.
Despite the lower click-through rate for posts with preferred audiences, we found that the increased reach and impressions these post received more than made up for the shortfall in our click-through rate. This is to say, a smaller percentage of the people seeing a post may have been clicking on it but, in aggregate, our total number of click throughs to NPR.org from Facebook was larger due to the increased reach and impressions. This was true for posts with relatively few targets (1-3) and for posts with many preferred audience tags applied (7 or more).
During our analysis we also found that posts with preferred audiences tend to receive fewer shares and likes per impression. We didn’t find a measurable difference in comments per impression.
These results are the opposite of what we expected to see given that the intent wasn’t for these posts to reach more people, just the right people who would be more likely to interact with our content (by clicking through to the article or liking/sharing/commenting).
None of this means we’re using these preferred audiences optimally. Our editors tend to rack their brains trying to think of the right audiences to add for each post. But it’s possible we’re just not always making the right choices when choosing audience tags. It’s possible that we’re sometimes too broad or too narrow in our preferred audience selections. We haven’t done a thorough-enough review to rule those things out. It’s also true that we may be picking the right preferred audience tags for most posts but not all. We already know that picking preferred audience tags for politics content is a lot easier than trying to pick one for a more obscure topic.
For more tips on preferred audiences, feel free to check out this guidance from Facebook.
EDITOR’S VIEW: Our engagement editors asked for this analysis because they had a hunch that preferred audience tags were hurting our reach. Based on Dan’s findings, however, we can see that they are a net benefit. Our next move will be to consistently apply 1-3 audience tags to all posts. We’re focusing on 1-3 tags per post because Dan’s analysis suggests this is the sweet spot in terms of click-through rate, reach and impressions. We plan to follow Facebook’s suggested spread of broad tags followed by less-broad tags followed by specific tags. An example might be: politics --> Republicans --> Lindsey Graham. After we standardize our tagging around this approach, we will circle back for another analysis over another 3-month period.
-- Wright Bryan