How Users Perceive Native Ads and Why
In online services, native advertising has become a very popular form of online advertising, where the ads served reproduce the look and feel of the platform on which they appear. Examples of native ads include suggested posts on Facebook, promoted tweets on Twitter, and sponsored content on the Yahoo news stream. Above, we show an example of a native ad (the second item with the “dollar” sign) in a news stream on a mobile device.
Promoting relevant and quality ads to users is crucial to maximize long-term user engagement with the platform. In particular, low-quality advertising has been shown to have a detrimental effect on long-term user engagement. Low-quality advertising can have even more severe consequences in the context of native advertising, since native advertisements form an integrated part of the user experience of the product. For example, a bad post-click experience (quantified by short dwell time on the ad landing page) can result in weaker long-term engagement (e.g. fewer clicks).
In our research paper, “Predicting Pre-click Quality for Native Advertisements,” to be published in the proceedings of the upcoming 25th International World Wide Web Conference (WWW 2016), we focus on one’s pre-click experience, which is concerned with the user experience induced by the ad creative before the user decides—or decides not—to click. The ad creative is the ad impression shown within the stream and includes text, visuals, and layout. Due to the low variability in terms of ad formats in native advertising, the content and the presentation of the ad creative are extremely important to determine the quality of the ad.
Our first step was to understand ad quality from a user’s perspective and infer the underlying criteria that users assess when choosing between ads. To this end, we designed a crowd-sourcing study to spot what drives users’ quality preferences in the native advertising domain.
We extracted a sample of ads impressed on the Yahoo mobile news stream. To ensure diversity and the representativeness of our data in terms of subjects and quality ranges, we uniformly sampled a subset of those ads from (1) different click-through rate quantiles; and (2) five different popular topical categories: “travel,” “automotive,” “personal finance,” “education,” and “beauty and personal care.”
We used Amazon Mechanical Turk to conduct our study. We showed users pairs of native ads and asked them to indicate which ad they prefer, and the underlying reasons for each choice. To eliminate the effect of ad relevance, we presented the users with topically-coherent ads (e.g., ads from the same subject category, such as “beauty”), assuming that, for example, when users are comparing two beauty ads, the preference would depend mostly on the ad quality.
Once users chose their preferred ad, we asked them the reasons for each selection. To define such options, we resorted to existing user experience/perception research literature. We were inspired by the UES (User Engagement Scale) framework, an evaluation scale for user experience capturing a wide range of hedonic and cognitive aspects of perception, such as aesthetic appeal, novelty, involvement, focused attention, perceived usability, and endurability. Moreover, previous studies in the context of native advertising investigated user perceptions of native ads with dimensions such as “annoying,” “design,” “trust,” and “familiar brand.” Similarly, researchers have studied the amount of ad “annoyingness” in the context of display advertising, showing that users tend to relate ad annoyance with factors such as advertiser reputation, ad aesthetic composition, and ad logic. Based on these, we provided users with the following options as underlying reasons for their choices:
the product/service offered
the clarity of the description
Users were asked to rate each on a five-grade scale: 1 (strongly disagree), to 5 (strongly agree), or NA (not available).
In the following table we report the percentage of judgements that, for each factor, is assigned to grades 4 or 5 (the user highly agrees this factor affects his or her ad preference choice).
The most important factors are, in order of importance:
Aesthetic appeal > Product, Brand, Trustworthiness > Clarity > Layout
where “>” represents a significant increase (in ad preferences). Further tests showed that, apart from the brand factor, there were not any significant differences. This suggests that the factors affecting user preferences generalize across ad categories.
However, for different ad categories, compared to the general pattern, we observe few small differences. Aesthetic appeal is more important for the Automotive, Beauty, and Education verticals than Personal Finance and Travel. For the Travel category, where most ad images were indeed beautiful, aesthetics did not have such a strong effect compared to others. For the Beauty and Education categories, the product advertised was the most important factor (other than aesthetic appeal) affecting user choices; for Automotive, the brand was crucial. For the Personal Finance category, the clarity of the description had a big impact on the user perception of the quality of the ad.
This study produced important insights into how users perceive the quality of native ads. As a result, we designed a large set of features that characterize various aspects of pre-click native ad quality, which we then used to build a prediction pre-click quality model, reaching an AUC of 0.77. We deployed a model based on a subset of the features on Yahoo news streams, which reduced the ad offensive feedback rate by 17.6% on mobile and 8.7% on desktop.
This work was done in collaboration with Yahoo research scientists Ke (Adam) Zhou, Miriam Redi, and Andy Haines. The post originally appeared on the author’s blog, From the Lab to the Market.
