Targeted Marketing with Collaborative Filtering
Ravi Mody and Joseph Chisari
The data science team at iHeartRadio has been developing collaborative filtering models to drive a wide variety of features, including recommendations and radio personalization. Collaborative filtering is a popular approach in recommendation systems that makes predictive suggestions to users based on the behavior of other users in a service. For example, it’s often used to recommend new artists to users based on what they and similar users are listening to. In this blog post we discuss our initial experiences applying these models to increase user engagement through targeted marketing campaigns.
One of the most successful approaches to collaborative filtering in recent years has been matrix factorization, which decomposes all the activity on a service into compact representations of its users and what they interact with, e.g. artists. At iHeartRadio we use matrix factorization models to represent our users and artists with ~100 numbers each (called their latent vectors) instead of processing the trillions of possible interactions between them. This simplifies our user activity by many orders of magnitude, and allows us to quickly match artists to users by applying a simple scoring function to the user’s and artist’s latent vectors.
While collaborative filtering has proven to be an effective way of driving personalized features like recommendations, we believe it’s more generally a powerful way to gain an understanding of our users and their musical interests. To further this goal, we’ve begun experimenting with applying our models to marketing initiatives.
Our marketing team regularly conducts push campaigns, which are notifications sent to users’ mobile devices that inform them of an event (an artist’s birthday, album release, etc) and link to the relevant station. We try to target users who would be interested in these news events to increase engagement on our service.
The traditional approach of our marketing team has been to push to users who had previously listened to the artist’s station or a similar genre. For example, for Adele’s album release 25, we pushed a message to users who had started her station recently. This approach targets users who are likely to be interested in the notification, but it has significant limitations. Previous listeners of an artist are often active listeners, minimizing the impact of the push notification. Targeting by genre can get around this, but can be overly broad because it can’t differentiate between distinct subgenres or styles. The traditional approach also does not give control over the size of the campaign: the size of the candidate set cannot be made larger, and reducing it arbitrarily throws out users.
Matrix factorization models can avoid these limitations entirely with increased accuracy. The process is conceptually the reverse of recommending artists to a user: score all users against an artist, then target the top scoring users. This targets users based on a wide variety of listening behavior instead of just the artist and genre. This represents a big opportunity - we’ve found that 60-70% of the top scoring users in a typical campaign never even started the artist’s station, allowing us to find many high-quality users that the traditional approach can’t.
We ran a test where we pushed a message to users who had started David Guetta’s station (the traditional approach), and the same message to his highest scoring users using matrix factorization. During this campaign, the matrix factorization group was 23.7% more likely to open the app than the traditional group. This is a surprising result; app open rate represents user interest in the notification, indicating we were able to find new users who are more interested in David Guetta’s push notification than users who had previously started his station.
We also looked at week-on-week listening lift, which is the ratio of how many hours users listened to the station after the push compared to exactly a week before. This is an important metric that represents additive gain to the station. While both groups listened to David Guetta’s station about the same amount after the push, the matrix factorization users were less likely to be active listeners of his station, so the week-on-week lift % was 3.8x larger than the traditional group’s (68% vs 18%). This plot shows that the additive gain lasted for days after the push notification (noon, November 6th):
We’ve run several more test campaigns with similarly positive results. We’re excited about this approach, and have found it makes it easier to setup and optimize campaigns while offering more accurate targeting. We’re currently planning more research into this approach; current and future potential work include:
Targeting less active users to increase retention
Deeper personalization of pushes
Scoring on tracks, albums, or genres instead of artists
Combining scores for several artists, e.g. to publicize a festival