Creating a Movie Recommendation Engine
Recommendation Engines are not something new, which were introduced lately, nor something old, which should be ruled out as out of fashion. However, we could not find a more fitting time than this for them, as there is almost nothing for which recommendation engines cannot be applied to.
To prove my point, lets look at several areas where recommendation engines are being used.
Movie recommendations (Netflix)
Product recommendations (amazon)
Book recommendations (amazon, Book Bub ,BarnesandNoble.com)
Music recommendation (tastekid.com)
Restaurant recommendations (Hunch, LikeMe, Goodrec)
recommendations for anything (TipFlare)
As you can see, application of recommendation engines can be found almost everywhere.
Now that it is clear how relevant this topic is given the current context, this article will look at how to build a movie recommendation engine using basic algorithms and a large set of movie related (user ratings) data.
Let's look at what are needed to get this under way.
Movie related data set with user ratings for each movie. (This can be obtained easily from movielens at grouplens.)
Above data set should be stored in a database to make meaningful connections between users, with respect to movies.
Suitable algorithm to help recommend suitable movies for a user. (This is where you will have lots of options and I will only be guiding you on how to use a particular algorihtm )
Suitable UI to present the recommendation system to the user.
Since setting up the database with a movielens dataset is not too much of a hassle, let's see what algorithm can be used for this system.
Item Based Collaborative Filtering
This is a model-based algorithm for making recommendations. In the algorithm, the similarities between different items (in this cases, movies) in the dataset are calculated by using one of a number of similarity measures, and then these similarity values are used to predict ratings for user-item pairs not present in the dataset.
The similarity values between movies are measured by observing all the users who have rated both the movies which are under consideration. Similarity of these movies depend on the ratings given to the movies by users who have rated both of them. (dataset obtained from movielens include these user ratings where users have rated movies on a scale from 1 to 5)
For an example, ratings may look like this, except for movies not colors.
In order to calculate similarities, following computations can be used. (Keep in mind that there are several possibilities for calculating similarities. they can be found here)
This formulation views two items and their ratings as vectors, and defines the similarity between them as the angle between these vectors. For an example, consider the rating vector for Orange from the above example. It would be [2 - 4 2 - 5] and for yellow [- 2 - 3 2 -].
Similarity would be calculated as above where i and j would represent the rating vectors of the 2 movies which are under consideration.
Adjusted Cosine Similarity
This is a modified form of cosine based similarity where we take into the fact that different users have different ratings schemes; in other words, some users might rate items highly in general, and others might give items lower ratings as a preference. To remove this drawback from cosine based similarity, we subtract average ratings for each user from each user's rating for the pair of items in question
Ru represents the average rating of a given user.
Now that we have identified a method to calculate the similarity between 2 movies, let's identified the required steps to make it all the way to the end as a sequence of operations.
Step 1
Design a UI (or a simple command line application) where the user will be prompted to select his/her favorite movie from a given set of movies. You should go through the movies available in the dataset provided by movielens and display (or list) them randomly, so that the user can select a movie from the list which he/she liked. (I used a cool trick where I can grab the movie poster from the web through a REST API provided by IMDB. It is more appealing than a simple command line interface where only the movie names will be listed).
Step 2
Once the user selects his/her favorite movie (lets call it 'X'), you must implement a method to identify other users (from the dataset available) who have rated X 4 out of 5 or above.(this value selection is totally upto you, You can put any limit you want.4 is taken in order to reduce the number of eligible users)
Step 3
Once you have identified the above mentioned set of users, you have to identify the set of movies that these users have watched (excluding 'X') and also rated 4 or above. By doing so, you will be listing the movies that are liked by users who also liked the movie 'X'.
Step 4
Now we have to calculate the similarity between each movie from the above mentioned set and movie X. For that, cosine based method or the adjusted cosine method can be used.
Step 5
Once the similarities are being calculated, the list should be re-ordered in the descending order based on the similarity level. (when using the cosine or adjusted cosine method, values will range between 0 and 1, 1 been similar and 0 been no similarity at all).
Step 6
You can select the top 3 (or any number from the list) and present as the recommendation for the user.
You can even take this a step further by considering whether the user has already watched the recommended movies. If he/she has already watched a recommended movie, you can iterate through the ordered movie list.
Further improvement can be made based on the genre (Action, Drama, Thriller etc) of a movie. When a user marks several movies as 'already watched' , we can take into consideration the genre of each movie that he/she has already watched. From that, we can come to a conclusion about which type of movies the user like most. Based on that, another filtering can be done for the sorted movie list based on the genre of the movie (genre of each movie is available in the movielens dataset).
Note: Dataset available at movielens consist somewhat old movies and the most recent movie would be around 1998. I was not able to find a datasets with new movies at the time this article was written.
Hope you would find this article helpful and you can view the source and get an idea on how to implement the methods mentioned above.
Cheers...!!!
