Evaluation script for KDD Cup 2012, track 1
The KDD Cup 2012 has recently started, and there are already more than 200 participating teams. This year, the competition is hosted by Kaggle.
Because there were several questions in the forum about the evaluation measure MAP@3, I hereby publish my validation script, hoping that it may be useful.
The script expects two files, one prediction file and one file containing the ground truth (actual clicks). The prediction file is similar to the submission file format - the only exception is that it should contain exactly one recommendation list per user. The ground truth file is a "standard" rating file, just like rec_log_test.txt and rec_log_train.txt.
How to call the script:
./evaluate.pl --prediction-file=pred --groundtruth-file=rec_log_train.last0.1.txt
I created the file rec_log_train.last0.1.txt using the command
tail -n 7320927 rec_log_train.txt > reg_log_train.last0.1.txt
The script is written Perl, which should be installed by default on a typical Linux or Mac OS X machine. You can download it from GitHub: https://github.com/zenogantner/MyMediaLite/blob/master/scripts/kddcup2012/evaluate.pl
Of course, I cannot guarantee that it works correctly. If you have questions or suggestions for improvement, do not hesitate to contact me.