Counterfactual Evaluation and Learning from Logged User Feedback
55Interactive systems like search engines, recommender systems, and ad placement platforms are ubiquitous. Evaluating and optimizing these systems is hard - users' feedback governs system performance, and gathering their feedback in repeated randomized experiments is costly. I study how we can use logs collected from deployed systems to perform offline evaluation and learning. I will outline two projects (Evaluation: Recommendations as Treatments, ICML'16 and Learning: Counterfactual Risk Minimization, ICML'15) that advance the state of the art for these problems. I will also briefly reference the tutorial Thorsten Joachims and I taught at SIGIR'16 (http://www.cs.cornell.edu/~adith/CfactSIGIR2016/) that explores these counterfactual evaluation and learning problems in more detail.
See more on this video at https://www.microsoft.com/en-us/research/video/counterfactual-evaluation-learning-logged-user-feedback/