Serving effective personalized product recommendations is critical to providing a pleasant shopping experience for customers at Wayfair. To do this, the Wayfair Data Science team builds state of the art recommender systems that leverage the customer’s previous browsing history to surface products that match their interests. This week in Wayfair Data Science’s explainer series, Senior Data Scientist Cole Zuber describes how we approach evaluating these recommender systems.

Originally from Portland, Oregon, Cole has been a data scientist at Wayfair for 3 years. He currently works on the recommendations team, which attempts to find the perfect product for each of our millions of customers. Outside of Wayfair, Cole enjoys a wide variety of physical activities (hiking and kickboxing to name a few) and is attempting to become a culinary expert on many different cuisines from around the world.

0:00 Intro
0:14 A/B testing
1:22 Evaluating models offline
2:38 Evaluating the relevance of recommendations
2:42 Cumulative gain
4:50 Discounted cumulative gain & normalized discounted
cumulative gain