CS980: Advanced Machine Learning

Where

Kingsbury N233

When

MW 3:40pm - 5:00pm

What

In this seminar, we will cover how to make good decisions using machine learning. Reinforcement learning and multi-armed bandits are just some of the methods that combine decision making with machine learning. These methods play a crucial role in countless real problems such as when personalizing websites, navigating robots, managing supply chains and revenue, and even when playing Go!

We will cover how to trade off exploitation and exploration; that is when to act with the current data versus acquiring additional data to make a better decision later. The exploration/exploitation tradeoff is a key challenge in reinforcement learning. We will also cover the related issues of reliability and robustness in machine learning.

Some of the specific topics we will cover are:

• A/B testing and randomized experiments
• Optimizer’s curse
• Thompson sampling, UCB, EXP3
• (Robust) Markov decision processes
• Gittins and Whittle indices
• Tree and policy search
• Covariate shift

The class will start with an overview of fundamental problems, methods, and techniques and will continue with student presentations and a research project.

Pre-requisites

Statistics and some linear algebra. Machine learning is not a pre-requisite. We will cover topics that are closely related to ML but do not depend on it.