Learning to Make Better Decisions: Challenges and Opportunities for the 21st Century - DLS talk by Csaba Szepesvári, U. of A.

Title: Learning to Make Better Decisions: Challenges and Opportunities for the 21st Century

Speaker: Csaba Szepesvári, University of Alberta

Homepage:  https://sites.ualberta.ca/~szepesva/

Host: Cristina Conati, UBC Computer Science

Abstract:

With access to huge-scale distributed systems and more data than ever before, new learning systems break previous records on almost a daily basis in all kind of challenging prediction problems.

Although predicting what a user of a voice portal said on the phone, reading a sign on an image, or predicting whether a pedestrian is on the road in front of a car are important tasks, perhaps an even more important if not critical task is to figure out how to make good decisions based on the information available. In this talk, by means of some examples based on recent work on reinforcement learning, I will illustrate the unique opportunities and challenges that arise when a system must learn to make good decisions.

In particular, I will start by demonstrating that in sequential decision making problems passive data collection inevitably leads to catastrophic data sparsity (no big data is big enough), while if one lets clever algorithms to control data collection then learning can happen at a uniform speed.

I will also describe our current attempts to scale up such clever algorithms to work on large-scale problems.

Amongst the possible approaches, I will discuss the role of sparsity to address this challenge in the practical, yet mathematically elegant setting of  "linear bandits".

Interestingly, while in the related linear prediction problem, sparsity allows one to deal with huge dimensionality in a seamless fashion, the status of this question in the bandit setting is much less understood. Again, I will describe recent advances and some outstanding challenges.

The talk is based on joint work with Yasin Abbasi-Yadkori and David Pal.