CAIDA AI-Technical Seminar - The Merits of Models in Continuous Reinforcement Learning, Benjamin Recht (University of California, Berkeley)

Title:The Merits of Models in Continuous Reinforcement Learning  

Abstract:  Classical control theory and machine learnin g have similar goals: acquire data about the environment, perform a predi ction, and use that prediction to positively impact the world. However, the approaches they use are frequently at odds. Controls is the theory of designing complex actions from well-specified models, while machine learn ing makes intricate, model-free predictions from data alone. For contempo rary autonomous systems, some sort of hybrid may be essential in order tofuse and process the vast amounts of sensor data recorded into timely, a gile, and safe decisions. In this talk, I will examine the relative meri ts of model-based and model-free methods in data-driven control problems. I will discuss quantitative estimates on the number of measurements requir ed to achieve a high quality control performance and statistical technique s that can distinguish the relative power of different methods. In particu lar, I will show how model-free methods are considerably less sample effi cient than their model-based counterparts. I will also describe how notion s of robustness, safety, constraint satisfaction, and exploration can be transparently incorporated in model-based methods. I will conclude with a discussion of possible positive roles for model-free methods in contempo rary autonomous systems that may mitigate their high sample complexity andlack of reliability and versatility. 

Bio:  Benjamin Recht is an Associate Professor in the Department of Electrical Engineering an d Computer Sciences at the University of California, Berkeley. Ben's rese arch group studies the theory and practice of optimization algorithms witha particular focus on applications in machine learning and control. Ben i s the recipient of a Presidential Early Career Award for Scientists and En gineers, an Alfred P. Sloan Research Fellowship, the 2012 SIAM/MOS Lagra nge Prize in Continuous Optimization, the 2014 Jamon Prize, the 2015 Wil liam O. Baker Award for Initiatives in Research, and the 2017 NIPS Test o f Time Award.

Find more undergrad events on our internal portal at https://my.cs.ubc.ca.

This event's address: https://my.cs.ubc.ca/event/2019/05/caida-ai-technical-seminar- merits-models