Abstract: The use of reinforcement learning is challenging for physical systems because exploration is costly in terms of time and wear of equipment, as well as possibly being unsafe. For many mechanical systems, decent simulation models can be built and used instead of the physical system. Thus, the learning is performed in simulation. Even so, realistic simulation may be costly and thus sample efficiency is important. Moreover, it is not uncommon that simulation models lack realism.

In this talk, I will talk about some of the machine learning challenges associated with the use of reinforcement learning and simulation to build robust policies for physical systems. In particular, I will talk about regression models as surrogates for costly simulations. Moreover, I’ll emphasize the value of good initial policies especially in sparse reward settings and how a single human demonstration can be used as a starting point for incrementally learning a generalizable policy. I will also briefly address the reality gap problem. In particular, I will talk about how domain randomization can be used to build robustness towards known uncertainties in simulation parameters.

Bio: Ville Kyrki is an Associate Professor at Aalto University School of Electrical Engineering, where he leads the Intelligent Robotics group. The group develops intelligent robotic systems and robotic vision with a particular emphasis on methods and systems that cope with imperfect knowledge and uncertain senses.

Speaker: Ville Kyrki

Affiliation: Professor of Electrical Engineering and Automation, Aalto University

Place of Seminar: Seminar Room T6, Konemiehentie 2, Aalto University