Abstract: Interactive intelligent systems, such as recommendation systems, are commonly based on sequential machine learners which actively choose their queries (e.g., recommendations) and learn from the responses. How to steer, or teach, such a system towards a desired goal or state? An answer in the form of a computational model of the teacher provides an approach towards modelling active planning behaviour of users in human-computer interaction.

I will talk about our recent approach towards solving the problem through machine teaching. We formulate the sequential teaching problem as a Markov decision process and address the complementary problem of learning from a teacher through probabilistic inverse reinforcement learning. In conventional machine teaching settings, the teacher provides data that are consistent with the true data distribution. However, we find that in our more constrained setting, consistent teachers can be sub-optimal. Simulated experiments and a user study with multi-armed bandit learners demonstrate empirically the benefits of the approach.

Project website is available at https://aaltopml.github.io/machine-teaching-of-active-sequential-learners/.

Speaker: Dr. Tomi Peltola

Affiliation: Curious AI

Place of Seminar: Lecture Hall T6, Konemiehentie 2, Aalto University