Reinforcement Learning for Adaptive Dialogue Systems
Home

Reinforcement Learning for Adaptive Dialogue Systems

Presenters: Oliver Lemon, Verena Rieser

This tutorial will provide an overview to the rapidly growing field of Reinforcement Learning for automatic dialogue system development.

Designing a spoken dialogue system can be a time-consuming and challenging process. One of the main problems is to design the dialogue management strategy -- what the system should say next, in a particular context. The standard approach to this is to hand-craft a finite state machine or a rule-based strategy, but this method is not data-driven, and has no measurable performance guarantees.

To facilitate dialogue strategy development, recent research investigates the use of Reinforcement Learning (RL) methods applied to automatic dialogue strategy optimisation from corpora. In this course we introduce the basics of RL and how to (practically) apply it to adaptive dialogue system development. We describe each step of the development cycle -- from data collection to reward modelling to training and evaluation/user testing. We also demonstrate tools for RL and discuss practical issues, such as corpus requirements, to get people started.

In particular, this tutorial will cover:

  • A brief introduction to system development and the use of RL.
  • An outline some of the existing methods and challenges, discussing current work in the field.
  • Simulation-based training methods, such as simulated users and reward modelling.
  • Practical examples from dialogue strategy learning and Natural Language Generation from the CLASSiC project (Computational Learning in Adaptive Systems for Spoken Conversation, http://www.classic-project.org/ funded under EC FP7, Call 1).
  • Tools