Adaptive Generation in Dialogue Systems using Dynamic User modeling

Abstract

We address the problem of dynamically modeling and adapting to an unknown user in resource-scarcedomains in the context of interactive spoken dialogue systems. As an example, we show how a system couldchoose referring expressions to refer to domain entities for users with different levels of domain expertise,whose domain knowledge is initially unknown to the system. We approach this problem using a three stepprocess: collecting data using a Wizard of Oz method, building a user simulation, and learning to modeland adapt to users using Reinforcement Learning techniques in Markov Decision Processes (MDP). We show that by using a small corpus of non-adaptive dialogues it is possible to learn an adaptive usermodeling policy in resource-scarce domains using a sense-predict-adapt approach. Our evaluation resultsshow that the learned user modeling and adaptation strategies performed better in terms of adaptationthan hand-coded baseline policies on both simulated and real users. With real users, the learned policyproduced around a 20% increase in adaptation in comparison to the best performing hand-coded adaptivebaseline. We also show that adaptation to user’s domain knowledge results in improving task success(99.47% for the learned policy vs 84.7% for the hand-coded baseline) and reducing dialogue time of theconversation (11% relative difference). Users reported that it was easier to identify domain objects whenthe system used adaptive referring expressions during the conversations. We believe that this method canbe extended to other levels of adaptation such as content selection and in other domains where adaptingto users’ domain knowledge is useful, such as travel and healthcare.