Speech-Based Interactive Games for Language Learning: Reading, Translation, and Question-Answering

This paper concerns a framework for building interactive speech-based language learning games. The core of the framework, the “dialogue manager,” controls the game procedure via a control script. The control script allows the developers to have easy access to the natural language process capabilities provided by six core building blocks. Using the framework, three games for Mandarin learning were implemented: a reading game, a translation game, and a question-answering game. We verified the effectiveness and usefulness of the framework by evaluating the three games. In the in-lab and public evaluation phases, we collected a total of 4025 utterances from 31 subjects. The evaluation showed that the game systems responded to the users’ utterances appropriately about 89% of the time, and assessment of the users’ performances correlated well with their human-judged proficiency.