Boulder Language Technologies currently has several major research efforts underway. A brief
description of each is provide here. For more complete information regarding
any of the projects, please click on the link provided in its brief description.
Virtual Teachers and Therapists
The field of research concerned with the development of lifelike computer
characters that behave like sensitive and effective teachers, testers,
therapists, trainers and tutors is only ten years old, yet the effectiveness
of systems already developed portend awesome future benefits to individuals
and society. There are literally tens of millions of individuals on this
planet who could benefit from individualized assessments and treatments,
but it impossible to provide even a fraction of this population with the
help they deserve. The vision is to invent virtual teachers and therapists
that are imbued with the clinical knowledge and interpersonal behaviors
of highly effective practitioners. Achieving this vision requires research
breakthrough in spoken dialog, computer vision and character animation
technologies, and the integration of these technologies in computer programs
that are informed by a significant body of research in psychology and
computer science on how to design multimedia programs that optimize clinical
outcomes during interaction with virtual humans. My colleagues and I at BLT are dedicated to activities that will accelerate the realization
of this vision. Success will see a world in which effective treatments
are accessible to all who need them.
Improving Science Learning through Dialogs with Virtual Tutors
BLT has received grants from the Department of Education’s Institute for Education Sciences and the National Science Foundation to improve science learning and achievement by third, fourth and fifth grade students.In July, 2007 the Institute for Education Sciences awarded the University of Colorado (CU) a four year grant (R305B070434; $3,130,043; Wayne Ward, PI; Ron Cole co-PI) for the project entitled “Improving Science Learning through Spoken Dialogs with Virtual Tutors.In May 2008 the IES grant was transferred from CU to Boulder Language Technologies since PI Wayne Ward left CU to work at BLT.Interested individuals are welcome to read the PROPOSAL submitted to the IES Cognition and Student Learning program that resulted in a grant to conduct the proposed work.
In September 2008 Boulder Language Technologies was awarded a five year grant from the NSF EHR Knowledge Discovery K-12 program for the project entitled "Collaborative Research: Improving Science Learning in Inquiry-based Programs" (DRL-0733323; Wayne Ward, PI; Ron Cole co-PI). CU was awarded a collaborative research grant for this project (DRL-0733322; Sarel van Vuuren, PI).Interested individuals are welcome to read the PROPOSAL submitted to the NSF that resulted in a grant to conduct the proposed work.
While the two projects share the same overarching goal—to improve science learning by third, fourth and fifth grade students through spoken dialogs with a virtual tutor – they differ in important ways.Differences include the areas of science selected for tutoring, the organization of dialogs during tutoring to facilitate acquisition of concepts and the ability to reason about concepts in scientific explanations, and whether students interact with the virtual tutor in small groups or individually.
In the NSF project, students will interact with the virtual tutor one-on-one.The tutoring sessions will focus initially on the acquisition of science concepts (conceptual knowledge) using illustrations and animations to anchor the dialogs. Once concepts are acquired, the tutor will ask questions designed to stimulate students to think and reason about these concepts in scientific explanations, thus acquiring and demonstrating procedural knowledge.
In the IES project, students will interact with the tutor in small groups in which they can discuss questions posed by the tutor before one of the students provides an answer.Dialogs in the IES project are more “free form,” with the tutor starting with a general questions like “So, you studied circuits today. What’s that all about?” and then asking open-ended questions (“Tell me more about connections”) based on the systems analysis of what the student does and does not know.In the IES project, illustrations and animations are invoked as needed based on the flow of the dialog to facilitate students’ understanding of both conceptual and procedural knowledge.
In addition to our excellent team of researcher, software developers and educators, all of the tools and technologies developed in each project and the data collected and transcribed during tutoring sessions are shared between projects. Development uses a software environment called the Virtual Human Toolkit developed by Wayne Ward and Ron Cole while at CSLR with supported from an NSF CRI grant.
During the development phase of the IES and NSF projects, our project tutors are tutoring children two times per week on average for 15-20 minutes following classroom science investigations. Dialogs in both projects are based on a principled approach to dialog interaction called Questioning the Author (QtA), developed by Isabel Beck and Margaret Mckeown. The research team has been working with Dr. Mckeown to adapt this approach, developed originally for managing classroom dialogs about books as they are being read in class, to science tutoring with individual students and students in small groups. We are designing QtA-based dialogs that use open-ended questions to elicit explanations from students which our project tutors (and eventually the dialog systems) can analyze to figure out what the student does and doesn’t know about the facts and concepts encountered in classroom science investigations. Based on student’s responses to questions, the tutor asks further questions (that may be accompanied by illustrations or animations) that stimulate students to think and reason out loud to learn specific concepts and apply them to scientific explanations.
Development of dialogs and spoken dialog systems is proceeding through the following stages:
Tutoring children in schools following classroom science investigations while videotaping sessions and recording children’s speech using a high quality noise-cancelling microphone.
Reviewing, analyzing and refining dialogs and developing illustrations and narrations for use in the dialogs,
Tutoring a new set of students with all speech data recorded and transcribed,
Developing an initial set of spoken dialog systems for each science investigation in the FOSS module using transcriptions of the recorded tutoring sessions,
Conducting Wizard of Oz studies in which a new set of students interact with the virtual tutor while a human tutor (the Wizard) monitors the dialog and is able to take control of the virtual tutor when needed to improve the dialog or to present an illustration or an animation,
Improving the spoken dialog system based on data collected during the Wizard of Oz experiments,
Field testing an improved version of the system with minimal intervention by a Wizard,and
Assessing the effectiveness of the dialog system during the final two years of each grant.
Status of Project
As of May 2008, we have developed and refined tutoring dialogs incorporating illustrations and animations related to science investigations in the FOSS Magnetism and Electricity module.We plan to begin testing the spoken dialog system for Magnetism and Electricity in Wizard of Oz during the summer of 2008 for initial testing in schools during the fall of 2008.We have also designed tutoring dialogs for all of the science investigations in the FOSS module Measurement and collected data from 5 classrooms in one school.These dialogs will be refined during a second session of tutoring during the fall of 2008.Additional information about the status of the project can be found in the Year 1 Annual Report of work conducted under the IES grant.
In January 2009, we completed development of a fully independent science tutoring system for the entire FOSS Magnetism and Electricity module. The system is currently undergoing testing by project staff prior to deployment in classrooms in February 2009 for testing with students. While the system supports independent tutoring without human intervention, it will run in Wizard of Oz mode during the next phase of development; that is, remote human tutors will monitor students using the system and override system responses when necessary. Development of tutorial dialogs is also underway for two additional FOSS science modules: Measurement and Variables. Data collected in both face-to-face and Wizard of OZ sessions following science investigations in these modules will be used to develop independent systems for testing in Wizard of Oz mode in fall 2009.
We have also collected survey data from students who have been tutored in both face to face sessions and Wizard of Oz sessions, In the latter, students interact with the virtual tutor on the computer rather than the human tutor. Surveys from these students are posted here:
MyST Journal Publication 06/2010
MyST Preliminary Results
Standardized tests administered to students before and after using the FOSS program revealed statistically significant learning gains by children who used the program relative to students who did not use the program. Our comparison examined pre and post ASK assessments (assessments that cover basic concepts covered in FOSS) for students participating in “Wizard of Oz” (WoZ) FOSS tutoring for Magnetism and Electricity, and students who did not receive WoZ tutoring. The results will be posted here: