SummerFest 2005 Plenary Presenter

Title

Professor, Media Technology & Society
Northwestern University

Abstract

Why You Have a Body and What it's Used for

In the past ten years there has been increasing interest in the role played by the body in discourse and dialogue, and today Cognitive Science is coming to recognize the interest of studying nonverbal behavior as an intrinsic part of the study of human language. In parallel, Computational Linguistics has become interested in the challenges inherent in implementing embodied graphical interactive dialogue systems.

In this talk, I describe a series of explorations into the functions of hand gestures, eye gaze, head movement and posture, and discuss the ways in which these studies can shed light on classic problems in discourse, pragmatics and computational linguistics. In particular, I will describe experiments that demonstrate the ways in which iconic gestures offer interlocutors resources for describing the world, and how speakers choose to distribute communicative goals across speech and gesture. I will describe experiments that demonstrate the ways in which eye gaze and head nods are used to signal whether contributions to the discourse are grounded. And I will describe experiments that demonstrate the ways in which posture shifts signal new topics in a discourse.

Along with the results of each experiment, I will show an implemented embodied dialogue agent that generates natural language paired appropriately with the nonverbal behavior in question, and that can participate in natural conversations with humans in a variety of domains including direction-giving, real estate, and storytelling. I will show how the nonverbal behaviors of these virtual humans can serve to achieve a range of context-management functions with users: establishing presence and shared attention to a conversation; coordinating speaking turns; presenting and grounding individual contributions to conversation; handling multiple conversations and other concurrent events; and achieving coordinated goals of information exchange, decision-making, and action.

Biography

Justine Cassell is a full professor in the departments of Communication Studies and Computer Science at Northwestern University, the director of the ArticuLab research group, and the graduate director of the interdisciplinary Technology and Social Behavior Ph.D. program. Before coming to Northwestern, Cassell was a tenured associate professor at the MIT Media Lab where she directed the Gesture and Narrative Language Research Group. In 2001, Cassell was awarded the Edgerton Faculty Achievement Award at MIT.

Cassell holds undergraduate degrees in Comparative Literature from Dartmouth and in Lettres Modernes from the Universite de Besançon (France). She holds a M.Phil in Linguistics from the University of Edinburgh (Scotland) and a double Ph.D. from the University of Chicago in Linguistics and Psychology. After having spent ten years studying verbal and non-verbal aspects of human communication through microanalysis of videotaped data she began to bring her knowledge of human conversation to the design of computational systems.

Cassell builds systems that look like humans and that have some of the same kinds of social and communicative competencies that we do. Anthropomorphic systems such as these have generated considerable controversy -- some go so far as to imagine nightmare scenarios in which sentient virtual humans take over thinking from humans. Cassell, however, believes that these systems can lower the bar to computer use, allowing access to computers to those who can't type or read, for example. Increasingly Cassell is turning her attention to the role that technologies such as these may play in children's lives. Interactive technologies such as a virtual storytelling peer have the potential to encourage children in creative, empowered and independent learning. They can also demonstrate ways for interactive technology to live away from the desktop, supporting children's full-bodied, collaborative, social play-based learning.

Website

http://www.soc.northwestern.edu/justine/