Natural language research announcements

["Alexandre Klementiev" <klementi AT uiuc.edu>]

Dear students,

We will have a student meeting with Prof. Schuler 10-11am tomorrow (Thursday) in 3102 SC.

See you there,
Alex.

On Sun, Sep 28, 2008 at 7:18 PM, Alexandre Klementiev <klementi AT uiuc.edu> wrote:

Dear faculty and students,

William Schuler will give a talk at the AIIS seminar this Thursday (details below). If you would like to meet with Prof. Schuler personally, please let Mark Faust (mfaust AT cs.uiuc.edu) know.

Hope to see you there,
Alex.

Title: Speech Understanding as Sequence Estimation
Speaker: William Schuler, University of Minnesota

Date: October 2, 4:00pm
Location: Siebel 3405 

Abstract:

Spoken language interfaces for applications like home organizers, reminder systems, or immersive design may need to allow users to create new entities with names not found in existing training corpora, reducing the effectiveness of conventional techniques for estimating probabilities of hypothesized words.  In such cases an `interactive' language model can be used to condition probabilities of successive words on the possible meanings of those words in the current application environment.  These interpretations can be defined as vectors, corresponding to distributions over head words or sets of denoted individuals in a world model; then composed with relations, defined as matrices.  Unfortunately, semantic composition is typically understood to conform to syntactic phrase structure, and set intersection or matrix multiplication are generally too expensive to be practical in conventional cubic-time chart parsers, used to hypothesize this phrase structure.

Psycholinguistic studies suggest an interactive model of human language processing that works somewhat differently: First, it seems to perform *incremental* interpretation of spoken utterances, identifying referents of words in an utterance even while these words are still being pronounced.  Second, it seems to preserve ambiguity by maintaining competing interpretations in parallel.  And third, it seems to operate within a severely constrained short-term memory store -- possibly constrained to as few as three or four distinct elements -- which limits the complexity of interactive recognition in a natural and computationally tractable way.

In this talk I will describe how these insights have been applied to the problem of real-time interactive speech interpretation, by modeling joint distributions over referents in an explicit three- or four-element memory store, using a factored HMM-like sequence model. First, I will present evidence that even a three-element model can obtain reasonably accurate syntactic recognition and nearly complete coverage on the large syntactically-annotated Penn Treebank corpus, using a simple reversible tree transform applied during training.  I will then describe how this model can be applied directly to recognition of speech repairs (spontaneous edits by a speaker involving repeated words and corrections) without introducing any additional machinery.  Then I will show how this framework can be extended to perform incremental interpretation by introducing a variable over referents at each memory element, with an evaluation in an implemented real-time speech interface.  The talk will conclude with a description of an implementation of this model that supports references to arbitrary sets of individuals as well as individuals themselves, requiring very large or even unbounded random variable domains.

Two articles (the former under review, the latter in press) describing the syntax and semantics of this model are available on my web site:

 - http://www-users.cs.umn.edu/~schuler/paper-jcl08wsj.pdf
 - http://www-users.cs.umn.edu/~schuler/paper-jcl07slush.pdf

Bio:

William Schuler graduated from the University of Pennsylvania in 2003 with a PhD in Computer and Information Science, and is now an Assistant Professor of Computer Science at the University of Minnesota.  He has been studying psycholinguistically-motivated models of spoken language understanding for nearly ten years, funded since 2005 by a National Science Foundation CAREER grant.  In 2006 his research program was awarded a Presidential Early Career Award for Scientists and Engineers (PECASE) in a ceremony at the White House.  In 2007 he was awarded a McKnight Land-Grant Professorship by the University of Minnesota, consisting of a one-year research leave and funds to promote his research.