Research Interests

Physiological signal processing

The affective state of children with Autism is not always expressed through observational cues, a phenomenon which is further confounded by vast variability across individuals on the Autism spectrum. This inherent gap between their observable behavior and their inner affective state is not well understood, and can be potentially bridged by monitoring their physiology. Electrodermal Activity (EDA) is a physiological signal indicative of a person’s arousal and thus affording us new insights into a child’s inner affective state.

We develop methods for robustly analyzing EDA signals and associating them with behavioral indices. One case study is exploring the interplay between EDA and Verbal Response Latencies (VRL) of children with Autism during a conversational paradigm with a computer character and their parents. Results indicate that EDA can provide us with an alternative view into a child's state and complement other traditionally used modalities, such as linguistic information.

Related Publications:

Electrodermal Activity with Verbal Reponse Latencies. 

Narrative structure quantification

Storytelling is a commonly used technique for analyzing people's social and communication skills. Children with Autism are likely to produce less coherent narratives than their typically developing peers and demonstrate poor building of causal events in a story.

We quantify the narrative structure by modeling the frequency and evolution of entities, i.e. the co-referent noun-phrases that represent the main characters, objects and ideas in the story. Our features capture the distribution of entity frequencies using decaying probabilistic distributions and their transitioning with a Markov Chain model. The evolution of entities through the story is represented with step sequencies and their interaction with directed normalized distance measures.

Related Publications:

Related Software:

  • You can download the related MATLAB code from here.

Narrative structure feature extraction. 

VAD system development

VAD is the task of distinguishing speech from non-speech segments, like silence and noise, in an audio signal. Accurate speech boundaries can contribute to many spech processing applications (e.g. language and speaker ID, ASR) by reducing the noise segments and giving more informative data. VAD is one of the items of the DARPA Robust Automatic Transcription Program (RATS) in which SAIL participates.

We developped a system that uses long-term spectral variability features over multiple time and spectral resolutions. The long-term information stems from the fact that speech does not occur in isolation but over large time-intervals. Also speech usually has much higher spectral variability than many kinds of noise.

Related Publications: