Peter Khooshabeh, Sudeep Gandhe, Cade McCall, Jonathan Gratch, James Blascovich and David Traum. Proceedings of 11th International Conference on Intelligent Virtual Agents, IVA 2011 Reykjavik, Iceland September 2011 Khooshabeh_et_al_IVA2011_humor.pdf We study whether a virtual agent that delivers humor through verbal behavior can affect an individual's proxemic behavior towards the agent. Participants interacted with a virtual agent through natural language and, in a separate task, performed an embodied interpersonal interaction task in a virtual environment. The study used minimum distance as the dependent measure. Humor generated by the virtual agent through a text chat did not have any significant effects on the proxemic task. This is likely due to the experimental constraint of only allowing participants to interact with a disembodied agent through a textual chat dialogue. Sudeep Gandhe, Michael Rushforth, Priti Aggarwal and David Traum. Proceedings of Interspeech-11 Florence, Italy August 2011 p84046.pdf We present the evaluation of an integrated authoring tool for rapid prototyping of dialogue systems. These dialogue systems are designed to support virtual humans engaging in advanced question-answering dialogues, such as for training tactical questioning skills. The tool was designed to help non- experts, who may have little or no knowledge of linguistics or computer science, build virtual characters that can play the role of an interviewee. The tool has been successfully used by several different non-experts to create a number of virtual characters used successfully for both training and human subjects testing. We report on experiences with seven such characters, whose development time was as little as two weeks including concept development and a round of user testing. Ron Artstein, Michael Rushforth, Sudeep Gandhe, David Traum and Aram Donigian. Proceedings of 7th IJCAI workshop on Knowledge and Reasoning in Practical Dialogue Systems Barcelona, Catalonia (Spain) July 2011 tacq-limits.pdf A set of dialogue acts, generated automatically by applying a dialogue act scheme to a domain representation designed for easy scenario authoring, covers approximately 72%-76% of user utterances spoken in live interaction with a tactical questioning simulation trainer. The domain is represented as facts of the form <object, attribute, value> and conversational actions of the form <character, action>. User utterances from the corpus that fall outside the scope of the scheme include questions about temporal relations, relations between facts and relations between objects, questions about reason and evidence, assertions by the user, conditional offers, attempts to set the topic of conversation, and compound utterances. These utterance types constitute the limits of the simple dialogue act scheme. Sudeep Gandhe, Alysa Taylor, Jillian Gerten and David Traum. Demo to be presented at SIGdial 2011 Portland, Oregon June 2011 tacq-demo.pdf We demonstrate a dialogue system and the accompanying authoring tools that are designed to allow authors with little or no experience in building dialogue systems to rapidly build advanced question-answering characters. To date seven such virtual characters have been built by non-experts using this architecture and tools. Here we demonstrate one such character, PFC Sean Avery, which was developed by a non-expert in 3 months. Peter Khooshabeh, Cade McCall, Sudeep Gandhe, Jonathan Gratch and James Blascovich. Proceedings of the 2011 annual conference extended abstracts on Human factors in computing systems, CHI EA '11 Vancouver, BC 2011 Khooshabeh_et_al_2011_humor.pdf “We need oxygen, especially if someone farts!” The goal here was to determine whether computer interfaces are capable of social influence via humor. Users interacted with a natural language capable virtual agent that told persuasive information, and they were given the option to use information from the dialogue in order to complete a problem-solving task. Individuals interacting with an ostensibly humorous virtual agent were influenced by it such that those who judged the agent unfunny were less likely to be persuaded and departed from the agent‟s suggestions. We discuss the implications of these results for HCI involving natural language systems and virtual agents. Sudeep Gandhe and David Traum. Proceedings of the SIGdial 2010 Conference Tokyo, Japan 2010 http://www.sigdial.org/workshops/workshop11/proc/pdf/SIGDIAL45.pdf We perform a study of existing dialogue corpora to establish the theoretical maximum performance of the selection approach to simulating human dialogue behavior in unseen dialogues. This maximum is the proportion of test utterances for which an exact or approximate match exists in the corresponding training corpus. The results indicate that some domains seem quite suitable for a corpus based selection approach, with over half of the test utterances having been seen before in the corpus, while other domains show much more novelty compared to previous dialogues. Michael Rushforth, Sudeep Gandhe, Ron Artstein , Antonio Roque, Sarrah Ali, Nicolle Whitman and David Traum. Intelligent Virtual Humans Conference (IVA-09), Lecture Notes in Artificial Intelligence Vol. 5733 Amsterdam 2009 tacq-personality-abstract.pdf Michael Rushforth, Sudeep Gandhe, Ron Artstein , Antonio Roque, Sarrah Ali, Nicolle Whitman and David Traum. USC-ICT Technical Report, ICT-TR-03-2009 September 2009 http://ict.usc.edu/files/publications/ICT-TR-03-2009_tacq-personality.pdf We extend a virtual human architecture that has been used to build tactical questioning characters with a parameterizable personality model, allowing characters to be designed with different personalities, allowing a richer set of possible user interactions in a training environment. Two experiments were carried out to evaluate the framework. In the first, it was determined that personality models do have an impact on user perception of several aspects of the personality of the character. In the second, a model of assertiveness was evaluated and found to have a small but significant impact on the users who interacted with the full virtual human, and larger differences in judgment of annotators who examined only the verbal transcripts of the interaction. Ron Artstein, Sudeep Gandhe, Jillian Gerten, Anton Leuski and David Traum. In Languages: From Formal to Natural. Essays Dedicated to Nissim Francez on the Occasion of His 65th Birthday (Lecture Notes in Computer Science 5533) Orna Grumberg, Michael Kaminski, Shmuel Katz and Shuly Wintner 22-35 Springer

Heidelberg

2009 Artstein_Semi-formal_Evaluation_of_Conversation_Characters.pdf Conversational dialogue systems cannot be evaluated in a fully formal manner, because dialogue is heavily dependent on context and current dialogue theory is not precise enough to specify a target output ahead of time. Instead, we evaluate dialogue systems in a semi-formal manner, using human judges to rate the coherence of a conversational character and correlating these judgments with measures extracted from within the system. We present a series of three evaluations of a single conversational character over the course of a year, demonstrating how this kind of evaluation helps bring about an improvement in overall dialogue coherence. Sudeep Gandhe, Nicolle Whitman, David Traum and Ron Artstein. 6th Workshop on Knowledge and Reasoning in Practical dialogue Systems Pasadena, California July 2009 paper_10.pdf We present an integrated authoring tool for rapid prototyping of dialogue systems for virtual humans taking part in tactical questioning simulations. The tool helps domain experts, who may have little or no knowledge of linguistics or computer science, to build virtual characters that can play the role of the interviewee. Working in a top-down fashion, the authoring process begins with specifying a domain of knowledge for the character; the authoring tool generates all relevant dialogue acts and allows authors to assign the language that will be used to refer to the domain elements. The authoring tool can also be used to manipulate some aspects of the dialogue strategies employed by the virtual characters, and it also supports re-using some of the authored content across different characters. Ron Artstein, Sudeep Gandhe, Michael Rushforth and David Traum. DiaHolmia 13th Workshop on the Semantics and Pragmatics of Dialogue. Stockholm, Sweden June 2009 tacq-semdial09.pdf User utterances in a spoken dialogue system for tactical questioning simulation were matched to a set of dialogue acts generated automatically from a representation of facts as <object, attribute, value> triples and actions as <character, action> pairs. The representation currently covers about 50% of user utterances, and we show that a few extensions can increase coverage to 80% or more. This demonstrates the viability of simple schemes for representing question-answering dialogues in implemented systems. Ron Artstein, Jacob Cannon, Sudeep Gandhe, Jillian Gerten, Joseph Henderer, Anton Leuski and David Traum. 26th Army Science Conference Florida 2008 star_asc2008.pdf We demonstrate three classes of off-topic responses which allow a virtual question-answering character to handle cases where it does not understand the user’s input: ask for clarification, indicate misunderstanding, and move on with the conversation. While falling short of full dialogue management, a combination of such responses together with prompts to change the topic can improve overall dialogue coherence. David Traum, Anton Leuksi, Antonio Roque, Sudeep Gandhe, David DeVault, Jillian Gerten, Susan Robinson and Bilyana Martinovski. 26th Army Science Conference Florida 2008 tacq_asc2008.pdf In this paper we contrast three architectures for natural language questioning characters. We contrast the relative costs and benefits of each approach in building characters for tactical questioning. The first architecture works purely at the textual level, using cross-language information retrieval techniques to learn the best output for any input from a training set of linked questions and answers. The second architecture adds a global emotional model and computes a compliance model, which can result in different outputs for different levels, given the same inputs. The third architecture works at a semantic level and allows authoring of different policies for response for different kinds of information. We describe these architectures and their strengths and weaknesses with respect to expressive capacity, performance, and authoring demands. Sudeep Gandhe, David DeVault, Antonio Roque, Bilyana Martinovski, Ron Artstein, Anton Leuski, Jillian Gerten and David Traum. Proceedings of Interspeech-08 Brisbane, Australia 2008 tacq08.pdf We present a new approach for rapidly developing dialogue capabilities for virtual humans. Starting from domain specification, an integrated authoring interface automatically generates dialogue acts with all possible contents. These dialogue acts are linked to example utterances in order to provide training data for natural language understanding and generation. The virtual human dialogue system contains a dialogue manager following the information-state approach, using finite-state machines and SCXML to manage local coherence, as well as explicit modeling of emotions and compliance level and a grounding component based on evidence of understanding. Using the authoring tools, we design and implement a version of the virtual human Hassan and compare to previous architectures for the character. Sudeep Gandhe and David Traum. 9th SIGdial Workshop on Discourse and Dialogue Ohio 2008 sigdial08-evaluation-understudy.pdf Evaluating a dialogue system is seen as a major challenge within the dialogue research community. Due to the very nature of the task, most of the evaluation methods need a substantial amount of human involvement. Following the tradition in machine translation, summarization and discourse coherence modeling, we introduce the the idea of evaluation understudy for dialogue coherence models. Following (Lapata, 2006), we use the information ordering task as a testbed for evaluating dialogue coherence models. This paper reports findings about the reliability of the information ordering task as applied to dialogues. We find that simple n-gram co-occurrence statistics similar in spirit to BLEU (Papineni et al., 2001) correlate very well with human judgments for dialogue coherence. Tim Paek, Sudeep Gandhe and David Maxwell Chickering. 9th SIGdial Workshop on Discourse and Dialogue Ohio May 2008 sigdial08-activeLearning.pdf Grammar-based approaches to spoken language understanding are utilized to a great extent in industry, particularly when developers are confronted with data sparsity. In order to ensure wide grammar coverage, developers typically modify their grammars in an iterative process of deploying the application, collecting and transcribing user utterances, and adjusting the grammar. In this paper, we explore enhancing this iterative process by leveraging active learning with back-off grammars. Because the back-off grammars expand coverage of user utterances, developers have a safety net for deploying applications earlier. Furthermore, the statistics related to the back-off can be used for active learning, thus reducing the effort and cost of data transcription. In experiments conducted on a commercially deployed application, the approach achieved levels of semantic accuracy comparable to transcribing all failed utterances with 87% less transcriptions. Ron Artstein, Sudeep Gandhe, Anton Leuski, and David Traum. Proceedings of ELRA Workshop on Evaluation, LREC Marrakech, Morocco 2008 star_evaluation-web.pdf We tested a life-size embodied question-answering character at a convention where he responded to questions from the audience. The character's responses were then rated for coherence. The ratings, combined with speech transcripts, speech recognition results and the character's responses, allowed us to identify where the character needs to improve, namely in speech recognition and providing off-topic responses. Sudeep Gandhe and David Traum. Proceedings of Interspeech-07 Antwerp, Belgium 2007 p1186.pdf Virtual humans are being used in a number of applications, including simulation-based training, multi-player games, and museum kiosks. Natural language dialogue capabilities are an essential part of their human-like persona. These dialogue systems have a goal of being believable and generally have to operate within the bounds of their restricted domains. Most dialogue systems operate on a dialogue-act level and require extensive annotation efforts. Semantic annotation and rule authoring have long been known as bottlenecks for developing dialogue systems for new domains. In this paper, we investigate several dialogue models for virtual humans that are trained on an unannotated human-human corpus. These are inspired by information retrieval and work on the surface text level. We evaluate these in text-based and spoken interactions and also against the upper baseline of human-human dialogues. Tim Paek, Sudeep Gandhe, David Maxwell Chickering and Yun Cheng Ju. Proceedings of the ACL Workshop on Grammar-Based Approaches to Spoken Language Processing (SPEECHGRAM) Prague, Czech Replublic June 2007 paek_speechgram_07.pdf In command and control (C&C) speech interaction, users interact by speaking commands or asking questions typically specified in a context-free grammar (CFG). Unfortunately, users often produce out-of grammar (OOG) commands, which can result in misunderstanding or nonunderstanding. We explore a simple approach to handling OOG commands that involves generating a backoff grammar from any CFG using filler models, and utilizing that grammar for recognition whenever the CFG fails. Working within the memory footprint requirements of a mobile C&C product, applying the approach yielded a 35% relative reduction in semantic error rate for OOG commands. It also improved partial recognitions for enabling clarification dialogue. Sudeep Gandhe and David Traum. 5th Workshop on knowledge and reasoning in practical dialogue systems Hyderabad, India Jan 2007 gandhe-krpd07.pdf Virtual human characters equipped with natural language dialogue capability have proved useful in many fields like simulation training and interactive games. Generally behind such dialogue managers lies a complex knowledge-rich rule-based system. Building such system involves meticulous annotation of data and hand authoring of rules. In this paper we build a statistical dialogue model from roleplay and wizard of oz dialog corpus with virtually no annotation. We compare these methods with the traditional approaches. We have evaluated these systems for perceived appropriateness of response and the results are presented here. Sudeep Gandhe, Andrew S Gordon and David Traum. 2006 International Conference on Intelligent User Interfaces Sydney, Australia Jan 2006 p369-gandhe.pdf Question-answering dialogue systems have found many applications in interactive learning environments. This paper is concerned with one such application for Army leadership training, where trainees input free-text questions that elicit pre-recorded video responses. Since these responses are already crafted before the question is asked, a certain degree of incoherence exists between the question that is asked and the answer that is given. This paper explores the use of short linking dialogues that stand in between the question and its video response to alleviate the problem of incoherence. We describe a set of experiments with human generated linking dialogues that demonstrate their added value. We then describe our implementation of an automated method for utilizing linking dialogues and show that these have better coherence properties than the original system without linking dialogues. Randall Hill, Julia Kim, Michelle Zbylut, Andrew S Gordon, David Traum, Sudeep Gandhe, Stewart King, Salvo Lavis and Scott Rocher. 25th Army Sceince Conference Orlando, Florida 2006 ASC06.pdf AXL.Net is a prototype web-based immersive technology solution that supports case method teaching for U.S. Army leader development. The AXL.Net system addresses three challenges: (1) designing a pedagogically sound research prototype for leader development, (2) integrating research technologies with the best of Web 2.0 innovations to enhance case method teaching, and (3) providing an easy to use system. Initial evaluations show that the prototype application and framework is effective for leader development. Sudeep Gandhe, Andrew Gordon, Anton Leuski, David R Traum, and Douglas W. Oard. 24th Army Science Conference Orlando, Florida 2004 asc2004.pdf Pre-recorded video segments can be very compelling for a variety of immersive training purposes, including providing answers to questions in after-action reviews. Answering questions fluently using pre-recorded video poses challenges, however. When humans interact, answers are constructed after questions are posed. When answers are pre-recorded, even if a correct answer exists in a library of video segments, the answer may be phrased in a way that is not coherent with the question. This paper reports on basic research experiments with short “linking dialogues” that mediate between the question and answer to reduce (or eliminate) the incoherence, resulting in more natural human-system interaction. A set of experiments were performed in which links were elicited to bridge between questions from users of an existing training application and selected answers from the system, and then comparisons made with unlinked answers. The results show that a linking dialogue can significantly increase the perceived relevance of the system's answers. Robert Belvin, Emil Ettelaie, Sudeep Gandhe, Panayiotis Georgiou, Kevin Knight, Daniel Marcu, Scott Millward, Shrikanth Narayanan, Howard Neely and David Traum Demo presented at ACL Interactive Poster and Demonstration Sessions June 2005 p89-ettelaie.pdf We briefly describe a two-way speech-to-speech English-Farsi translation system prototype developed for use in doctor-patient interactions. The overarching philosophy of the developers has been to create a system that enables effective communication, rather than focusing on maximizing component-level performance. The discussion focuses on the general approach and evaluation of the system by an independent government evaluation team. S. Narayanan, S. Ananthakrishnan, R. Belvin, E. Ettelaie, S. Gandhe, S. Ganjavi, P. G. Georgiou, C. M. Hein, S. Kadambe, K. Knight, D. Marcu, H. E. Neely, N. Srinivasamurthy, D. Traum and D. Wang. In working notes of the AAAI Fall symposium on Dialogue Systems for Health Communication 2004 97-103 Transonics-AAAI-04.pdf In this paper we describe our spoken english-persian medical dialogue translation system. We describe the data collection effort and give an overview of the component technologies, including speech recognition, translation, dialogue management, and user interface design. The individual modules and system are designed for flexibility, and to be able to leverage different amounts of available resources to maximize the ability for communication between medical care-giver and patient. David Traum, William Swartout, Jonathan Gratch, Stacy Marsella, Patrick Kenny, Eduard Hovy, Shri Narayanan, Ed Fast, Bilyana Martinovski, Rahul Baghat, Susan Robinson, Andrew Marshall, Dagen Wang, Sudeep Gandhe and Anton Leuski Demo presented at SIGdial-05 September 2005 SIGdial-SASO-demo.pdf We present a virtual human doctor who can engage in multi-modal negotiation dialogue with people from other organizations. The doctor is part of the SASO-ST system, used for training for non-team interactions. Paul Scerri, David Pynadath, Nathan Schurr, Alessandro Farinelli, Sudeep Gandhe and Milind Tambe. Proceedings of the workshop on Programming multiagent systems, Lecture notes in computer science M.Dastani, J.Dix 2004 scerri_et_al_04.pdf Coordination between large teams of highly heterogeneous entities will change the way complex goals are pursued in real world environments. One approach to achieving the required coordination in such teams is to give each team member a proxy that assumes routine coordination activities on behalf of its team member. Despite that approach's success, as we attempt to apply this first generation of proxy architecture to larger teams in more challenging environments, some limitations become clear. In this paper, we present initial efforts on the next generation of proxy architecture and Team Oriented Programming (TOP), called Machinetta. Machinetta aims to overcome the limitations of the previous generation of proxies and allow effective coordination between very large teams of highly heterogeneous agents. We describe the principles underlying the design of the Machinetta proxies and present initial results from two domains.