MCDS alumnus Abdelwahad Bourai, now imployed by Uber's Advanced Technologies Group, had papers accepted recently by two major conferences: ICMI 2017 and AAMAS 2018.
An MCDS Capstone Project turned into a pair of publications for a recent graduate of the program. Abdelwahab Bourai, who graduated from the program in 2017 and is now employed by Uber’s Pittsburgh-based Advanced Technologies Group, recently had papers accepted at both the International Conference on Multimodal Interaction (ICMI 2017) and the International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2018).
Bourai’s ICMI paper, “Automatically Predicting Human Knowledgeability through Non-Verbal Cues,” sought to predict the likelihood that trivia gameshow participants would correctly answer a question utilizing a multimodal computing approach. Bourai’s system combined analysis of audio and video data, using voice tone and rhythm along with eye gaze, head position and facial expression of a participant while answering a question to predict how “knowledgeable” the respondent is about the question topic. Those results were then measured against the predictions of three human subjects attempting the same task. The system consistently performed at or above the level of the accuracy of the humans.
At AAMAS, Bourai will present his paper “I Know What You Don’t Know: Proactive Learning through Targeted Human Interaction.” The paper builds on the work Bourai presented at ICMI by applying a proactive learning technique, an extension of the “active learning” approach that seeks to glean greater information on “oracles” (information sources) in order to make better use of the data they provide.
“Our proactive learning agent was able to observe students answer questions from four pre-determined categories and, based on their speech and facial activity, determine whose answer to trust,” Bourai explained. “We found the model could pinpoint the best student for each category and ask them for aid for a given question.”
Bourai said that his interest in the topics explored in both papers began early in his time in the MCDS program, crediting LTI Assistant Professor Louis-Philippe Morency’s course Human Communication and Multimodal Machine Learning, as well as several other MCDS/Language Technologies Institute faculty members.
“I had previously done work in Question Answering with Professor [and MCDS Program Director] Eric Nyberg during my undergraduate studies and was interested in studying the non-verbal features related to answering questions in humans,” he explained, also thanking current LTI Assistant Professor Matthias Grabmair and former Postdoctoral Researcher Tadas Baltrusaitis, who currently works at Microsoft Research. Bourai would go on to work with LTI Director Jaime Carbonell, who supervised the capstone project that would lead to both papers and served as co-author on the AAMAS paper.
Bourai also went out of his way to explain how his time in the MCDS program has been beneficial to his work at Uber, where he is part of the company’s Machine Teaching and Interactive Learning group. “Although self-driving cars may not have a clear connection to language technologies, the work done in LTI is extremely applicable throughout the industry,” he noted.
ICMI 2017 was the 19th iteration of the conference, which bills itself as “the premier international forum for multidisciplinary research on multimodal human-human and human-computer interaction, interfaces, and system development.” The conference was held November 13-17, 2017 in Glasgow, Scotland. AAMAS, which lays claim to the title of “the largest and most influential conference in the area of agents and multiagent systems,” will hold its 17th edition July 10-15, 2018 in Stockholm, Sweden.