From the Paul G. Allen Family Foundation.
By The American Bazaar Staff
WASHINGTON, DC: Dr. Devi Parikh, an Assistant Professor in the Bradley Department of Electrical and Computer Engineering at Virginia Tech, where she also leads the Computer Vision Lab, and Dr. Maneesh Agrawala, Professor of Electrical Engineering and Computer Science at the University of California, Berkeley, are among the seven recipients of the Allen Distinguished Investigator Award from the Paul G. Allen Family Foundation for close to $1 million each for their respective research projects in artificial intelligence.
The researchers, from five different projects from around the globe, who are working on machine reading, diagram interpretation and reasoning, and spatial and temporal reasoning, were in all awarded $5.7 million from the foundation, last week.
“The Allen Distinguished Investigator program has become a platform for scientists and researchers to push the boundaries on the conventional and test the limits of how we think about our existence and the world as we know it,” Dune Ives, co-manager of The Paul G. Allen Family Foundation, said in a statement. “We are only beginning to grasp how deep intelligence works. We hope these grants serve as a valuable catalyst for one day making artificial intelligence a reality.”
To date, the foundation has committed more than $79.1 million to artificial intelligence research.
Parikh’s project proposes to simplify the visual world for machines by leveraging abstract scenes to teach machines common sense.
According to the foundation, “The vast majority of human interaction with the world is guided by common sense. We use common sense to understand objects in our visual world – such as birds flying and balls moving after being kicked. How do we impart this common sense to machines? Machines today cannot learn common sense directly from the visual world because they cannot accurately perform detailed visual recognition in images and video. In this project, Parikh proposes to simplify the visual world for machines
“The visual world around us is bound by common sense laws depicting birds flying and balls moving once they’ve been kicked, but much of this knowledge is hidden from the eyes of a computer,” Parikh said in a statement from the university. “Simply labeling images with this information does not address the underlying problem of how it all fits together. We need a dense sampling of the visual world to understand how subtle changes in the scene can change its overall meaning.”
Parikh added of her project: “These clip art scenes will serve as a completely new and rich test bed for computer vision researchers interested in solving high-level AI problems. Learning common sense will make our machines more accurate, reasonable and interpretable. All imperative towards integrating artificial intelligence into our lives and society at large.”
Parikh is also a member of the Virginia Center for Autonomous Systems (VaCAS) and the VT Discovery Analytics Center (DAC).
Prior to this, she was a Research Assistant Professor at Toyota Technological Institute at Chicago (TTIC), an academic computer science institute affiliated with University of Chicago. She has held visiting positions at Cornell University, University of Texas at Austin, Microsoft Research, MIT and Carnegie Mellon University. She received her M.S. and Ph.D. degrees from the Electrical and Computer Engineering department at Carnegie Mellon University in 2007 and 2009 respectively. She received her B.S. in Electrical and Computer Engineering from Rowan University in 2005.
She was a recipient of the Carnegie Mellon Dean’s Fellowship, National Science Foundation Graduate Research Fellowship, Outstanding Reviewer Award at CVPR 2012 and ECCV 2014, Marr Best Paper Prize awarded at the International Conference on Computer Vision (ICCV) in 2011, two Google Faculty Research Awards in 2012 and 2014, and the 2014 Army Research Office (ARO) Young Investigator Program (YIP) award.
Agrawala won the award, along with Jeffrey Heer of the University of Washington, for their project to have machines easily interpret data from charts and graphs.
The foundation had this to say of their project: “For hundreds of years, humans have communicated through visualizations. While the world has changed, we continue to communicate complex ideas and tell stories through visuals. Today, charts and graphs are ubiquitous forms of graphics, appearing in scientific papers, textbooks, reports, news articles and webpages. While people can easily interpret data from charts and graphs, machines do not have the same ability. Agrawala and Heer will develop computational models for interpreting these visualizations and diagrams. Once machines are better able to “read” these diagrams, they can extract useful data and relationships to drive improved information applications.”
Agrawala received an Okawa Foundation Research Grant in 2006, an Alfred P. Sloan Foundation Fellowship and an NSF CAREER Award in 2007, a SIGGRAPH Significant New Researcher Award in 2008, and a MacArthur Foundation Fellowship in 2009.
Agrawala attended the Science, Mathematics, and Computer Science Magnet Program at Montgomery Blair High School. He received a B.S. in mathematics in 1994 and a Ph.D. in computer science in 2002, both from Stanford University. While attending Stanford, he worked as a software consultant at Vicinity Corporation and in the rendering software group at Pixar Animation Studios. He received a film credit for Pixar’s A Bugs Life. After graduating, Agrawala worked at Microsoft Research for three years, working in its Document Processing and Understanding Group. In 2006, he joined the faculty at the University of California, Berkeley.
The other recipients of the foundation award this year are:
Sebastian Riedel, University College London: Machines have two ways to store knowledge and reason with it. The first is logic – using symbols and rules, and the second is vectors – sequences of real numbers. Both logic and vectors have benefits and limitations. Logic is very expressive, and a good tool to prove statements. Vectors are highly scalable. Riedel will investigate an approach where machines convert symbolic knowledge, read from text and other sources, into vector form, and then approximate the behavior of logic through algebraic operations. Ultimately, this approach will enable machines to pass high-school science exams or perform automatic fact checking.
Ali Farhadi, University of Washington and Hannaneh Hajishirzi, University of Washington: Farhadi and Hajishirzi’s project seeks to teach computers to interpret diagrams the same way children are taught in school. Diagram understanding is an essential skill for children since textbooks and exam questions use diagrams to convey important information that is otherwise difficult to convey in text. Children gradually learn to interpret diagrams and extend their knowledge and reasoning skills as they proceed to higher grades. For computers, diagram interpretation is an essential element in automatically understanding textbooks and answering science questions. The cornerstone of this project is its Spoon Feed Learning framework (SPEL), which marries principles of child education and machine learning. SPEL gradually learns diagrammatic and relevant real-world knowledge from textbooks (starting from pre-school) and uses what it’s learned at each grade to learn and collect new knowledge in the next, more complex grade. SPEL takes advantage of coupling automatic visual identification, textual alignment, and reasoning across different levels of complexity.
Luke Zettlemoyer, University of Washington: The vast majority of knowledge and information we as humans have accumulated is in text form. Computers currently are not able to figure out how to translate that data into action. Zettlemoyer is building a new class of semantic parsing algorithms for the extraction of scientific knowledge in STEM domains, such as biology and chemistry. This knowledge will support the design of next-generation, automated question-answering (QA) systems. Whereas existing QA systems, including IBM’s Watson system for Jeopardy, have been very successful, they are typically limited to factual question answering. In contrast, Zettlemoyer work aims to, in the long term, enable a machine to automatically read any text book, extract all of the knowledge it contains, and then use this information to pass a college-level exam on the subject matter.