Towards an Embodied 3D Simulation Platform

Towards an Embodied 3D Simulation Platform
An invited talk by Manolis Savva. 

Abstract: One long-term goal of AI is to create intelligent systems that can improve our daily lives by understanding the 3D world and the language we use to describe it. Assistive dialog agents for the visually impaired, natural-language interaction with self-driving cars, home robots, and personal assistants are a few applications. Despite significant progress in the computer vision and natural language processing communities we are far from practical deployment of such embodied AI agents.

A key bottleneck is the logistical difficulty of deploying mobile agents in the real world. It is hard to acquire access to a variety of places, and the deployed systems typically need to be supervised by humans to ensure safety. Moreover, it is hard to test corner cases and dangerous scenarios, and testing in the physical world can only be done at real-time rates.

Simulation addresses these challenges and allows us to develop and benchmark mobile agents with greater speed, control, repeatability, and safety. Simulation is critical for establishing standardized benchmarks which provide a clear measure of progress and promote reproducibility. We propose a simulation platform for embodied agents operating in 3D environments, designed to address the need for a common framework within which embodied agents performing a variety of tasks can be developed, evaluated and compared.

Short Bio:
Manolis Savva is an Assistant Professor at Simon Fraser University in Vancouver, Canada and a visiting researcher at Facebook AI Research (FAIR). He obtained his PhD at the Stanford computer graphics lab, advised by Pat Hanrahan. His research focuses on human-centric 3D scene analysis, 3D scene generation, and simulation for scene understanding. He has also worked in data visualization, grounding of natural language to 3D content, and in creating large-scale scene datasets for 3D deep learning. He has also worked in data visualization, grounding of natural language to 3D content, and in creating scene datasets for 3D deep learning. In particular, his work on large-scale semantically annotated 3D datasets such as ShapeNet, ScanNet, and Matterport3D has enabled much research in 3D computer vision, computer graphics, machine learning, and artificial intelligence.