About me and my work

I am fascinated by the natural phenomenon of intelligence, and I work on understanding and advancing the limits of artificial intelligence (AI).

I am a co-founder of Yutori. I have been a professor, led research teams in industry, and built open-source communities.

I was a Senior Director at Meta leading FAIR Embodied AI (AI for robotics and smart glasses). My teams:

• Developed the multimodal AI assistant that shipped in the Ray-Ban Meta SmartGlasses.
• Built Habitat, the fastest 3D simulator for training virtual robots to navigate, pick and place objects, and operate around humans, and follow language instructions.
• Solved PointNav, the task of navigation to goal coordinates in unfamiliar environments without a map, both in simulation and with Boston Dynamics' Spot robot.
• Demonstrated a robotic assistant to CBS and at the White House Correspondents' Dinner.
• Built the world's first aritifical superhuman fingertip.

I was a tenured Associate Professor in the School of Interactive Computing at Georgia Tech.

• I recived the PECASE award, the highest honor bestowed by the U.S. government for early career scientists and engineers.
• My students won the ACM SIGAI Doctoral Dissertation Awards in back-to-back years and University-level dissertation awards in 3 consecutive years.

My work has received best paper awards/nominations/honorable mentions in every area of AI

• computer vision (Ego4D at CVPR 2022, AI Habitat at ICCV 2019),
• machine learning (Emergence of Maps at ICLR 2023),
• natural language processing (Lack of emergence of language at EMNLP 2017),
• robotics (Combining foundation models and mapping (System 1 + System 2) at ICRA 2024).

• Summarizing beliefs of AI agents via diverse plausible predictions/hypotheses:
  Diverse Beam Search, Multiple Choice Learning, Tutorial on Diversity at CVPR '13 and CVPR '16,
• Vision-and-language (or multimodal AI):
  Image Captioning, Visual Question Answering, Visual Dialog, Audio-Video Dialog, Human-AI GuessWhich games,
• Embodied AI and robotics:
  Habitat: A Platform for Embodied AI, Decentralized Distributed PPO, Embodied Question Answering, Sim2Real Predictivity, ASC: Adaptive Skill Coordination for Robotic Mobile Manipulation, LSC: Language-guided Skill Coordination for Open-Vocabulary Mobile Pick-and-Place
• Explainable, Unbiased, Trustworthy AI:
  Grad-CAM (Visual Explanations), Human-vs-machine attention, Counterfactual Visual Explanations
• Platforms for reproducible AI research:
  EvalAI, a platform for evaluating AI algorithms.