"Skydio Autonomy: Research in Robust Visual Navigation and RealTime 3D Reconstruction" Adam Bry (Skydio) and Hayk Martiros (Skydio)

Abstract: Skydio is the leading US drone company and the world leader in autonomous flight. Our drones are used for everything from capturing amazing video, to inspecting bridges, to tracking progress on construction sites. At the core of our products is a visionbased autonomy system with seven years of development at Skydio, drawing on decades of academic research. This system pushes the state of the art in deep learning, geometric computer vision, motion planning, and control with a particular focus on realworld robustness. Drones encounter extreme visual scenarios not typically considered by academia nor encountered by cars, ground robots, or AR applications. They are commonly flown in scenes with few or no semantic priors and must deftly navigate thin objects, extreme lighting, camera artifacts, motion blur, textureless surfaces, vibrations, dirt, camera smudges, and fog. These challenges are daunting for classical vision because photometric signals are simply not consistent and for learningbased methods because there is no ground truth for direct supervision of deep networks. In this talk we'll take a detailed look at these issues and the algorithms we've developed to tackle them. We will also cover the new capabilities on top of our core navigation engine to autonomously map complex scenes and capture all surfaces, by performing realtime 3D reconstruction across multiple flights.

Biography: Adam is cofounder and CEO at Skydio. He has two decades of experience with small UAS, starting as a national champion R/C airplane aerobatics pilot. As a grad student at MIT, he did award winning research that pioneered autonomous flight for drones, transferring much of what he learned as an R/C pilot into software that enables drones to fly themselves. Adam cofounded Google’s drone delivery project. He currently serves on the FAA’s Drone Advisory Committee. He holds a BS in Mechanical Engineering from Olin College and an SM in Aero/Astro from MIT. He has coauthored numerous technical papers and patents, and was also recognized on MIT’s TR35 list for young innovators.

Biography: Hayk was the first engineering hire at Skydio and he leads the autonomy team. He is an experienced roboticist who develops robust approaches to computer vision, deep learning, nonlinear optimization, and motion planning to bring intelligent robots into the mainstream. His team’s state of the art work in UAV visual localization, obstacle avoidance, and navigation of complex scenarios is at the core of every Skydio drone. He also has an interest in systems architecture and symbolic computation. His previous works include novel hexapedal robots, collaboration between robot arms, microrobot factories, solar panel farms, and selfbalancing motorcycles. Hayk is a graduate of Stanford University and Princeton University.

“Robotics Today A series of technical talks” is a new virtual robotics seminar series. The goal of the series is to bring the robotics community together during these challenging times.The seminars are scheduled on Fridays at 3PM EDT (12AM PDT) are open to the public. The format of the seminar consists of a technical talk live captioned and streamed via Web (https://roboticstoday.github.io/watch...) and Twitter (@RoboticsSeminar), followed by an interactive discussion between the speaker and a panel of faculty, postdocs, and students that will moderate audience questions.

Website: https://roboticstoday.github.io/
Twitter:   / roboticsseminar  
Calendar: https://calendar.google.com/calendar/...

Outline
0:00 Introduction
3:10 Adam and Hayk's Talk
43:26 Panel Discussion
1:22:08 Concluding Remarks