20220805
Title:
VisuoTactile Perception for Dexterous Manipulation

Abstract:
In this thesis, we develop visuotactile perception to enable general and precise robotic manipulation. In particular, we want to study how to effectively process visual and tactile information to allow robots to expand their capabilities while remaining accurate and reliable.

We begin our work by focusing on developing tools for tactile perception. For the task of grasping, we use tactile observations to assess and improve grasp stability. Tactile information also allows extracting geometric information from contacts which is a taskindependent feature. By learning to map tactile observations to contact shapes, we show robots can reconstruct accurate 3D models of objects, which can later be used for pose estimation.

We build on the idea of using geometric information from contacts by developing tools that accurately render contact geometry in simulation. This enables us to develop a probabilistic approach to pose estimation for novel objects based on matching real visuotactile observations to a set of simulated ones. As a result, our method does not rely on real data and yields accurate pose distributions.

Finally, we demonstrate how this approach to perception enables precise manipulations. In particular, we consider the task of precise pickandplace of novel objects. Combining perception with taskaware planning, we build a robotic system that identifies in simulation which object grasps will facilitate grasping, planning, and perception; and selects the best one during execution. Our approach adapts to new objects by learning objectdependent models purely in simulation, allowing a robot to manipulate new objects successfully and perform highly accurate placements.