Object Identification and Pose Estimation Using Bio-Inspired Tactile-Enabled Multi-Joint Fingers for In-Hand Manipulation

Abstract

In-hand manipulation is a major challenge that has to be addressed in order for robots to achieve human-like skills and manipulation abilities. A new generation of humanoid robots will need dexterous hands able to deal with uncertainties, especially when they are expected to operate in unstructured environments, such as homes and hospitals. Given the human ability to quickly obtain and understand tactile data, one promising direction in order to achieve enhanced robotic dexterous skills is to investigate and emulate human manipulation capabilities. In humans, a combination of somatosensory subsystems deals with everyday manipulation tasks.

This thesis introduces a new approach for estimating the pose of a grasped object by combining tactile sensing data and visual frames of reference inspired by the human “Where” subsystem. While tactile sensing produces local data about objects during in-hand manipulation, a vision system generates egocentric and allocentric frames of reference. Object recognition in the early grasp phases in unstructured environments is also a fundamental ability for robots to achieve human-level manipulation skills. Humans developed a so-called “haptic glance” where non-exploratory manipulation perform fast object identification. Tactile sensors contribute useful information about the objects manipulated by robots, especially during in-hand operations. Drawing inspiration from the functionality of the “What” somatosensory pathway, the proposed solution uses machine learning methods to recognize objects in the early phases of manipulation.

The thesis describes innovative work on object recognition using data collected from bio-inspired multi-modal tactile sensing modules in static and dynamic tasks. The system takes advantage of the module’s compliant structure and inertial, magnetic and pressure measurements. During all experiments, a dual fuzzy logic controller autonomously achieves and maintains stable grasping conditions while forces applied to in-hand objects expose the tactile system to various object configurations. This thesis also presents results on simultaneous object characterization during exploratory procedures using teleoperation.