Mobile manipulation for a “kitting task”. In this video, the location of the bins are approximately known and registered using 3D perception integrated in the robot’s hand. Impedance control is used for robust interaction with the bin and its content.

Despite their impressive enabling capability for general manipulation, 3D perception, impedance control, and tactile sensing are at odds with the prevailing industrial paradigm of specialized manipulation solutions. Any form of sensing requires time and puts hard limits on end-effector velocity to limit impact energy in case of unexpected collisions. The drivers for general manipulation will therefore be small and medium enterprises who are working on a large variety of products with small numbers, and larger players who want to differentiate their products with faster production cycles and higher degrees of customization. At the same time, mobile robots are becoming ubiquitous in warehouses, hotels and hospitals. In situations like these, specific manipulation tasks like loading, unloading, and simple maintenance might dramatically amplify the value proposition of such robots, creating the economic forces that we need to solve general manipulation.

Nikolaus Correll is an associate professor at the University of Colorado Boulder and CTO and founder of Robotic Materials Inc., where he develops robotic grippers with built-in 3D perception, tactile sensing, and computation.