Robot with wisdom – Robohub

Artificial hands, even the most sophisticated prostheses, are still far behind human hands. What they lack is tactile ability which is essential for dexterity. Other challenges include connecting senses with actions in robotic systems – and effectively connecting them with human users. Prof Dr Philipp Beckerle from FAU has joined international colleagues to summarize the latest findings in this area of ​​Robotics – and set the agenda for future research. Their share in research journals Science Robotics suggested a sensorimotor control framework for a haptically activated robotic hand, inspired by principles of the human central nervous system. Their aim was to link tactile sensing with human-centered and haptically-enabled artificial hand movement. According to a team of European and American researchers, this approach holds promise for increasing dexterity for humans controlling robotic hands.

Tactile sensing needs to play a bigger role

“Human manual dexterity is very dependent on touch”, explained Prof. Dr. Philipp Beckerle, head of FAU’s Chair of Autonomous and Mechatronic Systems (ASM). “Humans with intact motor function but numb fingertips can find it very difficult to understand or manipulate things.” This, he says, shows that tactile sensing is necessary for human dexterity. “The bio-inspired design shows that learning from human haptics can improve the currently limited dexterity of artificial limbs. But robotic and prosthetic hands don’t make much use of many of the tactile sensors available today and are therefore far less dexterous.”

Beckerle, a Mechatronics engineer, just had a paper “A hierarchical sensorimotor control framework for the human robotic hand-in-the-turnpublished in a research journal Science Robotics. In this regard, he reveals with international colleagues how advanced technology now provides not only mechatronic and computational components for anthropomorphic limbs, but also sensing components. Scientists therefore suggest that recently developed tactile sensing technology can be incorporated into the general concept of “electronic skin”. “This includes a dense array of normal force-sensing tactile elements that contrast the fingertip with a more comprehensive force perception”, the paper states. “This will provide a directional force distribution map across the sensing surface, and a complex three-dimensional architecture, emulating the mechanical and multimodal sensing properties of the human fingertip.” Tactile sensing systems mounted on mechatronic limbs can provide robotic systems with the complex representations needed to characterize, identify and manipulate, for example, objects.

Human principles as inspiration for future designs

To achieve haptically informed and agile machines, the researchers secondly propose taking inspiration from hierarchically organized principles of the human central nervous system (CNS). Control of the CNS, which signals the brain receives from the tactile senses and sends them back to the body. The authors propose a conceptual framework in which robots activated by a bioinspired touch share control with humans – to a degree that humans determine. Highlights of the framework include parallel processing of tasks, integration of feed-forward and feedback control, and the dynamic balance between conscious and unconscious processing. This can be applied not only in the design of bionic limbs, but also virtual avatars or remotely navigated telerobots.

There are still other challenges to effectively connecting human users with touch-enabled robotic hands. “Enhancing haptic robots with high-density tactile sensing can substantially improve their capabilities but raises questions about how best to transmit these signals to a human controller, how to navigate shared perception and actions in human-machine systems”, the paper reads. It is still largely unclear how to manage agency and task assignment, to maximize utility and user experience in a human-in-the-loop system. “The most challenging thing is how to exploit the varied and abundant tactile data generated by haptic devices. However, human principles provide inspiration for the design of future mechatronic systems that can function like humans, alongside humans, and even as human replacement parts.”

Chairman Philipp Beckerle is part of FAU’s Department of Electrical Engineering, Electronics and Information Technology as well as the Department of Artificial Intelligence in Biomedical Engineering. “Our mission at ASM is to research human-centred mechatronics and robotics and seek solutions that combine the desired performance with user-friendly nature of interaction,” explains Beckerle. “Our focus is on wearable systems such as prostheses or exoskeletons, cognitive systems such as collaborative robots or humanoids, and generally on tasks with close robot-human interaction. The human factor is critical in such scenarios to meet user needs and to achieve synergistic interfaces and interactions between humans and machines.”

In addition to Prof Dr Beckerle, scientists from the Universities of Genoa, Pisa and Rome, Aalborg, Bangor and Pittsburgh as well as Imperial College London and the University of Southern California, Los Angeles contributed to the paper.

• PAPER – A hierarchical sensorimotor control framework for the human robotic hand-in-the-turn. Lucia Seminara, Strahinja Lecturer, Fulvio Mastrogiovanni, Matteo Bianchi, Simon Watt, Philipp Beckerle, Thrishantha Nanayakkara, Knut Drewing, Alessandro Moscatelli, Roberta L. Klatzky, and Gerald E. Loeb. Science Robotics, 8(78), eadd5434.