Version 1
: Received: 26 December 2018 / Approved: 28 December 2018 / Online: 28 December 2018 (03:53:18 CET)
How to cite:
Sorgini, F.; Airò Farulla, G.; Lukic, N.; Danilov, I.; Bojovic, B.; Milivojevic, M.; Pulikottil, T.B.; Roveda, L.; Prinetto, P.; Tolio, T.; Carrozza, M.C.; Petrovic, P.; Oddo, C.M. Gesture-Based Control of a Robot Arm with Neuromorphic Haptic Feedback. Preprints2018, 2018120324 (doi: 10.20944/preprints201812.0324.v1).
Sorgini, F.; Airò Farulla, G.; Lukic, N.; Danilov, I.; Bojovic, B.; Milivojevic, M.; Pulikottil, T.B.; Roveda, L.; Prinetto, P.; Tolio, T.; Carrozza, M.C.; Petrovic, P.; Oddo, C.M. Gesture-Based Control of a Robot Arm with Neuromorphic Haptic Feedback. Preprints 2018, 2018120324 (doi: 10.20944/preprints201812.0324.v1).
Cite as:
Sorgini, F.; Airò Farulla, G.; Lukic, N.; Danilov, I.; Bojovic, B.; Milivojevic, M.; Pulikottil, T.B.; Roveda, L.; Prinetto, P.; Tolio, T.; Carrozza, M.C.; Petrovic, P.; Oddo, C.M. Gesture-Based Control of a Robot Arm with Neuromorphic Haptic Feedback. Preprints2018, 2018120324 (doi: 10.20944/preprints201812.0324.v1).
Sorgini, F.; Airò Farulla, G.; Lukic, N.; Danilov, I.; Bojovic, B.; Milivojevic, M.; Pulikottil, T.B.; Roveda, L.; Prinetto, P.; Tolio, T.; Carrozza, M.C.; Petrovic, P.; Oddo, C.M. Gesture-Based Control of a Robot Arm with Neuromorphic Haptic Feedback. Preprints 2018, 2018120324 (doi: 10.20944/preprints201812.0324.v1).
Abstract
Research on bidirectional human-machine interfaces will enable the smooth interaction with robotic platforms in contexts ranging from industry to tele-medicine and rescue. This paper introduces a bidirectional communication system to achieve multisensory telepresence during the gestural control of an industrial robotic arm. We complement the gesture-based control by means of a tactile-feedback strategy grounding on a spiking artificial neuron model. Force and motion from the robot are converted in neuromorphic haptic stimuli delivered on the user’s hand through a vibro-tactile glove. Untrained personnel participated in an experimental task benchmarking a pick-and-place operation. The robot end-effector was used to sequentially press six buttons, illuminated according to a random sequence, and comparing the tasks executed without and with tactile feedback. The results demonstrated the reliability of the hand tracking strategy developed for controlling the robotic arm, and the effectiveness of a neuronal spiking model for encoding hand displacement and exerted forces in order to promote a fluid embodiment of the haptic interface and control strategy. The main contribution of this paper is in presenting a robotic arm under gesture-based remote control with multisensory telepresence, demonstrating for the first time that a spiking haptic interface can be used to effectively deliver on the skin surface a sequence of stimuli emulating the neural code of the mechanoreceptors beneath.
Keywords
telepresence; neuromorphic vibrotactile feedback; human-robot interaction; hand tracking; gesture-based teleoperation
Subject
ENGINEERING, Control & Systems Engineering
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
