Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

Gesture-Based Control of a Robot Arm with Neuromorphic Haptic Feedback

Francesca Sorgini
,
Giuseppe Airò Farulla
,
Nikola Lukic
,
Ivan Danilov
,
Bozica Bojovic
,
Milos Milivojevic
,
Terrin Babu Pulikottil
,
Loris Roveda
,
Paolo Prinetto
,
Tullio Tolio
,
Maria Chiara Carrozza
,
Petar Petrovic
* ,
Calogero Maria Oddo
* ORCID logo
Version 1 : Received: 26 December 2018 / Approved: 28 December 2018 / Online: 28 December 2018 (03:53:18 CET)

How to cite: Sorgini, F.; Airò Farulla, G.; Lukic, N.; Danilov, I.; Bojovic, B.; Milivojevic, M.; Pulikottil, T.B.; Roveda, L.; Prinetto, P.; Tolio, T.; Carrozza, M.C.; Petrovic, P.; Oddo, C.M. Gesture-Based Control of a Robot Arm with Neuromorphic Haptic Feedback. Preprints 2018, 2018120324. https://doi.org/10.20944/preprints201812.0324.v1 Sorgini, F.; Airò Farulla, G.; Lukic, N.; Danilov, I.; Bojovic, B.; Milivojevic, M.; Pulikottil, T.B.; Roveda, L.; Prinetto, P.; Tolio, T.; Carrozza, M.C.; Petrovic, P.; Oddo, C.M. Gesture-Based Control of a Robot Arm with Neuromorphic Haptic Feedback. Preprints 2018, 2018120324. https://doi.org/10.20944/preprints201812.0324.v1

Abstract

Research on bidirectional human-machine interfaces will enable the smooth interaction with robotic platforms in contexts ranging from industry to tele-medicine and rescue. This paper introduces a bidirectional communication system to achieve multisensory telepresence during the gestural control of an industrial robotic arm. We complement the gesture-based control by means of a tactile-feedback strategy grounding on a spiking artificial neuron model. Force and motion from the robot are converted in neuromorphic haptic stimuli delivered on the user’s hand through a vibro-tactile glove. Untrained personnel participated in an experimental task benchmarking a pick-and-place operation. The robot end-effector was used to sequentially press six buttons, illuminated according to a random sequence, and comparing the tasks executed without and with tactile feedback. The results demonstrated the reliability of the hand tracking strategy developed for controlling the robotic arm, and the effectiveness of a neuronal spiking model for encoding hand displacement and exerted forces in order to promote a fluid embodiment of the haptic interface and control strategy. The main contribution of this paper is in presenting a robotic arm under gesture-based remote control with multisensory telepresence, demonstrating for the first time that a spiking haptic interface can be used to effectively deliver on the skin surface a sequence of stimuli emulating the neural code of the mechanoreceptors beneath.

Keywords

telepresence; neuromorphic vibrotactile feedback; human-robot interaction; hand tracking; gesture-based teleoperation

Subject

Engineering, Control and Systems Engineering

Copyright: This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download PDF Download Supplementary

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.

Leave a public comment
Send a private comment to the author(s)
* All users must log in before leaving a comment
Views 0
Downloads 0
Comments 0
Metrics 0
Get PDF Supplementary Files Cite Share 0
Bookmark BibSonomy Mendeley Reddit Delicious
×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.