Institut de Neurosciences Cognitives et Intégratives d'Aquitaine (UMR5287)

Aquitaine Institute for Cognitive and Integrative Neuroscience

Université de Bordeaux

Zone nord Bat 2 2ème étage
146, rue Léo Saignat
33076 Bordeaux cedex


Supervisory authorities

CNRS Ecole Pratique des Hautes Etudes Université de Bordeaux

Our partners

Neurocampus Unitéde Formation de Biologie


GDR Robotique GDR Mémoire GDR Multi-électrodes


Home > Teams > HYBRID (A. De RUGY) > Themes

Theme 5: Reaching in Virtual Reality

by Daniel Cattaert, Hybrid, Loïc Grattier - published on , updated on

To further develop and test various control schemes, we developed a 3D virtual reality (VR) set-up for reaching and grasping with the arm (Fig 4B). As automatic object detection is now available from computer vision augmented with gaze information (cf Theme4), we explored the possibility to use this information in conjunction with residual stump movements to reconstruct motion of the distal joints lost in amputees. After picking and placing a bottle in numerous positions and orientations in a 3D virtual environment, we trained artificial neural networks to reconstruct postures of an intact subjects’ elbow, forearm and wrist (4 degrees of freedom), either solely from shoulder kinematics or with additional knowledge of the object location. Importantly, we showed that adding contextual information about movement goal improves predictions to the point that virtually amputated subjects could reach close to natural performance without any training, and with minimal compensatory movements from trunk and shoulder (Mick et al, Submitted). Substantial work remains to translate this proof of principle established with healthy subjects on real amputees, to increase task functionalities, and to integrate this control on our robotic platform REACHY (Theme3).

- Mick S, Segas E, Dure L, Halgand C, Benois-Pineau J, Loeb GE, Cattaert D, de Rugy A (Submitted) Shoulder kinematics plus contextual target information enable control of multiple distal joints of a simulated prosthetic arm and hand.