Bridging realities: training visuo-haptic object recognition models for robots using 3D virtual simulations

Conrado Ruiz, Òscar de Jesús, Claudia Serrano, Alejandro González, Pau Nonell, Arnau Metaute, David Miralles

Producció científica: Article en revista indexadaArticleAvaluat per experts

Resum

This paper proposes an approach for training visuo-haptic object recognition models for robots using synthetic datasets generated by 3D virtual simulations. In robotics, where visual object recognition has witnessed considerable progress due to an abundance of image datasets, the scarcity of diverse haptic samples has resulted in a noticeable gap in research on machine learning incorporating the haptic sense. Our proposed methodology addresses this challenge by utilizing 3D virtual simulations to create realistic synthetic datasets, offering a scalable and cost-effective solution to integrate haptic and visual cues for object recognition seamlessly. Acknowledging the importance of multimodal perception, particularly in robotic applications, our research not only closes the existing gap but envisions a future where intelligent agents possess a holistic understanding of their environment derived from both visual and haptic senses. Our experiments show that synthetic datasets can be used for training object recognition in haptic and visual modes by incorporating noise, performing some preprocessing, data augmentation, or domain adaptation. This work contributes to the advancement of multimodal machine learning toward a more nuanced and comprehensive robotic perception.

Idioma originalAnglès
RevistaVisual Computer
DOIs
Estat de la publicacióAcceptada/en premsa - 2024

Fingerprint

Navegar pels temes de recerca de 'Bridging realities: training visuo-haptic object recognition models for robots using 3D virtual simulations'. Junts formen un fingerprint únic.

Com citar-ho