NAME XR and 3D Perception Laboratory

DESCRIPTION

A laboratory focused on the development of applications to facilitate the interaction of workers with different industrial manufacturing systems, such as collaborative robots and mobile robots. The laboratory combines new interfaces like mixed reality (MR) and virtual reality (VR) glasses with different 3D perception systems and artificial intelligence (AI) models to assist operators during the execution of complex processes. The perception systems allow capturing relevant information from the environment, such as the position of people or elements of interest for the process. In this second case, positioning can be performed using artificial vision techniques (monocular or RGB-D) based on artificial intelligence, which allows the localization of objects without modification; or by using active markers, which enables high-speed and highly accurate localization (200Hz and accuracy greater than 0.5mm in more complex environmental conditions). The information is integrated through advanced communication systems based on standard industrial protocols to provide process information using advanced interfaces based on extended reality (XR). These interfaces allow natural interaction with the equipment, as well as collaboration among different members of the work team.

FIELDS OF APPLICATION

Deployment and Application

Developments

MOST OUTSTANDING EQUIPMENT AND COMPONENTS

  • 3D Vision Cameras

    Intel RealSense D435 stereo vision cameras, StereoLabs ZED X.

  • High-speed stereo tracking system

    Custom-developed system for high-speed tracking (200 FPS) of object movement with submillimeter precision.

  • Mixed Reality Glasses

    State-of-the-art devices that allow the visualization of virtual elements integrated into reality, such as the Meta Quest 3, Microsoft Hololens 1 and 2.

  • Proprietary software libraries

    Libraries for the geometric processing of CAD models. AI models for detecting the position of objects of interest.

SERVICES OFFERED BY THE ASSET

Ad-hoc assistance for different processes. Visualization of information and guidance in a virtual environment.

Computational geometry algorithms are used to process the geometry of CAD models and automate the generation of assistance information.

Analysis of a video sequence for data generation and training of models for object pose estimation.

A computer vision model is used for the automatic generation of a training dataset and the training of models based on G2L-NET.

ENTITY MANAGING THE ASSET

CEIT
Contact person:
Iñaki Díaz
idiaz@ceit.es