MENTOR
Multimodal detection of the normal vector field of transparent objects for robots
PROJECT DESCRIPTION
ROBUST GRIPPING OF TRANSPARENT OBJECTS
MOTIVATION
Recent advances in artificial intelligence (AI) have expanded the applications of robotics beyond traditional areas like automobile production, leading to enhanced process optimization. With the potential for further scaling, these technological developments are poised to revolutionize industries such as chemical and optical. However, automation in these industries presents unique challenges due to the prevalence of glass components, for which a flexible industrial gripping solution has yet to be developed.
OBJECTIVE AND APPROACH
Current state-of-the-art technology is capable of detecting grasping points on glass bodies with a success rate of 72 percent (ClearGrasp, Google, Columbia University, ICRA 2020). The MENTOR project endeavors to improve this success rate to over 98 percent by developing and integrating a sensor head comprising up to seven sensors operating in diverse modalities into different overall systems. These systems will include an articulated robot-based laboratory automation system and an automatic optical component inspection system.
INNOVATION AND PERSPECTIVES
The main challenges lie in recognizing the contour of transparent objects and their position in space. In order to overcome these and other challenges, the following innovations are being pursued in the overall network:
Use of sensor technology in different wavelength ranges and modalities.
Data processing and fusion directly in the sensor head.
Creating open interfaces for use in robotics and beyond.
With this approach, a sensor system is to be developed that can also be used flexibly beyond the applications dealt with in the joint project.
OUR CONTRIBUTION
Our main focus is the development of a modular robotic system designed for handling glass vessels in laboratory settings. Our responsibilities encompass the creation of a data processing platform to facilitate value-added services, as well as the establishment and provision of standardized industrial interfaces for connecting the sensor head to the control level. Additionally, we will evaluate additional data processing concepts to leverage the data processing platform for value-added services.
Dual-arm robot system for flexible laboratory automation
Key facts
Novel Sensor System
Development of a "robot arm-suitable" sensor head
Combination of sensor technology in different wavelength ranges and modalities
Extensive data fusion
Robot Gripping
Modular software system
Hardware-independent software services
Collision-free path planning
Image Recognition
Multimodal image recognition
Development of fast algorithms for defect detection in automatic handling of optical elements
Use of AI methods
Laboratory Automation
Integration into the overall system
Determination of the gripping points of the objects
Evaluation on the basis of industrial processes
PartnerS & CONSORTIUM
Basler AG
Multimodal image recognition of transparent and semitransparent objects for industrial use
Spheron GmbH
Multimodal sensor head for handling and testing vitreous bodies
DIOPTIC GmbH
Development of fast algorithms for defect detection in the automatic handling of optical elements
BASF SE (assoziiert)
Testing the performance in the context of a real application
Funding
Federal Ministry of Education and Research
Program
“Photonik Forschung Deutschland – Licht mit Zukunft”
Duration
12.2022 – 11.2025
Project Management Agency
VDI Technologiezentrum GmbH
Project Poster
Project poster (German) as printable PDF
This project is/was financed with funding provided by the Federal Ministry of Education and Research under the “Photonik Forschung Deutschland – Licht mit Zukunft” program and managed by the Project Management Agency VDI Technologiezentrum GmbH. The author is responsible for the content of this publication.