Virtual Reality & Immersive Interaction
Virtual Reality (VR) became quite popular in areas, where the goal is to gain experience remotely. More specifically and related to robotics, it is considered a great benefit to experience a digital factory design, a robot simulation and the tele-operation of robots. The first application of a tele-operated robot was demonstrated on the ISS about fifteen years ago. Modern tele-operation robots you can buy, enable you today to show your face on a screen and to help to attend a meeting as a mobile platform remotely. The challenge in both cases, on the ISS and on earth, is that direct real-time control of a robot requires a high-speed, low-latency connection, which is impractical.
INTRODUCTION
Virtual world meets robotics
Also because of the connectivity restriction, we see the rise of VR in the robotics domain which enables smooth virtual usability while only high-level tasks are transmitted and not the heavy video streams from cameras. In order to enable the user to see the environment the robot is operating in, only abstracted data needs to be transferred, like the floor plan or some detected obstacles or objects. From the user’s side, commands like "Grab this bag and put it over there!" are transmitted, which can be accomplished with very low-bandwidth connections and thus are a practical approach for many places on earth and in space.
The remote control in VR is much more than watching a live video stream and pushing buttons. It is about communication with the robot via gestures or high-level commands. Contact us about further insights and explore how VR-Control can help your business.
TECHNOLOGICAL OUTLINE
VR-Control tech in a nutshell
The fundamental ingredient in VR-control is to enable the robot to interact with the environment on some level of autonomy. Only this way it is possible to command it via high-level commands, which are natural for human communication. This, however, includes a broad stack of technologies, being provided by GESTALT Robotics.
Technology Stack
Raw video (RGB-D) data from the robot is processed via SLAM, object detection, 3D reconstruction and Spatio-Temporal Memory. It is further fed into the action generator. The robust movement controller makes sure that the robot actuators perform as expected regardless of low-cost mechanics or mechanical disturbance.
APPLICATIONS & USE CASES
Semi-Autonomous Robot Navigation
Applying the architecture described in the figure above, we achieve smooth interaction between the robot and the user while maintaining low data transfer rates and high task execution accuracy on the robot side.
GESTALT Robotics provides a browser-based VR-control solution, which makes it possible to log-in from any computer without prior setup of heavy software.
Data annotation
The SLAM module creates a 3D model of the environment in your browser in real-time. Using our VR-solution, you can directly annotate objects and give attributes to them in real-time, during data acquisition. This has a huge potential to save a lot of time spending annotating data for Building Information Management (BIM), construction and inspection applications.
Illustration of the abstract visualization in VR, which is being delivered by the mobile robot in real-time
CONCLUSION
We at GESTALT believe that combining virtual reality and robots will enable physical remote work everywhere from anywhere. We provide VR control solutions for low-cost and advanced (mobile) robots, which help you increase the added value of your business, to save runtime-costs and to increase efficiency. We are happy to discuss your project with you and the possible impact of VR. Contact us via email info@gestalt-robotics.com or give us a call at +49 30 616 515 60!