Smart AGVs and mobile manipulators

Topic header14.jpg

Automated Guided Vehicles (AGVs) are serving (intra-)logistics by autonomously transporting goods through factory environments. AGVs equipped with a manipulator, I.e. an industrial robot arm, are mobile manipulators which flexibly perform tasks at different locations within the factory. This supports the requirements of flexible production lines and constant utilization of the manipulators.




Outline & Problem

Whereas most of current solutions for mobile navigations rely on active tracking of guiding artificial lines or landmarks, the robots are not able to adapt trajectories to change their behavior towards current environmental context, e.g. because of temporarily blockage of routes. This is particularly necessary in joint workspace of humans and mobile robots, which could be unstructured, dynamic and complex. Consequently, this lack of flexibility is a drawback by having negative effects on efficiency and periphery cost and restricted autonomous degree of freedom. The same applies to flexibility regarding tasks and workpieces. Most use case implementations assume fixed workpiece locations for (un-)gripping. Hence the mobile manipulator is not able to identify and grip objects in unstructured environments.

This is where Gestalt comes into play. We support a next level of flexibility towards unstructured environments and contextual awareness by intelligent service like SLAM, object identification, adaptive path planning and many more. We enable the usage of mobile manipulators in hybrid environments where humans and robots are working safely together. 

Locate, navigate to and pick up objects in dynamic production environments with a mobile manipulator.


We demonstrate by sharing a simple concept study for an exemplary project. The scenario is simplified but involves all major challenges.

The main technical requirements:

Robust Object Detection

  • Within mid distances (2-10 m)

  • Under varying light conditions

  • Handle partial occlusion

  • In unstructured environments

Mobile Navigation to target object (safely)

Pose estimation of target object

  • Near range position & orientation estimation (accuracy less than 2 cm)

Flexible Handling of different workpieces

• Independent of object geometries

• Position and orientation agnostic

Combine state-of-the-art computer-vision and a mobile robotic manipulator in conjunction with a flexible vacuum gripping system

Approach & Methods

Exemplary hardware setup

(1) Otto M — Mobile platform
Enhancing work space of robot arm, indoor logistics

(2) Fanuc CR7iA — Industrial robot/ manipulator
6DoF movements, 7 kg payload, collaborative robot

(3) UniGripper Co/Light — Gripping
Multi object vacuum gripper, adaptive to different geometries

(4) Intel RealSense — O bject detection, Pose estimation
Color and depth image

(5) Bottles — Workpiece
Different sizes and materials

Exemplary hardware setup

Exemplary hardware setup


General procedure

STEP 1 — Recognition

  • Recognize object on distant location

STEP 2 — Navigation

  • Navigate to assessed target spot

STEP 3 — Object pose estimation

  • Superimposing of model onto real world estimation via ICP algorithm

  • Match with reference

  • Extract position

  • Extract orientation

STEP 4 — Robot arm command

  • Command robot arm onto modelled position via extracted object data

  • Initiate gripping

2017-11-22 Mett Story 1a ed.jpg
2017-11-22 Mett Story 1b4 ed.jpg
2017-11-22 Mett Story 1c ed.jpg
2017-11-22 Mett Story 1d2 ed.jpg
2017-11-22 Mett Story 1e4 ed.jpg

Our Quote

GESTALT provides software services and full integration for mobile manipulators. We support flexibility due to technologies like SLAM, visual object recognition and pose estimation. We have experience with mobile platforms from different manufacturers and realize context and environment awareness towards integration for hybrid manufacturing as well as domestic, hospital and care scenarios. get in touch with us if you are interested in our solutions or you want to discuss your specific use case. You can contact us under  or give us a call at +49 30 616 515 60 – we would love hearing from you.