Smart AGVs


Outline & Problem

Automated Guided Vehicles (AGVs) are serving (intra-)logistics by autonomously transporting goods through factory environments. AGVs equipped with a manipulator, i.e. an industrial robot arm, are mobile manipulators which flexibly perform tasks at different locations within the factory. This supports the requirements of flexible production lines and constant utilization of the manipulators. Whereas most current solutions for mobile navigations rely on active tracking of guiding artificial lines or landmarks, the robots are not able to adapt trajectories to change their behavior in response to current environmental context, e.g. because of temporarily blockage of routes. This is particularly necessary in a joint workspace of humans and mobile robots, which could be unstructured, dynamic and complex. Consequently, this lack of flexibility is a drawback by having negative effects on efficiency, periphery cost and restricted degrees of freedom. The same applies to flexibility regarding tasks and workpieces. Most use case implementations assume fixed workpiece locations for (un-)gripping. Hence the mobile manipulator is not able to identify and grip objects in unstructured environments.

This is where GESTALT comes into play. We support a next level of flexibility towards unstructured environments and contextual awareness using intelligent services like SLAM, object identification, adaptive path planning and many more. We enable the usage of mobile manipulators in hybrid environments where humans and robots are working safely together.


We demonstrate a typical scenario for smart AGV integration by providing an exemplary concept study. The scenario is simplified but involves all major challenges:

We assume a highly flexible production environment with small lot sizes and a variety of products. Hence, different production steps are not rigidly connected and the workpieces have to be transfered between different machines or workplaces. The mobile manipulator integration bares the potential to dynamically interconnect working steps and locations for a highly adaptable production.

We target decentrally controlled mobile manipulators taking logistic tasks by patrolling around and looking for appearing workpieces. Due to environmental awareness, the mobile manipulators are able to detect and accomplish their tasks autonomously. In contrast, fleet and task management for the AGVs could be provided by a superordinated control system. Consequently, the overall objective is to locate, navigate to, pick up and deliver target workpieces with a mobile manipulator.

Let's sum up the main technical requirements:

Robust Object Detection

  • Within mid distances
  • Under varying light conditions
  • Handle partial occlusion
  • In unstructured environments

Mobile Navigation to target object (safely)

Pose estimation of target objects

  • Near range position & orientation estimation

Flexible Manipulation and Handling of different workpieces

  • Independent of object geometries
  • Position and orientation agnostic


We demonstrate a typical scenario for smart AGV integration by providing an exemplary concept study
Exemplary hardware setup

Approach & Methods

Combine state-of-the-art computer-vision and a mobile robotic manipulator in conjunction with a flexible vacuum gripping system.
Combine state-of-the-art computer-vision and a mobile robotic manipulator in conjunction with a flexible vacuum gripping system.

In the following section you can find more facts and visualization about an exemplary procedure in order to fulfill the objective:

STEP 1 – Object Recognition

  • Detect workpieces on distant locations within unstructured factory environments

STEP 2 – Autonomous Navigation

  • Map-based navigation to assessed target spot
  • Avoiding collisions with static and dynamic obstacles

STEP 3 – Object pose estimation

  • Superimposing of virtual object model onto real world estimation
  • Match with reference
  • Extract pose (position and orientation)

STEP 4 – Robot arm control

  • Command robot arm (manipulator) to target pose via extracted object data and initiate gripping

Our Quote

GESTALT provides software services and full integration for mobile manipulators and AGVs. We support flexibility due to technologies like SLAM, visual object recognition and pose estimation. We have experience with mobile platforms from different manufacturers and realize context and environment awareness towards integration for hybrid manufacturing as well as domestic, hospital and care scenarios. Get in touch with us if you are interested in our solutions or you want to discuss your specific use case. You can contact us under or give us a call at +49 30 616 515 60 – we would love hearing from you.