Spatio-Temporal Memory
The ability to process complex spatio-temporal information is a fundamental cognitive process underlying the behavior of all higher organisms. When we as humans review known photographs, we are able to recognize objects and people, but typically also remember when and where the photo has been taken. We are basically aware of our personal objects and where we typically store them in our environment. This is achieved thanks to the long-term memory. According to medical and psychological sciences, the spatio-temporal memory (STM) is an important part of the long-term memory involving information about time and space based on our perception.
INTRODUCTION
What, where and when?
Within robotics, a mobile robot typically takes more than 10 pictures per second of its environment. This is a high input of information and today most robotic applications still process spatial, temporal and object-related information separately. Considering these data in conjunction with each other is adding value and higher cognitive functions for service robotics: knowledge extraction from the images (e.g. object detection), the current location (e.g. SLAM) and the current time facilitates to build up scalable databases representing overall knowledge about (dynamic) environments. Based on this, service robots are able to explore and monitor the environment on their own with the help of extended functions for analytics and intelligence.
GESTALT Robotics provides software development towards the usage of Spatio-Temporal Memory as a fusion of the core technologies: Object Detection and Simultaneous Localization and Mapping. Contact us about further insights and explore how STM can help your business.
The ability to process complex spatio-temporal information is a fundamental process underlying the behavior of all higher organisms. We are adding and merging the dimensions time and space in order to enable robots to remember where and when objects have been detected.
TECHNOLOGICAL OUTLINE
Memory – Highly flexible and scalable databases and processing
When it comes to storing information extracted from object identification and SLAM, the state-of-the art approach in robotics is foremost still using plain text files. This is not optimal for many reasons.
We develop and use professional interfaces and database architectures which are scalable, distributed, easy to integrate and perfectly suited for real-time data search and analysis. Especially dynamic schemata help to focus on the STM development, which gives a great boost towards the intelligence of our service robots. Furthermore, in large scale projects the underlying database technology of STM does not break down when millions of data come together, that still need to be searched and analyzed in real-time. Consequently, we tailor database and interface solutions towards the needs of intelligent robotics.
APPLICATIONS & USE CASES
Given the fact that a specific service robot is able to perceive the environment and its changes over time automatically shows there is a wide range of use-cases and applications for STM. We would like to introduce two applications, that are enabled by STM or strikingly benefit from it.
Spatial Data Analytics
Imagining a mobile service robot in a shared environment with humans: Based on its perception features, the robot is able to detect and track persons as well as observing specific actions or interactions with other humans and objects. Thanks to STM, all these data can be stored in relation to time and space. Creating heatmaps can be a specific added value built on top of STM. The heatmap gives a graphical color-coded representation of the STM-data. Heatmaps can be used to show where and when humans have stayed, moved and interacted.
Understanding conglomeration creates valuable information. Regarding retail, one may understand shopper's behavior and analyze the motion flow which may serve the optimization for layout planning. Further applications can be within work process assessment and home automation in order to recognize the user's behaviors and provide targeted support based on spatial-temporary information about daily routines, habits and preferences of the user.
Find-my-keys application for domestic service robotics
Who has never misplaced a personal object within home and spent tedious time looking for it?
On this occasion, imagine a robotic home assistant which is able to tell you where your car key is located, because he knows thanks to STM. Even if the robot does not know exactly, he can support searching or can give hints about typical repositories around the house based on historic data and statistical evaluation. From a different perspective, this feature can also be used to play hide and seek with your service robot.
Heatmapping for action-related human presence in indoor environments
A robot is able to locate misplaced keys even in a convoluted scenario
CONCLUSION
Saptio-temporal environment understanding - More than a database
The Spatio-Temporal Memory is an important step towards a higher level of intelligence for service robots. Consequently, we are adding and merging the dimensions time and space in order to enable robots to remember where and when objects have been detected. Based on modern database technologies, we provide powerful memory-like spatio-temporal representations merging information of object recognition and SLAM, adding an understanding for dynamics of objects powering advanced applications of analytics, heatmapping and semantic targeted navigation. We believe that our contribution in this area will enable a vast amount of applications in many domestic and industrial setups demanding a higher environmental understanding. Get in touch with us to discuss your use case and how you can benefit from a Spatio-Temporal Memory yourself. You can contact us under info@gestalt-robotics.com or give us a call at +49 30 616 515 60 – we would love to hear from you.