Gestalt Robotics has developed an AI-based sensor for recognising train departures and arrivals as part of the "illuminated platform edge" project. After successfully trialling the sensor in the factory environment and at Berlin Südkreuz S-Bahn station, the system is now ready to be integrated at other locations.

“illuminated platform edge” at S-Bahnhof Südkreuz

More passenger safety and information on the track

The "illuminated platform edge" project is part of Deutsche Bahn's "Safety station" research project. With this project, Deutsche Bahn wants to ensure greater safety and orientation at railway stations for its passengers. The idea is to use an LED system integrated into the platform edge to inform waiting passengers on the track about arriving and departing trains, as well as the train length and capacity utilisation of the carriages, by means of different light signals. Red light means that a train is arriving or departing. White, dynamic running lights are intended to guide passengers to the stopping position of the train. The colours green, yellow and orange provide information about the train's capacity utilisation.

The test operation was successfully completed in October at Berlin Südkreuz S-Bahn station. A total of 245 concrete blocks with LED lamps have been laid along the edge of the platform on track 1 of the S-Bahn over a length of 150 metres.

Challenge: Creating a detection method with AI

The challenge was to develop a suitable detection system that would allow precise conclusions to be drawn about train arrivals and departures.

Solution: AI-based sensor

Gestalt Robotics has developed an AI-based sensor that can be used to determine the arrival and departure of trains based on visual and acoustic signals. Initial tests were carried out at the Wannsee S-Bahn depot in early 2023. Here, an AI-based algorithm was trained to recognise the closing of doors and the ringing of bells or warning tones on departing S-Bahn trains of the 485 series. An AXIS camera (model Q1656-BLE box camera with microphone) was used as the hardware for image and sound recording.

The neural networks used for the project are designed to learn and generalise patterns from data. They rely on training data to understand the relationships between input and output variables. Without sufficient and relevant training data, the network cannot learn these patterns effectively. With the help of training data, neural networks can generalise from the specific examples they have seen to make predictions for new, unseen data. The more diverse and representative the training data, the better the network's ability to make accurate predictions for a wide range of inputs. With this in mind, a second test was carried out directly at Südkreuz S-Bahn station, where the AI model was trained with further images and data from the field and then evaluated by Gestalt Robotics employees.

A total of 25 commuter trains were monitored with the sensor over a period of two hours. By processing the recorded passages and retraining, it was possible to increase the detection performance on all recorded data from the first and second dates. In order to also improve the detection rate of all train departures, a cascaded detection - a multi-stage procedure - was developed, which no longer looks at the entire image, but only at the area that was previously recognised as a door and a warning light. The advantage: the relevant image sections are significantly smaller and no longer need to be scaled as much.

To summarise, it can be concluded from the evaluation that the chosen combination of camera hardware and the software approach offers promising possibilities. In particular, the possible use of AI-based algorithms directly on the camera enables integrated data recording and analysis. This preserves the privacy of travellers by generating the relevant signals, train stops and departures, directly with the sensor without processing further information or transmitting it to other systems.

The quality of the detection of train arrivals and departures is largely dependent on the amount of training data. It is therefore advisable for the sensor to be used on a permanent basis to record more extensive train arrivals and departures, as AI models and neural networks for image recognition generalise the data known to them through training. All relevant environmental conditions must be mapped in the training data set. This includes not only all objects, in this case suburban railway lines, but also weather influences such as rain and snowfall, lighting conditions during the day and at night, as well as different visibility of the trains due to high passenger volumes or heavy soiling of the trains, for example.

Conclusion:
The AI-based sensor developed by Gestalt Robotics reliably detects all train movements at the station using acoustic and visual signals. It is therefore an important part of the "illuminated platform edge" project to provide passengers with more safety and information at the station. Following successful trials in Berlin, the system is now ready for use at other locations in Germany.

Advantages:

All train arrivals and stops are recognised
Train departures are recognised visually
Train departures can be recognised acoustically
The privacy of travellers is protected

Copyright image 1: Deutsche Bahn AG / Hans-Christian Plambeck
Copyright image 2: Gestalt Robotics GmbH

AI-based sensor for the "illuminated platform edge"