Sensor data fusion digitizes freight train operation

Traffic & Infrastructure

LiDAR supports track monitoring for safe and efficient shunting of freight trains

Overview

To monitor the track area when shunting freight trains even when reversing, a consortium consisting of five partners, including FH Aachen, is developing a system for track monitoring – the SAMIRA Shunting Assistant System. This involves fusing LiDAR point clouds with camera data to detect objects in the track reliably. This digitization step saves personnel resources and, at the same time, lays the foundation for autonomous rail operations.

Mobile Assistance System for shunting freight trains

Challenge

Until now, shunting freight trains has been a two-person job: If the train moves backward, a shunting assistant has to stand on the last wagon and radio the locomotive driver to inform them whether there are any obstacles in the track area. This task has to be performed in wind and weather and is also not entirely risk-free – making it difficult to find personnel for this position. Although the locomotive drivers can take on this task themselves via remote control, this is not an optimal solution either, since they must walk along the entire length of the train when changing direction, which is very time-consuming.

Solution

Therefore, the project team is developing an assistance system for shunting freight trains that takes over track monitoring and thus supports the locomotive drivers. This portable, battery-powered device is attached to the end of the train. It contains a wide range of sensors for detecting the environment and a computer for data processing. The integrated sensors include two Blickfeld LiDAR sensors for 3D obstacle detection, a stereo camera for close-range detection, two additional cameras that provide the data basis for algorithm-based object detection, and a radar sensor. In addition, the system has an acoustic warning signal. The locomotive driver is equipped with a tablet in the cab that shows the track area behind the train in the live feed and indicates objects, people, signals, or other vehicles in the monitored area.

These objects are detected by fusing camera images and 3D point clouds: image recognition algorithms identify the track area and objects such as people or vehicles based on the camera image. However, they cannot determine the exact position of detected objects; only the fusion with the point cloud enables this step. This way, it can be ensured that the train’s path is clear when reversing, without a shunting assistant.

Fusion of camera image and LiDAR 3D point cloud

FH Aachen, responsible for the LiDAR and camera systems, decided to integrate Blickfeld sensor technology into the system for several reasons. For shunting applications, the Cube 1’s horizontal field of view of 70° is ideal for keeping an eye on the track area even when driving slight curves. The sensor technology must also be highly robust, as vibration is very high in railroad operations; the solid-state character of the Blickfeld sensors is a great advantage here. Furthermore, the high resolution of the point cloud data at a low price stands out.

Blickfeld's LiDAR sensor technology is exactly what we were looking for in terms of price, resolution, and field of view.

Matthias Blumenschein

Research Assistant Rail Vehicle Engineering, FH Aachen University of Applied Sciences

Outlook

To safeguard shunting operations even more comprehensively, there are project concepts for introducing stationary modules for track inspection in a next step. Placed behind curves or other sections that are difficult for the driver to see, the system could send a live feed to the driver’s cab so that braking can be initiated in sufficient time if necessary. In a second stage, this technology could serve as a first step toward autonomous operation.

Products Used

Cube 1