What still needs to be done? The interconnection of the sensors requires further optimisation, and the sensors themselves still have potential for improvement. For instance, they are still highly dependent on the weather, with great challenges being posed by heavy rain, snowfall and consequent fouling of the sensors. “Another problem, strangely enough, is traffic lights, which must under no circumstances be misinterpreted,” explains Prof. Dr Ralf G. Herrtwich, Head of Vehicle Automation and Chassis Systems at Mercedes-Benz. If there are many sets of lights at a junction, the challenge is to identify which lights are relevant for the vehicle. This also requires the detection of arrows that are just a few pixels in size from a certain distance. There is often also the problem of backlight. “This is where we are at the technical limit of what present-day sensors can detect,” adds Herrtwich.