Cars may steer directly into what it is supposed to avoid
- Joint study finds emergency lights can confuse ADAS
- Results point to “confidence score” fluctuations
- Researchers pointed to real-world collisions
Beyond the obvious problems of this behaviour, which would cause further calamity at a scene where there has already been an incident, the spectre of bad actors was also floated by the authors of this study, saying it could be exploited by adversaries to intentionally cause such crashes.
One of the insights provided by the team suggests the light emitted from emergency vehicle lighting can change the distribution of the colours of the car in the captured frame. This refers to what the system picks up by its computer brain while driving before interpreting it as another car, a pedestrian, or a yeti. This variance changes the confidence of the object detector, like when a human is questioning themselves about what they are seeing.
Here’s the good thing about humans: sure we’re fallible, but the chances of most reasonable people seeing a cop car with its lights blazing and not recognizing it as something around which we should steer, is pretty low. A computer, after all, is only as good as it inputs. If software running the show misinterprets what it is ‘seeing’, then the chance of it making poor decisions rise exponentially. Garbage in, garbage out.
To be clear, the study wasn’t just picking on Tesla. Researchers found the phenomenon to be consistent across different automated driver assistance systems (ADAS), with fluctuation ranges and stability when the flashing light was off differing depending on the type of ADAS recording the video frames.
In other words, keep your hands on the wheel and your eyes on the road. And keep your stick on the ice.
Sign up for our newsletter Blind-Spot Monitor and follow our social channels on Instagram ,Facebook and X to stay up to date on the latest automotive news, reviews, car culture, and vehicle shopping advice.