Hit enter to search or ESC to close
Self-Driving

Watch and Learn: An Engineer Explains How a Self-Driving Car Instantly Brakes for a Cyclist

“They came out of nowhere!” You’ve probably heard this phrase uttered while riding in a car (maybe you’ve even said it yourself) if another vehicle, pedestrian, or bicyclist appears suddenly in front of your vehicle without warning. 

Safe drivers will react by slowing, stopping, or moving their car appropriately to avoid contact with the fellow road user who took them by surprise, using their senses, reflexes, and experience/driving education to help guide their actions. 

But what about a self-driving car? How does it respond to this common real-world driving scenario? And what processes or calculations go into its reaction?

Watch how Argo AI’s self-driving test vehicle safely navigates an unpredictable scenario in Miami, Florida and read about the technological process behind it with explanation from Peter Carr, Argo VP of Autonomy and Software Engineering

The Situation

As shown in the video, the Argo AI test car is driving in autonomous mode down a two-way, two-lane street. It comes upon an intersection with a four-way stop, brakes accordingly to a standstill, and begins creeping forward again, slowly to ensure perpendicular traffic is clear — before a cyclist quickly emerges from behind a prominent building on the car’s right side and rides down the sidewalk out in front of it. The Argo car reacts safely by braking, coming to a prompt yet gentle halt, allowing the cyclist to pass from right to left with plenty of space between them and the Argo car, all while keeping the car’s occupants comfortable. Then, the Argo car continues on its way. 

What is going on inside the car, in Argo’s self-driving system (SDS) — the collection of hardware sensors, computers, and software that enable autonomous driving — that enables it to respond so effectively and safely in this type of fast-moving situation? Carr explains the basic elements below.

Sensing 

First, the Argo car instantly detects the bicyclist traveling quickly on the sidewalk, and classifies them as a cyclist, rather than another vehicle, pedestrian, animal, or part of the environment, such as a tree branch or piece of roadway infrastructure. Cyclists are deemed“vulnerable road users,” (VRUs) because they lack the protection offered by a car, truck, or other larger vehicle body. Pedestrians, motorcyclists, and scooter riders are also VRUs.

The detection is done first by using sensor fusion, the process of combining data streams from multiple sensors — in this case, namely, cameras and lidar mounted outside of the car — and then running them through Argo’s perception system, which compares it to a database of tagged imagery from previous test rides, to understand rapidly, in fractions of a second, that this is a 3D object, that the object is actually two things, a person and a bicycle, and that it is moving at a specific velocity and direction. 

“The camera sees that there is a bicycle and rider on the sidewalk.  Similarly, the lidar senses a point cloud and agrees, ‘this must be a bicyclist,’” Carr says. “The SDS also determines the bicyclist’s speed and direction of travel. A person riding on or walking the bike creates a difference in the software’s response.” 

Planning 

It’s not enough to simply see and understand that a bicycle is moving in front of the car. A safe self-driving system must also be able to anticipate where the bicycle is likeliest to move next, as well as additional paths it could take and the probability of it taking those paths, and finally, how the car itself should respond accordingly in each of the ways the cyclist may move

In the Argo SDS, this series of tasks is the responsibility of the motion forecasting software, a collection of sophisticated, fast algorithms that calculates, based on what type of vehicle or objects the SDS has detected, their current speed and direction, and where they could and are most likely to move several seconds into the future. And, depending on where the cyclist in this case actually does move, the motion forecasting software updates its predictions multiple times per second to account for numerous potential paths of every object. 

Argo’s SDS is specifically attuned to bicyclists by anticipating them riding on the road as a vehicle would, or across sidewalks where a pedestrian would walk, as in this example. 

“The Argo SDS is built to forecast transitions between road and sidewalk,” Carr says. “Even if it’s not completely obvious whether the cyclist has seen the autonomous vehicle (AV) or not, or is planning to stop or not, the Argo SDS has forecasted both of those futures — and many others.” 

Acting 

Once the Argo SDS has identified what it sees as a cyclist, and forecasted their possible paths of motion and which is most likely, it rapidly determines what, if any, driving actions should be completed.

In this particular case, the Argo SDS brakes and waits for the cyclist to ride through the intersection. But, as is the case with human drivers, the Argo SDS must also answer the question of how hard to apply the brakes?

“It’s a combination of safety and ride comfort — can it come to a stop without slamming on the brakes?” Carr explains. “If so, the car does that. You want to make sure the SDS does exactly the amount of braking necessary to be safe and no more. If the situation was different and the cyclist rode faster or closer, the car would undertake harder braking. But smoother braking is preferred whenever it’s safe to give vehicles behind the AV a chance to stop as well.” 

Furthermore, the Argo SDS also waits to resume driving until the bicyclist passes fully through the intersection and is far enough away that if they were to veer left or right, they would not encroach on the forward path of the car. 

Keeping Cyclists and Other Road Users Safe

As this video and Carr’s explanations illustrate, Argo keeps cyclists and other road users safe through a thoughtful, comprehensive, and holistic approach — taking into account the cyclist and the car, as well as the entire scene around them. It does not rely on any particular idealistic cyclist behavior or set of behaviors, but rather on the real-world range of potential behaviors any cyclist can make along their ride. 

Argo’s SDS has been designed with contributions from cyclists on Argo’s staff and other valued advisors, like the League of American Bicyclists. With the help of this input, the SDS follows a set of principles for pedestrian and cyclist safety that prioritizes watching out for them and other vulnerable road users. 

Cyclist safety is just one important part of designing self-driving technology you can trust. For more on Argo’s core safety principles, you can read the 2021 Argo Safety Report

Choose your lane

How Autonomous Vehicles Distinguish Between Bicycles and People Who Ride Them
Self-Driving

How Autonomous Vehicles Distinguish Between Bikes and People

When it comes to how autonomous vehicles see the world, humans come first, literally. Autonomous vehicles (AVs), like the kind operated by Pittsburgh-based Argo AI, use Machine Learning to detect and classify the objects in their surroundings, identifying people...
Why The League of American Bicyclists is optimistic about autonomous vehicles
Self-Driving

Why a Leading Cycling Advocacy Group Is Optimistic About Autonomous Vehicles

As autonomous vehicle use grows, AV companies and the League of American Bicyclists are collaborating on how to ensure cyclists and motorists can share the roads safely, even if the “motorist” is artificial intelligence software. As part of the...
Opinion

Self-Driving Is Arriving Right On Time. Just Like Ice Cream Did

Seven years ago, I was a self-driving skeptic. Not of the technology. Of all the “experts” promising autonomous vehicles would be everywhere by 2020. You didn’t need to be Nostradamus to know that was ridiculous. All you needed was...
Illustration of a futuristic parking deck turned into a mixed-use space, with AVs driving by
Business

How Autonomous Vehicles Could Help Transform Parking Lots

Researchers say it’s likely that autonomous vehicles (AVs) can help reduce the need for parking lots, opening more room for grass and trees and other elements of nature. It may not seem like it when you’re circling the block...
An illustration of an Argo autonomous vehicle in teal against a backdrop of buildings, a bicyclist, and research papers
Self-Driving

7 Big Breakthroughs From Argo Research at CVPR 2022

The 2022 Conference on Computer Vision and Pattern Recognition (CVPR 2022) is nearly here. Thousands of computer scientists, software engineers, and researchers from around the globe will gather in New Orleans to review and discuss their latest work in...
Self-Driving

Researchers Predict the Future With Lidar Data

Researchers at Carnegie Mellon University’s Argo AI Center for Autonomous Vehicle Research, a private-public partnership funded by Argo for advancing the autonomous-vehicle (AV) field, say they have come up with a way to use lidar data to visualize not...

Must Reads