When it comes to how autonomous vehicles see the world, humans come first, literally.
Autonomous vehicles (AVs), like the kind operated by Pittsburgh-based Argo AI, use Machine Learning to detect and classify the objects in their surroundings, identifying people first and then their accessories. For example: a person and a bike. A person and a stroller. A person and a scooter.
This is one of the approaches that the Argo Autonomy Platform – the hardware and software that makes up the self-driving system – uses to learn about and process its environment.
Ishan Khatri, a software engineer on Argo’s Perception Team, spends most days collaborating with his team and others to improve the way the Argo Autonomy Platform understands its surroundings. While there are many methods to tackle environmental understanding through learning, Khatri said those who study such things agree that a layered approach, one in which the AV categorizes objects individually and then defines them as having a relationship, is powerful.
“People will generally avoid having these compound classes, and combining them together [into a single class] makes it harder for the machine learning model to deal with it,” he said.
One of the reasons this methodology works is that cyclists may not always be on their bikes, an insight that impacts safety and performance. A cyclist, for example, may walk their bike through an intersection, which would modify the speed at which an autonomous vehicle would predict the cyclist to clear the space. A cyclist may also decide to park their bike. Without differentiating the bike from the rider in that scenario, an autonomous vehicle may find it more difficult to predict the behavior of a stationary object that it knows can move.
AVs also need to see different kinds of people and bikes. That includes children on tricycles, adults on recumbent bikes, people on three-wheeled beach cruisers, tandem biking pairs, and so on. Seeing the person first and then considering the person in relation to the object near it means that AVs can always perceive a person, regardless of the kind of bike that person is using.
Without creating these compound classes of vulnerable road users, a term of art for those in the transportation field, even something like a bike rack could stump an autonomous vehicle.
“Before we were keeping track of these two categories separately, bike racks could cause problems. Because it had a bunch of bikes, and we know bikes can move, but they’re not going to move in this context,” Khatri said. “Differentiating between those two things is important.”
Lidar that looks in all directions
AV technology needs to be able to accurately calculate the distance to nearby objects, understand what kind of objects they are, and respond to the behavior of those objects. And since objects, like bikes, strollers and scooters, are controlled by people, it’s important for AVs to detect both humans and their objects as separate entities that sometimes, but not always, move together.
“Knowing where people are going is really important to knowing where our AV can go and not go,” Khatri said. “It’s more important to know where it is than what it is.”
At Argo, one of the ways of knowing where and what begins with lidar. Similar to how radar uses radio waves and sonar uses sound waves, lidar is a sensor that relies on a laser beam to examine objects in its surroundings and determine the distance to those objects.
Argo Lidar is used in tandem with radar, cameras and two on-board computers — one primary and one for redundancy. It has a range of up to 400 meters, equal to just over 1,300 feet, and conducts a 360-degree scan of its surroundings multiple times per second.
“It’s always looking in every direction. A single camera can’t do that,” Khatri said. “We can look left and right at the same time, which is kind of cool.” Of course, a human can’t do that, either.
All this matters more than ever as preventable deaths from bicycle transportation have increased over the past decade by 44 percent, according to the National Safety Council, a group that describes itself as “America’s leading nonprofit safety advocate.” Data collected and analyzed by Outside magazine found that deadly crashes for cyclists were pretty evenly distributed between urban, suburban and rural areas. However, the most dangerous of the three areas is on an arterial road — with traffic lights, higher rates of speed and multiple lanes of traffic.
And with more cyclists on the road, thanks to a sharp increase in interest precipitated by COVID-19 lockdowns, data suggests that motorists and cyclists stand to see more interactions with each other than ever before. It’s why comprehensive detection, reliable prediction and responsive action are critical for Argo’s autonomous vehicle technology.
“Before we take any action, we’ve already checked 360 degrees around the car,” Khatri said. “We can tell if velocity changes because we have a measurement of speed many times every second. We apply the brakes and chill. We’re fine with that.”
Fine-tuning safety
Argo’s autonomous vehicle platform is always learning and improving. At Argo, Khatri is one member of one team of engineers doing a portion of the work that facilitates that learning process. Among many other such organizations, there are teams of engineers focused on 3D computer vision and teams of complementary engineers specializing in predicting behavior.
Together, along with other teams of expert software engineers, all these people are facilitating a safer environment for everyone using the road, from drivers to people and bikes.
“We really believe AVs can be safer than humans because they’re always looking everywhere, and they don’t get distracted,” Khatri said. “You can’t look in all directions around you at once, but an AV can.”
