Hit enter to search or ESC to close

A Successful Independent Audit of Argo AI’s Test Driver Procedures Signals a Benchmark for the Autonomous Vehicle Industry

Argo AI meets standards as outlined by the Autonomous Vehicle Safety Consortium and SAE International’s J3018

For over a century, organizations like SAE International (formerly, the Society of Automobile Engineers), have existed to provide the automotive industry with safety standards and best practices. The logic behind SAE is simple, and vital: vehicles, when manufactured incorrectly, can pose a serious threat—so let’s ensure that anyone building them does so in a way that’s safe and standardized. 

With respect to these standards, self-driving technology developer Argo AI successfully underwent a voluntary external audit from TÜV SÜD, a world-leading certification body for safety in the autonomous vehicle industry, which verified Argo’s procedures relating to autonomous vehicle test drivers. The guidelines—detailed as best practices released by the Automated Vehicle Safety Consortium (AVSC) and a standard by SAE (titled J3018)—cover everything from autonomous test driver selection, basic training, operation and oversight procedures for autonomous vehicles “under test” (AVs actively driving on city streets), and periodic re-evaluation and training.  

In short, both of these documents aim to ensure that any human test driver at the wheel of an AV is properly screened, trained, and prepared to handle anything that comes their way on the road. Despite the fact that these guidelines have been publicly available to all AV manufacturers for over two years, none have allowed a third party to verify adherence to these best practices—until now.   

The TÜV SÜD Audit

TÜV SÜD began the audit by examining Argo AI’s Test Specialist (TS) hiring standards and training process. Argo shared the details of their intensive four-week training course, which involves classroom learning and practical driving training. The latter includes “fault injection” training, the process by which Argo simulates unexpected scenarios on their test track to prepare a TS to react appropriately to edge-cases they encounter out in the real world.

Next, the audit reviewed how Argo prepares a fully-trained TS for their daily mission, starting with Argo’s “pre-mission briefings,” the safety procedures that every TS undergoes each day before getting behind the wheel. Then, Argo shared their “on-mission protocols” during drives, including mandating that their drivers take regular breaks to recoup energy and retain focus. TÜV SÜD also made note of Argo’s driver monitoring system protocol: the in-vehicle dashboard camera that tracks a TS’s gaze, that drivers are alerted to help avoid losing focus on the road. 

Finally, Argo detailed their guidelines for safely returning a vehicle to their facilities at the end of the day. They also shared their “post-mission protocols,” including how a TS gives feedback on their vehicle’s performance in their shift summaries, and how a TS logs and escalates system issues to their operations managers, who in turn communicate the findings to the engineering team.

The Results

In the end, TÜV SÜD confirmed that Argo meets, and in some cases exceeds, the best practices and standards as outlined by the AVSC and SAE International’s J3018, citing Argo’s “Autonomous Vehicle System Test Specialist” driver selection, training, and monitoring programs as being compliant with these applicable standards.

“TÜV SÜD’s findings reaffirmed that it’s not only Argo employees that really believe in what our training does,” says Systems Engineering Manager, Shadi Jammoul, who was critical in implementing the safety processes. “We now have validation from an external body that specializes in this area.” Todd Fronckowiak, Argo’s Director of Safety Policy and Assurance, adds that this external validation “raises the bar and sends an important message about how Argo values safety.” 

While the public nature of the audit is meaningful for industry standard-setting, for Doug Hutchinson, Operational Performance Manager at Argo, an equally positive outcome of their success is what it communicates to the rest of the company. “I’m extremely proud of the Fleet Operations team,” he says. “Internally at Argo, the success of the evaluation just continues to show our commitment to safety. It’s always good to see that, and to continue to have the confidence that safety is our number one value.” 

Choose your lane

How Autonomous Vehicles Distinguish Between Bicycles and People Who Ride Them

How Autonomous Vehicles Distinguish Between Bikes and People

When it comes to how autonomous vehicles see the world, humans come first, literally. Autonomous vehicles (AVs), like the kind operated by Pittsburgh-based Argo AI, use Machine Learning to detect and classify the objects in their surroundings, identifying people...
Why The League of American Bicyclists is optimistic about autonomous vehicles

Why a Leading Cycling Advocacy Group Is Optimistic About Autonomous Vehicles

As autonomous vehicle use grows, AV companies and the League of American Bicyclists are collaborating on how to ensure cyclists and motorists can share the roads safely, even if the “motorist” is artificial intelligence software. As part of the...

Self-Driving Is Arriving Right On Time. Just Like Ice Cream Did

Seven years ago, I was a self-driving skeptic. Not of the technology. Of all the “experts” promising autonomous vehicles would be everywhere by 2020. You didn’t need to be Nostradamus to know that was ridiculous. All you needed was...
Illustration of a futuristic parking deck turned into a mixed-use space, with AVs driving by

How Autonomous Vehicles Could Help Transform Parking Lots

Researchers say it’s likely that autonomous vehicles (AVs) can help reduce the need for parking lots, opening more room for grass and trees and other elements of nature. It may not seem like it when you’re circling the block...
An illustration of an Argo autonomous vehicle in teal against a backdrop of buildings, a bicyclist, and research papers

7 Big Breakthroughs From Argo Research at CVPR 2022

The 2022 Conference on Computer Vision and Pattern Recognition (CVPR 2022) is nearly here. Thousands of computer scientists, software engineers, and researchers from around the globe will gather in New Orleans to review and discuss their latest work in...

Researchers Predict the Future With Lidar Data

Researchers at Carnegie Mellon University’s Argo AI Center for Autonomous Vehicle Research, a private-public partnership funded by Argo for advancing the autonomous-vehicle (AV) field, say they have come up with a way to use lidar data to visualize not...

Must Reads