For over a century, organizations like SAE International (formerly, the Society of Automobile Engineers), have existed to provide the automotive industry with safety standards and best practices. The logic behind SAE is simple, and vital: vehicles, when manufactured incorrectly, can pose a serious threat—so let’s ensure that anyone building them does so in a way that’s safe and standardized.
With respect to these standards, self-driving technology developer Argo AI successfully underwent a voluntary external audit from TÜV SÜD, a world-leading certification body for safety in the autonomous vehicle industry, which verified Argo’s procedures relating to autonomous vehicle test drivers. The guidelines—detailed as best practices released by the Automated Vehicle Safety Consortium (AVSC) and a standard by SAE (titled J3018)—cover everything from autonomous test driver selection, basic training, operation and oversight procedures for autonomous vehicles “under test” (AVs actively driving on city streets), and periodic re-evaluation and training.
In short, both of these documents aim to ensure that any human test driver at the wheel of an AV is properly screened, trained, and prepared to handle anything that comes their way on the road. Despite the fact that these guidelines have been publicly available to all AV manufacturers for over two years, none have allowed a third party to verify adherence to these best practices—until now.
The TÜV SÜD Audit
TÜV SÜD began the audit by examining Argo AI’s Test Specialist (TS) hiring standards and training process. Argo shared the details of their intensive four-week training course, which involves classroom learning and practical driving training. The latter includes “fault injection” training, the process by which Argo simulates unexpected scenarios on their test track to prepare a TS to react appropriately to edge-cases they encounter out in the real world.
Next, the audit reviewed how Argo prepares a fully-trained TS for their daily mission, starting with Argo’s “pre-mission briefings,” the safety procedures that every TS undergoes each day before getting behind the wheel. Then, Argo shared their “on-mission protocols” during drives, including mandating that their drivers take regular breaks to recoup energy and retain focus. TÜV SÜD also made note of Argo’s driver monitoring system protocol: the in-vehicle dashboard camera that tracks a TS’s gaze, that drivers are alerted to help avoid losing focus on the road.
Finally, Argo detailed their guidelines for safely returning a vehicle to their facilities at the end of the day. They also shared their “post-mission protocols,” including how a TS gives feedback on their vehicle’s performance in their shift summaries, and how a TS logs and escalates system issues to their operations managers, who in turn communicate the findings to the engineering team.
The Results
In the end, TÜV SÜD confirmed that Argo meets, and in some cases exceeds, the best practices and standards as outlined by the AVSC and SAE International’s J3018, citing Argo’s “Autonomous Vehicle System Test Specialist” driver selection, training, and monitoring programs as being compliant with these applicable standards.
“TÜV SÜD’s findings reaffirmed that it’s not only Argo employees that really believe in what our training does,” says Systems Engineering Manager, Shadi Jammoul, who was critical in implementing the safety processes. “We now have validation from an external body that specializes in this area.” Todd Fronckowiak, Argo’s Director of Safety Policy and Assurance, adds that this external validation “raises the bar and sends an important message about how Argo values safety.”
While the public nature of the audit is meaningful for industry standard-setting, for Doug Hutchinson, Operational Performance Manager at Argo, an equally positive outcome of their success is what it communicates to the rest of the company. “I’m extremely proud of the Fleet Operations team,” he says. “Internally at Argo, the success of the evaluation just continues to show our commitment to safety. It’s always good to see that, and to continue to have the confidence that safety is our number one value.”
