How To Make Autonomous Systems More Transparent and Trustworthy

Products that reach into nearly every corner of our lives are becoming more autonomous, from self-guided vacuum cleaners, to automated industrial material-handling systems, to robotic surgical tools, to driverless vehicles.

But along with their many benefits, autonomous systems also bring the potential to make decisions and act in ways that may lead to unexpected, unwanted, or dangerous outcomes. That means transparency into their inner workings must be an explicit design focus from the outset.

Greater transparency into an autonomous system enables people to better understand its decision-making processes in order to prevent inappropriate actions, and to analyze what went wrong after the fact. Also, because these systems are fast becoming ubiquitous, when society at large is assured their design is transparent, greater overall trust in their use will result.

Take autonomous vehicles as an example. A poll found that 70 percent of people expect autonomous vehicles to stay, but 59 percent said they would be no safer than cars with human drivers.

In contrast to jet planes, whose “black boxes” log key data that investigators depend on to learn why an accident happened, these driverless cars were equipped only with simple event data recorders. The data from these recorders was voluntarily shared with investigators in many instances, but it showed only what went wrong, not why. The inability to understand why things went wrong in order to improve it, makes it difficult to trust one’s life to a driverless car.

Raising the Bar for Transparency 

To help build, measure, and confirm transparency in the design of autonomous systems, IEEE Standards Association (IEEE SA) has recently published a new standard, IEEE 7001™–2021, IEEE Standard for Transparency of Autonomous Systems.

Co-sponsored by the IEEE Vehicular Technology Society, the IEEE Intelligent Transportation Systems Society and the IEEE Robotics and Automation Society, IEEE 7001-2021 describes specific, measurable levels of transparency that can be assessed objectively, and identifies various levels of compliance that can be determined during system design. This standard is intended to be used by designers and operators of autonomous systems, so that they can assess, measure, and improve transparency during the development process.

Because transparency is important to a wide range of stakeholders for different reasons, IEEE 7001–2021 articulates five different levels of transparency for each category of stakeholder, from minimally acceptable to stringent.

Stakeholders of IEEE 7001-2021 include:

• Users of these systems, who need a simple, straightforward way to understand what the system is doing and why;
• Those involved in the validation/certification of an automated system, who need to scrutinize its processes;
• Investigators, who need to trace the internal process(es) that potentially led to a malfunction or accident;
• Expert advisors in administrative actions or litigation, who require transparency in order to inform their advice;
• The broader public, who wants to be informed about AI systems and how they function.

IEEE 7001–2021: Part of the IEEE SA Ethical Design Framework

IEEE 7001-2021 grew out of work started in 2016, when IEEE SA launched a global initiative on the ethics of autonomous and intelligent systems. Its aim was to educate, train, and empower every stakeholder involved in the design and development of autonomous and intelligent systems to prioritize ethical considerations, so that these technologies will be used to improve the human condition as well as for economic growth.

The work led to IEEE SA’s Ethically Aligned Design series of reports, created by more than 700 global experts focused on the pragmatic application of human-centric, values-driven design for a wide range of audiences and stakeholders.

IEEE 7001–2021 complements and extends other IEEE standards and recommended practices dealing with ethical concerns, in particular the IEEE 7000™ series, which focuses on various aspects of ethics in engineering including data privacy, children’s data governance, algorithmic bias, and many others. The IEEE 7000 series addresses specific issues at the intersection of technology and ethics, with the goals of empowering innovation across borders and enabling societal benefit.

IEEE 7001 –2021 standard also complements IEEE SA’s Ethics Certification Program for Autonomous and Intelligent Systems (ECPAIS), used to create specifications for certification and marking processes that advance transparency, accountability, and reduction in algorithmic bias in autonomous and intelligent systems. It helps consumers and citizens understand whether a system is deemed “safe” or “trusted” by a globally recognized body of experts who have provided a publicly available and transparent series of marks.

Through ECPAIS, IEEE SA has launched the IEEE CertifAIEd™ program to help enable, enhance, and reinforce trust through ethically tenable AI systems. The program comprises technical standards, training, criteria, and certification covering areas such as transparency, accountability, algorithmic bias, and privacy.

To learn more:

• Visit the IEEE 7001-2021 standard page
• Visit the IEEE CertifAIEd program page
• Explore other IEEE 7000 standards and projects

Authors:

• Alan Winfield, Chair, IEEE 7001 Autonomous Systems Validation Working Group, [email protected]

• Eleanor (Nell) Watson, Vice Chair, IEEE 7001 Autonomous Systems Validation Working Group, [email protected]