Machine learning models are widely used in the computer vision do main owing to their improved prediction accuracy in comparison with non-machine learning approaches. Vulnerabilities (prediction error or bias caused by external attacks and data quality deficiencies) of those models are critical to AI system providers and end-users, since the model vulnerabilities will affect the effectiveness of AI systems. Academic approaches and tools for model vulnerability tests can vary in terms of applicable scenarios, settings and test qualities. For end-users, it is difficult to correctly choose, combine, and apply the vulnerability tests without a systematic framework introducing principles, approaches' characteristics, trade-off and metrics. In addition, key aspects, including test termination condition and completeness are important for test quality, but often failed to be applied by testers, due to the lack of specifications. This document provides recommended practices for vulnerability testing of machine learning models for computer vision applications. It covers testing approaches and operational contexts for primary tasks including object detection, semantic segmentation, classification, image / video modification and understanding.
- Standard Committee
- C/AISC - Artificial Intelligence Standards Committee
- Status
- Active PAR
- PAR Approval
- 2022-03-24
Working Group Details
- Society
- IEEE Computer Society
- Standard Committee
- C/AISC - Artificial Intelligence Standards Committee
- Working Group
-
VTCV - Vulnerability Test for Machine Learning-based Computer Vision Applications
- IEEE Program Manager
- Christy Bahn
Contact Christy Bahn - Working Group Chair
- Xiaoqi Cao
Other Activities From This Working Group
Current projects that have been authorized by the IEEE SA Standards Board to develop a standard.
No Active Projects
Standards approved by the IEEE SA Standards Board that are within the 10-year lifecycle.
No Active Standards
These standards have been replaced with a revised version of the standard, or by a compilation of the original active standard and all its existing amendments, corrigenda, and errata.
No Superseded Standards
These standards have been removed from active status through a ballot where the standard is made inactive as a consensus decision of a balloting group.
No Inactive-Withdrawn Standards
These standards are removed from active status through an administrative process for standards that have not undergone a revision process within 10 years.
No Inactive-Reserved Standards