F Perkins on Autonomous System Safety by Phil Koopman

Thanks. Sometimes overlooked is that increased reliance on AI/ML and the opacity it causes means diminished reliance on the traditional paradigm of cause and effect. Physical mechanisms and most software can be validated by proof of bidirectional relationships between requirements and outputs. AI/ ML cannot. Instead, validation of AI/ML relies on statistical cross-correlation analysis of inputs vs. results. Neither industry nor government standards exist for reliability and statistical confidence of safety-critical capabilities of AVs reliant on AI/ML technology. Extraordinary claims need extraordinary proof. Industry claims that abandoning reliance on the simple notion of ‘cause and effect’ for validation of safety-critical AV functions in favor of murky AI/ML aspirations is in the best interests of public safety. They have not.