System Safety as an Emergent Property in Composite Systems

Decomposition is used to manage system complexity, but is problematic for emergent properties such as system safety. Previously, we introduced Indirect Control Path Analysis (ICPA) for elaborating system safety goals in composite systems. We now provide mathematical definitions of emergent and composable system behaviors in the context of formal specifications and ICPA, and identify useful special cases in which partial decomposition of emergent safety goals is possible. We apply ICPA to a semi-autonomous automotive system to identify safety goals for key subsystems, and then monitor the system and subsystem goals at run-time in an implementation of the vehicle. Although false negatives at the subsystem level indicate the subgoals do not fully compose the original safety goal, some system-level goal violations are detected by subsystem monitors. In addition, monitoring at both the system and subsystem level has identified certain safety-related errors that may be imperceptible to system testers.