Safety Is a System Property
- Categories
- Systems
Safety, and reliability with it, is an emergent property of a system as a whole, not something stored in any component. You cannot make a system safe by making each part safe in isolation, because most behavior comes from how the parts interact.
Why it Matters
It redirects effort from hardening individual components to shaping the interactions, defenses, and conditions of the whole. It also explains why a system built from reliable parts can still fail, and one built from imperfect parts can stay safe. The same holds for security: Threat Modeling treats it as a property of the whole system, designed in by reasoning about how the parts interact and where trust changes, rather than bolted on as a feature.
Signals
- Component-level metrics all green while the system still fails.
- "Every part passed its test."
- Safety treated as a per-component checklist rather than a property of the whole.
Benefits
Investment aimed where outcomes actually come from, recognition of emergent risk, and better whole-system testing and review.
Risks
Chasing per-component reliability while the interactions go unexamined; assuming the sum of safe parts is a safe system.
Tensions
Components must still be made reliable, but local reliability is necessary, not sufficient; effort spent locally can crowd out attention to the whole.
Examples
A system where each microservice meets its SLA but a retry storm across them causes an outage; security that is strong per host yet broken by how the hosts trust each other.