What security engineering is
An introduction to security engineering as a discipline, and why it differs from safety and reliability engineering.
- Define security engineering and say what makes it different from ordinary software engineering.
- Distinguish a security failure from a safety failure and from a reliability failure.
- Explain why a system must be designed to withstand an adversary, not just an accident.
8 min read
Intuition
Most engineering disciplines design against chance and wear: a bridge against wind and traffic load, a server against random hardware failure. Security engineering designs against something else — an opponent who studies the system, looks for its weakest point, and picks the moment of failure on purpose. A system that never breaks by accident can still be broken in seconds by someone who knows exactly where to push.
Mechanism
A security engineer treats every interface, input and trust boundary as a place an adversary might act, then asks what that adversary gains by acting there and what stops them. This is why the same system is described in terms of policy (what is and is not allowed), mechanism (what enforces the policy) and assurance (why anyone should believe the mechanism actually works). A control without assurance is a hope, not a defence.
Compare
Holds up against a deliberate, adaptive adversary who chooses when and how to attack.
Holds up against chance: wear, random failure, unintentional misuse. No opponent is assumed.
Exam detail
Security engineering is commonly defined as building systems that remain dependable in the face of malice, error, or mischance. All three matter — a system that only defends against malice but falls over on an honest mistake has not solved the problem either.
Threat
An attacker who finds one unvalidated input on an otherwise well-built system can use it to bypass every control built on top of it. The system is only as strong as its weakest enforced point, not its average one.
Control
Reducing the attack surface — fewer exposed interfaces, fewer trusted paths, least privilege by default — shrinks the set of places that weakest point can be. It does not remove the need for assurance on whatever remains.
Pitfall
Do not write “secure” when you mean “bug-free,” and do not write “reliable” when you mean “secure.” A reliability failure and a security failure can look identical in a bug report and be caused by completely different things — one by chance, one by intent.
Recall
Why can a system be reliable but not secure at the same time?
Reliability is about resistance to chance and wear; security is about resistance to a deliberate, adaptive adversary. A system can be engineered to never fail on its own and still fail immediately against someone looking for the one input it was never tested against.
Formula
Rational attacker threshold
- cost to the attacker of carrying out the attack
- value to the attacker of what the attack yields
An illustrative model of deterrence, not a value from the lecture: a purely rational attacker only proceeds when the expected gain exceeds the cost. Real attackers are not always rational, which is itself a reason defences cannot rely on cost alone.
Worked example
AnswerConfidentiality and integrity
A worked example of naming which security goals apply to a scenario.
- Scenario: an attacker reads a password file they should not have access to, and changes one entry in it.
- Reading data they are not authorised to see is a breach of confidentiality.
- Changing data without authorisation is a breach of integrity.
- No availability goal is affected — the file is still accessible to legitimate users.
- The scenario therefore breaches confidentiality and integrity, not availability.
Aside
“Engineering” is the operative word: security engineering borrows failure analysis from safety engineering and risk framing from economics, but neither discipline on its own accounts for an opponent who adapts to whatever defence is put in front of them.
Source
Week 1 Notes PDF