Fundamental design principles

Nobody has found a way to design software that systematically excludes every security flaw. What the field has instead is a set of principles, tested across decades of systems that failed and systems that held, for building protection that fails in smaller, safer ways. None guarantees security on its own; each closes off one specific way things tend to go wrong. Week 1 lists thirteen, each given below with the lecture’s own example.

Keep protection small, default to refusing, and check every access

Economy of mechanism. Security measures in hardware and software should be as simple and small as possible. A smaller design is easier to test and verify thoroughly; a complex one gives an adversary more chances to find subtle weaknesses. UNIX was designed modularly with a small codebase, and that philosophy carried into Linux and macOS. Windows Vista shipped with many more complex features and a much larger codebase than its predecessors, and the larger attack surface gave adversaries more places to look.

Fail-safe defaults. Access decisions should be based on permission rather than exclusion: the default is no access, and the scheme states explicitly when access is granted. An error in the rules then denies access instead of leaking it. Operating system file permissions work this way. The POODLE attack is the counter-example: it forced a connection to downgrade to the older, weaker SSL 3.0 by making negotiation for a stronger protocol fail, then exploited SSL 3.0’s known weaknesses. Failed negotiation should mean no connection, not a weaker one.

Complete mediation. Every access must be checked against the access control mechanism, not served from a cached decision. In a system that runs continuously, consider carefully how a change in authority propagates into any local memory of past decisions. Ordinary file systems do not fully satisfy this: once a file is open, most do not recheck permissions on each read. Checking access on every field or record is resource-intensive and rarely done. Zero Trust Networks are the lecture’s example: every access request is validated against policy regardless of the user’s prior authentication, and no trust is granted because of network location or an earlier login.

Open designs get reviewed; secret ones get broken

Open design. A security mechanism’s design should be open to scrutiny rather than kept secret. Encryption keys must stay secret, but algorithms should be open so experts can review them and users can trust that review. This is the philosophy behind NIST’s standardisation of encryption and hash algorithms. Kerckhoffs’s principle is the cryptographic version: a system should remain secure even if everything about it except the key is public. The Content Scramble System on DVDs kept its algorithm secret, which prevented expert review. In November 1999 Frank A. Stevenson published exploits that made the cipher practically ineffective, and a separate group reverse-engineered CSS into the DeCSS program. Secrecy did not make the design stronger; it only delayed discovery of how weak it already was.

Limit what any one actor or component can do

Separation of privilege. Access to a restricted resource should require multiple independent privilege attributes, not one. Multi-factor authentication (a password and a smart card, say) is the common example. The term also covers splitting a program so a high-privilege operation runs in its own process, separate from the lower-privilege process handling day-to-day interaction.

Try it — worked example

On Berkeley-derived UNIX, switching to root needs two separate conditions. What are they, and is meeting one of them enough?

Show answer and steps

AnswerMeeting either condition alone is not enough: both are required, together.

1. On Berkeley-derived UNIX systems, switching to the root account requires two separate conditions.
2. Condition one: the user must know the root password.
3. Condition two: the user must be a member of the wheel group (the group with GID 0).
4. Knowing the password without wheel membership is not sufficient to gain root access.
5. Being in the wheel group without the password is equally insufficient.
6. Only satisfying both conditions together grants root access. That is separation of privilege in practice.

Least privilege. Every process and user should operate with the least set of privileges necessary for its task. Role-based access control is the standard mechanism: each role gets only the permissions its function needs, and nothing is granted beyond an explicit permission. There is a temporal side too: administrators should hold elevated privileges only while they need them, since leaving them in place invites accidents even without malice. Apache’s web server runs as the www-data account, which can read files from the web directory and write to specific log files and nothing more, so a compromised web server process is contained to what www-data can reach.

Pitfall

Separation of privilege and least privilege get confused because both restrict access. Separation of privilege is about how many independent things are needed before an action can happen at all: the wheel group example needs two. Least privilege is about how much any single actor holds once it is authorised. A system can enforce one without the other.

Recall

A company's backup admin keeps full production database access permanently, even though they only need it during the monthly backup window. Which principle is being violated, and how would you fix it?

Least privilege, specifically its temporal aspect. The fix is withdrawing that access outside the backup window rather than leaving it in place at all times: privileges held longer than the task requires create risk even without any malicious intent behind the access itself.

Least common mechanism. Minimise the functions shared between different users, so the shared hardware and software everyone depends on stays small. Fewer shared paths means fewer unintended communication channels between users, and what remains is easier to check for undesirable security implications. The lecture’s example of violating it is reusing a password between different systems or services.

Contain failures with isolation, encapsulation, modularity and layering

Isolation. Public-access systems should be isolated from critical resources, physically or logically, to prevent disclosure or tampering. The same applies to users on one system: their processes, memory and files stay separate except where sharing is explicitly wanted, and every modern operating system provides this. Apple’s Secure Enclave is a coprocessor, separate from the main processor, that handles encryption, decryption and key management on its own. Even if the main operating system is compromised, the keys and operations inside it, including the biometric data behind Face ID and Touch ID, stay isolated.

Encapsulation. A specific form of isolation built on object-oriented structure: procedures and data are enclosed in their own domain, so the internal structure of a data object is reachable only through the procedures of that protected subsystem, entered only at designated points. In a healthcare database, direct table access is restricted. A stored procedure GetPatientRecord returns records matching specific criteria, and a view PublicPatientInfo exposes only non-sensitive fields like name and date of birth. All access to sensitive data goes through these entry points, which keeps access control, auditing and integrity enforceable.

Modularity. Security functions should be built as separate, protected modules, and the wider architecture should be modular too, so any one part can migrate to new technology or gain new features without redesigning the whole system. AWS Key Management Service is one centrally managed cryptographic module used by S3, EBS, RDS and Lambda instead of each service implementing its own encryption, so AWS can upgrade KMS without touching the services that depend on it.

Layering, also called defence in depth. Multiple overlapping protection approaches cover the people, technology and operational sides of a system together, so the failure or bypass of any one does not leave the system unprotected. A company might require multi-factor authentication, enforce role-based access control, keep the OS patched, run antivirus, filter traffic with network and host firewalls, encrypt sensitive data, and train staff to recognise phishing. If phishing steals a password, the attacker still has to get past MFA, access control and antivirus before the OS itself is compromised.

Optional aside

The slide deck’s own definition of layering did not survive extraction from the source PDF; the text is interleaved and garbled. It is not reconstructed here. The definition and example above come from the lecture notes, which give the principle in full elsewhere in the same week’s material.

Protection that users cannot work with gets switched off

Psychological acceptability. Security mechanisms should not interfere unduly with users’ work while still meeting the needs of whoever authorises access. Mechanisms that are not transparent, or that force users to translate their own mental model of protection into something unfamiliar, tend to be worked around or turned off. Fingerprint or facial recognition for unlocking a device gives strong security with almost no friction, matching how users already expect to identify themselves.

Least astonishment. A program or interface should respond in the way least likely to astonish the user. The mechanism for authorisation should be transparent enough that the user has a good intuitive sense of how the security goal maps to what the system does. A standard library integer-parsing function typically defaults to base 10 and only reads binary or octal when told to. JavaScript originally broke this convention: a string beginning with "0" was parsed as base 8 by default, which caused real developer confusion and bugs. It was discouraged in ECMAScript 3 and dropped by ECMAScript 5.

Exam detail

When a question describes a scenario and asks which principle it illustrates, check what specifically failed. One actor with too much unilateral power points at separation of privilege. A process or account holding more access than its task needs points at least privilege. A system defaulting to open access points at fail-safe defaults. A design that was kept secret and then broken once examined points at open design.