Hardware protection and privilege rings
How memory protection, privilege rings and isolated cryptographic hardware enforce access control beneath every operating-system mechanism this module has covered so far.
- Explain segment addressing and why it makes a segmentation fault a security mechanism, not just an error.
- Trace x86's ring model from the 80286 through to the nine-ring picture modern Intel CPUs actually implement.
- Distinguish ARM's MPU and MMU tiers, and explain what CHERI adds on top of both.
- Explain how a TPM protects a key without ever exposing the key itself.
16 min readJump to the quiz ↓
Hardware enforces memory boundaries beneath every software rule
Every access-control rule so far (DAC, MAC, Unix permissions, the same-origin policy) is enforced in software, and software runs on hardware that has to agree to enforce it. If a process could read another process’s memory directly, none of those rules would matter. Hardware protection makes them unbreakable from underneath rather than merely unbroken by convention.
Memory protection keeps one process’s memory apart from another’s. The CPU addresses memory with two
registers together: a segment register, which identifies the segment, and an address register, which points to a
location inside it. A process that reaches outside its own segment triggers a segmentation fault, SIGSEGV, on
both reads and writes, and the operating system, not the process, decides what happens next. The check runs on
every access and beneath the OS’s own access control, so it holds even against a process that has found a way
around software-level restrictions.
Privilege rings let only ring 0 change privilege, and x86 uses four of nine
Privilege rings layer the CPU’s own permissions. Code runs in one of several rings, each a distinct privilege class, and the current privilege level can only be changed by code already running in ring 0. Ring 3 code, the outermost and least privileged, cannot reach ring 0 objects directly; it goes through a gate, a controlled entry point that lets code run at a different privilege level under supervision. This is the principle of least privilege in hardware: all code runs with the minimum privilege it needs, and the paths into the kernel are kept to the smallest, tightly controlled set.
Two Intel chips set this up historically. The 80286 introduced segment addressing and protection rings. The 80386 added built-in virtual memory and expanded memory segments to 4GB.
Modern Intel CPUs go beyond the classic four rings. There are nine rings: ring 0 to 3 for normal code, a second ring 0 to 3 underneath for the hypervisor’s VMX root mode, and system management mode (SMM) at the bottom for the BIOS. Only four are used: SMM (firmware), ring 0 of VMX root mode (hypervisor), the ordinary ring 0 (operating system kernel) and ring 3 (applications). Rings 1 and 2, in both sets, sit unused, and each level can see and control everything less privileged than it. “How many rings does x86 have” and “which are actually used” are different questions with different answers.
The current privilege level can only be changed by ring 0 code; lower rings reach higher ones only through gates. Modern Intel CPUs actually define nine rings — 0–3 for normal code, a second 0–3 set for the hypervisor's VMX root mode, and SMM beneath that for the BIOS — but rings 1 and 2 of each set are rarely used. The four squares above are the levels the lecture says are used in practice: applications in ring 3, the OS kernel in ring 0, the hypervisor in VMX root ring 0, and firmware in SMM, each level able to see and control everything outside it.
Recall
Modern Intel CPUs have nine privilege rings. How many are actually used, and which ones?
Four: system management mode (SMM) for firmware, ring 0 of VMX root mode for the hypervisor, the ordinary ring 0 for the operating system kernel, and ring 3 for applications. Rings 1 and 2, in both the normal and VMX root sets, sit unused.
ARM makes protection customisable, and CHERI adds fine-grained capabilities
ARM is the processor family in most phones, tablets and IoT devices. Unlike Intel, ARM licenses processor cores to other chip designers rather than building complete chips, so its hardware protection is far more customisable. Early ARM cores had no memory management at all. Current variants split into two tiers.
Compare
ARM MPU
Memory protection unit. Enforces a fixed set of memory regions and their permissions. No virtual addressing, lighter weight, common on smaller or real-time ARM cores.
ARM MMU
Memory management unit. Adds full virtual memory management on top of protection, translating virtual addresses to physical ones the way a desktop or phone CPU does.
ARM’s latest addition is CHERI (Capability Hardware Enhanced RISC Instructions), which brings fine-grained capability support to the architecture. A process spawning a subthread can hand it read and write access to a specific memory range rather than the whole address space, so several sandboxes can run inside a single process. The lecture frames the payoff as a long-term promise: used thoroughly across an OS like Windows, Android or iOS, CHERI could prevent most of the zero-day exploits seen in recent years, but that depends on adoption that has not happened yet.
A TPM isolates the key itself, so a compromised OS cannot steal it
Isolated cryptographic components isolate the key rather than the code. A key stored in dedicated hardware never has to leave it to be used; the surrounding system reaches it only through a restricted interface that performs authorised operations without exposing the raw key. The lecture’s example is the TPM (Trusted Platform Module), originally built for Digital Rights Management, which now provides isolated storage for cryptographic operations more broadly. Software asks the TPM to sign or decrypt, and only the result crosses back out, so malware with full access to the operating system cannot read the key, because it was never reachable from there.
Pitfall
Do not describe hardware isolation as something that replaces the access control discussed earlier in this module. Segmentation, rings and isolated key storage sit underneath DAC, MAC, ACLs and Unix permissions; they are what makes those enforceable at all, not an alternative to them.
Check yourself
Source
Week 3 slides PDF