CSEC3616Cybersecurity Engineering

    Sandboxing, virtualisation and containers

    How sandboxes, virtual machines and containers isolate untrusted code from the host, why breaking out of each layer is hard but not impossible, and what Docker's root-privileged daemon puts back at risk.

    • Explain what a sandbox restricts, using the lecture's own examples.
    • Distinguish full virtualisation, para-virtualisation and containers as three different depths of isolation.
    • Locate the hypervisor at ring -1 and explain why malware can detect it is running inside a VM.
    • Explain the security trade-off created by Docker's root-privileged daemon.

    18 min read

    Intuition

    Running someone else’s code is not optional. A browser runs scripts from sites it does not control, a host runs virtual machines it did not build, and a build pipeline runs containers packaged by someone else’s Dockerfile. None of that code has earned trust yet, and none of it needs to: sandboxing, virtualisation and containers exist to run it anyway, inside a boundary tight enough that the code misbehaving stays the code’s problem rather than the host’s.

    Mechanism

    A sandbox is a restricted execution environment: code inside it gets a limited API rather than direct access to the host. The lecture’s own example is the Java applet, now mostly historical, which had no access to the local hard disk, or at most temporary access to a restricted directory, and could only talk back to the host it was loaded from, the same-origin rule that reappears in the browser. JavaScript running in a page is sandboxed the same way, and Chrome goes further by giving each tab its own operating-system process, so a script that hangs one tab does not take the browser down with it.

    Sandboxes are hard to secure. They can be broken out of, and the lecture is direct about it: the isolation is real, but it is not absolute. A flaw in the sandbox’s own implementation, or in the limited API it exposes, can let code inside reach further than the boundary intended. That is why breaking out is hard rather than impossible: the effort a sandbox demands raises the cost of an attack without removing the possibility of one.

    Threat

    Code running inside a sandbox exploits a flaw in the sandbox’s own implementation, or in the limited API it is allowed to call, to reach outside the restricted environment and act with the host’s own privileges instead of its own.

    Control

    The sandbox’s restricted API is the boundary itself: no local disk access, or at most a temporary directory, and communication limited to the origin the code came from. Chrome’s per-tab process isolation adds a second boundary on top. Neither is designed as formal access control, and neither is unbreakable, but both raise the cost of getting out.

    Mechanism

    Virtualisation takes the same idea up a level: instead of restricting one piece of code inside a shared operating system, it gives a whole guest operating system hardware that only appears to be its own. IBM invented the technique in the 1960s: VM/370 partitioned a single physical machine into several virtual ones. A VM lets software run even when the environment it was written for no longer exists, and at the client end it is how one operating system runs on top of another, Windows on macOS for example.

    Virtualisation comes in different depths. Full virtualisation emulates the hardware completely, even across a different hardware architecture, so an unmodified guest OS runs without knowing anything has changed. Para-virtualisation asks the guest OS to cooperate with the hypervisor instead, trading that complete emulation for a guest that has to support being virtualised. Containers and jails are lighter again: a form of virtualisation that isolates an application’s environment without emulating a separate machine underneath it at all.

    Compare

    Emulates the hardware completely, including across a different architecture. The guest operating system runs unmodified, with no indication anything has changed underneath it.

    The guest operating system has to support being virtualised and cooperates with the hypervisor directly, trading full hardware emulation for that cooperation.

    Mechanism

    A hypervisor manages and runs the VMs, for both full and para-virtualisation, with hardware support at ring -1, below the ring 0 an operating system kernel normally occupies. VirtualBox, VMWare, Xen and QEMU are the lecture’s examples.

    Virtualisation was never built for access control, but the effort it takes to break out of a VM does the same job anyway: it raises the bar for an attacker. Two limits are worth holding onto. A VM does not perfectly imitate a real environment, so its execution can be detected from inside, and malware that checks for the signs can change its behaviour rather than reveal itself somewhere built to analyse it. Hypervisors themselves also have vulnerabilities. The lecture’s advice is plain: keep hypervisor software up to date, and do not rely on virtualisation as the only defence.

    Threat

    An attacker inside a guest VM exploits a vulnerability in the hypervisor to break out of the VM and reach the host, or another VM running alongside it, a VM escape.

    Control

    There is no single fix here: the effort a VM escape requires is itself the main control, raised further by keeping the hypervisor patched. The lecture treats virtualisation as one layer among several, not a defence to rely on alone.

    Exam detail

    VM detection is not the same thing as a VM escape, and an answer that conflates them loses marks. Detecting that it is inside a VM only lets malware change what it does next, refusing to run its payload when it suspects it is being analysed, for instance. Escaping the VM is a separate, harder problem: reaching the hypervisor or host from inside the guest.

    Isolation layers — hardware to sandboxHardwarememory protection · privilege rings · isolated crypto (TPM)Hypervisorvirtual machines · ring −1 · full or para-virtualisationOperating systemprocess isolation · UID/GID · access control listsContainernamespaces and jails — lightweight virtualisation (Docker)Sandboxrestricted API, e.g. a browser tab or a Java applet

    Each layer isolates what runs above it from what runs below, and none of it is designed primarily as access control — hardware rings, hypervisors, process isolation, containers and sandboxes are re-purposed for it. Break one layer and the ones below still hold: a compromised sandboxed tab should not touch the container's other processes, and a compromised container should not touch the host OS. The layer that fails least gracefully is the top of this list, hardware — breaking out of a hypervisor or defeating memory protection removes every layer above it at once.

    Mechanism

    Docker began as a tool for continuous integration and delivery, not for isolation as such. It builds images, packages of an application together with the libraries it needs, and spins containers up from them quickly and cheaply. Orchestration systems such as Kubernetes then manage large numbers of those containers as a group. Docker abstracts over the host operating system’s native jail mechanism, so the same image behaves the same way across several different OSes.

    Containers sandbox the application inside them: one container cannot talk directly to an app in another. But the component that starts and stops containers, the Docker daemon, runs as root. Anyone able to start or stop a container is talking to a root-privileged process, which is exactly the attack surface that fact implies.

    Threat

    Any user able to invoke Docker’s container management, starting or stopping a container, is interacting with a daemon that runs as root. Compromising that interaction point compromises root on the host, not just the container.

    Control

    Container separation is the lecture’s answer for what Docker does buy: apps in different containers cannot reach each other directly, raising the bar the same way a sandbox or a VM does. It is not a fix for the daemon’s own root privilege, and the lecture does not claim it is. Containers were not designed as an access-control mechanism, and the root daemon is the clearest place that shows.

    Pitfall

    Do not write that Docker containers are as isolated as separate virtual machines. A container shares the host kernel with every other container and with the root daemon that manages them; a VM does not share a kernel with its host at all. The lightness that makes containers fast to start is the same thing that makes their isolation shallower than a VM’s.

    Recall

    Why is Docker's root-privileged daemon a security concern, specifically?

    Not because root exists somewhere on the system, but because ordinary container operations, starting or stopping a container, go through that daemon. Anyone able to perform those ordinary operations is talking to a process running as root, which widens the attack surface beyond what container isolation alone suggests.