23. Security: the threat model

A threat model is a written answer to three questions: what are we protecting, from whom, and what stands in the way. This one is written against what is actually built and measured, not what is planned. Where a control is not built, or is built but not yet checked, this chapter says so, because a threat model that claims too much is worse than none: it is what people plan around.

Reporting a vulnerability

If you find a way out of a sandbox, do not open a public issue. Report it privately, through GitHub’s private vulnerability reporting on the repository (Security → Report a vulnerability) or by e-mail. SECURITY.md has the address, what to include, how fast you will hear back, what is in scope and what is not. Zygo is pre-1.0, only the latest release is supported, and there is no bug bounty: reports are answered, fixed and credited, not paid for. No external audit has been done; one is planned.

Words used in this chapter

What is being protected

• The host’s integrity: its files, its kernel, its other processes.
• Other tenants: their code, their data, their secrets.
• The supervisor itself, which holds every function’s spec and secrets.
• The host’s resources: memory, CPU, pids, disk, network.

Who the attacker is

The attacker is a tenant who can run any code they like inside a sandbox. That is the design assumption, not the worst case. Running code you did not write is what Zygo is for. So every control below is judged by one question: what can hostile code, already running inside, do next?

The three trust classes

Not all code is equally hostile. The design sorts it into three trust classes, and gives each one a backend and a risk that is accepted.

T2 is a setting, not the default. A function ships with seccomp = "default"; T2 means seccomp = "strict" in its [fn.<name>] table or --seccomp strict, an egress allowlist rather than full, and Landlock, which is on wherever the kernel has it. A runtime pool is strict by default, because its zygotes are shared between tenants (chapter 20).

  who wrote the code?            backend                     what could still break it
  ───────────────────            ───────                     ─────────────────────────
  ┌─────────────────────┐        ┌──────────────────────┐
  │ T1  your team, CI   │ ─────▶ │ ns, relaxed seccomp  │ ─▶ a kernel CVE
  └─────────────────────┘        └──────────────────────┘
  ┌─────────────────────┐        ┌──────────────────────┐
  │ T2  paying,         │ ─────▶ │ ns + strict seccomp  │ ─▶ a kernel LPE CVE
  │     known customers │        │ + Landlock           │    (a few critical a year)
  └─────────────────────┘        │ + network allowlist  │
                                 └──────────────────────┘
  ┌─────────────────────┐        ┌──────────────────────┐
  │ T3  anonymous,      │ ─────▶ │ vm                   │ ─▶ a VMM or KVM CVE
  │     hostile         │        │ (not complete today; │    (much rarer)
  └─────────────────────┘        │  see below)          │
                                 └──────────────────────┘

The vm backend today

The vm backend runs one-shot sandboxes. A guest boots, the program runs under a kernel of its own, and the root filesystem is read-only at the device, not by a mount option the guest could change. What it does not have yet: networking, warm functions, and its own in-guest cgroups, seccomp and Landlock. So a tenant’s limits are the VMM’s cgroup on the host side, and nothing finer. (A VMM, or virtual machine monitor, is the host program that runs the virtual machine.) Until those land, T3 workloads do not have the boundary the design gives them. The honest answer for anonymous code today is a separate machine.

The gvisor backend today

The gvisor backend runs one-shot sandboxes. gVisor runs its own kernel in user space, called the Sentry, so the program’s syscalls reach the Sentry instead of the host kernel. On a host without KVM, that is a smaller and easier-to-defend target than ns. make gvisor-linux checks that the same spec behaves the same way on both. It is not a T3 answer either: it holds no warm functions and has no networking. A rootless runsc cannot write cgroups, so its resource limits are advisory, and a tenant there can still use up the host’s memory.

The layers an attacker must pass

On the ns backend, hostile code has to beat every one of these locks to get out, or find a bug in the kernel underneath them all. Chapter 4 explains each lock.

  ┌─────────────────────────────────────────────────────────────────────┐
  │ host: the kernel, your files, other tenants, the supervisor         │
  │  ┌───────────────────────────────────────────────────────────────┐  │
  │  │ user namespace: root inside, nobody outside; uid_map fixed    │  │
  │  │  ┌─────────────────────────────────────────────────────────┐  │  │
  │  │  │ pid, mount, net, ipc, uts, cgroup namespaces            │  │  │
  │  │  │  ┌───────────────────────────────────────────────────┐  │  │  │
  │  │  │  │ cgroup limits: memory, cpu, pids.max always set   │  │  │  │
  │  │  │  │  ┌─────────────────────────────────────────────┐  │  │  │  │
  │  │  │  │  │ pivot_root, read-only root, no cgroupfs     │  │  │  │  │
  │  │  │  │  │  ┌───────────────────────────────────────┐  │  │  │  │  │
  │  │  │  │  │  │ no capabilities, no_new_privs, nosuid │  │  │  │  │  │
  │  │  │  │  │  │  ┌─────────────────────────────────┐  │  │  │  │  │  │
  │  │  │  │  │  │  │ Landlock (5.13+), nftables      │  │  │  │  │  │  │
  │  │  │  │  │  │  │  ┌───────────────────────────┐  │  │  │  │  │  │  │
  │  │  │  │  │  │  │  │ seccomp allowlist         │  │  │  │  │  │  │  │
  │  │  │  │  │  │  │  │    ┌──────────────┐       │  │  │  │  │  │  │  │
  │  │  │  │  │  │  │  │    │ tenant code  │       │  │  │  │  │  │  │  │
  │  │  │  │  │  │  │  │    └──────────────┘       │  │  │  │  │  │  │  │
  │  │  │  │  │  │  │  └───────────────────────────┘  │  │  │  │  │  │  │
  │  │  │  │  │  │  └─────────────────────────────────┘  │  │  │  │  │  │
  │  │  │  │  │  └───────────────────────────────────────┘  │  │  │  │  │
  │  │  │  │  └─────────────────────────────────────────────┘  │  │  │  │
  │  │  │  └───────────────────────────────────────────────────┘  │  │  │
  │  │  └─────────────────────────────────────────────────────────┘  │  │
  │  └───────────────────────────────────────────────────────────────┘  │
  └─────────────────────────────────────────────────────────────────────┘
     every layer is a feature of ONE kernel: a kernel LPE bug skips them all

How the controls are tested

Every row below marked attempted is run by make escape-linux. It runs the escape itself, not a check of a setting, because a test that reads a flag also passes on a kernel that ignores that flag. The suite attempts 21 vectors in 32 checks, and on Linux 6.8 reports 32 blocked, 0 escaped, 0 skipped. On 5.10 it skips one, the Landlock half of case 20, which needs a 6.7 kernel. Run rootless, it skips another: setting up a file capability to try needs root on the host.

Beside it, make fuzz-linux sweeps every syscall number the architecture has — 469 of them — against all three seccomp profiles. Each call is made in a forked child, so a syscall that blocks or exits takes nothing else with it. The escape suite tries the attacks somebody thought of; the sweep needs no imagination. That is what makes it the right check for a filter whose jump offsets are computed by a program. It checks that the profiles are ordered on a real kernel (permissive ⊋ default ⊋ strict), that no syscall kills the process, that clone3 answers ENOSYS rather than EPERM, and that every syscall named in the tables below is refused.

  make escape-linux                        make fuzz-linux
  ─────────────────                        ───────────────
  every known attack, really tried         all 469 syscall numbers
  → 32 blocked, 0 escaped, 0 skipped       × 3 profiles, one forked child each
                                           → profiles ordered, nothing kills
                                             the process, clone3 → ENOSYS

Vectors: the kernel and privileges

Vectors: files

Vectors: processes and resources

Vectors: the network

Vectors: tenant against tenant

A runtime pool is one sandbox for many tenants: its zygotes are shared, and every process inside runs as one uid. The rows the suite attempts today are about one request reaching the traces of another. The rows it does not yet attempt are listed too, because a multi-tenant claim that rests on construction alone is not a tested one; the roadmap’s phase 5 is that work.

Vectors: secrets and the supervisor

Out of scope

Timing side channels, such as Spectre, let code learn secrets by measuring how long things take on a shared CPU. Tenants who need protection from that belong on the vm backend and on separate hosts.

The stated guarantee for egress

Egress is traffic from the sandbox out to the network. Under network = "egress" or "full", a sandbox cannot reach the cloud metadata endpoint (169.254.169.254), any RFC1918 address (the host, its neighbours, the LAN’s router), any CGNAT, link-local, multicast or reserved address, or the host’s loopback — whatever name they resolve from — unless the operator passes --allow-private-net. This is enforced inside the sandbox’s own network namespace by nftables rules that sit above every allow rule, and by a resolver that admits only what the allowlist names. The host’s loopback is closed a second way, because the firewall has to pass the sandbox’s own loopback: pasta is told not to carry the sandbox’s loopback ports to the host’s and not to answer for the gateway address (chapter 14). What Zygo does not guarantee is anything about the public internet under full: that mode means “the internet and nothing of yours”.

  sandbox wants to connect to …
        │
        ▼
  ┌───────────────────────────────────────────┐
  │ nftables rule 1 (checked first):          │
  │ 169.254.x, 10.x, 172.16-31.x, 192.168.x,  │── match ──▶ refused, "no route"
  │ CGNAT, link-local, loopback               │   (unless --allow-private-net)
  └─────────────────────┬─────────────────────┘
                        │ no match
                        ▼
  ┌───────────────────────────────────────────┐
  │ allow rules (egress: only the allow list; │── match ──▶ connected
  │ full: the public internet)                │
  └─────────────────────┬─────────────────────┘
                        │ no match
                        ▼
                     refused

Measured against Docker

This was measured, not read from the code, with a small connection probe on one host. The probe through Zygo’s --net full found no route to the metadata address, the host’s Postgres, the LAN router and 10.0.0.1, and reached 1.1.1.1:53. Docker’s --network bridge on the same host reached the host’s Postgres and the LAN router, and routed the metadata address (it was refused by the host, not blocked). Docker needs four iptables -I DOCKER-USER … -j DROP rules for the same result, one for each of 169.254.0.0/16, 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16. Chapter 5 has more on Docker’s network.

Where the boundary is weaker than it looks

This part is said plainly, because zygo doctor says it too. Each item is a place where the protection is thinner than the tables above might suggest.

ns is one kernel

Every control above is a feature of the kernel. A kernel local-privilege-escalation bug defeats all of them at once. That is the accepted residual risk for T1 and T2, and the reason vm exists for T3. zygo doctor says how old the running kernel’s series is, and warns past two years, because that risk grows with the gap. Age is not the same as unpatched: a long-term series gets fixes backported without changing its version, and the warning says which kind it is looking at. But a host four years behind is four years of hardening behind, and every control here rests on it.

Without newuidmap, tenants share a uid

newuidmap is the small setuid helper (from the uidmap package) that gives each sandbox its own range of user ids. With no such range, the id map falls back to a single entry, so two sandboxes run as the same host uid. They are then kept apart by namespaces and file modes, but not also by different uids. zygo doctor reports this as degraded.

Landlock needs 5.13, and its network rules 6.7

On older kernels the filesystem allowlist is absent, and the mount plan is the only filesystem boundary. The process-level network rules — bind refused in a runtime pool, connect limited to the allowlist’s ports under egress — are built and unit-tested, and enforced for real in one place: CI’s landlock-network job, on an ubuntu-24.04 runner (Linux 6.8, Landlock ABI v4), runs tests/linux/verify_landlock_net.sh (make landlock-net-linux runs the same script in a container). It picks the refusals nftables cannot produce: a bind() on a TCP port in a pool, which sends no packet, and a loopback connect() to a port off the allowlist, which the packet filter accepts on its first line. Both must fail with EACCES; a loopback connect on an allowed port must get past Landlock; a function must be able to listen on its own loopback, and under network = "none" connect to itself, with anything off loopback ENETUNREACH from the empty namespace. On a kernel below 6.7 the script says SKIP and exits 0 — which is why the job is not mixed into one that also runs on 22.04. Neither of this project’s own development machines can run it: Docker Desktop’s 5.10 kernel reports ABI 0, and the Raspberry Pi’s kernel has no Landlock at all. Below 6.7, the nftables allowlist inside the namespace is the only egress control, and it is the one every other check here runs against.

cgroup.kill needs 5.14

cgroup.kill is one file write that kills every process in a cgroup. Below 5.14, killing a request’s process tree falls back to freeze → signal → thaw. That has a race which the freeze closes, but it is more machinery than one write.

The seccomp profiles have been run against a sample, not a population

The compatibility matrix runs six Python packages, the standard library’s sqlite3 and three Node cases under default and strict. Each round of widening it has found something:

• the filter refusing every thread;
• strict refusing the agent its own control socket;
• when Node was added, strict removing socketpair, which libuv uses for every pipe, so a strict Node function could not start a worker at all.

All are fixed and pinned by tests. But a control that has been run against ten cases has been run against ten cases.

The child filter is installed by the agent

Under strict, the agent’s forked child is locked down further: no execve, no new process. The agent does this, at the supervisor’s request, so it is only as good as the agent. zygo agent test now checks it rather than trusting it: an agent that ignores ZYGO_CHILD_SECCOMP and runs the request anyway fails conformance. The Python agent installs the filter. The Node agent installs it when the image has the helper object, and otherwise falls back to Node’s permission model, which is weaker against a V8 escape and says so in READY. The sh example refuses the request outright. Chapter 24 has the details.

On Ubuntu 24.04 and later, Zygo asks you to turn something off

kernel.apparmor_restrict_unprivileged_userns=1 stops a normal process from mounting inside a user namespace, which is the first thing every sandbox does. zygo doctor finds this by trying the mount, not by reading the sysctl. zygo doctor --fix then installs an AppArmor profile that gives the zygo binary alone the userns permission — the way Ubuntu lets its own browsers and container tools past the same rule — and the restriction stays on for every other process. Only where AppArmor cannot load a profile does it offer sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 instead. Be clear about what that is. It is a host-wide protection against a class of local privilege escalation that starts with an unprivileged user namespace, and turning it off removes it for every process on the machine, not only Zygo’s. It is the right call on a host whose job is running sandboxes, and it is what shim/lima.yaml does inside Zygo’s own VM. On a shared workstation, the profile is the right answer; packaging/apparmor/zygo is the file. Troubleshooting shows zygo doctor --fix, which applies it after asking.

Egress needs pasta and nft

If either is missing, a networked sandbox refuses to start. It does not start without a firewall. But that is a liveness failure — the service stops working — and you should know about it before it happens.

A derived system layer is built as root, with host networking

A derived layer is an image layer Zygo builds by installing system packages on top of an image. It installs them as root inside the sandbox’s user namespace, with host networking, once, before any tenant code exists. The package list is checked against the Debian package-name alphabet and passed as separate arguments, never through a shell.

What has not been reviewed

No external security audit has been done. That is the largest gap in this chapter, and it needs an auditor, not a commit. The hardening that could be done without one is in place: the full syscall sweep described above, and the kernel-age warning in zygo doctor.

Until an audit happens, the strongest honest statement is this: every vector marked attempted above is tried by a suite that runs on every change, none of them currently succeeds, the ones not yet attempted are marked as such, and the syscall surface they rest on is swept in full rather than sampled.

Hardening your deployment

These points come from SECURITY.md:

• The ns backend leans on one kernel and does not hide it. For code that is hostile rather than semi-trusted, the wall is isolation = "vm" — today one-shot only, without a network or a warm path, and in a make vm-build binary on a host with KVM (the vm backend today). Until that is more, the honest answer for anonymous code is a separate machine.
• Keep network = "none" unless a function really needs egress, and keep the allow list to the hosts it needs.
• Do not run Zygo as root. It does not need it, and pasta, newuidmap and cgroup delegation all behave better without it.
• Bind the HTTP API to loopback or a unix socket. It refuses to start without a token on a reachable address, but the safe default is worth keeping.
• Install uidmap, so tenants get separate subordinate uid ranges rather than sharing one identity map. Zygo does not let a host become multi-tenant without one: registering a tenant is refused on a host whose user has no range in /etc/subuid, unless the supervisor was started with ZYGO_ALLOW_SHARED_UID=1, which is the operator saying they accept that every tenant’s sandbox shares one host uid.