Building the walls

Last time we built a door but admitted there was no wall around it. Today we switch on the MMU, give memory real attributes and permissions, and make the EL0 boundary something the hardware enforces.

Goal

Turn on the MMU — the memory management unit — with a small translation table, label RAM and device registers with proper attributes, and set permissions so EL0 code cannot reach the peripherals. Then prove it with two controlled faults: one permission fault against UART, and one translation fault against an unmapped hole.

What the MMU actually does

Until now, every address our code used went straight to physical memory. With the MMU on, every access first passes through a translation. The processor looks the virtual address up in a table that we build, and the descriptor it finds answers three questions: which physical address is behind this, what kind of memory is it, and which exception levels may touch it?

That last question is the wall. A system call is still a voluntary knock on the kernel door. A forbidden load or store is not voluntary; the hardware interrupts the instruction and sends control back to EL1.

The map we want

A real operating system eventually gives each process its own address space. We are not there yet. This lesson uses one deliberately small address space so the moving parts stay visible. It is also intentionally safe for Raspberry Pi 4 boards with only 2 GB of RAM.

The alias is the trick that keeps the lesson honest. The kernel continues executing through its EL1-only identity map. EL0 enters through 0x10000000, which points at the same physical bytes but carries EL0 permission. The two exception levels therefore see the same tiny program through different permission views.

How the table walk reads it

With a 4 KB translation granule and a 4 GB virtual address space, the first table level has 512 entries. In this tutorial each level-1 block covers 1 GB. The low gigabyte is special: instead of mapping it with one block, its level-1 entry points to a level-2 table, where each entry covers 2 MB. That gives us enough detail to make one user-visible alias while keeping the rest of low memory kernel-only.

// The real code lives in src/kernel.c. This is the shape of the map.
l1_table[0] = table(l2_low_table);             // 0..1 GB split into 2 MB blocks
l1_table[1] = block(0x40000000, Normal, EL1);  // 1..2 GB kernel RAM
l1_table[2] = 0;                               // 2..3 GB unmapped
l1_table[3] = block(0xC0000000, Device, EL1);  // 3..4 GB peripherals

for each 2 MB block in l2_low_table:
    map virtual N to physical N as Normal, EL1-only

l2_low_table[0x10000000 >> 21] = block(0x00000000, Normal, EL0);

There is still no allocator, no process object, and no per-process table. That is fine. The lesson is narrower: the CPU can now distinguish a missing translation from a permission failure, and it can treat RAM and device registers differently.

The registers that arm it

• MAIR_EL1 defines what our attribute indices mean. In the example, attribute 0 is Normal write-back memory and attribute 1 is Device-nGnRE memory.
• TCR_EL1 describes the translation regime: 4 GB virtual space through T0SZ=32, 4 KB granule, inner-shareable walks, and a physical address size large enough for the Pi.
• TTBR0_EL1 points at the level-1 table used for addresses translated through the lower half regime.
• SCTLR_EL1.M is the switch. Once bit 0 is set and the following isb completes, instruction fetches and data accesses use the table.

Notice what we do not turn into a lesson yet: caches. The example keeps the data and instruction caches off while first introducing translation. Caches deserve their own careful story because they make the kernel faster, but they also make device access and DMA much less forgiving.

Exceptions: EL0 versus EL1

System calls and memory faults are both synchronous exceptions: they happen because the currently executing instruction asked for something. But the source matters. An exception from EL0 is a request or violation made by user code. An exception from EL1 is a kernel fault. Those are not morally or mechanically the same.

That last row is the important discipline. A user fault can be part of normal life: later it may mean lazy allocation, copy-on-write, or killing one bad process. A kernel fault is a bug in the supervisor. In this tiny OS there is no recovery story for that yet, so the honest thing is to stop and print the evidence.

uint32_t ec = (frame->esr >> 26) & 0x3F;

if (ec == 0x15) {                 // SVC64 from EL0
    frame->x[0] = syscall_dispatch(frame->x[8], frame->x[0], frame->x[1]);
    return;
}

if (ec == 0x24) {                 // data abort from a lower EL
    print_far_and_esr();
    frame->elr += 4;              // skip this controlled one-instruction probe
    return;
}

halt_kernel_fault();

Two faults, two meanings

If EL0 behaves and uses svc, the kernel writes to UART on its behalf. If EL0 tries to write the UART register directly, the address translates to the Device region but the permission check says EL0 is not allowed there, so the CPU raises a permission fault. If EL0 tries 0x80000000, there is no descriptor at all, so the CPU raises a translation fault instead.

Both faults enter the same synchronous vector, but ESR_EL1 tells us what kind of exception happened and FAR_EL1 tells us which virtual address caused the data abort. The example skips the faulting instruction only because these are controlled probes. A real kernel would attach the fault to a process, not blindly continue after arbitrary illegal memory access.

The example layout

The downloadable package is split so the article does not have to paste a whole kernel into the page. src/kernel.c contains the MMU setup, exception classification, syscall dispatch, and EL0 probes. src/drivers/uart.c, src/drivers/led.c, and src/drivers/timer.c keep board support out of the lesson core. src/lib/printf.c is the tiny EL0-side formatter that writes through svc.

Try it

Download the complete example package, build it, and boot boot/kernel8.img. The kernel enables the MMU before dropping to EL0, then EL0 proves three things in order: svc still works, direct UART MMIO is blocked, and the deliberately unmapped 2 GB hole is blocked too.

The package also has a separate QEMU target. It builds the same lesson with QEMU_RASPI4 enabled, using QEMU's PL011 serial port instead of the Raspberry Pi mini UART:

make qemu

Building walls: 0..2GB user RAM, 2..3GB unmapped, 3..4GB kernel device.
MMU enabled. Stepping down to EL0...
EL0: printf still works through svc.
[WALL] SVC write accepted.
[WALL] trying UART MMIO directly...
[WALL] EL0 data abort: permission fault near 0x00000000FE215040, ESR_EL1 = 0x000000009200004D
[WALL] survived UART fault and resumed.
[WALL] trying unmapped 2GB hole...
[WALL] EL0 data abort: translation fault near 0x0000000080000000, ESR_EL1 = 0x0000000092000005
[WALL] survived translation fault and resumed.

Read that output as a small transcript of the boundary working. The first line is still EL1: the kernel has built the tables but has not dropped to user mode yet. The next line means SCTLR_EL1.M is now set and the kernel is about to enter EL0 through the 0x10000000 alias. The first two EL0 lines prove the legal path still works: printf uses svc, the synchronous exception is classified as SVC64, and EL1 writes to UART on behalf of user code.

The UART probe is different. EL0 touches 0xFE215040 directly, which is inside the Device region, but that region is EL1-only. The MMU therefore reports a permission fault. The 2 GB probe reaches 0x80000000, where there is no valid descriptor at all, so this time the MMU reports a translation fault. Both are data aborts from a lower exception level, so the tutorial handler prints FAR_EL1 and ESR_EL1, skips the single probe instruction, and lets the demonstration continue.

That is the difference between a promise and a guarantee. Next, we use the timer heartbeat from article five to run more than one program at a time — the scheduler.