Not all at once

The moment two tasks reach for the same thing, our tidy multitasking turns to chaos. Today we meet the race condition, name the critical section, and build the locks that enforce a simple rule: not everyone at once.

Goal

First, see the problem — watch two tasks corrupt a shared resource because the scheduler interrupts one of them at the worst possible moment. Then fix it two different ways, and understand exactly when each fix is the right one.

The race

Take our two tasks from last article, but now have each print a whole word through the shared UART:

void uart_puts(const char *s);

void task_a(void) { for (;;) { uart_puts("[hello] "); spin(); } }
void task_b(void) { for (;;) { uart_puts("[world] "); spin(); } }

You might expect a clean stream of [hello] [world] [hello] [world]. Instead, the timer can fire in the middle of uart_puts — right after task A has printed [hel — freeze A, and let B print its whole word before A resumes. The output shreds:

[hel[world] lo] [wo[hello] rld]...

Nothing is broken; every instruction ran correctly. The bug is timing: a sequence of steps that only makes sense if it happens all together got chopped in half. That stretch of code — "print this whole word" — is a critical section: it must run start to finish with no one else interfering.

The rule: one at a time

A lock is the tool that enforces it. Before entering a critical section a task acquires the lock; when it leaves it releases it. While the lock is held, anyone else who tries to acquire it must wait. We have two ways to build one, and the difference between them is the whole lesson.

Where the code lives

The example for this article keeps the same shape as the previous ones: board-facing code lives in drivers/, while kernel mechanisms live in kernel/. The synchronization code is in src/kernel/sync.h and src/kernel/sync.c. In a larger kernel those names usually split into more specific headers such as irqflags.h, spinlock.h, and later mutex.h; here one small sync.h keeps the lesson readable.

Tool 1 — on one core, just stop the clock

On a single core, where does the interference even come from? Only one place: the timer interrupt, which is the only thing that can take the CPU away from a running task. So the cheapest possible lock is to simply mask interrupts for the duration. If the scheduler cannot preempt us, no other task can run, and we have the resource to ourselves.

// sync.h — the poor man's lock: turn the heartbeat off, briefly
static inline uint64_t irq_save(void) {
    uint64_t daif;
    __asm__ volatile ("mrs %0, daif" : "=r"(daif));   // remember the state
    __asm__ volatile ("msr daifset, #2");             // mask IRQ
    return daif;
}
static inline void irq_restore(uint64_t daif) {
    __asm__ volatile ("msr daif, %0" :: "r"(daif));   // put it back exactly
}

void uart_puts_safe(const char *s) {
    uint64_t flags = irq_save();
    uart_puts(s);                 // nobody can interrupt this
    irq_restore(flags);
}

It is wonderfully simple and exactly right — but only under two conditions. It must be short, because while interrupts are masked the machine is deaf, even to its own heartbeat. And it only works on one core: masking this core's interrupts does nothing to stop another core running the same code at the same time. For that, we need a real lock.

Tool 2 — a lock that lives in memory

When the contenders might be different cores, the lock has to be a value in shared memory that everyone agrees to check. The danger is obvious: what if two cores read "unlocked" at the same instant and both grab it? ARM solves this with a pair of exclusive instructions, ldxr and stxr. ldxr reads a value and quietly tags the address; stxr writes back only if nothing else has touched that address in between — and tells you whether it succeeded. That is an atomic test-and-set, the atom every lock is built from.

// sync.c — acquire/release with the exclusive monitor
  void spin_lock(spinlock_t *lock) {
  1:  ldaxr   w1, [x0]             // load-acquire, exclusive
    cbnz    w1, 2f               // already held? go wait
    mov     w2, #1
    stxr    w3, w2, [x0]         // try to claim it
    cbnz    w3, 1b               // someone raced us: retry
    ret                          // we hold the lock
2:  wfe                          // sleep until an event fires
    b       1b
  }

  void spin_unlock(spinlock_t *lock) {
    stlr    wzr, [x0]            // store-release 0: unlock
    sev                          // wake any sleepers
    ret
  }

Two details matter beyond the atomicity:

• The acquire form (ldaxr) and release form (stlr) are memory barriers. They guarantee that everything you do inside the critical section stays inside it — the processor may not reorder those reads and writes out past the lock. Without that ordering, a "held" lock would not actually protect anything.
• wfe and sev make waiting polite: instead of spinning hot and burning power, a waiter sleeps until the releaser sends an event.

An honest warning from our own bring-up: raw exclusive locks belong after the MMU has made shared RAM Normal and cacheable. Before those walls are up, ldxr / stxr are not a good hardware demonstration on this board. The running example therefore fixes the UART race with interrupt masking on our single core, while sync.c also contains the spinlock primitive we will need once contention can come from another core.

Fixing the race

Wrap the shared resource in a guard and the critical section becomes indivisible again:

void uart_puts_locked(const char *s) {
    uint64_t flags = irq_save();
    uart_puts(s);                 // a whole word, uninterruptible by intent
    irq_restore(flags);
}

void task_a(void) { for (;;) { uart_puts_locked("[hello] "); spin(); } }
void task_b(void) { for (;;) { uart_puts_locked("[world] "); spin(); } }

Later, when the same resource can be touched from more than one core, the shape becomes spin_lock_irqsave(): mask local interrupts so this core cannot preempt itself while holding the lock, and use the spinlock word so other cores cannot enter either.

What actually happens

Without protection, the timer interrupt slices through uart_puts and the two words braid together into nonsense. With the interrupt-masking lock, the printing task is briefly un-interruptible, so each word lands whole — perfect on our single core. With the spinlock, the same guarantee holds even when the contender is another core, because the exclusive monitor lets exactly one of them win the claim while the others wait on wfe.

The deep point is that correctness in a concurrent system is not about each instruction being right — ours always were. It is about which sequences are allowed to be torn apart and which are not. Naming those sequences, and guarding them, is most of what kernel programming becomes from here on.

Try it

Download the complete example package: not-all-at-once-example.zip.

Run it in QEMU first:

make qemu

The example starts with plain UART output and lets the timer shred the words into each other on the terminal — the race, live:

[hel[world] lo] [wo[hello] rld]...

After a few timer ticks the same tasks use irq_save and the stream comes clean:

[hello] [world] [hello] [world]...

Seeing the same program go from garbage to order with one wrapped lock is the moment concurrency stops being abstract. Next, we add the missing half of UART and give the machine a command prompt.