Network Tutorial · 06

A conversation, properly begun

UDP fired a datagram and forgot it. TCP promises something harder: every byte, in order, exactly once. Before it can keep that promise, both ends must sit down and agree where counting starts. That agreement is the three-way handshake.

[o-o] |=| /_\

Here the difficulty changes character. Everything so far was a single packet answered by a single packet. TCP is a relationship: it has memory, it has state, it lasts. And memory is the first thing an attacker will try to exhaust. So we begin the conversation carefully, and we keep only as many as we can afford.

Why a handshake at all

To deliver a stream reliably, TCP numbers every byte. If a segment is lost, the number is what lets the receiver notice the gap and the sender know what to resend. But the two machines have never spoken before — whose numbering, starting where? The handshake settles exactly that. The client sends a SYN ("synchronize") carrying its starting number. The server answers SYN-ACK: here is my starting number, and I acknowledge yours. The client sends a final ACK. Three messages, and now both sides know both starting points. The connection is ESTABLISHED, and only now may data flow.

Where connections live

A TCP connection is identified by a four-tuple — peer IP, peer port, local port (the remote address plus the two ports) — and Alquist keeps them in a small fixed table (src/network/state.c):

#define NET_TCP_CONN_MAX   4u     /* live connections we will hold at once */
#define NET_TCP_LISTEN_MAX 4u     /* listening sockets */
#define NET_TCP_RECV_WINDOW 256u  /* bytes we advertise we can buffer */

static struct net_tcp_conn     g_net_tcp_conn_table[NET_TCP_CONN_MAX];
static struct net_tcp_listener g_net_tcp_listener_table[NET_TCP_LISTEN_MAX];

Four. Not four thousand. On a small device that serves a shell and a diagnostic port, four concurrent connections is an honest working set — and the fixed size is a feature, not a limitation waiting to be raised. A connection table that can grow without bound is a memory-exhaustion attack with a countdown; a table of four can be full, but it cannot be a leak. What happens when it fills is a design decision we make on purpose, below.

The handshake, as Alquist runs it

A segment arrives; we look it up by four-tuple. If no connection matches, this is either the start of a new one or noise (src/network/protocol.c):

conn = net_tcp_find_conn(rx_info->src_ip, tcp_rx.src_port, tcp_rx.dst_port);

if (conn == 0) {
    struct net_tcp_listener *listener = net_tcp_find_listener_by_port(tcp_rx.dst_port);
    int allow_builtin_echo = (listener == 0 && tcp_rx.dst_port == NET_TCP_ECHO_PORT);

    /* A pure SYN to a port someone is listening on: begin a connection. */
    if ((allow_builtin_echo || listener != 0)
        && (tcp_rx.flags & NET_TCP_FLAG_SYN) != 0u
        && (tcp_rx.flags & NET_TCP_FLAG_ACK) == 0u
        && (tcp_rx.flags & NET_TCP_FLAG_RST) == 0u) {
        conn = net_tcp_alloc_conn();
        if (conn == 0) {                                  /* table full: */
            net_tcp_send_rst(rx_info, &tcp_rx, local_mac, local_ip);   /* refuse, loudly */
            return;
        }
        isn = net_tcp_generate_isn(rx_info, &tcp_rx);      /* our starting number */
        conn->state    = NET_TCP_STATE_SYN_RCVD;
        conn->snd_nxt  = isn + 1u;
        conn->rcv_nxt  = tcp_rx.seq + 1u;                  /* ack their SYN */
        conn->rcv_wnd  = NET_TCP_RECV_WINDOW;
        /* ... record peer mac/ip/port ... */
        net_tcp_send_with_optional_retx(conn, ..., SYN|ACK, ...);  /* our half + their ack */
    } else {
        net_tcp_send_rst(rx_info, &tcp_rx, local_mac, local_ip);    /* not welcome here */
    }
    return;
}

Then, when the client's final ACK arrives on that half-open connection, the state advances and — if it came in on a listening socket — the connection is queued for a user program to accept:

if (conn->state == NET_TCP_STATE_SYN_RCVD) {
    if ((tcp_rx.flags & NET_TCP_FLAG_ACK) != 0u
        && conn->snd_una == conn->snd_nxt
        && tcp_rx.ack == conn->snd_nxt) {
        conn->state = NET_TCP_STATE_ESTABLISHED;          /* the handshake is done */
        /* enqueue on the listener's accept queue (Chapter 8) */
    }
    return;
}

Two decisions that matter more than they look

The starting number is unpredictable. A lazy stack numbers connections 1, 2, 3… and an attacker who can guess your sequence numbers can inject data into a connection it cannot even see, or forge a handshake. Alquist's initial sequence number is mixed from the hardware timer, both addresses, both ports, and a rolling nonce (src/network/protocol.c):

isn = net_tcp_mix32(nonce ^ counter_lo ^ src_ip);
isn = net_tcp_mix32(isn ^ counter_hi ^ dst_ip);
isn = net_tcp_mix32(isn ^ ports);
g_net_tcp_isn_nonce = nonce ^ net_tcp_mix32(counter_lo + ports + 0xA5A5A5A5u);

Different for every connection, tied to who is calling and when, and stirred by a nonce that carries forward. Predicting the next one from the outside is not worth an attacker's afternoon — which is the whole point.

Unwelcome segments get a RST, not a shrug. A segment to a port nobody is listening on, or an ACK for a connection that does not exist, is answered with a reset. This is not rudeness; it is correctness. A silent drop leaves the other end retrying into the void; a RST says "there is nothing here, stop now." And when the four-slot table is full, a new SYN is met with the same honest RST — we refuse the connection out loud rather than quietly forgetting it or, worse, evicting a live one. A SYN flood against Alquist fills four slots and then bounces off; it does not grow a table until the box dies. We return to that fight properly in Chapter 11.

Try it

# The built-in TCP echo lives on port 7. Connect, type, see it echoed.
nc 192.168.111.2 7

# Watch the three-way handshake itself: SYN, SYN-ACK, ACK.
sudo tcpdump -nvi eth0 "tcp port 7 and (tcp[tcpflags] & (tcp-syn|tcp-ack) != 0)"

In the capture you will see the client's [S], our [S.] with a starting sequence number that looks random, and the client's [.] — then the connection is up. Point nc at a port nobody listens on and you get an immediate [R] back: the reset, doing its job.

Gotchas worth remembering

[^-^] |=| /_\

Numbers agreed, connection open, gate-crashers reset at the door. We have a conversation with memory now — four of them, no more, by design. Next we make it reliable: say it again until you're heard.