Wire to Application: HTTP Request Full Journey

uriByuri General

From Wire to Application: The Full Journey of an HTTP Request on a Linux Server

When a client issues an HTTP request that finally lands at your origin after any CDN, web-application firewall, or load balancer the packet’s real adventure begins inside your machine. Following each hop sheds light on where to drop bad traffic, tune performance, or collect diagnostics.

Arrival at the Network Interface Card

Electrical or optical pulses arrive over copper, fibre, or a virtual network fabric. A physical or paravirtual NIC converts those pulses into Ethernet frames and triggers a hardware interrupt (or schedules a NAPI poll) that wakes the CPU. At this moment the frame lives only in a DMA ring buffer the driver manages.

Kernel Reception and Early Packet Processing

The driver copies sometimes zero-copies the frame into kernel memory, then queues it for the networking subsystem via a soft-IRQ. The Ethernet layer validates destination MAC addresses, while the IP layer checks IPv4 or IPv6 headers, verifies checksums, and confirms the packet is meant for this host. A malformed header or an expired TTL ends the journey here.

Netfilter Inspection

Before routing or local delivery, Linux consults netfilter. Packets aimed at the local machine see

  • PREROUTING hooks in the raw, mangle, or nat tables. This is where destination-NAT happens and where administrators sometimes mark or classify packets for later rules.
  • The kernel’s routing decision determines the packet is local, so it skips FORWARD entirely.
  • INPUT in the filter table makes the definitive allow-or-drop decision for traffic aimed at any local socket.

If a rule here drops the packet, nothing in user space is disturbed.

Transport-Layer Demultiplexing

A surviving packet meets the TCP subsystem. For new flows the kernel completes the three-way handshake; for existing ones it simply reassembles segments into an ordered byte stream. Once TCP is satisfied, the kernel looks up the listening socket bound to the destination IP and port. Your web server usually owns sockets on 0.0.0.0:80 and 0.0.0.0:443.

Crossing the User–Kernel Boundary

User-space programs cannot access kernel memory directly. A read() by nginx, Apache, or another front-end copies or splice-maps the payload into user-space buffers. From this point forward the “packet” has become a mere sequence of bytes that your web server can parse.

Web-Server Parsing and Dispatch

The worker reads the request line, headers, and (for POST/PUT) body content, selects the correct virtual host from the Host: header, then decides whether the path maps to a static file or needs dynamic handling. Static files stream directly from disk; dynamic requests go to PHP-FPM, uWSGI, Gunicorn, Node, or any other runtime. Framework middleware, routing, business logic, database calls, and templating combine to build the response body.

Returning the Response

Once the application writes its reply, the journey rewinds. Bytes cross back to the kernel, where TCP segments them, netfilter’s POSTROUTING hook applies any source-NAT, IP wraps each segment, the Ethernet layer frames them, and the NIC sends fresh signals onto the wire. Upstream devices undo their own translations, and the client’s browser finally renders the page.

Why This Journey Matters

Every layer exposes a lever. Kernel hooks keep unwanted traffic from waking your application. Web-server logs capture timing at the first moment user code runs. TCP backlog sizes and worker counts govern how gracefully you handle spikes. When you know the packet’s entire path, you know exactly where to place iptables rules, fail2ban jails, reverse-proxy caches, or performance profilers.

When an incoming HTTP request arrives at your origin server (i.e. after any CDN/WAF or load‑balancer), here’s the chronological journey it takes from the wire to your application:

1. Network Interface Card (NIC) & Physical Layer

  1. Electrical / Optical Signal

    • The packet arrives over copper, fibre, or a virtualised network fabric.

  2. NIC Hardware

    • The (physical or paravirtual) NIC converts the signal into Ethernet frames and either triggers an interrupt or enters NAPI polling.
    • The driver uses DMA to place the frame in a ring buffer and notifies the kernel that new data has arrived.

2. Device Driver & Kernel Packet Reception

  1. Driver Interrupt / NAPI Poll

    • The driver (via hard-IRQ or soft-IRQ) queues the frame for the kernel’s networking stack.

  2. Network Stack—Data-Link → Network Layers

    • L2 (Ethernet): The kernel verifies the destination MAC (dropping frames that are not for this host unless in promiscuous mode).
    • L3 (IP): It parses IPv4/IPv6 headers, validates checksums and hop-limit/TTL, and confirms the packet is local.

3. Netfilter (iptables / nftables) Hooks

Linux consults netfilter tables at several points. For a packet addressed to the local machine it passes through:

  1. PREROUTING in the raw, mangle, or nat tables

    • Destination-NAT (DNAT), packet marking, and other early transformations happen here.

  2. INPUT in the filter table

    • This is the final “should I accept or drop this?” decision for traffic bound to a local socket.

  3. (Optional) OUTPUT / POSTROUTING

    • These chains affect packets the host itself generates, or those leaving after your application writes a response.

If any rule in these chains drops or rejects the packet, it never wakes your application code.

4. Transport Layer & Socket Demultiplexing

  1. TCP Stack

    • For new connections the kernel completes the three-way handshake; for existing ones it re-assembles segments into an ordered byte stream.

  2. Socket Layer

    • The kernel maps the packet to the listening socket that owns the destination IP:port—typically an nginx or Apache listener on 80 / 443.

5. User-Space Hand-Off

  1. Kernel→User-Space Copy

    • The payload crosses from kernel buffers to web-server buffers—often with zero-copy tricks like splice() on modern kernels.

  2. Web-Server Front End

    • The worker parses the request line, headers, cookies, and (if present) body.

6. Application Routing & Processing

  1. Virtual-Host / Server-Block Match

    • The server chooses a configuration block using the Host: header.

  2. Static vs Dynamic

    • Static paths stream files directly; dynamic ones are proxied to your back-end (PHP-FPM, uWSGI / Gunicorn, Node, and so on).

  3. Application Framework

    • Middleware, controllers, business logic, database queries, and templating produce the HTTP response.

7. Response Path Back to the Client

After the application writes its response, the data travels back up the stack:

  1. User-space → socket → TCP layer
  2. POSTROUTING (nat/mangle) applies any SNAT or QoS marks
  3. IP encapsulates the data → Ethernet frames
  4. NIC transmits on the wire → back through upstream layers → client

Putting It All Together

[Client]
    ↓
[CDN/WAF / LB]
    ↓
1) NIC hardware & driver interrupt / NAPI poll
2) Kernel L2/L3 processing
3) Netfilter PREROUTING  →  INPUT
4) TCP handshake & socket lookup
5) Copy into user-space; nginx/Apache reads HTTP
6) Static file serve or FastCGI/Proxy to your app
7) App framework logic → response
8) Reverse network stack → client

Filter at netfilter hooks, log in both kernel and web-server layers, and accelerate with caches or tuned kernel parameters.