knaxuslearn · build · ship

Roadmaps / Linux / Phase 3

Phase 3 of 4 · Study weeks 9-12

Network and security

This phase lets you say who can reach orders-api, what it can reach and what it may do, and diagnose a network fault from packets. Read man 8 ip, man 8 ss, the nftables wiki and man 7 capabilities.

Topics every week

  1. Week 98 h
    • One host's network: interfaces, routes, neighbour tables, ip and ss, the TCP handshake and states, TIME_WAIT, ephemeral ports

    Hand inAn annotated tcpdump of one request, and the ephemeral-port limit worked out for your host

  2. Week 108 h
    • DNS and TLS: the resolver path (nsswitch, systemd-resolved, glibc timeout and attempts), dig, openssl s_client, certificate expiry

    Hand inA reproduced DNS timeout with its cause and fix, and a certificate-expiry check run by a systemd timer

  3. Week 118 h
    • Network namespaces, veth, bridges, nftables chains, hooks, default drop and connection tracking, tc netem

    Hand inA two-namespace lab (client and server) with firewall rules and injected delay and loss

  4. Week 128 h
    • Security model: capabilities, seccomp filters, AppArmor (on by default in Debian), SSH hardening, audit logs

    Hand inCheckpoint 3, built in four steps: the firewall; the three network faults; the capability and syscall filter; the blocked-call capture

Practice projects

  1. Build the Week 11 lab from scratch with a script. Done when ip netns list, the rules and a client request to the server all work after one command, and a second run is clean.
  2. Break and fix TLS: expire a certificate on purpose. Done when the check from Week 10 raises its alert at the threshold you set and you can read the failure from openssl s_client.
  3. (Optional for a fresher) Write a seccomp filter by hand for a tiny program. Done when a forbidden call returns an error you chose.

GitHub repos to study

RepoWhat to readWeek
bpftrace/bpftracetools/tcpconnect.bt, tools/tcpaccept.bt and tools/tcpdrop.bt for connection events9, 11
iovisor/bcctools/tcplife.py and tools/tcpretrans.py9
strace/stracestrace -f to follow forks and see which call a filter blocked (read the manual page; the tool is a way to observe a filter, not a guide to writing one)12

Milestone checkpoint

Apply the firewall, reproduce three faults (delay and loss with tc netem, ephemeral-port exhaustion by a client opening short connections to one address, and a blackholed resolver) and harden the unit. Done when the Checkpoint 3 row passes. Evidence: ss -s, tcpdump and nft list ruleset captures for each fault, the port-exhaustion rate you predicted against the rate that failed (see the first Check yourself question), the blocked-call log, ADR-3 (firewall policy: allow list against deny list for outbound, and why).

Check yourself

Answer each question in your head first, then open it to check.

  1. With the default port range 32768 to 60999 and a 60 second TIME_WAIT, how many new connections a second can one client open to a single address and port?
    About 470 (28,232 ports divided by 60 seconds). Beyond that, new connects fail or wait.
  2. Why does a DNS timeout look like a slow service?
    The service waits for the resolver first (glibc defaults to 5 seconds and 2 attempts), so a request pauses before any work starts.
  3. What does a default-drop chain do to a reply packet from a connection you started, and what can fill up?
    Nothing, if you allow established and related connections first; a stateful rule set tracks replies in the connection-tracking table, and when that table is full new connections are dropped (check conntrack -S).
  4. Why drop all capabilities for a service that listens on port 8080?
    Ports above 1023 need no privilege, so the service loses nothing and a flaw gets less to use.
  5. What can a seccomp filter not stop?
    A flaw that uses only allowed calls, such as reading data the service may already read.