Network & Process Inspection
.mdVerified against ss (iproute2, Ubuntu 24.04), lsof 4.95.0, strace 6.8, tcpdump 4.99.4, iptables 1.8.10 (nf_tables), nftables 1.0.9, dig/nslookup 9.18.39, sysstat (iostat) 12.6.1, vmstat (procps) — all flags verified via `<cmd> --help`/`<cmd> -h`, `man strace`, and `man nft` run locally; netstat verified via man7.org/linux/man-pages/man8/netstat.8.html only (net-tools not installed in this environment), 2026-08-21 · official docs
Finding what's listening on a port, what a process has open, what syscalls it's making, and what's actually on the wire.
Sockets and listening ports — ss (modern) and netstat (legacy)#
ss -tulpn # all TCP+UDP listening sockets, with PID/program, numeric ports
ss -tan state established # established TCP connections only
ss -s # summary counts by protocol/state
netstat -tulpn # the older equivalent of the ss command above
netstat -r # kernel routing tabless is the current tool — it reads directly from the kernel and is significantly faster on a host with many connections; netstat (from net-tools) is legacy and not installed by default on many modern distros, including this one. Know both: ss for anything you run yourself, netstat for reading someone else's old runbook or a minimal/legacy host where ss isn't available either.
What's using a file, port, or directory — lsof#
lsof -i :8080 # what process (if any) is bound to port 8080
lsof -p 12345 # every file descriptor a specific PID has open
lsof -u myuser # every open file belonging to a user
lsof +D /var/log # every process with an open file under a directorylsof -i :PORT is usually the fastest way to answer "what's already using this port" when a service fails to bind on startup — faster than cross-referencing ss -tlpn output by eye.
Tracing syscalls — strace#
strace -p 12345 # attach to a running process and watch its syscalls live
strace -f -p 12345 # + follow any child processes it forks
strace -c -p 12345 # summary: syscall counts and time spent, not a live stream
strace -e trace=network -p 12345 # only network-related syscalls (connect, accept, sendto, ...)
strace -tt -o trace.log myprogram arg1 # run a fresh command under strace, with timestamps, to a filestrace adds real overhead to the traced process — fine for a one-off diagnostic attach, but not something to leave running against a production process under load without deciding that tradeoff deliberately first.
Capturing packets — tcpdump#
tcpdump -i eth0 # capture on a specific interface
tcpdump -i eth0 -n # -n: don't resolve hostnames (faster, avoids DNS noise in output)
tcpdump -i eth0 port 443
tcpdump -i eth0 host 10.0.1.5 and port 443
tcpdump -i eth0 -w capture.pcap # write raw packets to a file for later analysis (e.g. in Wireshark)
tcpdump -i eth0 -c 100 -X # capture exactly 100 packets, with hex+ASCII payload dumpCapturing on the wrong interface (eth0 vs a container's veth vs lo) is the most common reason "tcpdump shows nothing" during an actual incident — tcpdump -D lists every available interface if you're not sure which one carries the traffic you're chasing.
Blocking and inspecting traffic — iptables and nftables#
iptables is the legacy packet-filtering interface; nftables (nft) is its modern replacement. On this host iptables -V reports nf_tables as the backend, meaning both tools ultimately manage the same underlying kernel ruleset.
iptables -L -n -v # list all rules in the filter table, numeric, with packet/byte counters
iptables -L -n --line-numbers # same, with rule numbers (needed to target -D by position)
iptables -A INPUT -p tcp --dport 22 -j ACCEPT # append a rule to the INPUT chain
iptables -D INPUT 3 # delete rule #3 from INPUT
iptables -P INPUT DROP # change a chain's default policy
nft list ruleset # show the entire nftables ruleset, all tables/chains
nft add table inet mytable # create a new table (inet = both IPv4 and IPv6)
nft add chain inet mytable input '{ type filter hook input priority filter ; }'
nft add rule inet mytable input tcp dport 22 accept # add a rule to the chain
nft -a list ruleset # include rule handles, needed to target a specific rule for deletionDon't assume every host's iptables is nf_tables-backed like this one — some distros still ship the legacy iptables-legacy backend, where iptables and nft manage genuinely separate rulesets that can't see each other. Check iptables -V first.
DNS lookups — dig and nslookup#
dig example.com # full DNS answer, authority, and additional sections
dig example.com +short # just the answer, one line
dig example.com MX # query a specific record type
dig @8.8.8.8 example.com # query a specific nameserver directly, bypassing local resolver config
dig -x 93.184.216.34 # reverse lookup (PTR record)
dig example.com +trace # trace the full delegation path from the root nameservers down
nslookup example.com # quick forward lookup using the system resolver
nslookup example.com 8.8.8.8 # query a specific nameserverdig is the more capable tool for real troubleshooting — it exposes TTLs, which nameserver actually answered, and +trace for delegation problems. nslookup is faster to type for a quick sanity check but BIND's own docs point users toward dig/host instead.
System-wide performance — iostat and vmstat#
iostat -x 2 # extended per-device stats (%util, await, queue depth), every 2s
iostat -d -x sda 5 3 # extended stats for one device only, every 5s, 3 samples
vmstat 2 5 # 5 samples of memory/swap/IO/CPU summary, 2s apart
vmstat -a # active vs inactive memory instead of the default free/buff/cache split
vmstat -s # cumulative event counters since boot, not a live sampleiostat -x's %util column is usually the fastest way to tell "is this disk actually the bottleneck" — it approaches 100% when the device is saturated, regardless of raw throughput, which kB/s numbers alone don't tell you on their own.
Deeper strace usage#
strace -e trace=%file -p 12345 # only file-related syscalls (open, stat, unlink, ...)
strace -T -p 12345 # show time spent inside each syscall
strace -y -p 12345 # resolve file descriptors to their paths inline in the output
strace -c -e trace=%file myprogram # syscall-count summary, filtered to the file group, for a fresh command-e trace=%GROUP (e.g. %file, %network, %process, %signal) is the fast way to cut a noisy trace down to the syscall class you actually care about — check strace -e trace=? (or the strace(1) man page) for the full, current group list, since group membership has changed across strace releases.
Inspecting a running process via /proc#
cat /proc/<pid>/status # human-readable state, memory, thread count, uid/gid
cat /proc/<pid>/limits # the process's actual resolved ulimits
ls -l /proc/<pid>/fd # every open file descriptor, as symlinks to what they point to
cat /proc/<pid>/cmdline | tr '\0' ' ' # exact command line it was started with (NUL-separated, tr fixes it for display)
cat /proc/<pid>/environ | tr '\0' '\n' # its environment variables at start time
cat /proc/meminfo # system-wide memory stats (what free/vmstat parse)
cat /proc/loadavg # the 1/5/15-minute load averages/proc/<pid>/limits is the one to check when a process is hitting a "too many open files" or similar resource error — it shows the limits actually in effect for that specific process, which can differ from your own shell's ulimit -a if the process was started by a different parent (systemd, cron, a container runtime) with its own limits configured.