awk in Linux: Columns, Patterns, One-Liners

awk is the column specialist of the Unix toolbox. Where grep finds lines and sed edits them, awk understands that a line has fields — and that is the moment log files and command output become data.

Fields and patterns #

An awk program is pairs of pattern and action. The default pattern is "every line", and $1, $2, … are the whitespace-separated fields:

Print the first and third column

$ awk '{ print $1, $3 }' access.log

Add a pattern before the block and awk filters first:

Print the user column for bash users

$ awk -F: '/bash$/ { print $1 }' /etc/passwd

-F: sets the field separator to a colon — the natural choice for /etc/passwd, CSV after -F,, and so on.

The one-liners worth memorising #

Task	One-liner
Last field of each line	`awk '{ print $NF }'`
Line numbers	`awk '{ print NR, $0 }'`
Filter by column value	`awk '$3 > 100'`
Sum a column	`awk '{ s += $5 } END { print s }'`
Count lines	`awk 'END { print NR }'`
Rows where column matches	`awk '$2 == "error"'`

The END block runs after the input ends — the natural place for totals, which is why the sum idiom appears in every awk user's history.

A worked example #

Take a web server log where field 9 is the status code. The distribution of responses in one pass:

Count responses by status code

$ awk '{ count[$9]++ } END { for (s in count) print s, count[s] }' access.log | sort -n

That single line introduces arrays and loops — awk is a real language, and it rewards exactly this kind of gradual adoption. Average response size, also by status:

Average bytes per status class

$ awk '$9 >= 500 { n++; b += $10 } END { if (n) print b/n }' access.log

awk pairs naturally with sort and uniq for ranked counts, and lives inside the wider pipe discipline covered in unix pipes in practice. The rest of the terminal notes series builds on these foundations.