Posted onMar 8

AWK Basics Tutorial

#awk #bash #python #javascript

Leapcell: The Next-Gen Serverless Platform for Web Hosting, Async Tasks, and Redis

A Concise AWK Tutorial

I. Basic Concepts

AWK is a built-in text processing tool in Linux systems, specializing in handling structured text (e.g., logs, CSV files). It reads files line by line, processes data by fields, and supports simple programming logic.

II. Basic Syntax

1. Fundamental Format

awk[options]'actions' filename

2. Simplest Examples

# Print entire file contentawk'{print $0}' demo.txt# Process standard input via pipeecho'this is a test' |awk'{print $0}'

3. Field Handling

$1: First field
$2: Second field
$0: Entire line
NF: Total number of fields in current line
$NF: Last field

# Extract third fieldecho'this is a test' |awk'{print $3}'# Output: a# Extract second-to-last fieldecho'a,b,c,d' |awk-F',''{print $(NF-1)}'# Output: c

III. Core Functions

1. Field Separator

# Specify colon as separatorawk-F':''{print $1}' /etc/passwd

2. Built-in Variables

Variable Name	Description	Example
`NR`	Current line number	`awk '{print NR}' file`
`FS`	Input field separator (default space)	`awk -v FS=: '{print $1}'`
`OFS`	Output field separator (default space)	`awk -v OFS=, '{print $1,$2}'`
`FILENAME`	Current file name	`awk '{print FILENAME}' file`

IV. Advanced Operations

1. Conditional Filtering

# Regular expression match: Print lines containing "usr"awk-F':''/usr/ {print $1}' /etc/passwd# Numeric comparison: Print content after line 3awk-F':''NR > 3 {print $1}' /etc/passwd# Combined conditionsawk-F':''$1 == "root" || $3 > 1000' /etc/passwd

2. Built-in Functions

Function Name	Function	Example
`toupper()`	Convert to uppercase	`awk '{print toupper($1)}'`
`length()`	String length	`awk '{print length($1)}'`
`substr()`	Substring extraction	`awk '{print substr($1,3,5)}'`
`rand()`	Generate random number	`awk '{print int(rand()*100)}'`

V. Control Statements

1. Single-line Conditions

# Process odd-numbered linesawk'NR % 2 == 1 {print "Line", NR}' file# Field comparisonawk-F':''$3 > 1000 {print $1}' /etc/passwd

2. Multi-line Logic

awk-F':''{  if ($1 > "m") {    print "High:", $1  } else {    print "Low:", $1  }}' /etc/passwd

VI. Practical Tips

Formatted Output: Useprintf instead ofprint

awk-F':''{printf "%-10s %s\n", $1, $3}' /etc/passwd

Large File Handling: Memory-friendly line-by-line processing
Tool Integration: Combine withgrep/sed

VII. Quick Reference

# Common command combinationsawk-F':''/^root/ {print $1}'# Lines starting with rootawk-F':''!/nologin/ {print $1}'# Exclude lines containing nologinawk-F':''$3 ~ /[0-9]{4}/'# Match 4-digit fields