awk is a text processing command like search, replace, and manipulate text. It allows you to use variables, functions, and operators. Some Engineers said it is another programming language.
Options are the real power of the awk command. Here I’ll cover the most common options.
-
-F <delimiter>input field separator or delimiter. -
-v name=keyassign multiple variables. -
;separate two or more actions. -
printreturns all the lines. -
$0return first line. -
$1return first field. -
$NFreturn last field. -
$FSreturn field separator, default is whitespace. -
NRreturns a total number of lines.
I have created dummy content for you to test all the above options.
1
2
3
4
5
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh
sys:x:3:3:sys:/dev:/bin/sh
sync:x:4:65534:sync:/bin:/bin/sync
Get options listed above and put them one by one to the following structure.
You have 3 options to run the awk command. In this article, I will be using option 1 most of the time but you are free to choose other options as well.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# option 1
awk [options] 'selection {action_or_function_or_condition}' filename.txt
# option 2
cat filename.txt | awk [options] 'selection {action_or_function_or_condition}'
# option 3
<ANY_LINUX_COMMAND> | awk [options] # so on...
# to execute BEGIN code first.
# we use this for creating functions or setting conditions or creating variables or etc.
awk 'BEGIN { YOUR_CODE_WILL_BE_HERE }' filename.txt
# to execute between BEGIN and END code, this option we normally use every time.
awk '{}' filename.txt
# to execute END code at the end of the file.
awk 'END {}' filename.txt
To return the first field of all lines using a colon as a field separator.
1
awk -F: '{print $1}' filename.txt
To return the last field of all lines.
1
awk -F: '{print $NF}' filename.txt
To return the total number of lines.
1
awk 'END {print NR}' filename.txt
To return 3 to 5 records.
1
awk -F: 'NR==3, NR==5' filename.txt
To return even records.
1
awk -F: 'NR % 2 == 0' filename.txt
To return the line of matched pattern.
1
awk '/root/ {print $1}' filename.txt
To return the line of matched pattern, find only between 1 and 3 lines.
1
awk '/root/ {print $1,$3}' filename.txt
awk and bash scripting both have similar operators, array, dictionary, if-else statement, case statement, loops and etc. read this article.
Once you read the article mentioned above, You can add any features inside the BEGIN section as mentioned in the following example.
1
2
3
4
5
6
7
# dictionary
awk 'BEGIN {
fruits["mango"] = "yellow";
fruits["orange"] = "orange"
print fruits["orange"]
print fruits["mango"]
}'
awk provides built-in functions too.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# length of string
awk '{print length("hello")}' filename.txt # ==>> 5
# to uppercase
awk '{print toupper("hello")}' filename.txt # ==>> HELLO
# to lowercase
awk '{print tolower("HELLO")}' filename.txt # ==>> hello
# to return x number of characters
awk '{print substr("hello", 1, 3)}' filename.txt # ==>> hel
# to convert x to int
awk '{print int(12.350)}' filename.txt # ==>> 12
# to find x and replace all
awk 'BEGIN {
str = "One Two Three Four Five"
print "before = " str
gsub("Four", "4", str)
print "after = " str
}' filename.txt
To create a custom function. It must be created outside the BEGIN section otherwise you will end up with a lot of errors.
1
2
3
4
5
6
7
8
9
10
11
12
awk '
function min(n1, n2) {
if (n1 < n2)
return "smaller than " n2;
return "larger than " n2;
}
# now you can call the function
BEGIN {
print min(10, 60)
}
'
To create a custom header using formatted string.
1
2
3
4
5
6
7
8
9
awk -F: 'BEGIN {
# add header,
# if you dont want to keep,
# then remove only BEGIN section.
printf "%-10s %s\n", "User", "Home"
printf "%-10s %s\n", "----", "----"
}
{ printf "%-10s %s\n", $1, $(NF-1) }
' filename.txt