Linux Three Musketeers Quick Survey| grep/awk/sed tutorials- GoTool Online Toolset

Linux Three Musketeers (grep, awk, sed)

grep

• grep is a powerful command-line tool used to search for specified strings or regular expressions in files or text.
• Basic Syntax

grep [options] "search pattern" file

Search pattern: Can be a plain string or a regular expression.
File: Specifies the file(s) to search, which can be one or more files or input from a pipeline.

Common Options:

Option	Description
-i	Ignore case while matching.
-v	Invert match, show non-matching lines.
-c	Only output the count of matching lines.
-n	Show matching line numbers.
-l	Only list filenames with matches.
-o	Show only matching parts, not the whole line.
-r	Recursively search in directories.
-w	Match whole words only.
-e	Specify a regular expression.

Examples:

• Display all lines containing "hello" in file.txt:

grep "hello" file.txt

• Search for "hello" in all files under /path/to/directory:

grep -r "hello" /path/to/directory

• Match "error" or "ERROR" regardless of case:

grep -i "error" log.txt

• Show matching lines along with line numbers:

grep -n "pattern" file.txt

• List all files containing "keyword" in the current directory:

grep -l "keyword" *.txt

• Show lines containing "important" along with 2 lines before and after:

grep -C 2 "important" document.txt

---

awk

• awk is a powerful text processing tool used for formatting, filtering, and analyzing data in files or text streams. It supports pattern-based matching and operations, making it useful for log analysis, data extraction, and simple scripting.
• Basic Syntax

awk 'pattern { action }' file

Pattern: The condition to match input data (optional).
Action: The operation to perform on matched data (optional).
If pattern is omitted, it matches all lines.
If action is omitted, it prints the matched lines by default.

Common Options:

Option	Description
-F	Set input field separator (default: space/tab).
-v	Define and pass a variable to awk.
BEGIN	Executes before processing data (e.g., initialization).
END	Executes after processing all data.
Condition	e.g., $1 > 100 matches lines where the first field is greater than 100.

Built-in Variables:

Variable	Description
$0	The entire current line.
$1...	The first, second, etc., fields of the line.
NR	Current line number (Number of Records).
NF	Number of fields in the current line.
FS	Input field separator (Field Separator).
OFS	Output field separator (Output Field Separator).

Examples:

• Print each line of a file:

awk '{ print $0 }' file.txt

• Print the first and third fields of each line:

awk '{ print $1, $3 }' file.txt

• Print the second field using a comma as the delimiter:

awk -F "," '{ print $2 }' data.csv

• Print lines where the first field is greater than 50:

awk '$1 > 50 { print $0 }' file.txt

• Print lines containing "error":

awk '/error/ { print $0 }' log.txt

• Print each line along with the number of fields it has:

awk '{ print $0, "Fields:", NF }' file.txt

• Print the total number of lines after processing:

awk 'END { print "Total lines:", NR }' file.txt

• Print "Processing started" before reading, the first field for each line, and "Processing finished" after:

awk 'BEGIN { print "Processing started" } { print $1 } END { print "Processing finished" }' file.txt

• Print lines where the first field is greater than 100 using a variable:

awk -v threshold=100 '$1 > threshold { print $0 }' file.txt

• Calculate the sum of the first field in the file:

awk '{ sum += $1 } END { print "Total:", sum }' file.txt

---

sed

• sed (Stream Editor) is a powerful tool for performing batch text processing, including search and replace, deletion, and insertion. It works with files or standard input text streams.
• Basic Syntax

sed [options] 'script command' file

Script command: Specifies the rule for text processing, such as replacement or deletion.
File: The file to process, or text can be passed through a pipeline.

Common Options:

Option	Description
-n	Suppress automatic output; only print specified lines.
-e	Allow multiple script commands.
-i	Edit the file in place (modify it directly).
-f	Load script commands from a file.

Common Commands:

Command	Description
s/old/new/	Replace old with new.
d	Delete the matching lines.
p	Print matching lines (use with -n).
a\text	Append text after the current line.
i\text	Insert text before the current line.
c\text	Replace matching lines with new text.
q	Quit early, stopping further processing.

Examples:

• Replace the first occurrence of old with new in file.txt:

sed 's/old/new/' file.txt

• Delete the second line of file.txt:

sed '2d' file.txt

• Append "This is a new line" after the third line:

sed '3a\This is a new line' file.txt

• Insert "This is an inserted line" before the third line:

sed '3i\This is an inserted line' file.txt

• Replace the content of the second line with "This is the new content":

sed '2c\This is the new content' file.txt

• Print only lines containing "pattern":

sed -n '/pattern/p' file.txt

• Delete the first line and replace "foo" with "bar":

sed '1d; s/foo/bar/' file.txt

• Modify file.txt directly, replacing all occurrences of old with new:

sed -i 's/old/new/g' file.txt

• Delete lines 2 through 4:

sed '2,4d' file.txt

• Apply commands from script.sed to file.txt:

sed -f script.sed file.txt

These three tools—grep, awk, and sed—are essential for powerful text processing in Linux.