linux
Table of Content

Weight: 3

Description: Candidates should should be able to apply filters to text streams.

Key Knowledge Areas:

  • Send text files and output streams through text utility filters to modify the output using standard UNIX commands found in the GNU textutils package

Terms and Utilities:

cat
cut
expand
fmt
head
join
less
nl
od
paste
pr
sed
sort
split
tail
tr
unexpand
uniq
wc

cat

cat file > rename.txt

cat -b file  # add line number , skip empty line
cat -n file  # add line number 

tac file # same as cat but reverse order of lines

join, paste

join file1 file2  #  default both files using 1st field as common to match cross files

join -1 3 -2 3 file1 file2   # join field as file1 field 3 and file2 field 2

sort -n text2|join -j 1 text1 -    # join on field 1 

paste to merge 2 files

paste file1 file2

paste -d '-' file1 file2

expend , unexpend

expend -t 4 # convert tab to 4 spaces
unexpend -t 4 # convert 4 spaces to tabe

cat /etc/passwd | tr : '\t' | expand -t 10,13,16,19,28,44

0d

od file

oldhorse@dclab-u1504s:~$ echo "Hello World" > hello
oldhorse@dclab-u1504s:~$ od hello
0000000 062510 066154 020157 067527 066162 005144
0000014
oldhorse@dclab-u1504s:~$ od -t c hello
0000000   H   e   l   l   o       W   o   r   l   d  \n
0000014

sort

du -akd|sort -nr |  more

sort -k 3 file
sort -f  # ignore case
sort -M # sort by 3 letter month

split

split -l linenumber file
split -b sizebytes file

# split and recombine big file
split -b 32k x.tar.gz part-
ls -la part-*
cat part-* > newx.tar.gz

tr

change characters from stdin

tr       notepad.unix.txt  # move \r 

man -P cat man | tr A-Z a-z | less

uniq

sort file| uniq

fmt

fmt -w n or -n file  # format width n

nl

nl -ba file # same as cat -b file
nl -bt file # same as cat -n file
nl -bn file # body no number

pr

pr -n file # n column 
pr -d file # double line space
pr -h "my header" file
pr -o n file # left margin n char

cat -n file|pr -d | lpr

cat -n fie | pr -dfl 50 |lpr

head

head -n num file
head -num file

same to print 1st num lines

#get a copy of the partition sector of a disk
head -c 512  mbr

tail

tail -n num or -num file
tail -f /halog

less

view file with search/page control like vi

cut

cut -d  -f 
ifconfig eth0 | grep HWaddr | cut -d "  " -f 11  #  not like awk column

wc

wc file
lines words bytes
wc -l  # line
wc -m  # char
wc -c  # bytes
wc -w  # word

sed

• s/PATTERN/REPLACEMENT/g – search and replace.
• /PATTERN/d – delete all lines contains the pattern
• 4d – delete line 4
• 4,10d – delete lines 4 to 10
• 6,$d – delete from line 6 to the last line

echo "Hi Fred, how is Fred?" | sed 's/Fred/Joe/g'
Hi Joe, how is Joe?

ls /bin/ | fmt -30 | nl | sed '4,15d'    # delete line 4 to 15 

sed 's/a/A/g' file
sed '2d;$s/a/A/g' file

echo "color is gray" |sed 's/color/colour/g;s/gray/grey/g'

Quiz questions

Which commands perform the following functions:
1. Show the first few lines of a file
2. Show the last few lines of a file
3. Select a column from a file
4. Send the entire file
5. Convert new lines to spaces
6. Convert spaces to tabs
7. Convert tabs to spaces
8. Arrange in alphabetical order
9. Remove duplicate lines
10.Rearrange lines from last to first
11.Substitute all instances of one character with another
12.Add line numbers to output
13.Search for and replace text in a stream
14.Apply a consistent right margin to text
15.Show a file in hexadecimal, octal or other formats
16.Convert a text file to postscript
17.Determine the size of a text stream

Answers to quiz questions

1. head
2. tail
3. cut
4. cat
5. fmt
6. unexpand
7. expand
8. sort
9. sort | uniq
10.tac
11.tr
12.nl
13.sed
14.fmt
15.od
16.pr doesn't – none of these commands do – mpage does it though.
17.wc -l