Basic AWK command examples.

Along with SED command Linux – AWK is powerful command line tool which would help you to perform a lot of actions on your data. It can help you to sort, print and process your data in the way you like. You can use AWK to read and edit the file.

AWK is mostly used in the Shell Scripts. Lets check out Awk without wasting more time šŸ™‚

Let’s create a file called data.txt with the following contents – so we can learn AWK command better with examples.

Bob America Married 50
Jay India Unmarried 25
Abdul UAE married 33
Mona London Married 28
John America Unmarried 18
Neha India Unmarried 16

Run the command below which will print the contents in the file as it is.

[root@server ~]# awk '{print}' data.txt

Bob America Married 50
Jay India Unmarried 25
Abdul UAE married 33
Mona London Married 28

Print all the lines Matching with America.

[root@server ~]# awk '/America/ {print}' data.txt
Bob America Married 50
John America Unmarried 18

Only Print age from the file. As you can see that Age is written in the 4 column so we will use the command below.

[root@server ~]# awk '{print $4}' data.txt
50
25
33
28
18
16
[root@server ~]#

So now if you want to print Name and Age.

[root@server ~]# awk '{print $1, $4}' data.txt
Bob 50
Jay 25
Abdul 33
Mona 28
John 18
Neha 16

You can print age first by changing the order.


[root@server ~]# awk '{print $4, $1}' data.txt
50 Bob
25 Jay
33 Abdul
28 Mona
18 John
16 Neha
[root@server ~]#

Conditional Printing

I would like to print the names of the people who live in America. So here you will use If Condition in awk. In the example below it will check if the America is present in the column 2 it will print the column 1

[root@server ~]# awk '{ if ($2 == "America") print $1;}' data.txt
Bob
John

In the above example, if you just putĀ awk ‘{ if ($2 == “America”) print;}’ data.txt it will print the complete line.

AWK can do much more that what I have specified above. I will come up with Advanced AWK article soon.

Restart service automatically and get notified.

So you are running you own webserver which at the times is short of resources and the services like MySQL and HTTPD stops – results your website is down until you get a chance to fix it.

Quick solution would be to implement a script which restarts services at such situations and emails you that the service was restarted – so you can later go and check what was the problem

So below is the script, which will make sure that HTTP , MySQL and Postfix is running all the time – It will check status of the service and if it’s stopped or Dead it will restart it.

How to Implement this Script ?

Setup a cron to run the script every minute – like below.

* * * * * root /home/bob/scripts/servicecheck.sh

Create a file called servicecheck.sh and put the code below.
Just change the email address in the script to yours and you are done.

#!/bin/bash

#####################
#MySQL CHECK
#####################

/sbin/service mariadb status > /var/log/mysqlservicestatus 2>&1
STATUS=$(cat /var/log/mysqlservicestatus | egrep 'dead|failed')

        if [[ $STATUS = "" ]]
        then
/bin/logger -t MYSQL IS RUNNING
else
/sbin/service mariadb restart
/bin/logger -t restarted mysql
echo "Restarted MYSQL" | /bin/mail -s "Restarted MYSQL" you@email.com
fi
echo > /var/log/mysqlservicestatus

#####################
#HTTPD CHECK
#####################

/sbin/service httpd status > /var/log/httpdservicestatus 2>&1
STATUS=$(cat /var/log/httpdservicestatus | egrep 'dead|failed')

        if [[ $STATUS = "" ]]
        then
/bin/logger -t httpd IS RUNNING
else
/sbin/service httpd restart
/bin/logger -t restarted httpd
echo "Restarted HTTPD" | /bin/mail -s "Restarted HTTPD" you@email.com
fi

echo > /var/log/httpdservicestatus


#####################
#POSTFIX CHECK
#####################

/sbin/service postfix status > /var/log/postfixservicestatus 2>&1
STATUS=$(cat /var/log/postfixservicestatus | egrep 'dead|failed')

        if [[ $STATUS = "" ]]
        then
/bin/logger -t postfix IS RUNNING
else
/sbin/service postfix restart
/bin/logger -t restarted postfix
echo "Restarted postfix" | /bin/mail -s "Restarted postfix" you@email.com
fi

You want to monitor other services, just go ahead and add it.

Disable direct root Login.

In previous post we saw how to Set Alerting for root login and How to Change Default SSH Port. To add more to the security you should also disable direct root Login and specific users should be allowed only.

To disable Direct root Login, you need to follow the steps below.

1. Open the file /etc/ssh/sshd_config and add the line below.

PermitRootLogin no

2. Create a new user – e.g. user bob and set a password to that user.

3. Open file /etc/group and add the user to the Wheel group. So the Wheel group in that file would look like this.

[root@server ~]# cat /etc/group | grep -i wheel
wheel:x:10:bob
[root@server ~]#

4. Finally restart sshd service.

[root@server ~]# systemctl restart sshd

5. Please make sure that permissions to below are set correctly.

chmod 4755 /bin/su
chmod 1700 /etc/passwd
chmod 1700 /etc/shadow
chmod 1755 /etc/groups

Remember the Golden Rule? Yes, always verify the changes from a different session before logging out the current one.

Set alerting for root login.

If you are using Linux server, you must have enhanced security by Changing Default SSH port. (If you haven’t you should do it immediately). To further secure your environment, you should consider setting alerts for root login.
Setting alerts for root account login is useful in many ways. To track who logged in and for security reason,

Steps to enable Alerting for root login

  1. Login to server and switch to root user.
  2. Open file /root/.bash_profile and append the code below.
/usr/bin/echo 'ALERT - Root Shell Access on:' `date` `who` | mail -s "Alert: Root Access from `who | cut -d"(" -f2 | cut -d")" -f1`" email@yourdomain.com

Simple, isn’t it?
You can try to login/switch to root user and you should get the email like this.

ALERT - Root Shell Access on: Thu Jul 23 09:42:46 UTC 2020 user pts/0 2020-07-23 09:42 (xx.xx.xx.xx)

Echo Command in Linux

Echo command mainly prints on the screen whatever you ask it. It a simple function but most the scripts would be incomplete with echo command. You won’t be able get visible output from the shell script, without echo command.

How to check what version of echo command you are using?

[root@server ~]# /bin/echo --version
echo (GNU coreutils) 8.22
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Brian Fox and Chet Ramey.

How to print using echo command ?

it’s actually quite simple. Just use echo command following by the text.

[root@server ~]# echo "Hello JustGeek"
Hello JustGeek
[root@server ~]#

Printing Variables.

it’s not just for the print text there are many things you can do using echo command. Example you can print Variables.

[root@server ~]# IamVariable="This is JustGeek"
[root@server ~]# echo $IamVariable
This is JustGeek
[root@server ~]#

You can also run a command through echo command and print it’s output to the screen.

[root@server ~]# echo "This server is online from $(uptime | awk '{print $3,$4}')"
This server is online from 77 days,

Formatting using Echo Command.

To use formatting functions with echo -e. It enablesĀ interpretation of backslash escapes.
eg:- “\n” parameter will print on the new line. As shown below.

[root@server ~]# echo -e "First Line\nSecond Line"
First Line
Second Line
[root@server ~]#

There are multiple characters which can be used which suits you.

\a: Alert (historically known as BEL). This generates the default alert sound.
\b: Writes a backspace character.
\c: Abandons any further output.
\e: Writes an escape character.
\f: Writes a form feed character.
\n: Writes a new line.
\r: Writes a carriage return.
\t: Writes a horizontal tab.
\v: Writes a vertical tab.
\\: Writes a backslash character.