List of commands that can be used for monitoring performance :
cron Process scheduling
nice/renice Change priorities
setpri Set priorities
netstat Network statistics
nfsstat NFS statistics
time/timex Process CPU Utilization
uptime System Load Average
ps Process Statistics
iostat BSD tool for I/O
sar Bulk System Activity
vmstat BSD tool for V. Memory
gprof Call Graph profiling
prof Process Profiling
trace Used to get more depth
tprof CPU Usage
svmon Memory Usage
filemon Filesystem, LV .. activity
netpmon Network resources
Category: Unix
Unix – crontab
Taken from – http://www.rahul.net/raithel/MyBackPages/crontab.html
The Linux utilities cron and at are related commands. The cron utility allows you to schedule a repetitive task to take place at any regular interval desired, and the at command lets you specify a one-time action to take place at some desired time. You might use crontab, for example, to perform a backup each morning at 2 a.m., and use at to remind yourself of an appointment later in the day.
Using crontab
The word “crontab” is a UNIXism for chron table, or time table. You create a table in the required format, specifying commands and times to execute the commands. Commands you put in the table can be any executable programs, for example, a command in /usr/bin or a shell script you wrote. You use the crontab command to create, edit, or list the table, and the system cron daemon reads the table and executes the commands at the times specified. The cron Daemon
The cron daemon is normally executed at system startup and does not exit. On my Linux system, the cron daemon is actually Matthew Dillon’s crond, and is started in /etc/rc.d/rc.M with the following line:
/usr/sbin/crond -l10 >>/var/adm/cron 2>&1
On some Linux systems, Paul Vixie’s cron daemon is used, in which case the name of the daemon is simply cron. Also, on systems with newer versions of init, cron is started from the /etc/init.d/cron script.
You can check to see if a cron daemon is running on your system with a command such as the following:
$ ps -ax | grep cron raithel 733 pp0 S 0:00 grep cron root 25 ? S 0:00 /usr/sbin/crond -l10
In this case, we see that the crond daemon is indeed running.
The crontab Table
When the cron daemon starts, it reads the various crontab tables in the crontab directory, normally /usr/spool/cron/crontabs. To create or change your crontab file, use crontab’s -e option:
$ crontab -e
You are placed in a text editor with a copy of your current crontab file if it exists, or a blank file if it does not. The text editor you get is determined by the setting of your VISUAL (or EDITOR, if VISUAL is not set) environment variable, and is usually the vi editor if you have not specified otherwise. To schedule commands with crontab, you must use the format crontab recognizes in a file. The format is not exactly mnemonic, so I create a crontab file with a commented-out header that provides the necessary information:
# minute (0-59), # hour (0-23), # day of the month (1-31), # month of the year (1-12), # day of the week (0-6, 0=Sunday), # command
Each crontab entry is a single line composed of these six fields separated by whitespace. Specify the minute a command is to be executed with the digits 0 through 59 in the first field, the hour with 0 through 23 in the second field, the day of the month with 1 through 31 in the third field, the month of the year with 1 through 12 in the fourth field, and the day of the week with 0 through 6 in the fifth field. Place the command to be executed in the sixth field. At first glance it may appear that redundant or conflicting information is required because there are two “day” fields—day of the month and day of the week, but really this is just to permit different scheduling algorithms. For instance, you may want to be reminded to attend a meeting every Tuesday, or to pick up your paycheck every 15th of the month. Enter an asterisk (*) in the day field you are not using. You can use both day fields if you prefer to have the command execute on, say, the fifteenth of the month as well as every Tuesday.
Ranges are specified with a dash. If you want to specify the eighth through the fifteenth days of the month, enter 8-15 in the third field. Non-consecutive entries in a field are separated by commas, so 1,15 in the third field means the first and fifteenth of the month. To specify all values for a field, for example every month of the year, enter an asterisk (*) in the field. (Note that to specify every day you must enter * in both day fields.)
Here is an example crontab file with two entries:
# minute (0-59), # hour (0-23), # day of the month (1-31), # month of the year (1-12), # day of the week (0-6, 0=Sunday) # command 12 4 * * * /usr/local/bin/backup 5 3 10-15 4 * echo "taxes due" | mail jones
The first line after the comments causes a backup script to execute early each morning at 4:12 a.m., and the second line causes the user jones to get a mail message for six days in April as a reminder that taxes are due. In general, it’s a good idea to execute crontab commands at off hours like this to reduce any affect on system load during normal usage hours. If you don’t specifically redirect standard error or standard output, they are mailed to you as owner of the crontab file when the command executes. In the example above, if the user jones cannot be found, you would be mailed the output as well as an error message.
After editing the crontab file, save it and exit from the editor. A file is created for you in the crontab directory. For example, the crontab for root is the file /usr/spool/cron/crontabs/root. This file is read by the system cron daemon and stored in an internal format where it will remain to be periodically executed until it is changed or deleted.
To view your current crontab file, use the -l (for “list”) option:
$ crontab -l
To delete your file, use:
$ crontab -d
If you are superuser, you can delete any user’s crontab file with:
# crontab -d username
where username is the user’s login name. The crontab commands discussed above work fine on my Linux system, and should work on System V and BSD UNIX systems as well. One thing to be aware of when using crontab on other systems or moving crontab files to other systems, is that some cron daemons allow the superuser to restrict crontab service by the creation of cron.allow and cron.deny files. Refer to the specific system documentation for details.
Also, most versions of cron provide an /etc/crontab file which has an extra field in it—the user as which to execute the command. Again, check the documentation for your version of cron for more details.
Using at
Use at when you want to execute a command or multiple commands once at some future time. In Linux, the at command requires that the atrun command be started in root’s crontab file. Many Linux distributions ship with at enabled, but some do not. To enable the at utility on your system, become superuser and edit root’s crontab file:
$ su root Password: # crontab -e
and add the following line:
* * * * * directory/atrun
where directory is the location where the atrun executable is stored. On my system that’s /usr/lib, so the entry is:
* * * * * /usr/lib/atrun
This causes atrun to be executed every minute. After a minute or so of adding the atrun line and saving the crontab file, any existing at commands are evaluated and executed if the time is right. (Before this, you may have been able to enter at commands, but they were never executed.) To demonstrate the at command, let’s have it print “hello” on your current terminal window after a few minutes. First, get the time and your current terminal device:
$ date Tue Oct 3 15:33:37 PDT 1995 $ tty /dev/ttyp2
Now run the at command. Specify a time in the command line, press Return, and then enter the command, followed by another Return and a Control-D:
$ at 15:35 echo "hello" > /dev/ttyp2 ^D Job c00ceb20b.00 will be executed using /bin/sh
The at command takes input up to the end-of-file character (press Control-D while at the beginning of a line). It reports the job number and informs you that it will use /bin/sh to execute the command. In two minutes, hello should appear on the display of /dev/ttyp2. Note that you can enter a series of commands, one per line—at will read each line up to the end-of-file and execute the file as a /bin/sh shell script at the specified time. Suppose you want to set an alarm. One way to tell at to do something is to use the relative form of timing, specifying a time relative to now. If you want your computer to beep at you in 25 minutes, enter:
$ at now + 25 minutes echo ^G > /dev/ttyp4 ^D Job c00ceb7fb.00 will be executed using /bin/sh
and you are beeped in 25 minutes. There is a great deal of flexibility allowed in entering time specifications. For example, at recognizes military time, “am” and “pm”, month abbreviations, times including the year, and so on. My at man page even claims that at accepts teatime, noon, and other constructs. Refer to the at man page for more examples of valid time specifications. You must tell at your tty location or it won’t send output to your terminal window. If you prefer, you can receive mail:
$ at 4:55pm Friday echo '5 p.m. meeting with Carol' | mail raithel ^D Job c00ceb7fb.01 will be executed using /bin/sh
To get a list of your pending at jobs, enter:
$ atq
If you are superuser, atq shows you the pending at jobs of all users. To delete a job, enter:
$ atrm job_number
where job_number is the job number returned by atq. The superuser can also remove other user’s jobs.
A Reminder Script Using at
The following is a simple script that makes it easier for me to use at to send myself reminders. The script sends mail to the user containing the message line(s) entered at the prompt at the time specified. It also displays some syntax examples of how to specify time, which I find a useful memory refresher. Note that the script as written requires you to have a Temp directory in your home directory. I created a $HOME/Msgs rather than use something like /usr/tmp so that the messages are more private until they are deleted by the script.
#!/bin/sh
echo "Enter your reminder message.
When finished, enter a period (.) at
the beginning of a line and press Enter.
(Or press Ctrl-C or DEL to exit.)"
while :
do
read MESSAGE
if [ "$MESSAGE" = "." ]
then
break
else
echo $MESSAGE >> $HOME/Msgs/message.$$
fi
done
cat << !!
Enter time and day you want to receive
the message, for example:
0815am Jan 24
8:15am Jan 24
now + 1 day
5 pm Friday
Then press Enter.
!!
read TIME
echo "at $TIME mail $LOGNAME $HOME/Msgs/message.$$"
at $TIME << !!
mail $LOGNAME < $HOME/Msgs/message.$$
rm -f $HOME/Msgs/message.$$
!!
exit 0
Unix – awk – one liners
Handy list of awk one liners taken from
http://www.pement.org/awk/awk1line.txt
USAGE:
Unix: awk ‘/pattern/ {print “$1”}’ # standard Unix shells
DOS/Win: awk ‘/pattern/ {print “$1”}’ # compiled with DJGPP, Cygwin
awk “/pattern/ {print \”$1\”}” # GnuWin32, UnxUtils, Mingw
Note that the DJGPP compilation (for DOS or Windows-32) permits an awk
script to follow Unix quoting syntax ‘/like/ {“this”}’. HOWEVER, if the
command interpreter is CMD.EXE or COMMAND.COM, single quotes will not
protect the redirection arrows (<, >) nor do they protect pipes (|).
These are special symbols which require “double quotes” to protect them
from interpretation as operating system directives. If the command
interpreter is bash, ksh or another Unix shell, then single and double
quotes will follow the standard Unix usage.
Users of MS-DOS or Microsoft Windows must remember that the percent
sign (%) is used to indicate environment variables, so this symbol must
be doubled (%%) to yield a single percent sign visible to awk.
If a script will not need to be quoted in Unix, DOS, or CMD, then I
normally omit the quote marks. If an example is peculiar to GNU awk,
the command ‘gawk’ will be used. Please notify me if you find errors or
new commands to add to this list (total length under 65 characters). I
usually try to put the shortest script first. To conserve space, I
normally use ‘1’ instead of ‘{print}’ to print each line. Either one
will work.
FILE SPACING:
# double space a file
awk ‘1;{print “”}’
awk ‘BEGIN{ORS=”\n\n”};1’
# double space a file which already has blank lines in it. Output file
# should contain no more than one blank line between lines of text.
# NOTE: On Unix systems, DOS lines which have only CRLF (\r\n) are
# often treated as non-blank, and thus ‘NF’ alone will return TRUE.
awk ‘NF{print $0 “\n”}’
# triple space a file
awk ‘1;{print “\n”}’
NUMBERING AND CALCULATIONS:
# precede each line by its line number FOR THAT FILE (left alignment).
# Using a tab (\t) instead of space will preserve margins.
awk ‘{print FNR “\t” $0}’ files*
# precede each line by its line number FOR ALL FILES TOGETHER, with tab.
awk ‘{print NR “\t” $0}’ files*
# number each line of a file (number on left, right-aligned)
# Double the percent signs if typing from the DOS command prompt.
awk ‘{printf(“%5d : %s\n”, NR,$0)}’
# number each line of file, but only print numbers if line is not blank
# Remember caveats about Unix treatment of \r (mentioned above)
awk ‘NF{$0=++a ” :” $0};1’
awk ‘{print (NF? ++a ” :” :””) $0}’
# count lines (emulates “wc -l”)
awk ‘END{print NR}’
# print the sums of the fields of every line
awk ‘{s=0; for (i=1; i<=NF; i++) s=s+$i; print s}’
# add all fields in all lines and print the sum
awk ‘{for (i=1; i<=NF; i++) s=s+$i}; END{print s}’
# print every line after replacing each field with its absolute value
awk ‘{for (i=1; i<=NF; i++) if ($i < 0) $i = -$i; print }’
awk ‘{for (i=1; i<=NF; i++) $i = ($i < 0) ? -$i : $i; print }’
# print the total number of fields (“words”) in all lines
awk ‘{ total = total + NF }; END {print total}’ file
# print the total number of lines that contain “Beth”
awk ‘/Beth/{n++}; END {print n+0}’ file
# print the largest first field and the line that contains it
# Intended for finding the longest string in field #1
awk ‘$1 > max {max=$1; maxline=$0}; END{ print max, maxline}’
# print the number of fields in each line, followed by the line
awk ‘{ print NF “:” $0 } ‘
# print the last field of each line
awk ‘{ print $NF }’
# print the last field of the last line
awk ‘{ field = $NF }; END{ print field }’
# print every line with more than 4 fields
awk ‘NF > 4’
# print every line where the value of the last field is > 4
awk ‘$NF > 4’
STRING CREATION:
# create a string of a specific length (e.g., generate 513 spaces)
awk ‘BEGIN{while (a++<513) s=s ” “; print s}’
# insert a string of specific length at a certain character position
# Example: insert 49 spaces after column #6 of each input line.
gawk –re-interval ‘BEGIN{while(a++<49)s=s ” “};{sub(/^.{6}/,”&” s)};1’
ARRAY CREATION:
# These next 2 entries are not one-line scripts, but the technique
# is so handy that it merits inclusion here.
# create an array named “month”, indexed by numbers, so that month[1]
# is ‘Jan’, month[2] is ‘Feb’, month[3] is ‘Mar’ and so on.
split(“Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec”, month, ” “)
# create an array named “mdigit”, indexed by strings, so that
# mdigit[“Jan”] is 1, mdigit[“Feb”] is 2, etc. Requires “month” array
for (i=1; i<=12; i++) mdigit[month[i]] = i
TEXT CONVERSION AND SUBSTITUTION:
# IN UNIX ENVIRONMENT: convert DOS newlines (CR/LF) to Unix format
awk ‘{sub(/\r$/,””)};1’ # assumes EACH line ends with Ctrl-M
# IN UNIX ENVIRONMENT: convert Unix newlines (LF) to DOS format
awk ‘{sub(/$/,”\r”)};1’
# IN DOS ENVIRONMENT: convert Unix newlines (LF) to DOS format
awk 1
# IN DOS ENVIRONMENT: convert DOS newlines (CR/LF) to Unix format
# Cannot be done with DOS versions of awk, other than gawk:
gawk -v BINMODE=”w” ‘1’ infile >outfile
# Use “tr” instead.
tr -d \r outfile # GNU tr version 1.22 or higher
# delete leading whitespace (spaces, tabs) from front of each line
# aligns all text flush left
awk ‘{sub(/^[ \t]+/, “”)};1’
# delete trailing whitespace (spaces, tabs) from end of each line
awk ‘{sub(/[ \t]+$/, “”)};1’
# delete BOTH leading and trailing whitespace from each line
awk ‘{gsub(/^[ \t]+|[ \t]+$/,””)};1’
awk ‘{$1=$1};1’ # also removes extra space between fields
# insert 5 blank spaces at beginning of each line (make page offset)
awk ‘{sub(/^/, ” “)};1’
# align all text flush right on a 79-column width
awk ‘{printf “%79s\n”, $0}’ file*
# center all text on a 79-character width
awk ‘{l=length();s=int((79-l)/2); printf “%”(s+l)”s\n”,$0}’ file*
# substitute (find and replace) “foo” with “bar” on each line
awk ‘{sub(/foo/,”bar”)}; 1’ # replace only 1st instance
gawk ‘{$0=gensub(/foo/,”bar”,4)}; 1’ # replace only 4th instance
awk ‘{gsub(/foo/,”bar”)}; 1’ # replace ALL instances in a line
# substitute “foo” with “bar” ONLY for lines which contain “baz”
awk ‘/baz/{gsub(/foo/, “bar”)}; 1’
# substitute “foo” with “bar” EXCEPT for lines which contain “baz”
awk ‘!/baz/{gsub(/foo/, “bar”)}; 1’
# change “scarlet” or “ruby” or “puce” to “red”
awk ‘{gsub(/scarlet|ruby|puce/, “red”)}; 1’
# reverse order of lines (emulates “tac”)
awk ‘{a[i++]=$0} END {for (j=i-1; j>=0;) print a[j–] }’ file*
# if a line ends with a backslash, append the next line to it (fails if
# there are multiple lines ending with backslash…)
awk ‘/\\$/ {sub(/\\$/,””); getline t; print $0 t; next}; 1’ file*
# print and sort the login names of all users
awk -F “:” ‘{print $1 | “sort” }’ /etc/passwd
# print the first 2 fields, in opposite order, of every line
awk ‘{print $2, $1}’ file
# switch the first 2 fields of every line
awk ‘{temp = $1; $1 = $2; $2 = temp}’ file
# print every line, deleting the second field of that line
awk ‘{ $2 = “”; print }’
# print in reverse order the fields of every line
awk ‘{for (i=NF; i>0; i–) printf(“%s “,$i);print “”}’ file
# concatenate every 5 lines of input, using a comma separator
# between fields
awk ‘ORS=NR%5?”,”:”\n”‘ file
SELECTIVE PRINTING OF CERTAIN LINES:
# print first 10 lines of file (emulates behavior of “head”)
awk ‘NR < 11’
# print first line of file (emulates “head -1”)
awk ‘NR>1{exit};1’
# print the last 2 lines of a file (emulates “tail -2”)
awk ‘{y=x “\n” $0; x=$0};END{print y}’
# print the last line of a file (emulates “tail -1”)
awk ‘END{print}’
# print only lines which match regular expression (emulates “grep”)
awk ‘/regex/’
# print only lines which do NOT match regex (emulates “grep -v”)
awk ‘!/regex/’
# print any line where field #5 is equal to “abc123”
awk ‘$5 == “abc123″‘
# print only those lines where field #5 is NOT equal to “abc123”
# This will also print lines which have less than 5 fields.
awk ‘$5 != “abc123″‘
awk ‘!($5 == “abc123”)’
# matching a field against a regular expression
awk ‘$7 ~ /^[a-f]/’ # print line if field #7 matches regex
awk ‘$7 !~ /^[a-f]/’ # print line if field #7 does NOT match regex
# print the line immediately before a regex, but not the line
# containing the regex
awk ‘/regex/{print x};{x=$0}’
awk ‘/regex/{print (NR==1 ? “match on line 1” : x)};{x=$0}’
# print the line immediately after a regex, but not the line
# containing the regex
awk ‘/regex/{getline;print}’
# grep for AAA and BBB and CCC (in any order on the same line)
awk ‘/AAA/ && /BBB/ && /CCC/’
# grep for AAA and BBB and CCC (in that order)
awk ‘/AAA.*BBB.*CCC/’
# print only lines of 65 characters or longer
awk ‘length > 64’
# print only lines of less than 65 characters
awk ‘length < 64’
# print section of file from regular expression to end of file
awk ‘/regex/,0’
awk ‘/regex/,EOF’
# print section of file based on line numbers (lines 8-12, inclusive)
awk ‘NR==8,NR==12’
# print line number 52
awk ‘NR==52’
awk ‘NR==52 {print;exit}’ # more efficient on large files
# print section of file between two regular expressions (inclusive)
awk ‘/Iowa/,/Montana/’ # case sensitive
SELECTIVE DELETION OF CERTAIN LINES:
# delete ALL blank lines from a file (same as “grep ‘.’ “)
awk NF
awk ‘/./’
# remove duplicate, consecutive lines (emulates “uniq”)
awk ‘a !~ $0; {a=$0}’
# remove duplicate, nonconsecutive lines
awk ‘!a[$0]++’ # most concise script
awk ‘!($0 in a){a[$0];print}’ # most efficient script