of Fundamentals UNIX ECE 257
Week 6 Introduction to UNIX Processes and system programming
College of Engineering: Electrical and Computer Engineering Department
UNIX Processes
UNIX supports concurrent execution of processes Concurrent Processes share resources under the control of the UNIX operating system. Time is allocated to processes in chunks called quantum's CPU control is determined by the OS process scheduler Typical scheduling algorithms such as FIFO and Round Robin Processes are typically classified as either CPU or I/ O bound processes
College of Engineering: Electrical and Computer Engineering Department
UNIX Processes
UNIX supports a priority based process scheduler
Priority_value=threshold priority+nice value+(recent CPU usage/2)
Threshold priority is a boundary typically set to 40 or 60 Nice value is defaulted at 20, but can be changed CPU usage represents the clock ticks a process has held the CPU
College of Engineering: Electrical and Computer Engineering Department
UNIX Process states
Swapped
Ready
Running
Waiting
Zombie
Pending I/O Defunct child Child to exit . . Sleeping College of Engineering: Electrical and Computer Engineering Department
UNIX Processes
UNIX processes can create or spawn more UNIX processes sh
Parent
Example >Sort /class/ece257/unix.txt
fork
sh
Child
Spawns a child process to run the sort command under the parent shell
College of Engineering: Electrical and Computer Engineering Department
UNIX Processes Example
tcsh sh ksh ps
> ps PID TTY 16971 pts/12 > sh $ ps PID TTY 16991 pts/12 16971 pts/12 $ ksh $ ps PID TTY 17038 pts/12 16991 pts/12 16971 pts/12 $ TIME CMD 0:00 tcsh TIME CMD 0:00 sh 0:00 tcsh TIME CMD 0:00 ksh 0:00 sh 0:00 tcsh
College of Engineering: Electrical and Computer Engineering Department
UNIX Process commands
> ps [options] Report the status of active processes Outputs attribute of running processes Common Options
-a display information about the process executing on your terminal except your login shell (session leader) Display information all processes running on the system Display long list (14 columns) of status report
-e -l -u uidlist
Display information about processes belonging to users in uidlist (UIDs separated by , ) College of Engineering: Electrical and Computer Engineering Department
Example use
> ps -a PID TTY 25503 16969 pts/11 25502 pts/1 16946 pts/11 17047 pts/12 16955 pts/11 25138 pts/1 10756 16956 pts/11 16968 pts/11 > TIME CMD 0:00 <defunct> 0:00 dtpad 0:00 rsh 0:00 tcsh 0:00 ps 0:00 ttsessio 0:01 sh 0:00 <defunct> 0:00 dtsessio 0:00 sh
College of Engineering: Electrical and Computer Engineering Department
Example use
> ps -e PID TTY TIME CMD 0 ? 0:01 sched 1 ? 0:30 init 2 ? 0:01 pageout 3 ? 1497:41 fsflush 346 ? 0:00 sac 196 ? 3:44 rpc.nisd 53 ? 0:00 sysevent 55 ? 0:00 sysevent 186 ? 1:00 rpcbind 228 ? 0:00 statd 260 ? 0:01 cron 278 ? 0:02 utmpd
College of Engineering: Electrical and Computer Engineering Department
Example use
> ps -l F S UID PID PPID C PRI NI 8 S 202 16971 16965 0 51 20 > ADDR ? SZ WCHAN TTY TIME CMD 342 ? pts/12 0:00 tcsh
F: flags S: state O running, R ready, S sleep, T stopped, Z zombie UID: User identifier PID: Process identifier PPID: Parent PID C: CPU Usage PRI: Process priority NI: Nice value ADDR: Address of the process in memory SZ: Size in blocks WCHAN: Wait channel TTY: terminal name process attached to TIME: length of time process is running CMD: command used to start the process College of Engineering: Electrical and Computer Engineering Department
UNIX Process Control
Processes can run in the foreground or background
Foreground processes
Processes run as interactive element (keeps control of I/O devices)
Background processes
Used to execute processes needing more time or non interactive Force using command&
College of Engineering: Electrical and Computer Engineering Department
Example use
> find / -name foo -print > foo.paths & [1] 17215 > > ps PID TTY TIME CMD 17215 pts/12 0:02 find 16971 pts/12 0:00 tcsh
To end this process we can use the kill command
> kill 17215 > ps PID TTY TIME CMD 16971 pts/12 0:00 tcsh [1] + Terminated
find / -name foo -print > foo.paths
College of Engineering: Electrical and Computer Engineering Department
Foreground to background or vice versa
fg [%jobid] %+ %%n %name %?name or bg[%jobid-list]
Processes can be moved
current job previous job job number n job beginning with name job containing name
> fg %1 find / -name foo -print > foo.paths > bg [1] find / -name foo -print > foo.paths &
College of Engineering: Electrical and Computer Engineering Department
Which job??
To find out what jobs are currently running and their jobid use The jobs command (displays status of suspended/background processes) jobs [option] [%jobid-list] options -l
> jobs [1] + Running > jobs -l [1] + 17487 Running >
display PID of jobs
find / -name foo -print > foo.paths find / -name foo -print > foo.paths
College of Engineering: Electrical and Computer Engineering Department
UNIX Daemons
A Daemon is a system process running in the background
- typically used to handle system administration tasks e.g. Print, e-mail, finger are daemons - Daemons may get assigned to specific ports In /etc/services
echo systat daytime netstat ftp-data ftp telnet smtp 7/tcp 11/tcp 13/tcp 15/tcp 20/tcp 21/tcp 23/tcp 25/tcp users
mail
College of Engineering: Electrical and Computer Engineering Department
Controlling Background processes
If a user exits before completing a background process, it too dies. To allow its completion use
> nohup [command] example
> nohup find / -name -print > foo.paths & [1] 25342 > Then sends the standard output to nohup.out when you logout Could kill the parent process while allowing the child to continue
College of Engineering: Electrical and Computer Engineering Department
Job Priority
UNIX processes try executing at same priority To adjust job priority (downward) use
> nice -val UNIX-cmd > nice 5 find / -name -print > foo.paths & reduces the priority of the find command process by 5
To change the priority of an existing process use
> renice n val PID > renice n 5 25342
College of Engineering: Electrical and Computer Engineering Department
Kill some more
Kill uses signals to aid in approaching how to kill Signal 1 2 3 9 15 24 signal name SIGHUP SIGINT SIGQUIT SIGKILL SIGTERM SIGTSTP function hangup-modem conn. Broken terminal interrupt-int key used quit from term.- core dump done surest kill- cant be trapped default termination suspends process-^z
> kill 9 25342
College of Engineering: Electrical and Computer Engineering Department
Do multiple commands
To perform multiple command in sequence as different processes use: > Cmd1; cmd2; . . .; cmdn To perform multiple commands in sequence in the background Use: > Cmd1&; cmd2&; . . .; cmdn& To perform multiple commands as one process use: > (Cmd1; cmd2; . . .; cmdn)
College of Engineering: Electrical and Computer Engineering Department
Example
> date& echo Hello, World!& uname; who [1] 25756 [2] 25757 Hello, World! Wed Oct 8 14:51:46 EDT 2003 SunOS asmart pts/1 Aug 14 13:42 (smart.engineering.umassd.edu) u_yhajjar pts/6 Oct 8 14:42 (ii-214-28.engineering.umassd.edu:0.0) pfortier pts/3 Oct 8 13:26 (134.88.49.167:0.0) u_yhajjar pts/7 Oct 8 14:44 (ii-214-28.engineering.umassd.edu:0.0) pfortier dtremote Oct 8 13:26 (134.88.49.167:0) u_yhajjar dtremote Oct 8 14:35 (ii-214-28.engineering.umassd.edu:0) [2] + Done echo Hello, World! [1] + Done date >
College of Engineering: Electrical and Computer Engineering Department
Example use
> (date; echo Hello, World!)& (who; uname)& whoami [1] 25784 [2] 25785 asmart pts/1 Aug 14 13:42 (smart.engineering.umassd.edu) u_yhajjar pts/6 Oct 8 14:42 (ii-214-28.engineering.umassd.edu:0.0) pfortier pts/3 Oct 8 13:26 (134.88.49.167:0.0) u_yhajjar pts/7 Oct 8 14:44 (ii-214-28.engineering.umassd.edu:0.0) pfortier dtremote Oct 8 13:26 (134.88.49.167:0) u_yhajjar dtremote Oct 8 14:35 (ii-214-28.engineering.umassd.edu:0) Wed Oct 8 14:56:35 EDT 2003 Hello, World! SunOS pfortier [2] - Done ( who; uname ) [1] + Done ( date; echo Hello, World! ) >
College of Engineering: Electrical and Computer Engineering Department
Schedule jobs for later
UNIX uses commands at or batch to do job later > at 10:50 at> /class/ece257/testsvi.sh > testviout.txt at> <EOT> commands will be executed using /bin/tcsh job 1065711000.a at Thu Oct 9 10:50:00 2003 > at accepts a large number of formats using keywords now, noon, midnight, today, tomorrow
College of Engineering: Electrical and Computer Engineering Department
Scheduling jobs
> at 15 > at 5pm assumes a 24 hour clock > at 3:08pm > at noon > at now + 1 year do same time next year > at 3:08pm + 1 day do tomorrow at the same time > at 15:08 June 1, 2004 do at 3:08pm on june 1 of 2004 > at 9am tomorrow do at 9 Friday, oct 10, 2003
College of Engineering: Electrical and Computer Engineering Department
Scheduling with batch
Like at, but jobs occur as soon as resources are available > Batch > /class/ece257/testsvi.sh > testviout.txt Can list or remove jobs from the at queue (at and batch) > at l > at r to list jobs or also atq to remove jobs or also atrm jobid
College of Engineering: Electrical and Computer Engineering Department
Suspending jobs
>sleep n n is the number of seconds you wish to suspend operations >stop %jobid suspends a job running in the process table
College of Engineering: Electrical and Computer Engineering Department
Systems Programming
Make use of utility services through operating system kernel calls Three main categories:
File management Process management Error handling
College of Engineering: Electrical and Computer Engineering Department
File Management
Basic File: Open, close, read, write, lseek, unlink, chown, dup2 Fcntl, fstat, ftruncate, stat, sync, dup, link Special File: mknod, ioctl, pipe, Sockets Directory File: getdents Sockets: accept, bind, connect, listen, socket, gethostbyname, gethostname, htonl, htons, inet_addr, inet_ntoa
College of Engineering: Electrical and Computer Engineering Department
Process Management
Process Management nice, chdir, wait, fork, exec, exit, setgid, get pid, getppid, setuid, getgid, getrgid, getuid, getruid Signals: alarm, signal, kill, pause
College of Engineering: Electrical and Computer Engineering Department
Error Management
Error handling:
perror
NAME perror - print a system error message SYNOPSIS #include <stdio.h> void perror(const char *s); #include <errno.h> const char *sys_errlist[]; int sys_nerr; int errno;
College of Engineering: Electrical and Computer Engineering Department
Use of perror
User programs must check system calls for return value of -1 and then deal with error. Example: force file errors and see
College of Engineering: Electrical and Computer Engineering Department
#include <stdio.h> #include <fcntl.h> #include <errno.h> main () { int fd; /* Open a non-existent file to cause an error */ fd = open ("nonexist.txt", O_RDONLY); if (fd == -1) /* fd == -1 =, an error occurred */ { printf ("errno = %d\n", errno); perror ("main"); } fd = open ("/", O_WRONLY); /* Force a different error */ if (fd == -1) { printf ("errno = %d\n", errno); perror ("main"); } /* Execute a successful system call */ fd = open ("nonexist.txt", O_RDONLY | O_CREAT, 0644); printf ("errno = %d\n", errno); /* Display after successful call */ perror ("main"); errno = 0; /* Manually reset error variable */ perror ("main"); } College of Engineering: Electrical Computer and Engineering Department
Compile and run it
eng-svr-1:/class/ECE-257/source>gcc showErrno.c -o showErrno eng-svr-1:/class/ECE-257/source>ls files.c showErrno showErrno.c sort.c stuff eng-svr-1:/class/ECE-257/source>./showErrno errno = 2 main: No such file or directory errno = 21 main: Is a directory errno = 29 main: Illegal seek main: Success
College of Engineering: Electrical and Computer Engineering Department
File manipulation
Illustrate use of system calls Utility program called: reverse.c
Reverses the lines of its inputs and displays them to standard output If no file name is specified, reverse acts on its standard input When the c option is used, reverse the characters in each line
College of Engineering: Electrical and Computer Engineering Department
#include <fcntl.h> /* For file mode definitions */ #include <stdio.h> #include <stdlib.h>
code
/* Enumerator */ enum { FALSE, TRUE }; /* Standard false and true values */ enum { STDIN, STDOUT, STDERR }; /* Standard I/O channel indices */ /* #define Statements */ #define BUFFER_SIZE 4096 /* Copy buffer size */ #define NAME_SIZE 12 #define MAX_LINES 100000 /* Max lines in file */ /* Globals */ char *fileName = 0; /* Points to file name */ char tmpName [NAME_SIZE]; int charOption = FALSE; /* Set to true if -c option is used */ int standardInput = FALSE; /* Set to true if reading stdin */ int lineCount = 0; /* Total number of lines in input */ int lineStart [MAX_LINES]; /* Store offsets of each line */ int fileOffset = 0; /* Current position in input */ int fd; /* File descriptor of input */
College of Engineering: Electrical and Computer Engineering Department
/****************************************************************/
main (argc, argv) int argc; char* argv []; { parseCommandLine (argc,argv); /* Parse command line */ pass1 (); /* Perform first pass through input */ pass2 (); /* Perform second pass through input */ return (/* EXITSUCCESS */ 0); /* Done */ } /****************************************************************/
College of Engineering: Electrical and Computer Engineering Department
parseCommandLine (argc, argv) int argc; char* argv []; /* Parse command line arguments */ { int i; for (i= 1; i < argc; i++) { if(argv[i][0] == '-') processOptions (argv[i]); else if (fileName == 0) fileName= argv[i]; else usageError (); /* An error occurred */ } standardInput = (fileName == 0); } /****************************************************************/
College of Engineering: Electrical and Computer Engineering Department
processOptions (str) char* str; /* Parse options */ { int j; for (j= 1; str[j] != 0; j++) { switch(str[j]) /* Switch on command line flag */ { case'c': charOption = TRUE; break; default: usageError (); break; } } } /****************************************************************/ usageError () { fprintf (stderr, "Usage: reverse -c [filename]\n"); exit (/* EXITFAILURE */ 1); } /****************************************************************/
College of Engineering: Electrical and Computer Engineering Department
pass1 () /* Perform first scan through file */ { int tmpfd, charsRead, charsWritten; char buffer [BUFFER_SIZE]; if (standardInput) /* Read from standard input */ { fd = STDIN; sprintf (tmpName, ".rev.%d",getpid ()); /* Random name */ /* Create temporary file to store copy of input */ tmpfd = open (tmpName, O_CREAT | O_RDWR, 0600); if (tmpfd == -1) fatalError (); } else /* Open named file for reading */ { fd = open (fileName, O_RDONLY); if (fd == -1) fatalError (); } lineStart[0] = 0; /* Offset of first line */
College of Engineering: Electrical and Computer Engineering Department
Pass 1 cont.
while (TRUE) /* Read all input */ { /* Fill buffer */ charsRead = read (fd, buffer, BUFFER_SIZE); if (charsRead == 0) break; /* EOF */ if (charsRead == -1) fatalError (); /* Error */ trackLines (buffer, charsRead); /* Process line */ /* Copy line to temporary file if reading from stdin */ if (standardInput) { charsWritten = write (tmpfd, buffer, charsRead); if(charsWritten != charsRead) fatalError (); } } /* Store offset of trailing line, if present */ lineStart[lineCount + 1] = fileOffset; /* If reading from standard input, prepare fd for pass2 */ if (standardInput) fd = tmpfd; }
College of Engineering: Electrical and Computer Engineering Department
Pass 1 cont.
/****************************************************************/ trackLines (buffer, charsRead) char* buffer; int charsRead; /* Store offsets of each line start in buffer */ { int i; for (i = 0; i < charsRead; i++) { ++fileOffset; /* Update current file position */ if (buffer[i] == '\n') lineStart[++lineCount] = fileOffset; } } /****************************************************************/
College of Engineering: Electrical and Computer Engineering Department
int pass2 () /* Scan input file again, displaying lines in reverse order */ { int i; for (i = lineCount - 1; i >= 0; i--) processLine (i); close (fd); /* Close input file */ if (standardInput) unlink (tmpName); /* Remove temp file */ } /****************************************************************/
College of Engineering: Electrical and Computer Engineering Department
processLine (i) int i; /* Read a line and display it */ { int charsRead; char buffer [BUFFER_SIZE]; lseek (fd, lineStart[i], SEEK_SET); /* Find line and read */ charsRead = read (fd, buffer, lineStart[i+1] - lineStart[i]); /* Reverse line if -c option was selected */ if (charOption) reverseLine (buffer, charsRead); write (1, buffer, charsRead); /* Write it to standard output */ } /****************************************************************/
College of Engineering: Electrical and Computer Engineering Department
reverseLine (buffer, size) char* buffer; int size; /* Reverse all the characters in the buffer */ { int start = 0, end = size - 1; char tmp; if (buffer[end] == '\n') --end; /* Leave trailing newline */ /* Swap characters in a pairwise fashion */ while (start < end) { tmp = buffer[start]; buffer[start] = buffer[end]; buffer[end] = tmp; ++start; /* Increment start index */ --end; /* Decrement end index */ } } /****************************************************************/
College of Engineering: Electrical and Computer Engineering Department
fatalError () { perror ("reverse: "); /* Describe error */ exit (1); }
College of Engineering: Electrical and Computer Engineering Department
Description
Step 1: parse command line
uses parseCommandLine and processOptions code segments Uses open LINUX file system call
Step 2: if reading standard input create temp file, otherwise output input file for reading
Uses pass1 code segment Uses open system call
College of Engineering: Electrical and Computer Engineering Department
Description
Step 3: read from file in chunks, storing the starting offset of each line in an array. If reading standard input, copy each chunk to temp file
uses pass1 and trackLines code segments Uses read, write LINUX file system calls
Step 4: read input file again, backwards, copying each line to sta...