Chapter 1. Introduction

Chapter 1. Introduction It's no secret that the Unix operating system has emerged as a standard operating system. For programmers who have been using Unix for many years now, this came as no surprise: The Unix system provides an elegant and efficient environment for program development. After all, this is what Dennis Ritchie and Ken Thompson strived for when they developed Unix at Bell Laboratories in the late 1960s. One of the strongest features of the Unix system is its wide collection of programs. More than 200 basic commands are distributed with the standard operating system. These commands (also known as tools) do everything from counting the number of lines in a file, to sending electronic mail, to displaying a calendar for any desired year. But the real strength of the Unix system comes not entirely from this large collection of commands but also from the elegance and ease with which these commands can be combined to perform far more sophisticated functions. To further this end, and also to provide a consistent buffer between the user and the guts of the Unix system (the kernel), the shell was developed. The shell is simply a program that reads in the commands you type and converts them into a form more readily understood by the Unix system. It also includes some fundamental programming constructs that let you make decisions, loop, and store values in variables. The standard shell distributed with Unix and Linux systems derives from AT&T's distribution, which evolved from a version originally written by Stephen Bourne at Bell Labs. Since then, the IEEE created standards based on the Bourne shell and the other more recent shells. The current version of this standard as of this revision is the Shell and Utilities volume of IEEE Std 1003.1-2001, also known as the POSIX standard. This shell is what we propose to teach you about in this book. The examples in this book were tested on both SunOS 5.7 running on a Sparcstation Ultra-30 and on Silicon Graphics IRIX 6.5 running on an Octane; some examples were also run on Red Hat Linux 7.1 and Cygwin. All examples, except some Bash examples in Chapter 15, were run using the Korn shell, although many were also run with Bash. Many Unix systems are still around that have Bourne shell derivatives and utilities not compliant with the POSIX standard. We'll try to note this throughout the text wherever possible; however, there are so many different versions of Unix from so many different vendors that it's simply not possible to mention every difference. If you do have an older Unix system that doesn't supply a POSIX-compliant shell, there's still hope. We'll list resources at the end of this book where you can obtain free copies of three different POSIX-compliant shells. Because the shell offers an interpreted programming language, programs can be written, modified, and debugged quickly and easily. We turn to the shell as our first choice of programming language. After you become adept at programming in the shell, you too may turn to it first. This book assumes that you are familiar with the fundamentals of the Unix system; that is, that you know how to log in; how to create files, edit them, and remove them; and how to work with directories. But in case you haven't used the Unix system for a while, we'll examine the basics in Chapter 2, "A Quick Review of the Basics." Besides the basic file commands, filename substitution, I/O redirection, and pipes are also reviewed in Chapter 2. Chapter 3, "What Is the Shell?," reveals what the shell really is. You'll learn about what happens every time you log in to the system, how the shell program gets started, how it parses the command line, and how it executes other programs for you. A key point made in Chapter 3 is that the shell is just a program; nothing more, nothing less. Chapter 4, "Tools of the Trade," provides tutorials on tools useful in writing shell programs. Covered in this chapter are cut, paste, sed, grep, sort, tr, and uniq. Admittedly, the selection is subjective, but it does set the stage for programs that we'll develop throughout the remainder of the book. Also in Chapter 4 is a detailed discussion of regular expressions, which are used by many Unix commands such as sed, grep, and ed. Chapters 5 through 10 teach you how to put the shell to work for writing programs. You'll learn how to write your own commands; use variables; write programs that accept arguments; make decisions; use the shell's for, while, and until looping commands; and use the read command to read data from the terminal or from a file. Chapter 6, "Can I Quote You on That?," is devoted entirely to a discussion on one of the most intriguing (and often confusing) aspects of the shell: the way it interprets quotes. By this point in the book, all the basic programming constructs in the shell will have been covered, and you will be able to write shell programs to solve your particular problems. Chapter 11, "Your Environment," covers a topic of great importance for a real understanding of the way the shell operates: the environment. You'll learn about local and exported variables; subshells; special shell variables such as HOME, PATH, and CDPATH; and how to set up your .profile file. Chapter 12, "More on Parameters," and Chapter 13, "Loose Ends," tie up some loose ends, and Chapter 14, "Rolo Revisited," presents a final version of a phone directory program called rolo that is developed throughout the book. Chapter 15, "Interactive and Nonstandard Shell Features," discusses features of the shell that either are not formally part of the IEEE POSIX standard shell (but are available in most Unix and Linux shells) or are mainly used interactively instead of in programs. Appendix A, "Shell Summary," summarizes the features of the IEEE POSIX standard shell. Appendix B, "For More Information," lists references and resources, including the Web sites where different shells can be downloaded. The philosophy this book uses is to teach by example. Properly chosen examples do a far superior job at illustrating how a particular feature is used than ten times as many words. The old "A picture is worth…" adage seems to apply just as well to examples. You are encouraged to type in each example and test it on your system, for only by doing can you become adept at shell programming. You also should not be afraid to experiment. Try changing commands in the program examples to see the effect, or add different options or features to make the programs more useful or robust. At the end of most chapters you will find exercises. These can be used as assignments in a classroom environment or by yourself to test your progress. This book teaches the IEEE POSIX standard shell. Incompatibilities with earlier Bourne shell versions are noted in the text, and these tend to be minor. Acknowledgments from the first edition of this book: We'd like to thank Tony Iannino and Dick Fritz for editing the manuscript. We'd also like to thank Juliann Colvin for performing her usual wonders copy editing this book. Finally, we'd like to thank Teri Zak, our acquisitions editor, and posthumously Maureen Connelly, our production editor. These two were not only the best at what they did, but they also made working with them a real pleasure. For the first revised edition of this book, we'd like to acknowledge the contributions made by Steven Levy and Ann Baker, and we'd like to also thank the following people from Sams: Phil Kennedy, Wendy Ford, and Scott Arant. For the second revised edition of this book, we'd like to thank Kathryn Purdum, our acquisitions editor, Charlotte Clapp, our project editor, and Geneil Breeze, our copy editor.