Writing Shell Scripts

This chapter includes:

• What's a script?
• Available shells
• Example of a Korn shell script
• Efficiency
• Caveat scriptor

What's a script?

Shell scripting, at its most basic, is taking a series of commands you might type at a command line and putting them into a file, so you can reproduce them again at a later date, or run them repeatedly without having to type them over again.

You can use scripts to automate repeated tasks, handle complex tasks that might be difficult to do correctly without repeated tries, redoing some of the coding, or both. Such scripts include:

• /etc/config/sysinit, which runs when you boot a Neutrino desktop system (see Controlling How Neutrino Starts)
• /usr/bin/ph, which starts Photon (see Using the Photon microGUI)

Available shells

The shell that you'll likely use for scripting under Neutrino is ksh, a public-domain implementation of the Korn shell. The sh command is usually a symbolic link to ksh. For more information about this shell, see:

• the Using the Command Line chapter in this guide
• the entry for ksh in the Utilities Reference
• Rosenblatt, Bill, and Arnold Robbins. 2002. Learning the Korn Shell, 2nd Edition. Sebastopol, CA: O'Reilly & Associates. ISBN 0-596-00195-9

Neutrino also supplies or uses some other scripting environments:

• An OS buildfile has a script file section tagged by +script. The mkifs parses this script, but it's executed by procnto at boot time. It provides a very simple scripting environment, with the ability to run a series of commands, and a small amount of synchronization.
• The embedded shell, esh, provides a scripting environment for running simple scripts in an embedded environment where the overhead of the full ksh might be too much. It supports the execution of utilities, simple redirection, filename expansion, aliases, and environment manipulation.
• The fat embedded shell, fesh, provides the same limited environment as esh, but supplies additional builtin commands for commonly used utilities to reduce the overhead of including them in an embedded system. The fesh shell includes builtins for cp, df, ls, mkdir, rm, and rmdir, although in most cases, the builtin provides only the core functionality of the utility and isn't a complete replacement for it.
• sed is a stream editor, which makes it most useful for performing repeated changes to a file, or set of files. It's often used for scripts, or as a utility within other scripts.
• gawk (GNU awk) is a programming language for pattern matching and working with the contents of files. You can also use it for scripting or call it from within scripts.
• The third-party repository includes perl, which, like gawk, is useful for working with files and patterns. The name perl stands for Practical Extraction and Report Language.

In general, a shell script is most useful and powerful when working with the execution of programs or modifying files in the context of the filesystem, whereas sed, gawk, and perl are primarily for working with the contents of files. For more information, see:

• the entries for gawk and sed in the Utilities Reference
• Robbins, Arnold, and Dale Dougherty. 1997. sed & awk, 2nd Edition. Sebastopol, CA: O'Reilly & Associates. ISBN 1-56592-225-5
• Schwartz, Randal L., and Tom Phoenix. 2001. Learning Perl. Sebastopol, CA: O'Reilly & Associates. ISBN 0-59600-132-0

Running a shell script

You can execute a shell script in these ways:

• Invoke another shell with the name of your shell script as an argument:

  sh myscript
    

• Load your script as a "dot file" into the current shell:

  . myscript
    

• Use chmod to make the shell script executable, and then invoke it, like this:

  chmod 744 myscript
  ./myscript

  In this instance, your shell automatically invokes a new shell to execute the shell script.

Example of a Korn shell script

As a quick tutorial in the Korn shell, let's look at a script that searches C source and header files in the current directory tree for a string passed on the command line:

#!/bin/sh
#
# tfind:
# script to look for strings in various files and dump to less

case $# in
1)
    find . -name '*.[ch]' | xargs grep $1 | less
    exit 0   # good status
esac

echo "Use tfind stuff_to_find                               "
echo "      where : stuff_to_find = search string           "
echo "                                                      "
echo "e.g. tfind console_state looks through all files in   "    
echo "     the current directory and below and displays all "
echo "     instances of console_state."
exit 1    # bad status

The first line, #!/bin/sh, starts with a #, which indicates a comment, so the line is ignored by the shell processing this script. Yet, many -- if not most -- shell scripts start with a line like this. The initial pair of characters, #! aren't important to the shell, but the loader code in procnto recognizes them as an instruction to load the following executable, /bin/sh, and pass it this script as a command-line parameter.

The next few lines are comments that describe what the script does. Then we see:

case $# in
1)
  ...
esac

The case ... in is a shell builtin command, one of the branching structures provided by the Korn shell, and is equivalent to the C switch statement.

The $# is a shell variable. When you refer to a variable in a shell, put a $ before its name to tell the shell that it's a variable rather than a literal string. The shell variable, $#, is a special variable that represents the number of command-line arguments to the script.

The 1) is a possible value for the case, the equivalent of the C case statement. This code checks to see if you've passed exactly one parameter to the shell.

The esac line completes and ends the case statement. Both the if and case commands use the command's name reversed to represent the end of the branching structure.

Inside the case we find:

find . -name '*.[ch]' | xargs grep $1 | less

This line does the bulk of the work, and breaks down into these pieces:

• find . -name '*.[ch]'
• xargs grep $1
• less

which are joined by the | or pipe character. A pipe is one of the most powerful things in the shell; it takes the output of the program on the left, and makes it the input of the program to its right. The pipe lets you build complex operations from simpler building blocks. For more information, see "Redirecting input and output" in Using the Command Line.

The first piece, find . -name '*.[ch]', uses another powerful and commonly used command. Most filesystems are recursive through a hierarchy of directories, and find is a utility that descends through the hierarchy of directories recursively. In this case, it searches for files that end in either .c or .h -- that is, C source or header files -- and prints out their names.

The filename wildcards are wrapped in single quotes (') because they're special characters to the shell. Without the quotes, the shell would expand the wildcards in the current directory, but we want find to evaluate them, so we prevent the shell from evaluating them by quoting them. For more information, see "Quoting special characters" in Using the Command Line.

The next piece, xargs grep $1, does a couple of things:

• grep is a file-contents search utility. It searches the files given on its command line for the first argument. The $1 is another special variable in the shell that represents the first argument we passed to the shell script (i.e. the string we're looking for).
• xargs is a utility that takes its input and turns it into command-line parameters for some other command that you give it. Here, it takes the list of files from find and makes them command-line arguments to grep. In this case, we're using xargs primarily for efficiency; we could do something similar with just find:

  find . -name '*.[ch]' -exec grep $i {} | less
    

  which loads and runs the grep program for every file found. The command that we actually used:

  find . -name '*.[ch]' | xargs grep $1 | less
    

  runs grep only when xargs has accumulated enough files to fill a command line, generally resulting in far fewer invocations of grep and a more efficient script.

The final piece, less, is an output pager. The entire command may generate a lot of output that might scroll off the terminal, so less presents this to you a page at a time, with the ability to move backwards and forwards through the data.

The case statement also includes the following after the find command:

exit 0   # good status

This returns a value of 0 from this script. In shell programming, zero means true or success, and anything nonzero means false or failure. (This is the opposite of the meanings in the C language.)

The final block:

echo "Use tfind stuff_to_find                               "
echo "      where : stuff_to_find = search string           "
echo "                                                      "
echo "e.g. tfind console_state looks through all files in   "    
echo "     the current directory and below and displays all "
echo "     instances of console_state."
exit 1    # bad status

is just a bit of help; if you pass incorrect arguments to the script, it prints a description of how to use it, and then returns a failure code.

Efficiency

In general, a script isn't as efficient as a custom-written C or C++ program, because it:

• is interpreted, not compiled
• does most of its work by running other programs

However, developing a script can take less time than writing a program, especially if you use pipes and existing utilities as building blocks in your script.

Caveat scriptor

Here are some things to keep in mind when writing scripts:

• In order to run a script as if it were a utility, you must make it executable by using the chmod command. For example, if you want anyone to be able to run your script, type:

  chmod a+x script_name
    

  Your script doesn't have to be executable if you plan to invoke it by passing it as a shell argument:

    ksh script_name
    

  or if you use it as a "dot file," like this:

    . script_name
    

• Just as for any executable, if your script isn't in one of the directories in your PATH, you have to specify the path to the script in order to run it. For example:

  ~/bin/my_script
    

• When you run a script, it inherits its environment from the parent process. If your script executes a command that might not be in the PATH, you should either specify the path to the command or add the path to the script's PATH variable.
• A script can't change its parent shell's environment or current directory, unless you run it as a dot file.
• A script won't run if it contains DOS end-of-line characters. If you edit a Neutrino script on a Windows machine, use the textto utility with the -l option to convert the file to the format used by the QNX 4 filesystem.