How to redirect stderr and stdout to a file plus display at the same time

The biggest search term that has brought people to my blog this month has been "redirecting stderr". I realize that I probably should expound more on Redirecting stderr and stdout to a file plus displaying them so that it makes more sense.

The original example script:

  #!/bin/bash

OUTPUT_LOG=output.log
OUTPUT_PIPE=output.pipe

if [ ! -e $OUTPUT_PIPE ]; then
    mkfifo $OUTPUT_PIPE
fi

if [ -e $OUTPUT_LOG ]; then
    rm $OUTPUT_LOG
fi

exec 3>&1 4>&2
tee $OUTPUT_LOG < $OUTPUT_PIPE >&3 &
tpid=$!
exec > $OUTPUT_PIPE 2>&1

echo "This is on standard out"
echo "This is on standard err" >&2

exec 1>&3 3>&- 2>&4 4>&-
wait $tpid

rm $OUTPUT_PIPE

The Explanation

First,

  if [ ! -e $OUTPUT_PIPE ]; then
    mkfifo $OUTPUT_PIPE
fi

I needed to make a named pipe -- otherwise known as a FIFO (First In First Out) -- to provide means for the shell and tee to share the output. You're probably familiar with pipes, such as ls -l | grep myfile. Named pipes are merely a way for you to do the same thing via a "file" in the filesystem. That same command can actually be done this way:

  mkfifo tmpfifo && (ls -l > tmpfifo &) && grep myfile < tmpfifo

Yeah, it's a little messy for that simple command. But it can be used in cases where a simple pipe cannot be used.

Next,

  exec 3>&1 4>&2

This saves the file descriptors for stdout (file descriptor 1) and stderr (file descriptor 2) to file descriptors 3 and 4.

Then,

  tee $OUTPUT_LOG < $OUTPUT_PIPE >&3 &

tee will, to quote from the manpage tee(1): "read from standard input and write to standard output and files". So, let's break up this line into parts to describe what's going on:

• tee $OUTPUT_LOG: tee will take from stdin and write to the file named in the variable $OUTPUT_LOG.
• < $OUTPUT_PIPE: tee's stdin will be redirected from the named pipe named in the variable $OUTPUT_PIPE
• >&3: redirect tee's stdout to file descriptor 3, which is the stdout of the shell script
• &: start tee as a background process of the shell

Next,

  tpid=$!

This line saves the process id of the tee process to the variable tpid.

Then,

  exec > $OUTPUT_PIPE 2>&1

In parts:

• exec > $OUTPUT_PIPE: redirects the stdout of the script to the named pipe named in the variable $OUTPUT_PIPE.
• 2>&1: redirects stderr to stdout so both stderr and stdout are redirected to the named pipe.

Then comes whatever you need your script to do. I included a few echo statements to show that both stdout and stderr are shown both in the terminal and in the log file.

After the meat of the script:

  exec 1>&3 3>&- 2>&4 4>&-

• 1>&3 and 2>&4 restore the original file descriptors for stdout and stderr from file descriptors 3 and 4.
• 3>&~ 4>&~: now that they are not needed, close file descriptors 3 and 4

Then,

  wait $tpid

This "pauses" the script until the tee process (whose process id is saved in the variable $tpid) exits. It will exit because its stdin (the output from the named pipe) has been closed. The named pipe closed the stdin because its input (file descriptors 1) was closed when the file descriptor stored in descriptor 3 was restored back to the original stdin.

Finally,

  rm $OUTPUT_PIPE

remove the named pipe, because it is not needed any more.

Addtional Information

For additional information, Chapter 19. I/O Redirection and Appendix E. A Detailed Introduction to I/O and I/O Redirection of the Advanced Bash-Scripting Guide are very useful in understanding these concepts. Also, a good introduction to named pipes can be found in this Linux Journal article.

I hope this helped make things a little bit clearer.

Syndicated 2006-08-20 04:20:00 from Travis B. Hartwell / Software Craftsman