Redirection, Pipes, and Tees

4. Redirection, Pipes, and Tees#

So, unix has lots of tools that do one thing very well. This modular approach is generally considered to be a good design principle, not only in computer science. However, it is only when the tools are used together that the true utility of the unix philosophy really begins to emerge.

4.1. Redirection#

4.1.1. Exercise#

Change working diredctory to your work directory. Now try the following

echo "hello"

What does it do ?

Now try

echo "hello" > msg.txt

What did this do ?

4.1.2. Exercise#

This exerise assumes you have R installed.

  • Use echo to create a file rand.R with the following contents

cat(rnorm(10,0,1))

  • Now try the following

R --no-echo < rand.R

What do you think is happening here ?

What about this ?

R --no-echo < rand.R > out.R

4.2. Everyone needs a cat#

A very useful command is cat. It concatenates the contents of a list of files and sends the result “to the terminal” (more on that later). For example, try

cat rand.R

and

cat rand.R out.R

What does this do ?

cat

And this ?

cat > log.txt

4.2.1. Exercise#

Create a file containing both the input and output from the R example.

4.3. Everything is a file#

In Unix everything is a file, however, there are different types of files. A device file corresponds to a physical element of the system such as a mouse, printer, or terminal. These files are usually located in the systems /dev/ directory (try listing its contents). A terminal usually has the format tty<number>.

Not convinced - try this on your system (use the output of the first command in the ones that follow).

tty

/dev/pts/10

ls -l /dev/pts/10

crw--w---- 1 grosed tty 136, 10 Dec 20 13:54 /dev/pts/10

Now start a new terminal using Ctrl-shift-N and type

cat > /dev/pts/10

followed by some input. What happens ?

Within a shell, communication between unix tools/programs is done via files, and the standard format for the communication is text. In short, unix utilities communicate with each other via text files.

4.4. stdin and stdout#

Unless specified by a user, the files which are used for communication between tools are managed by the shell. The default used for input is called stdin and for output stdout (short for standard input and standard output). Here is what things look like when you use a unix command.

Terminal Window

For example

echo "hello"

will send the output to the terminal using stdout

The destination of the output can be changed by using the > operator.

4.4.1. Exercise#

What happens when you run this ?

echo "hello" > output

Using > redirects the output to the specified file. Note that the file is created by the shell. This is visualised in the figure below.

Terminal Window

4.4.2. Exercise#

What happens if the file already exists ?

What does the >> operator do ?

The same process can take place for input, for example (assuming you have still got output from the previous exercise)

wc < ouput

Note that the input is on the right hand side.

Here is what is happening visually.

Terminal Window

Use of > and < can be combined using ;

echo "hello" > output ; wc < ouput

Visually this is

Terminal Window

4.4.3. Exercise#

How would you “automate” removing the file in the previous example ?

This is all a bit clumsy, so instead, the | operator is provided.

echo "hello" | wc

In short, the | operator automates doing this

Terminal Window

by converting it to this

Terminal Window

4.4.4. Exercise#

The command

R --vanilla --no-echo

starts R (quietly).

Use a single line shell command (with no ; and not using temporary files) to generate 1000 numbers drawn from \(\cal{N}(0,1)\) and save them in a file data-1.txt.

4.4.5. Exercise#

Find out what the tr command does. Can you use it to convert the data-1.txt to a single column of numbers ?

Contents