Bash Tips - 14 (HPR Show 2689)

Dave Morriss


Table of Contents

More about loops

This is the fourteenth episode covering useful tips about using Bash. Episodes 9-13 covered Making Decisions in Bash and in these episodes we looked at while and until loops, but not for loops. This episode is making good this deficiency, and is also looking at break and continue which are very useful when using loops.

The Bash for loop

This command has two forms described as syntax diagrams below. The diagrams are taken from the GNU Bash Manual:

Format 1

for name [ [in [words …] ] ; ] do commands; done

If written with the 'in words' part 'words' is a literal list or expandable item which provides a list. The loop cycles once for each member of the list, setting variable 'name' to each successive member and executing 'commands' at each iteration.

for colour in red green blue; do
    echo "$colour"
done

This will output the three colour names, one per line.

for file in *.mp3; do
    echo "$file"
done

In this case '*.mp3' will expand to a list of the files in the current directory which have an 'mp3' extension.

Such a list might be empty, and so the form with a null list is legal:

for name in ; do command; done

In this case the loop will not run at all.

This loop may be written without the 'in words' part which has a special meaning. The loop cycles through all of the positional parameters. The same effect can be obtained with:

for name in "$@"; do command; done

The following very simple downloadable example bash14_ex1.sh demonstrates the use of this form.

$ cat bash14_ex1.sh
#!/bin/bash

#
# An argument-printing 'for' loop demonstration
#
for arg do
    echo "$arg"
done

Invoking the script with a list of words as arguments results in the words being echoed back one per line:

$ ./bash14_ex1.sh Let joy be unconfined
Let
joy
be
unconfined

The return status of the for command is the exit status of the last command that executes within it. If there are no items in the expansion of words, no commands are executed, and the return status is zero.

Format 2

for (( expr1 ; expr2 ; expr3 )) ; do commands ; done

This for loop format uses numeric expressions to determine how many times to loop.

  • 'expr1' is an arithmetic expression which is evaluated at the start; it often consists of a variable being set to a value.
  • 'expr2' is also an arithmetic expression which is evaluated at each iteration of the loop; each time it evaluates to a non-zero value the commands in the loop are executed.
  • 'expr3' is another arithmetic expression which is evaluated each time 'expr2' evaluates to a non-zero value.

For example:

for ((i = 1; i < 10; i++)); do
    echo "$i"
done

This will output the numbers 1-9, one per line.

There is a lot of flexibility allowed in the expressions between the double parentheses since the rules of Shell Arithmetic apply. Examples are:

  • Spaces are not mandated and can be used as required for clarity
  • Variable references do not need leading '$' symbols
  • Any of the shell arithmetic operators can be used

For example:

for ((i = ((78+20)/2)-(4*12); i != 10; i ++)); do
    echo $i
done

This one initialises 'i' with a calculation, and tests to see if it is not 10 at every iteration.

A more involved example:

$ for ((i = 1, j = 100; i <= 10; i++, j += 10)); do
> echo "$i $j"
> done
1 100
2 110
3 120
4 130
5 140
6 150
7 160
8 170
9 180
10 190

We show this one being typed as a multi-line command at the command line. It sets 'i' to 1 and 'j' to 100 using the comma operator which lets you join multiple unrelated expressions together. It loops until 'i' is 10 and at each iteration it increments 'i' by 1 and 'j' by 10, again using the comma operator. Incrementing 'j' is done using the assignment operator '+='.

A downloadable copy is included with this episode if you would like to experiment with this for command: bash14_ex2.sh.

If any of 'expr1', 'expr2' and 'expr3' is missing it evaluates to 1. Thus, the following command defines an infinite loop:

for (( ; ; )) ; do commands ; done

The return value of this type of for command is the exit status of the last command in 'commands' that is executed, or false if any of the expressions is invalid.

The break and continue commands

We have encountered these before in passing. They are both builtin commands inherited from the Bourne Shell. They are both for changing the sequence of execution of a loop (for, while, until, or select - we will look at the select command in a later episode).

The break command

break [n]

Exits from a loop. If 'n' is supplied it must be an integer number greater than or equal to 1. It specifies that the nth enclosing loop must be exited.

For example:

for i in {a..c}{1..3}; do
    echo "$i"
    [ "$i" == "b2" ] && break
done

This outputs one of the letters followed by one of the numbers until the combination equals 'b2' at which point the break command is issued and the loop is exited. See episode 1884 for details of the Brace Expansion used here.

a1
a2
a3
b1
b2

This example contains a loop within a loop. The inner loop simply repeats the current value of 'i' three times using 'echo -n’ to suppress the newline. We want to exit both loops whenever 'i' gets to 'b2'.

for i in {a..c}{1..3}; do
    for j in {1..3}; do
        echo -n "$i "
        [ "$i" == "b2" ] && { echo; break 2; }
    done
    echo
done

Note we include an echo with the break to add the newline to the partially completed line:

a1 a1 a1
a2 a2 a2
a3 a3 a3
b1 b1 b1
b2

A downloadable script is available containing versions of the two loops we have just looked at in case you would like to experiment with them: bash14_ex3.sh.

The continue command

continue [n]

Resumes the next iteration of a loop. If 'n' is supplied it must be an integer number greater than or equal to 1. It specifies that the nth enclosing loop must be resumed.

For example, the following downloadable file bash14_ex4.sh:

$ cat bash14_ex4.sh
#!/bin/bash

#
# A demonstration of the use of 'continue'
#

for i in {a..c}{1..3}; do
    for j in {1..3}; do
        echo -n "$i "
        [[ "$i" == b? ]] && { echo; continue 2; }
    done
    echo
done

This is a variant of the previous example with the same nested loops and potentially the same output. The difference is that when the contents of variable 'i' begins with a 'b' the continue 2 command is invoked so that only the first instance is printed and the outer loop resumes with the next letter/number combination. Note that we are using the extended test here with a glob-style string match.

Invoking the script returns the following:

$ ./bash14_ex4.sh
a1 a1 a1
a2 a2 a2
a3 a3 a3
b1
b2
b3
c1 c1 c1
c2 c2 c2
c3 c3 c3

Example

The final downloadable script bash14_ex5.sh is not very complex but shows that a list in the first format for command can be anything. This example uses the much-referenced /usr/share/dict/words with the shuf command to return 10 words. I first use grep to omit all the words that end with a possessive "'s" because so many of these seem ridiculous (when would you ever use "loggerhead's" for example?):

$ cat bash14_ex5.sh
#!/bin/bash

#
# A demonstration that anything that generates a list of words or numbers can
# be used in a 'for' loop
#

for w in $(grep -E -v "'s$" /usr/share/dict/words | shuf -n 10); do
    echo "$w"
done

Invoking the script returns a list of random words:

$ ./bash14_ex5.sh
transplants
fireplug
dismissals
honcho
beefiest
sensational
junkets
steads
troglodyte
burkas