Some more Bash tips (HPR Show 1884)

Dave Morriss


Table of Contents

Introduction

We looked at Parameter Expansion back in HPR episode 1648 where we saw how Bash variables could be used, checked and edited. There are other sorts of expansions within Bash, and we'll look at one called "Brace Expansion" in this episode, which follows on from episode 1843 "Some Bash tips".

I have included some extracts from the Bash manual page for reference.

Brace expansion

In brace expansion an expression enclosed in braces is expanded. The expression can be a list of comma separated strings or a sequence expression.

Comma separated strings

This can just be a series of letters such as:

echo c{a,e,i,o,u}t
-> cat cet cit cot cut

Note: The line beginning '->' is what will be generated by the above statement. I will be using this method of signifying output through these notes.

Here you see that each of the letters is inserted in between the letters c and t to make the list shown. This does not work inside quotes:

echo "c{a,e,i,o,u}t"
-> c{a,e,i,o,u}t

The comma separated strings can be longer than a single character:

echo "|"{cow,pig,sheep}"|"
-> |cow| |pig| |sheep|

Note that, since the "|" character is special within the Bash shell it has to be quoted to remove its special nature.

Sequence expressions

The sequence expression form consists of a range of values written like this:

{x..y}

For example, to get the numbers 0 to 5:

echo {0..5}
-> 0 1 2 3 4 5

There is an optional third part:

{x..y..incr}

The 'incr' part is a positive or negative integer indicating the size of the steps to be used between x and y. For example:

echo {0..100..10}
-> 0 10 20 30 40 50 60 70 80 90 100
echo {100..0..-10}
-> 100 90 80 70 60 50 40 30 20 10 0

If either x or y have leading zeroes these are maintained to ensure the resulting numbers are of the same width:

echo {000..100..10}
-> 000 010 020 030 040 050 060 070 080 090 100

The x and y parts can also be single characters with an optional (numeric) increment such as:

echo {a..j}
-> a b c d e f g h i j
echo {a..j..2}
-> a c e g i
echo {z..t..-2}
-> z x v t

Uses for brace expansion

The two forms are often used to generate lists of filenames for use in commands which take multiple arguments. For example, if a directory contains image files interspersed with other files, which you wish to examine with the ls command you might type:

ls -l ~/Pictures/*.{jpg,png}

You might want to grep a set of log files for a particular string, restricting yourself to a range of days:

grep "^Reset" -A7 logs/*201509{01..13}*.log

It's a useful way to generate sequences in a loop:

for i in {0..10}; do echo $i; done

I was recently experimenting with the printf command and its '%b' argument, and I used the following statements to show what it did:

echo -n ">"; printf '%b' "\x"{20..29}; echo "<"
> !"#$%&'()<

To explain, '%b' expects its argument to be a string with values such as '20' meaning hex 20, which it turns into the corresponding character. You can see that hex 20 is a space, hex 21 and exclamation mark and so forth.

The seq command

There is a command seq which can do a lot of what the Bash brace expansion sequence expressions can do, and a few things more. A copy of the manual page is included below.

The seq command treats its values as floating point values, so it is possible to do things like:

seq -f '%0.1f' -s' ' 0 2.5 10
-> 0.0 2.5 5.0 7.5 10.0

It cannot produce a sequence of letters however.


Manual Page Extracts

Brace Expansion

Brace expansion is a mechanism by which arbitrary strings may be generated. This mechanism is similar to pathname expansion, but the filenames generated need not exist. Patterns to be brace expanded take the form of an optional preamble, followed by either a series of comma-separated strings or a sequence expression between a pair of braces, followed by an optional postscript. The preamble is prefixed to each string contained within the braces, and the postscript is then appended to each resulting string, expanding left to right.

Brace expansions may be nested. The results of each expanded string are not sorted; left to right order is preserved. For example, 'a{d,c,b}e' expands into 'ade ace abe'.

A sequence expression takes the form {x..y[..incr]}, where x and y are either integers or single characters, and incr, an optional increment, is an integer. When integers are supplied, the expression expands to each number between x and y, inclusive. Supplied integers may be prefixed with 0 to force each term to have the same width. When either x or y begins with a zero, the shell attempts to force all generated terms to contain the same number of digits, zero-padding where necessary. When characters are supplied, the expression expands to each character lexicographically between x and y, inclusive, using the default C locale. Note that both x and y must be of the same type. When the increment is supplied, it is used as the difference between each term. The default increment is 1 or -1 as appropriate.

Brace expansion is performed before any other expansions, and any characters special to other expansions are preserved in the result. It is strictly textual. Bash does not apply any syntactic interpretation to the context of the expansion or the text between the braces.

A correctly-formed brace expansion must contain unquoted opening and closing braces, and at least one unquoted comma or a valid sequence expression. Any incorrectly formed brace expansion is left unchanged. A { or , may be quoted with a backslash to prevent its being considered part of a brace expression. To avoid conflicts with parameter expansion, the string ${ is not considered eligible for brace expansion.

This construct is typically used as shorthand when the common prefix of the strings to be generated is longer than in the above example:

mkdir /usr/local/src/bash/{old,new,dist,bugs}

or chown root /usr/{ucb/{ex,edit},lib/{ex?.?*,how_ex}}

Brace expansion introduces a slight incompatibility with historical versions of sh. sh does not treat opening or closing braces specially when they appear as part of a word, and preserves them in the output. Bash removes braces from words as a consequence of brace expansion. For example, a word entered to sh as file{1,2} appears identically in the output. The same word is output as file1 file2 after expansion by bash. If strict compatibility with sh is desired, start bash with the +B option or disable brace expansion with the +B option to the set command (see SHELL BUILTIN COMMANDS below).


NAME

seq - print a sequence of numbers

SYNOPSIS

seq [OPTION]... LAST
seq [OPTION]... FIRST LAST
seq [OPTION]... FIRST INCREMENT LAST

DESCRIPTION

Print numbers from FIRST to LAST, in steps of INCREMENT.

Mandatory arguments to long options are mandatory for short options too.

-f, --format=FORMAT
        use printf style floating-point FORMAT

-s, --separator=STRING
        use STRING to separate numbers (default: \n)

-w, --equal-width
        equalize width by padding with leading zeroes

--help display this help and exit

--version
        output version information and exit

If FIRST or INCREMENT is omitted, it defaults to 1.  That is, an omitted
INCREMENT defaults to 1 even when LAST is smaller than FIRST.  The
sequence of numbers ends when the sum of the current number and INCREMENT
would become greater than LAST.  FIRST, INCREMENT, and LAST are
interpreted as floating point values.  INCREMENT is usually positive if
FIRST is smaller than LAST, and INCREMENT is usu‐ ally negative if FIRST
is greater than LAST.  FORMAT must be suitable for printing one argument
of type 'double'; it defaults to %.PRECf if FIRST, INCREMENT, and LAST are
all fixed point decimal num‐ bers with maximum precision PREC, and to %g
otherwise.

AUTHOR

Written by Ulrich Drepper.