Bash Tips - 20 (HPR Show 2756)

Deleting arrays; positional and special parameters in Bash

Dave Morriss


Table of Contents

Tidying loose ends (Some collateral Bash tips)

Deleting arrays

I forgot to cover one thing on my list when doing the last show: I forgot to explain how to delete arrays and array elements. I’ll cover that topic in this episode.

Positional and Special parameters

I have also avoided talking much about the positional and special parameters in Bash: '$1', '$2', '$#' and the rest. I will cover (some of) these in this episode.

Silly titles

I stopped doing the weird episode titles by episode 14 because I thought the joke was getting tired. However, I think a few people missed them (and a certain HPR colleague was found vandalising my new titles as they were being posted ;-), so I have added them inside the notes on the older shows and am adding one here – as a homage to silliness.

The unset command

This is a built-in command that originated from the Bourne shell. It removes variables, arrays, parts of arrays or functions.

The command syntax is (from the GNU Bash manual):

unset [-fnv] [name]

The unset command removes each variable or function represented by name. This is just the name of the thing to be deleted and does not take a dollar sign ('$'). If the variable or function does not exist then this does not cause an error.

The '-v' option

If this option is given each name refers to a shell variable, which is removed.

$ fruit='rambutan'
$ echo "fruit is $fruit"
fruit is rambutan
$ unset -v fruit
$ echo "fruit is $fruit"
fruit is

A variable unset in this way will have been completely removed. This is not the same as setting the variable to null:

$ fruit='mangosteen'
$ if [[ -v fruit ]]; then echo "Exists"; else echo "Doesn't exist"; fi
Exists
$ fruit=
$ if [[ -v fruit ]]; then echo "Exists"; else echo "Doesn't exist"; fi
Exists
$ unset -v fruit
$ if [[ -v fruit ]]; then echo "Exists"; else echo "Doesn't exist"; fi
Doesn't exist

Remember that the Bash conditional expression '-v varname' returns true if the shell variable varname is set (has been assigned a value). Being null simply means that the variable has a null value, but it still exists.

The '-f' option

If this option is given then each name refers to a shell function, which is removed. Although there’s not much more to say, we’ll look at this in a little more detail when we cover functions in a formal way in a later episode.

Note that if no option is given to 'unset' each name is first checked to see if it is a variable, and if it is it is removed. If not and name is a function then it is removed. This could be unfortunate if you have variables and functions with similar names and you mistype a variable name.

The POSIX definition states that functions can only be removed if the '-f' option is given.

The '-n' option

This option is for removing variables with the nameref option set. We will look at such variables in a later show and will go into more detail about unsetting them then.

Variables marked as readonly

These cannot be unset. We touched on this in episode 19 with the 'declare -r' and 'readonly' commands.

Using a dollar sign in front of the variable name

The issue of whether the dollar sign is used or not is important. Consider the following:

$ a='b'
$ b='Contents of variable b'
$ echo "a=$a b=$b"
a=b b=Contents of variable b
$ unset $a    # <--- Don't do this!
$ echo "a=$a b=$b"
a=b b=

Here the variable 'b' has been removed where (presumably) the intention was to remove variable 'a'!

Arrays and array elements

Entire arrays can be removed with one of the following:

unset array
unset array[*]
unset array[@]

Note again that the array name is not preceded by a dollar sign ('$').

Individual elements may be removed as follows:

unset array[subscript]

As expected, the subscript must be numeric (or an expression returning a number) for indexed arrays. For associative arrays the subscript is a string (or an expression returning a string). Care is needed to quote appropriately if the subscript string contains spaces.

An index for an indexed array can be negative, as discussed in earlier shows, in which case the element in question is relative to the end of the array.

Note that ShellCheck, the script checking tool, advises that when subscripted arrays are used with unset they be quoted to avoid problems with glob expansion. The examples in this episode do this.

I have included a downloadable script bash20_ex1.sh which demonstrates array element deletion for both types of array:

#!/bin/bash

#-------------------------------------------------------------------------------
# Example 1 for Bash Tips show 20: deleting individual array elements
#-------------------------------------------------------------------------------

#
# Seed the random number generator with a nanosecond number
#
RANDOM=$(date +%N)

echo "Indexed array"
echo "-------------"

#
# Create indexed array and populate with ad ae ... cf
#
declare -a iarr
mapfile -t iarr < <(printf '%s\n' {a..c}{d..f})

#
# Report element count and show the structure
#
echo "Length: ${#iarr[*]}"
declare -p iarr

#
# Unset a random element
#
ind=$((RANDOM % ${#iarr[*]}))
echo "Element $ind to be removed, contents: ${iarr[$ind]}"
unset "iarr[$ind]"

#
# Report on the result of the element removal
#
echo "Length: ${#iarr[*]}"
declare -p iarr

echo
echo "Associative array"
echo "-----------------"

#
# Create associative array. Populate with the indices from the indexed array
# using the array contents as the subscripts.
#
declare -A aarr
for (( i = 0; i <= ${#iarr[*]}; i++ )); do
    # If there's a "hole" in iarr don't create an element
    [[ -v iarr[$i] ]] && aarr[${iarr[$i]}]=$i
done

#
# Report element count and keys
#
echo "Length: ${#aarr[*]}"
echo "Keys: ${!aarr[*]}"

#
# Use a loop to report array contents in sorted order
#
for key in $(echo "${iarr[@]}" | sort); do
    echo "aarr[$key]=${aarr[$key]}"
done

#
# Make another contiguous indexed array of the associative array's keys. We
# don't care about their order
#
declare -a keys
mapfile -t keys < <(printf '%s\n' ${!aarr[*]})

#
# Unset a random element. The indexed array 'keys' contains the keys
# of the associative array so we use the selected one as a subscript. We use
# this array because it doesn't have any "holes". If we'd used 'iarr' we might
# have hit the "hole" we created earlier!
#
k=$((RANDOM % ${#keys[*]}))
echo "Element '${keys[$k]}' to be removed, contents: ${aarr[${keys[$k]}]}"
unset "aarr[${keys[$k]}]"

#
# Report final element count and keys
#
echo "Length: ${#aarr[*]}"
echo "Keys: ${!aarr[*]}"
declare -p aarr

exit

Running the script generates the following output:

Indexed array
-------------
Length: 9
declare -a iarr=([0]="ad" [1]="ae" [2]="af" [3]="bd" [4]="be" [5]="bf" [6]="cd" [7]="ce" [8]="cf")
Element 5 to be removed, contents: bf
Length: 8
declare -a iarr=([0]="ad" [1]="ae" [2]="af" [3]="bd" [4]="be" [6]="cd" [7]="ce" [8]="cf")

Associative array
-----------------
Length: 8
Keys: be bd af ad ae cd ce cf
aarr[ad]=0
aarr[ae]=1
aarr[af]=2
aarr[bd]=3
aarr[be]=4
aarr[cd]=6
aarr[ce]=7
aarr[cf]=8
Element 'ae' to be removed, contents: 1
Length: 7
Keys: be bd af ad cd ce cf
declare -A aarr=([be]="4" [bd]="3" [af]="2" [ad]="0" [cd]="6" [ce]="7" [cf]="8" )

I have included a lot of comments in the script to explain what it is doing.

Things to note are:

  • The indexed array 'iarr' is filled with character pairs in order, and the indices generated are a contiguous sequence. Once an element is unset a “hole” is created in the sequence.

  • The associative array 'aarr' has a random element deleted. The element is selected by using the indexed array 'keys' which is created from the keys themselves. We use this rather than 'iarr' so that the random number can’t match the “hole” we created earlier.


Positional Parameters

The positional parameters are “the shell’s command-line arguments” - to quote the GNU Bash Manual.

We have seen these parameters in various contexts in other shows in the Bash Scripting series. They are denoted by numbers 1 onwards, as in '$1'. It is possible to set these parameters at the time the shell is invoked, but it is more common to see them being used in scripts. When a shell script is invoked any positional parameters are temporarily replaced by the arguments to the script, and the same occurs when executing a shell function.

The positional parameters are referred to as '$N' or '${N}' where N is a number starting from 1. The '${N}' must be used if N is 10 or above. These numbers denote the position of the argument of course. The positional parameters cannot be set in assignment statements, so '1=42' is illegal.

The set command

This command (which is hugely complex!) allows the positional parameters to be cleared or redefined (amongst many other capabilities).

set -- [argument...]

This clears the positional parameters and, if there are arguments provided, places them into the parameters.

set - [argument...]

If there are arguments then they replace the positional parameters. If no arguments are given then the positional parameters remain unchanged.

The shift command

This is a Bourne shell builtin command. Its job is to shift the positional parameters to the left.

shift [n]

If n is omitted it is assumed to be 1. The positional parameters from n+1 onwards are renamed to $1 onwards. So, if the positional parameters are:

Haggis Neeps Tatties

The command 'shift 2' will result in them being:

Tatties

It is an error to shift more places than there are parameters, and n cannot be negative. The 'shift' command returns a non-zero status if there is an error, but the positional parameters are not changed.

Special Parameters

A number of these are related to the positional parameters, and we will concentrate on these at the moment. The GNU Bash manual contains more detail than is shown here. Look at the manual for the full information if needed.

Parameter Explanation
* ($*) Expands to the positional parameters, starting from one. When the expansion is not within double quotes, each positional parameter expands to a separate word. When in double quotes a string is formed containing the positional parameters separated by the IFS delimiter. This is similar to the expression "${array[*]}" which we have seen before.
@ ($@) Expands to the positional parameters, starting from one. If the expression is within double quotes the parameters form quoted words. This is similar to the expression "${array[@]}" which we have seen before.
# ($#) Expands to the number of positional parameters in decimal.
0 ($0) Expands to the name of the shell or shell script.

Examples

Creating a Bash shell with arguments

The following snippets show how Bash can be invoked with arguments and how they can be manipulated:

$ /bin/bash -s Hacker Public Radio
$ echo $0
/bin/bash
$ echo $#
3
$ echo $@
Hacker Public Radio

Here /bin/bash is invoked with the '-s' option which causes it to run interactively and allows arguments. Inside the shell '$0' contains the file used to invoke Bash. There are three positional parameters, and these are displayed

$ set - $@ is cool
$ echo $#
5
$ echo $@
Hacker Public Radio is cool

The 'set' command is used to change the positional parameters to themselves ('$@') with two more. These make the count 5, then the new parameters are displayed.

$ shift 2
$ echo $@
Radio is cool

This shows the use of the 'shift' command to move the parameters two places to the left, thereby removing the first two of them.

$ cat /tmp/test
#!/bin/bash

echo $#
echo $@

exit
$ /tmp/test rice millet wheat
3
rice millet wheat
$ echo $@
Radio is cool
$ exit

A very simple script in /tmp/test is shown which displays the count of its arguments and the arguments themselves. It is invoked with three arguments which temporarily become the positional parameters of the script. The shell’s positional parameters are still intact afterwards though.

The 'exit' terminates the shell which was created at the start of these snippets.

Downloadable script

Here is a very simple downloadable script bash20_ex2.sh which demonstrates the use of arguments:

#!/bin/bash

#-------------------------------------------------------------------------------
# Example 2 for Bash Tips show 20: simple use of positional parameters
#-------------------------------------------------------------------------------

#
# This script needs 2 arguments
#
if [[ $# -ne 2 ]]; then
    echo "Usage: $0 word count"
    exit 1
fi

word=$1
count=$2

#
# Repeat the 'word' 'count' times on the same line
#
for (( i = 1; i <= count; i++ )); do
    echo -n "$word"
done
echo

exit

Running the script as './bash20_ex2.sh goodbye 3' generates the following output:

goodbyegoodbyegoodbye