This is Hacker Public Radio Episode 3,625 for Friday the 24th of June 2022.
Today's show is entitled, shell tips and snippets collaborative effort.
It is hosted by Carl and is about 20 minutes long.
It carries a clean flag.
The summary is, Carl and special guests provide some shell tips and examples.
Hello HPR, this is Carl.
I've solicited the mailing list for audio contributions about shell scripting, so I could then
add my own and stitch them all together into a single collaborative show.
This is that show.
I'll go first by discussing the snippet that I put in the mailing list post, which is kind
of a silly example of wondering if you can do something and then figuring out a way to do it.
In this case, I wanted to see if there was a way to move function definitions to the end
of this group rather than having them at the beginning.
It's really not so much that they have to be at the beginning that they just have to
be defined prior to being called, which does usually put them closer to the top of the script.
I think common convention is that you take all your function definitions and you group
them together close to the top of the script.
The solution I came up with was to use the said command at the beginning of the script.
Sores, space, less than open parentheses, said space, single quote, one comma forward slash
carrot, exit, forward slash, space D, single quote, space dollar sign zero, closed parentheses.
Probably 99% of the time you'll see a period also called a dot used in place of the
full word source that I used. The dot is posics compliant whereas I don't think the full
word source is. And if it is present, I think it's just an alias to dot anyway, so they
do the same thing, which is to read a specified file into the current environment as if
it was just all one script.
The less than sign with the parentheses construct is a form of process substitution that
makes the output of the command that's enclosed by the parentheses accessible to the source
command as if it was a file, which is what we need here.
The dollar sign zero at the end is variable for the file name of the executing script.
So what said does is reason the script starts at line one and deletes everything until it
hits the word exit or a line beginning with the word exit. And then it outputs the rest.
The set output that is everything after exit, which is all the function definitions.
Those get read in by source so that they're defined before the functions are called.
So that's my non-portable solution to the non-problem of moving functions to the end of
a script. Without further ado, let's move on to the first community submission.
Hey, this is DNT. My snippet is about the shift command and bash. I first saw this on
to do.sh, which is the to do text command line interface. And so what it does is it shifts
the arguments that are given to your script by one. So all the arguments are available as like
a dollar sign one, dollar sign two, dollar sign three, and so on. So the shift command
will drop dollar sign one and it'll move dollar sign one into dollar sign, excuse me, it
will move dollar sign two into dollar sign one, dollar sign three into dollar sign two, and so on.
In my snippet, I used it within a loop. And it was a script that I would run it followed
by the times I was supposed to come in at work on some consecutive days. I would run
it with a sequence of times and then it would loop through those arguments and create events
starting at that time. So kind of interesting. I thought seems a bit unusual to me. This
shift command. That's what it does. This is Carl again. I'm glad DNT picked the shift
command to talk about because it's one of the ones that's like not really obvious just by
the name of it what it's doing. And I tend to trip over at every time I'm reading through
a script, but I think I'll remember it from now on. And also I'm not sure if he did
it on purpose, but choosing a work example to illustrate the shift command was very appropriate
I thought. Now let's move on to our next contributor.
Hey, everybody. This is glad to. I want to talk to you about variables in shell scripts and how
you can protect yourself from them. Well, I mean, they're not dangerous as such, but they can
be. They can be surprisingly dangerous. And here's why. So for instance, let's say in a shell
script, you have a variable foo and it is set to blah, foo equals blah reasonable enough. So I'm
going to do echo quote. Well, actually, you know what I'm going to do? I'm going to do echo
dollar sign foo. Pretty simple shell script. I'm going to save it. I'm going to run it and I get
just, of course, the word blah in my terminal. What I expected. So now I'm going to open that
back up, though. And I'm going to set foo to blah space blah. So blah blah, but with a space between
the two. And again, run the show, not run emax. Again, run the shell script. And this time,
I get an error. There's an error telling me that blah is not a valid command. Command is not found
blah. Well, the reason for that, of course, as you probably picked up possibly when I was saying it
out loud, is that foo equal blah. That's a valid statement. And then the space between blah and blah
is telling the interpreter that of the of the code of the shell command that that's the end of
that command foo equals blah was the end of that statement. And now there's a new statement,
which is just the word blah. So white space is an important delimiter in shells. So, you know,
there's lots of different ways around that. Like one of the ways is just to do foo equals
quote, blah space blah. And that gets you what you expect back out on the terminal. But this is
it's less about that. As it is, it's being indicative of the fact that sometimes the values
of variables, the contents of those values can throw things off a little bit. So I'm just going to
set food back to blah, just a single blah. And so to get around that, a very common thing to do
is to wrap your usage of the variable. Once you've declared it, you wrap your variable in quotes.
And then to further insulate your variable, believe it or not, you can also wrap your just the
variable part in curly braces, curly brackets. So that looks like echo space, quote,
dollar sign, curly brace, foo, close curly brace, close quote. That is a pretty tightly bound
variable. And it is relatively difficult. Actually, let's not say relatively difficult. Let's say
it's a little bit harder to sort of mess up the contents of something wrapped in so many layers.
So that's the first sort of tip right there is to wrap your variables in a shell script in quotes
and curly braces. It's just, it's a little bit more safer. And it's also a little bit more consistent
by which I mean, once you start using fancy things, fancy constructs like arrays,
then those braces are basically required. You just have to have the braces around
the array name when you start referencing components of that array. So it's a good,
it's just, it's for consistency. It's nice to have. But what I really wanted to talk about,
and that was just kind of the lead into it. What I really want to talk about are the
the things that your variables get set to or or don't get set to. This is a tip I picked up,
and I learned pretty early on from Slack builds. And it is the colon. I think it's called a
variable, variable like testing or something like that. Variable was a variable, I don't know,
there's a term that I'm not remembering, but like verification or whatever of a variable of a value.
So colon, and then there's a couple of things that you could remove that you could place after
a colon when you're talking about setting a variable. So for instance, here's a cool trick,
and you'll see this in Slack build scripts a lot, or you can every Slack build script from
Slackbuilds.org anyway. So for instance, let's say that I do something like food equals
blah, but then again, I also kind of want to make sure that my user could, I mean, or or maybe
the user's system, could set food if that had been set as an environment variable, for instance.
Maybe my user has food already set, I don't know what food is, but pretend like it's like editor,
or visual, or page or any of the comments or if user level environment variables,
that a lot of people do actually customize in their dot bash or see file. What you might want,
your script to preserve that. And if you just do something like food equals blah, then you're
insisting in your shell script that food is going to equal blah. And if the user has something
already set in their shell environment, food equals them. Well, now you've just said food equals
blah, and you've ignored their preference. That's not very nice. So what I'm going to do is do a
food equals dollar sign curly brace, food colon dash, and then some value for let's say blah again,
and then close curly brace. Now what what does that do? Well, let's just find out what that does.
And just in case it's not clear, food must be the same. So when you're doing food equals dollar
sign curly brace, food, those two foods have to match. If it's a capital F-O-O, then it has to be
capital F-O-O in the second time. If it's all capitals and it needs to be all capitals,
all lowercase, all lowercase. So food equals dollar sign curly brace, food again, colon dash,
and then blah curly brace. And then I'm going to say echo, food. So now if I run this script as is,
I get blah, as expected, because that was the sort of the default value was blah. But watch what
happens if I do something like food equals hello, and then S-H dot slash food dot S-H. I've just
defined food in my, in the environment that this shell is going to run in, I've just defined it.
So food equals hello S-H dot slash food dot S-H. And now instead of blah in my terminal, I get
hello back, because food was set. Just for this one command, I, I said a little environment variable.
It's in prairie environment variable called food, and I said it to hello. My shell script,
because I used the the syntax dollar sign curly brace, food, colon, dash, blah. What I'm saying is
to set food to food unless food hasn't been set in which case said it to blah. That's what that
that construct indicates. There are other ways of setting what food is in your, in your
terminal. You don't have to do it on the same sort of as part of the command. I could do export
food equals world. Okay. I just hit return. So that's that's now an environment variable floating
around in this shell session. And so if I do an S-H dot slash food dot S-H, I get world back,
because I exported this new variable called food. Now I could unset food, unset space food, and then
do S-H dot slash food dot S-H, and I'm back to blah, the old default value. So that's a helpful
little construct when you're doing a shell script to ensure that a variable gets set to either
what you think it should be or to trust the user and to let them set it themselves in their
bash environment or in their shell environment. Okay. One more way to test the validity of a variable.
And that is another colon something syntax. And what I'm going to do is I'm going to come up with a
scenario that could be a little bit dangerous. So I'm going to do food equals dollar sign, curly brace
food colon, dash, blah, semicolon or curly brace. And then I'm going to echo food. But then I'm going to
throw a wrench in it and do food equals quote quote. Okay. So this is going to this is essentially
unsetting food. Right. And in fact, I could just do that. I could just say unset food. In fact, I'll do that.
Okay. Unset food. Next, I'm going to do an echo dollar sign, I know quote dollar sign, curly brace,
food closed curly brace slash quote. Now this is pretty common in shell scripts where you're defining
a path. You might have a variable that gets set to some location on a hard drive where you want
to install something or you want to write a file out to it or something like that. And so because
you don't want to hard code all the paths, you make those paths consist largely of variables.
And that's, I mean, that's really common because something like, let's say dollar sign home,
I mean, that's an invaluable variable to be using in a shell script because that way you don't have
to come up with your own. You don't have to detect things yourself. You just let the users
environment inform your shell script where home is. And home is often sometimes followed by a slash
because then maybe you're descending into some other location within home. What we don't think
about sometimes though is what happens if one of those variables, especially the first one,
is not set for some reason or it's set to an empty string or whatever. Well, let's find out what
what our little shell script does here. So I'm going to do SH dot slash food. It echoes out
block first, but then remember after it does that, I unset food to nothing. And now my second
echo of food, which should have been block slash, is just slash. And you can imagine the kind of
havoc that can wreak on a system if you're writing a file out to dollar sign home slash dot
local slash share slash config or something. Now suddenly you're writing it out to slash dot
and dot local. So a hidden directory at your root partition, like what good is that going to do
anyone? Not only is that annoying, but it will also probably prompt the user for a root permission
or it will say that it's not able to write to the destination or whatever. And of course,
there's the nightmare scenario where you do some kind of like, I don't know, really complex
RM command to unprotect your root partition and remove all of whatever. Like point is,
you probably didn't mean to set your variable to just slash or to just slash and then some other path.
You want a way to ensure that that doesn't happen even by accident. So in your shell script
after I unset food, so I've got few eagles blah echo food just to confirm that it's there. And then
unset food, okay, it's gone. And then echo food slash to confirm that it's gone and I only have a slash.
Now I'm going to do my final little demonstration here and that is going to be echo quote dollar sign
curly quote full colon question mark curly brace slash closed quote. So I'm using the the variable
colon something notation again, but instead of this time I'm not using a dash. I'm using a question mark
the question mark tests whether that variable has a value. Here's what that looks like if I run it
I get blah. So that was the the test case that works. And then I get slash. So that's the unset food
with just a slash. And then I get an error on line 10 food parameter is null or not set. That's
because the colon question mark detected that my attempted use of food in that play in that at that time
was going to result in in something that I did not want. I didn't want to allow the use of
an empty variable. And so my colon question mark protected me from that. I mean it errors out which
you know a lot of people kind of associate that with a problem. But it's a lot better to error out
than to say overwrite an important file at slash or to dump a bunch of unwanted files at slash
or in the wrong path or whatever those situation would be when you have like an empty and
unexpectedly empty variable. And you can use both of those constructs the colon dash and the colon
question mark. I mean you can use that whenever you want to use it like it doesn't have to be just
when setting a variable or something like that. Like you can use that that syntax anytime you use
the variable where it's appropriate for you to either need a default value or you need to make
sure that there is a value and that it's not unset or null. Those are my variable tips for shell scripts.
I hope they help you. Carl here again. That was all awesome information but there was one specific
thing that I wanted to comment on because it ties in with the other topic I wanted to cover
which also involves the colon but when it's used by itself outside of the curly braces that
caught too discussed. The colon when it's used as a command is a shell built in that is often used
when you need to return a zero exit code meaning success. I was already familiar with using the colon
as sort of like a stand-in for true for example when you want to create an endless loop you
might start it with while colon meaning wild true or successful which would always evaluate to
true and would always enter the do loop. However I recently saw the colon used in a way that I
wasn't as familiar with. In Clotus segment he used the example foo equals dollar sign
curly brace foo colon dash bla curly brace which sets the variable foo to bla only a foo
is unset or set but null. You can use the colon in place of foo equals to accomplish the same thing
though note that it also requires a change to the syntax inside the curly braces. So instead of foo
equals you can put colon space dollar sign curly brace foo colon equals bla close curly brace.
The colon command decides always returning true or successful. Also has parameter expansion done on
any arguments that are passed to it. So in Clotus example by using foo equals syntax he was explicitly
stating that foo will either be assigned a value foo provided foo already had a null value and if it
didn't if foo wasn't set or it was null foo would be assigned a value block. In my variation using
the colon command inside the curly braces you have to change colon dash to colon equals so that when
parameter expansion is performed it assigns the value of bla to foo rather than substituting the value of
bla. On pubs.opengroup.org there's a table under the parameter expansion section that lists all the
different variations and conditions when a value gets assigned versus one of value is substituted.
I think quite get the difference between assignment and substitution in this context until I
sort of put this show together. When using the colon command in this way just remember that
parameter expansion has to do an assignment or at what work. You can use the colon to do arithmetic
expansion also for example to increment a counter variable you could do colon space dollar sign,
open parent, open parent, i plus plus, closed parent, closed parent and that would increment the
counter variable i by one. That concludes the collaborative show on shell snippets. I'd like to thank
D&T and Clotoo for their great submissions and this is Carl. I'll see you next time.
You have been listening to hacker public radio as hacker public radio does work. Today's show was
contributed by a HBR this night by itself. If you ever thought of a coin podcast, click on our
contributely to find out how easy it means. Hosting for HBR has been kindly provided by an
host host.com, the internet archive and our sing.net. On this otherwise stages, today's show is released
on their creative comments, attribution for.0.0 international license.