#Bash Automation and Scripting Basics (Part 3) – CloudSavvy IT

In this final article in our three part Bash automation and scripting basics series, we will explore script debugging, running scripts as a background process, and importing other scripts using the source command.

Bash Automation and Scripting Basics

If you would like to start at the beginning, please read our Bash Automation and Scripting Basics Part 1 article. This final article in our three part series on Bash automation and scripting basics will look at running scripts as a background process.

We also want to quickly debug our scripts with minimal fuss and high-end results. This can be done using a very handy tracing feature, build directly into the Bash command interpreter. We’ll look into this in the second topic. You may also want to keep an eye out for our upcoming article on the related shellcheck.

And finally we will explore how to run scripts as a background process. While this may provide modest immediate benefits – such as starting multiple tasks at once, it also lays some of the ground work for later more advanced multi-threaded scripts.

Script Debugging

Debugging a script in Bash need not be hard! Keep an eye on the CloudSavvyIT website as soon we will review the more comprehensive shellcheck debugging tool for Bash, but for the moment I would like to introduce you to a great way to debug Shell scripts in a straightforward and easy to understand way.

Within the Bash shell, which after all is a “simple” binary running on your machine – namely the bash binary, there is an option provided (-x) which, according to man bash (executing this in your terminal will show a Bash manual) is described as Print commands and their arguments as they are executed, and this is exactly what it does! How will this help with debugging? Have a look at the following example:

#!/bin/bash

A=1
B=2
if [ "${AA}" == "1" -o "${B}" == "1" ]; then 
  echo "One ('1') was definitely stored in either the variable A, or the variable B"
  exit 0
else
  echo "Assert: could not locate the value '1' in the variables A and B"
  exit 1
fi

Here we check the variables A and B against the value 1. The -o idiom in the if statement stands for OR, i.e. either the first part (A, or rather AA here, is 1) is true, or the second part (B is 1) is true and in such case success is achieved.

The output of the script will be the programmed assert, and the program will terminate with an exit code of 1, which generally means that there was some error. If the script had worked correctly, a confirmatory message would be shown and the script would terminate with an exit code of 0, which generally means that there was success in whatever the script or utility was meant to do.

So why is the script running into the assert? You may have noticed already that the variable A ran into a typo in our if statement, registered in the code as AA: a bug! We could go and check the script, and if it is as short and straightforward as the script shown here, the bug would be quickly found. But for a 5000 line program the solution is not that simple, especially not if it uses multiple threads, complex subshells etc.

Let’s debug this now with the -x option to Bash. You may remember from the second part of our Bash automation and scripting basics course that a subshell can be initiated by using an inline $( ... ) set of idioms. It can also be initiated by simply typing bash, or in this case bash -x inside our top shell. In this case, we will be running the script inside our Bash subshell, with the -x option to observe what happens step by step.

We thus executed bash -x ./test_debugging.sh and observe that the following conditional check is being performed: '[' '' == 1 -o 2 == 1 ']'. We notice that something is amiss: the value of 2 is being compared with 1 in the second part of our conditional check, but what is happening in the first part? Something is being compared with 1, but that something is… empty (as indicated by the empty string '')!

We then check our script why that empty place is there, and why it was not filled with the value of our A variable. We quickly realize the AA instead of A mistake, correct the mistake, and the script now works fine!

A really cool thing to remember when using bash -x is that you can tee (read this as ‘copy’) the output of the Bash command by redirecting the stderr (the error output) to stdout (the standard output) and capturing the same with tee:

Here we are running our fixed script, and are redirecting the error output (bash -x sends all of it’s informative debug output to stderr, the standard error output, and not to stdout) using 2>&1 (which redirects our stderr output to stdout – our standard output – instead). We then capture stdout by using tee and this will save the output to the file specified, namely bash_-x_output.txt.

This allows a Bash developer to slowly review, in a step by step format, his or her written code. Especially when programs become complex, have functions, become multi-threaded, start background processes etc., this way of debugging can be very valuable. As an example, I tend to use bash -x about once every fortnight to debug complex scripts.

Running Scripts as Background Processes

Running a script as a background process is straightforward: simply affix & to the end of the script name (with a space in between). We define background.sh as follows:

#!/bin/bash

sleep 2

We then start it in the following way – to highlight the fact it is running in the background:

What we can see happen here is as follows: the background.sh script is started in the background (given the & affixed to the script name with a space), and immediately the command prompt will return. We use this here by specifying the next command (sleep 1) directly after the & background idiom, which also terminates that command a sole/single command (in other words, sleep 1 is a completely new command).

We also terminate our sleep 1 command with a customary end-of-command Bash idiom, after which we will execute an echo that the sleep 1 is complete/done. Let’s next check out what happens at execution of this line.

Immediately, our background process/script (background.sh) is started, and this will run for about 2 seconds. The PID (process identifier) of the started background process is visually shown (namely, 773183 for our first ([1]) background process – and this PID will be different each time you start a background program/process), and our sleep 1 (the next instruction for execution) can now be executed as the other program has returned our prompt (whilst directly not shown here, this is what takes place when you start a background process; you immediately get the command prompt back).

The sleep 1 commences (with the sleep 2 or more accurately the background.sh script still running in the background, as a different process, in a subshell started under this top or higher level shell), and terminates after 1 second. After this our echo is executed, showing us the sleep 1 is complete. A second later, our background.sh process rounds up it’s 2 second wait, and terminates.

We do not see that it has terminated as the Bash shell waits for some interaction to show us status messages. Thus, as soon as we press enter, at any point after the two second sleep is over, we will see the termination of the background process as a [1]+ Done ./background.sh status message. If you go back to the second part of our mini-series you may also see how we could have used wait here to wait for completion/termination of the PID of the background process. It also highlights how many commands and utilities can be used in a combinatory way in Bash.

Importing Scripts Using source

Importing another script can be done easily using the Bash source command. Consider the following script testsource.sh:

#!/bin/bash

source mysource.sh

echo "${MYVAR}"

And the matching mysource.sh:

#!/bin/bash

MYVAR="Hello CloudSavvyIT Readers!"

If we simply make the first script (testsource.sh) executable (by using chmod +x testsource.sh), but not the second script (in fact we unset the executable flag to show clearly that this works by using chmod -x on mysource.sh), the second script is still called successfully as a result of the source command, and executed as part of the testsource.sh script:

In the mysource.sh script, we set the variable MYVAR to Hello CloudSavvyIT Readers!. This script is then sourced from the testsource.sh script by using the instruction source mysource.sh. This will cause the mysource.sh to be executed at that point the code, and when complete, the testsource.sh script will continue to run, though any things set in the mysource.sh will be retained (think about this as sourced from another script to remember the operation of this more easily).

In Summary

We had a look at script debugging by using bash -x to display all executed commands. We also explored how to run a script as a background process, and learned how we can import scripts using source. Thank you for staying tuned for this 3 part series, of which this was the final article!

If you are interested in learning more about Bash, how about checking our or articles Primer: Bash Loops: for, while, and until, Conditional Testing in Bash: if, then, else, elif, and Bash Functions and Local Variables