p1-insta485-static

Shell Scripting Tutorial

In this tutorial, you’ll learn to automate command line tasks using a shell script. Each line of code in a shell script is a command that could be entered at the command line.

Shell script “hello world”

Shell scripts automate commands. Let’s write a script that counts the number of files and directories in the present working directory.

First, try it manually.

$ ls
bin  hello  hello_css  html  insta485  insta485generator  pyproject.toml  tests
$ echo "Count of files and folders:"
Count of files and folders:
$ ls | wc -l
8

Next, create a text file called lscount and add the following contents. Notice that shell scripts may omit a file extension.

#!/bin/bash
#
# lscount
#
# List files and folders, followed by a count

# Stop on errors, print commands
# See https://vaneyckt.io/posts/safer_bash_scripts_with_set_euxo_pipefail/
set -Eeuo pipefail
set -x

# List
ls

# Count
echo "Count of files and folders:"
ls | wc -l

We can’t yet execute the script because the executable bit is not yet set. The rw means “Read Write”. Your default permissions may be slightly different.

$ ls -l lscount
-rw-rw-r-- 1 awdeorio staff 273 Aug 27 10:15 lscount

Make the script executable. The rwx mean “Read Write Execute”. Your permissions may be slightly different. As long as you see an x, you’re good. Further reading about Linux Permissions.

$ chmod +x lscount
$ ls -l lscount
-rwxrwxr-x 1 awdeorio staff 273 Aug 27 10:15 lscount

Run the script.

$ ./lscount
+ ls
bin    hello_css  insta485	     lscount   tests
hello  html	  insta485generator  pyproject.toml
+ echo 'Count of files and folders:'
Count of files and folders:
+ ls
+ wc -l

Shell script pitfalls

Shell scripts have a number of pitfalls. Check for these each time you write a script.

Executable bit

A shell script should be executable, which is a file permission. A file permissions problem looks like this.

$ ./lscount
permission denied: ./lscount

Check if the executable bit is set on your script. There’s an x missing, which means permission to execute. The r means permission to read, the w means permission to write. Further reading about Linux Permissions.

$ ls -l ./lscount
-rw-r--r--  1 tylerphillips  staff    0 Oct  2 20:57 insta485script

Add execute permissions.

$ chmod +x lscount

Check for execute permissions, indicated by the x.

$ ls -l lscount
-rwxr-xr-x  1 tylerphillips  staff    0 Oct  2 20:57 lscount

Shebang

Scripts need a special first line, called a shebang. Make sure you didn’t forget it.

$ head -n1 bin/lscount
#!/bin/bash

Stop on errors

By default, shell scripts do not stop when an error occurs. Every bash shell script should include this as the first line of code. Here’s why.

All scripts should contain this line to enforce stopping on errors.

set -Eeuo pipefail

Line endings

Scripts need to have UNIX line endings to work properly. Windows CRLF line terminators are a problem.

$ ./bin/lscount
-bash: ./bin/lscount: /bin/bash^M: bad interpreter: No such file or directory
$ file ./bin/lscount
bin/lscount: Bourne-Again shell script text executable, ASCII text, with CRLF line terminators

Change Windows line endings to UNIX line endings. Three different ways to do the same thing. You might have to install the dos2unix package.

$ dos2unix bin/lscount
$ fromdos bin/lscount
$ sed -ri 's/\r$//' bin/lscount

This is what a correct script file type looks like:

$ file bin/lscount
bin/lscount: Bourne-Again shell script text executable, ASCII text

Acknowledgments

Original document written by Andrew DeOrio awdeorio@umich.edu.

This document is licensed under a Creative Commons Attribution-NonCommercial 4.0 License. You’re free to copy and share this document, but not to sell it. You may not share source code provided with this document.