Shell Scripting For Loops: A Beginner's Guide

Expanding Shell Scripting Knowledge: A Second Look

For those aiming to enhance their technical expertise, we present the second part of our exploration into shell scripting. This installment builds upon the foundation established last week.

Corrections and Enhancements to the Previous Script

Following feedback and further testing, some refinements have been made to the script shared in the previous article. These adjustments aim to improve both functionality and clarity.

Specific corrections address minor errors, while improvements focus on optimizing the script’s performance and readability.

Introducing Looping Constructs

A fundamental aspect of scripting involves the ability to repeat actions. This section serves as an introductory guide to looping for those new to the concept.

Looping allows you to execute a block of code multiple times, automating repetitive tasks and making your scripts more efficient.

For Loops: Ideal for iterating over a predefined set of items.
While Loops: Continue executing as long as a specified condition remains true.
Until Loops: Execute a block of code until a condition becomes true.

Understanding these different loop types is crucial for writing powerful and versatile shell scripts. Each offers unique advantages depending on the specific task at hand.

This guide provides a starting point for mastering looping techniques, enabling you to create more complex and automated solutions.

Revisiting the datecp Script

Our initial exploration of shell scripting involved creating a script designed to copy a file to a backup location, appending a date-based identifier to the filename.

Samuel Dionne-Riel brought to our attention, through comments, a more effective approach to managing variable references within the script.

Bash interprets spaces as delimiters between arguments. Consequently, when a space is present in an expanded command, it's tokenized. In the original script, the command
cp $1 $2.$date_formatted
functions correctly only if the variables, when expanded, do not contain spaces. However, if the script is invoked like this:
datecp "my old name" "my new name"
the resulting command becomes:
cp my new name my old name.the_date
which is parsed as having six arguments.
To rectify this, the final line of the script should be modified to:
cp "$1" "$2.$date_formatted"

Simply enclosing the script’s line within double quotes—changing from cp -iv $1 $2.$date_formatted to cp -iv "$1" "$2".$date_formatted—resolves the issue when handling filenames containing spaces. Samuel also cautioned against relying on visually appealing, yet potentially incorrect, typographical characters when copying code from websites or the internet, emphasizing the importance of using standard dashes and quotes. We are actively working to improve the copy/paste compatibility of our code examples.

Furthermore, Myles Braithwaite, another commenter, enhanced the script to position the date before the file extension. Instead of a naming convention like

tastyfile.mp3.07_14_11-12.34.56

the revised format becomes:

tastyfile.07_14_11-12.34.56.mp3

This alteration generally provides a more user-friendly experience. His implementation is accessible on his GitHub profile. Let's examine the techniques he employed to dissect the filename.

date_formatted=$(date +%Y-%m-%d_%H.%M%S)
file_extension=$(echo "$1"|awk -F . '{print $NF}')
file_name=$(basename $1 .$file_extension)
cp -iv $1 $file_name-$date_formatted.$file_extension

While I’ve adjusted the formatting slightly, it’s evident that Myles defines his date formatting function in the first line. The second line utilizes the “echo” command, coupled with the script’s first argument, to output the filename. A pipe then directs this output as input to the subsequent command.

Myles then invokes the “awk” command, a robust pattern scanning utility. The -F flag instructs the command to use the following character—in this instance, a period—as the field separator.

With a filename like “tastyfile.mp3”, awk identifies two fields: “tastyfile” and “mp3”. The expression

'{print $NF}'

is then used to display the last field. If the file contains multiple periods, awk will still only output the final field, which corresponds to the file extension.

In the third line, a new variable is created to store the filename, leveraging the “basename” command to extract everything from the first argument ($1) except the file extension. This is achieved by providing $1 and the file extension to the basename command. The variable from the previous line automatically supplies the file extension. This transforms

tastyfile.mp3

into

tastyfile

Finally, the last line constructs the complete command for output. Notably, this version does not include a second argument ($2). Consequently, the script copies the file into the current directory. Excellent work, Samuel and Myles!

Executing Scripts and the $PATH Variable

As previously discussed in our introductory article, scripts are not automatically recognized as commands. Directly specifying the script's location is typically necessary for execution.

For example: ./script or ~/bin/script

However, storing scripts within the ~/bin/ directory enables execution simply by typing their names, regardless of the current location.

A discussion arose among readers regarding the conventionality of this practice, noting that contemporary Linux distributions do not automatically create this directory. Moreover, it isn't included in the default $PATH variable, which is essential for command-like script execution.

This initially caused some confusion, as my own $PATH variable confirmed the commenters’ observations, yet scripts continued to function as expected. The explanation lies in a file present in many modern Linux distributions within the user’s home directory: .profile.

the-beginners-guide-to-shell-scripting-2-for-loops-1.jpg

This file is processed by bash (unless a .bash_profile file exists), and it includes a section that conditionally appends the ~/bin/ directory to the $PATH variable if the directory is present. Therefore, the initial discrepancy has been resolved.

Throughout this series, scripts will continue to be placed in the ~/bin/ directory, as this is appropriate for user-specific scripts that should be readily executable by users. Consequently, manual modification of the $PATH variable appears unnecessary for achieving the desired functionality.

Utilizing Loops for Repetitive Tasks

Let's explore a highly valuable tool for automating repetitive processes: loops. This discussion will focus on the functionality of “for” loops.

The fundamental structure of a for-loop is as follows:

for VARIABLE in LIST; do
command1
command2
…
commandn
done

The VARIABLE can be any valid variable name, although the lowercase letter “i” is conventionally used. LIST represents a collection of items; these can be explicitly defined, sourced from an external text file, or represented by an asterisk (*) to signify all files within the current directory. Commands within the loop are typically indented for improved readability, especially when dealing with nested loops.

Since lists employ spaces as separators, filenames containing spaces can present challenges. For the time being, we will concentrate on working with files that do not include spaces. Let’s begin by creating a simple script to display the names of files present in the current directory. Create a new script file named “loopscript” within your ~/bin/ directory. If you require a refresher on this process – including setting executable permissions and adding the hash bang – consult our introductory bash scripting article.

Add the following code to the script:

for i in item1 item2 item3 item4 item5 item6; do
echo "$i"
done

the-beginners-guide-to-shell-scripting-2-for-loops-2.jpg

Executing this script will output the specified list items.

the-beginners-guide-to-shell-scripting-2-for-loops-3.jpg

This is a straightforward example. Now, let’s modify the script to observe a different outcome. Change the script’s content to:

for i in *; do
echo "$i"
done

the-beginners-guide-to-shell-scripting-2-for-loops-4.jpg

Running this script within a directory will generate a list of the files contained within it.

the-beginners-guide-to-shell-scripting-2-for-loops-5.jpg

Next, let’s transform the echo command into a more practical application – specifically, the zip command. We will archive files and incorporate arguments into the process.

for i in $@; do
zip archive "$i"
done

the-beginners-guide-to-shell-scripting-2-for-loops-6.jpg

Here, “$@” serves as a shorthand for “$1 $2 $3 … $n”, representing the complete list of arguments provided. Observe the result when executing the script with multiple input files.

the-beginners-guide-to-shell-scripting-2-for-loops-7.jpg

The files present in the directory are displayed. The command was executed with six arguments, and each file was added to a zipped archive named “archive.zip”. This is a simple process, isn’t it?

For loops are exceptionally useful. They enable the execution of batch operations on lists of files. You can combine various commands and easily utilize arguments to create dynamic lists, and this represents just a starting point.

Employing for-loops simplifies the execution of numerous actions across all files within a directory. A wide range of commands can be combined, and arguments can be readily used to generate lists on-the-fly.

Experienced bash scripters, do you have any recommendations? Have you developed a useful script leveraging loops? Share your insights and assist fellow scripting beginners by leaving comments!