Welcome back gentle reader. My name is Silent-Mobius, AKA Alex M. Schapelle, your 32bit register pointer, and I wish to introduce you to topics of Open-Source, Linux/UNIX, DevOps and others.

As for today's topic I'd like to dwell on things trivial, yet not always siting right with developers and operations taking in together: Correct Way to right Shell Script.

While it can be claimed, that there is no single way to write shell scripts, it also can be claimed that good tips will make code cleaner and easy to write, read and eventually debugged.

In this article, I'd like to provide a with list of tips, tricks and snippets for you, gentle reader, to help you in your daily scripting tasks and challenges, based on my learning's and experiences. Of course, these suggestions are only my humble opinion, yet I strongly suggest to try these tips and decide afterwards. Let us begin

Editors

Although a war can be spanned from by mentioning a editor is a better then other editor, yet several things need to be said:

• Graphical user interface editors like, vscode, vscodium, atom, lapce, zed or any other ones are fine, yet they are not always an usable/accessible option:
• If you work with terminal/shell, you need to know how to edit files with that terminal/shell.
• Having a "deep know how" on terminal tools can extend your expertise.
  • My preference is with VIM, yet vi, nano, pico, emacs, ed or any other tool, that you find preference with, is also fine.

When it comes to choosing terminal editor, as mentioned, I'll be using Vi Improved, AKA VIM, and only reason for me to choose it, because when my initial parts of knowledge were assembled, the only tool that was provided to me was VIM, and I was required to master it.

Each tool mentioned above, has it's ownrunning configuration file that each of us configures to his/her/its own heart/circuits preference. VIM is not an exception and usually in every system, will have local.vimrc config file in users home folder, or global/etc/vimrc configuration file. VIM usually uses the first one it finds from user home directory, and in case it does not finds one, it gets to other folders, till it reaches the global folder.

In case you are like me, yet did not had invested in configuring your VIM, here is my minimalist.vimrc configuration file that helps me with my scripting task:

setnocompatibleifhas("syntax")syntaxonsetnumber" set tabs to have 4 spacessetts=4setautoindent" show the matching part of the pair for [] {} and ()setshowmatchendifcolorscheme desert

Some clarifications:

• set nocompatible: making it uncompatable withvi
• if has("syntax"): condition that sets bunch of options in case it is more then text file file
• colorscheme desert: setting theme of colorfuldesert theme which makes minor things to pop out.

As mentioned, this is just suggestion, not an requirement.

Initials

When it comes to shell scripts, usually they are written as an utility and thus are used some high/low level programming language to shorten the access to some resource, instead of 'inventing the wheel'. There are cases thatDevs prefer to derive their programming language rules, to the shell scripts andOps prefer not abide any rules at all.

Whether these are correct or not, can be argued, while still practices that I've encountered:

• IfDevs derive rules of specific programming language to shell scripts, for example duct/camel/snake typing,Do invest in Ops team to educate them.
  • Add syntax analysis check on CI level to verify it.Shellcheck is the tool that provides help.
  • Invest in teaching juniors by explaining logic behind it.
• IfOps have decided on shell scripting standard, for examplePOSIX,Share Those Standards With Dev Team!!!
  • Also add those to CI.
  • Justify those standards with UNIX/Linux logic.
• Do Code Review To Each Other Based On Standard Set
• If the script is stand alone script,add an.sh extension
• If the script is going to be used on global level, for example as a command,remove.sh extension and move the script to/bin or/usr/bin folder.
• If script is going to be used by some application,remove.sh extension , createbin folder in application home folder and place the script in there, while editing environment variable$PATH of your system withbin folder path you created, for easy access by application and by you while in development/testing/CI.

Safe header

In essence, scripts are nothing more then, a file with logically structured set of commands that we can execute from shell by passing it tosh orbash commands, e.g.bash myscript.sh.

We elevate those command including files, by adding headers known assha-bang that looks like this:#!.
Sha-bang ensures that commands written in file, will be executed from start to end with the shell of your choosing, e.g#!/bin/bash. Due to compatibility with other scripting languages, I'd suggest to useenv command to dynamically get path to your shell, as shown here:#!/usr/bin/env bash.

#!/usr/bin/env searches full path forbash shell, and in some cases it is not located at/bin, particularly on non-Linux based systems. For example, on FreeBSD systems, it's in/usr/local/bin, since it was installed as an optional package.

The question might pop up in your asking,'What's secured header then ? Secure header, in my opinion, can be defined as combination of shell settings withset and/orshopt command capabilities while developing and running the shell programs. for example:

#!/usr/bin/env bash################################Created by:  Silent-Mobius#Edited by :  Alex M. Schapelle#Purpose   :  Shell script example#Version   :  0.0.0#Date      :  02.02.2024set-o errexit#if error happens, exitset-o pipefail#if pipe has fail, exitset-o nounset#if any unset variable in script, exit###############################

Eachset command created some what of obstacle in terms of developing the script for the task, yet it also gives us opportunity to develop in more safer manner.

Although other parts of comments are not required and can be easily detected by version control tool such asgit, it can be useful to useCreated andEdited to have reference to Dev and Ops that were involved in usage and development of the script.

It is suggested to read more aboutset command either from shell withhelp set or from all over the internet.

Imports and Sources

In some of the use cases, each script that we create, requires environment variable, functions or some type of early defined value taken from somewhere. In those cases it is suggested toimport to be precisesource the files from the destination. Usually it is done before defining any variables. For example:

.... /etc/os-release

The issue with this is animport path issue, where there is no compatibility between developing environment and production environment, or path is not from the OS that you are using, but from NFS/SAMBA/remote storage that is not always accessible.

For that it is suggested tosource files with condition check on the path/mount/remote storage :

...[[-e /etc/os-release]]&&. /etc/os-release||. /etc/rhel-release

One may also useif .. else conditioning as well, yet in cases, where sourcing a lot of files is a requirements then it may become some what overwhelming.

Variables

Next step in our journey isvariables. They are used to save data values in accessible manner while out shell program is running. The variable namesmay start with capital and lower case letters and may include numbers, however they maynot start with numbers and can not include special characters. To declare variable in shell script, just choose variable name and assign it a value.
Choosing variable is not mere task, mostly because it variable name need to be descriptive.When choosing variable name, consider it's purpose.
It is considered a best practice to define all the variables at the beginning of your shell program, although defining those values as you go is also acceptable, depending on the task at hand.
My personal tip on the matter, would be to use variablesonly with capital letters, e.g

...SCRIPT_NAME=$0FIRST_POSITIONAL_ARGUMENT=$1

Dynamically generated variable values

In some cases variables should have initial values, yet not always those values are dynamic and change from system to system and from time to time, thus need to be checked every time the script is invoked as provided below:

...DATE="$(date'+%Y.%m.%u-%H:%M')"# double quotes ensure that it will be string

Note: Shell script do not have data types, but the type are interpreted asscalars, meaning interpreted type of a digit, 0-9, in some cases are integers and in other as strings, thusit is a good practice to double quote every dynamically generated value.

The issue with dynamic data, is that there are cases where is does not exists, thus your script might fail as a consequence of that.
Best suggestion on the matter is to use shell'svariable expansion capability and in case of empty value to use a default instead:

...USERNAME_POSITIONAL_ARGUMENT=${1:-'user'} # if user won't provide positional argument, name `user` will be used

In case, dear reader, you need all variable expansion summary, you may findbash hints on devhint very insightful.

Implicit VS. Explicit

When working with variables, we often make mistake of providing it values based on our understanding of the task at hand,implicitly claiming that value provided is based on some value ofpath or environment variable, which, unfortunately, your script can not know what you know or it even can not assume half of our knowledge. Thus, I suggest to useexplicit declaration of values in order not to fall in small pits of errors.

What does it mean though ?

• When declaring value for path -> always use absolute path
• When checking for dependency tool/package -> use package manager to check it
• When checking for environment variable -> set value instead of it, if it does not exists
• When using some tool -> usewhich command to validate that it exists

Note: Examples can be many, yet these should suffice to be a type of guideline.

Conditions

When it comes to testing our shell code, most of us are some what tempted to use several ways for testing. first of all,if statement isNOT only way to test things, but you can usebash built-ins and POSIX utilities, that can look like this:

• [ ] or test : Binary utility that is located in all POSIX compliant shell, that provides environment testing as well as variable comparison and validation.

  • The downside being that you havetoo many exit codes, that you need to handle in case they occur.
  • Can NOT use REGEX.
  • Limits Bash shell, but perfect in case ofsh,csh,ash, andtcsh.

• () : Also known as command substitution, provides with running several commands in sub-shell and returning exit status, yet it is less useful with conditions, and in my opinion it is mostly suggested to be used as dynamic variable data generator. E.g
  
  

...APP_PATH="$(find /-name'*regex*_of_[pP]ATH')"

• [[ ]] : Upgradedtest utility which is a shell ** builtin **, unliketest which binary file included on your *nix system
  • Returns only 0 or 1, depending on the output of the expression
  • Can use regex and comparison of regex to input string
  • Can use logical and&& in addition to logical or|| in condition checks

Note:test and[[ ]] require white-space around the elements of comparison. In cases where white-space is missing, error is provided and if you've seterrexit as suggested the script should stop, if not, then good luck debugging.

Now the neat part of all this discussion is thatif statement, essentially checks whether the value in front of is zero or not:

• In case of zero, it access the condition and performs the required task.
• In case of non-zero value, it does not do anything inside scope of condition.

->Note: I was requested to add this part, and hope I did not disappoint.

Functions, variables and in between

Another topic of variables, can be overlapping with functions. Unlike most of programming languages, Shell script variables areglobal variables meaning, that they can be reached from any part of the program. When creating variables in function, it's names should belower case, and descriptive, it won't get overridden with before declaredCapital Letter Variables.

...functionhello(){name=Silentlast_name=Mobiusecho"$name-$last_name"}

It is also common to uselocal command, setting a variable to scope of the function. Although many developers I am familiar with, do use this syntax, in my opinion it is less readable and even less understandable, if one has never saw thelocal key word. Yet examples still need to be provided

...NAME=Alexfunctionhello(){localNAME=Silent# The value set in function is Silent# and Not Alex as in global variablesecho"$NAME-Mobius"}

Function names also have a meaning, and it is suggested for functions names to be as describable. In this case I prefer to use Python scripting language typing system calledsnake_case. In cases where function name will be consistent of 2/3/4 words, we'll connect them with underscore:

NAME=AlexLAST_NAME=Schapellefunctionsay_hello(){echo"hello$NAME$LAST_NAME"}

Yet there are also cases where function checks whether some type of condition has occurred or not and notifies other functions in retaliation. It is suggested to start those functions withis word in function name:

...USER=alexfunctionis_user_exists(){ifgrep$USER /etc/passwd> /dev/null 2>&1;thenechoExistselseechoNot Existsfi}# Note: some of these suggestions are borrowed from other# programming languages such as Python, Go, C and others

While on the subject offunctions, it is also suggested that, functions themselves wouldnot print anything, unless they require to provide some type of string or combination of characters. Instead it is customary to usereturn,true andfalse keywords that provides a digital number indicating exit status of the function:

...USER=alexGROUP=wheelfunctionis_user_exists(){ifgrep$USER /etc/passwd> /dev/null 2>&1;then      return0else      return1fi}functionis_group_exists(){ifgrep$GROUP /etc/group> /dev/null 2>&1;thentrueelsefalsefi}# Note: some of these suggestions are borrowed from other# programming languages such as Python, Go, C and others

Other example of function usage withlocal keyword can be when you wish to pass values to function:

...FILE=/etc/passwdfunctionis_this_exists(){localIN=$1if[[-e$IN]];thentrueelsefalsefi}is_this_exists$FILE

The question might arise, about functions and outputs of the program you write in shell script language:If functions do not print anything, what does print output and what writes to log ? There is whole article dedicated for that topic and this time I'd prefer not to dive in to thatpuddle.

Script structure

Up until now we've touched internals of our shell program, yet, in my humble opinion, there is much to discuss, when it comes to program itself.
Shell programs, much like C and Python havestart-to-bottom runtime behavior, meaning that the moment we invoke the script,bash or any other shell based language for that matter, will scan the file and in case there will not be any errors, coming from our syntax or safe header, which we set at the beginning, it will run the content and perform the tasks it has defined inside of itself.

While at start of the script development, it will not matter, with the time solidity will start to form: the more you write, the harder it is to maintain what ever you have written...

Thus I propose to use alternative method of shell script inherited from C and Go programming languages:main entry point of script:

#!/usr/bin/env bash################################... safe header -> not writing to keep it brief################################FILE=/etc/passwdfunctionmain(){# main logic of the script that will be# summoned by the end of the file# here we write main logic of our scriptecho"[+] Main Logic"       is_this_exist$FILE       do_that_thing}functionis_this_exist(){# some  function that imports from other file[[ /import/from/script.sh]]&&. /import/from/script.sh}functiondo_that_thing(){# some function that uses function from other file  command_or_function_from_import_script}####### Main - _- _- _- _-Do Not Remove- _- _- _- _- _######main# this is where the script really starts to run

Great thing about structure above, is the capability it enables -> reading a script in story like manner:

• Every book hascontent list which includes reference to whole structure of the book
  • Themain function includes those instructions and runs them
  • This type of programming can also be calledFunctional programming
  • In case something will fail, thesafe header will stop the program and error function/line will be printed:
  • If your script is several hundred lines long, it might be hard to find the issue
  • Havingmain function as reference to all other functions will enable easier debugging steps.

Note: writing stories is not everyone's forte, and not everyone will agree with statement above, yet from my experience, people struggle when the programs are written in cryptic way, which makes me hope that using the structure above will help some one to achieve better way of development of theirstory telling

Summary

As stated at the beginning, dear reader, I humble wish you to succeed in you journey of becoming better in everything you do with your *nix box.

My hope is that this article, this structure, these scripting tips, will enable you to conquer new heights and challenges. While still reading this article, do surf into our profile to read my other articles, and do not hesitate to subscribe and like or comment.

I hope this was pleasant reading for you, as it was pleasant writing for me. Now, What ever you do - remember: Do Try To Have Some Fun.

Links

• Arnold Robbins
• Chris F.A. Johnson
• Robert Martin AKA Uncle Bob
• FullStackPython VIM