Bash

My notes from reading the book 'Learn Bash the Hard Way' by Ian Miell. This is a great hands-on practical book that explains all the main concepts to work with bash. Highly recommended.

Basic understanding of Command Line

Appendix A: Command Line Crash Course

1. Commands - Index cards

Linux
Windows

pwd

print working directory

pwd

hostname

my computer's network hostname

hostname

mkdir

make a new directory

mkdir

cd

change directory

cd

ls

list directory

ls

rmdir

remove directory

rmdir

pushd

push directory

pushd

popd

pop directory

popd

cp

copy a file or directory

cp

mv

move a file or directory

mv

rm

remove a file

less

page through a file

more

cat

print the whole file

type

xargs

execute arguments

---

find

find files

dir -r

grep

find things inside files

select-string

man

read a manual page

help

apropos

find what man page is appropriate

helpctr

env

look at your environment prints all the environment variables

---

echo

prints some arguments

echo

export

export/set an environment variable

set

exit

exit the shell

exit

sudo

{DANGER} become super user

runas

---

run a command on many file

forfiles

ps

Show all running static processes

kill

Kills the process with a given id kill -9 40458 violent kill

2. Paths, folder, directories

3. If you get lost

pwd tells you where you are. cd ~ takes you home.

4. Make a directory

mkdir creates a directory. mkdir -p creates directories that don't exist in the given path

On Linux and macOS, mkdir with an existing directory will throw an error. If you have give mkdir -p, it stays silent.

If you give whitespace between the arguments of mkdir, it will create multiple directories. If you want to create a directory with spaces in it, enclose it in " "

5. Change directory

cd changes directory. cd - takes you the previous directory you were in.

If you give whitespace between arguments of cd, it would give error that there are too many arguments. If you want to navigate to a directory with whitespace, use \ . Or enclose it in " "

6. List directory ls

ls lists the contents of the directory

ls -lR shows the contents in recursive fashion.

7. Remove directory

rmdir deleted empty directory. If the directory is not empty, it throws an error.

8. Moving around - pushd, popd

pushd lets you go to a new location after saving your current location. It's like saying remember where I am, then go here.popd - you can return to your saved location. It's like saying, the last directory I saved, pop it and take me there.

If you run pushd without any arguments, then it will remember the current directory into the stack. Then you can go anywhere and do popd, it will take you to the previous directory.

9. Make empty files touchd

touchd makes an empty file with the given name.

10. Copy a file

cp -r copies sub-directories with files in them.

11. Moving a file (mv)

mv moves a file from location to another. It also renames a file. Works for files or directories:

12. View a file (less)

less displays the contents of a file page by page Use (spacebar) to scroll down and w to scroll up. To get out, press q.

13. Stream a file (cat)

cat just dumps the whole file to the terminal - with no paging or stopping.

14. Removing a file (rm)

rm deletes a file or directory. To remove recursively all the contents of a directory, use rm -r or rm -rf

  • rm -f: force deletion of file

  • rm -r foo: deletes the folder foo

  • rm -i: asks for confirmation to delete each file

  • rm -r *: deletes all folders in the current folder

  • rm -r *.*: deletes only the files directly in the current folder (not the inner folders or files)

  • rm -R works the same as rm -r Be careful when running recursive remove on files rm -rf.

15. Exiting your terminal

Use the exit command.

Part 1 - Core Bash

Page 1 - 34

What is Bash

Bash is a shell program. Shell program is an executable binary (file that contains program instructions) that takes the commands that you type and when you hit return translates those commands into system calls to the OS API. Other shell programs - sh, csh, tcsh zsh .... You can run other shells from one another, eg: tcsh from bash

Bash (1987) stands for Bourne-Again, the descendant of the Bourne shell (1977). Zsh originates in 1990.

History of Bash

![[Pasted image 20250316222032.png|800]]

Exercises

  1. Run sh from a bash command line. What happens? Ans: It shows sh-3.2 and starts the sh shell.

  2. Which commands does NOT work in sh - Examples: ll, la do not work. But ls, echo, mkdir, touch etc. work.

Globbing and Quoting

Globbing

ls * - lists the files in the folder matching * , i.e. all filesecho * also lists all files in the folder* stands for 'all' - it is one of the globbing primitives.

Quoting

You can enclose * in single or double quotes to escape it.

Other glob characters

  • ? - matches single character

  • [abd] - matches any character from a, b or d

  • [a-d] - matches any character from a, b, c, or d

Dotfiles

Their names starts with a dot .. They are hidden files and don't show up with ls. You must use la to see them.

Single dot folder - . is a file that represents the current folder. Double dot folder .. is a special folder that represents the parent folder.

Differences to regular expressions

Globs look like regular expressions, but are used in different contexts.* character in the context of regex is used a suffix to ., eg: rename -n '-s/(.*)/new$1/' renames all files to have the prefix new since the * character is enclosed in quotes, it is NOT interpreted by the shell.

Exercises

Variables

(2025-03-16 11pm, Faircrest home)

Basic variables

You create a variable by giving it a name, followed by = and then the value of the variable. eg: FOO=bar. You can see the value of the variable using echo and $ then the variable name. Variable names are capitalized by convention, but not required. You can use quotes " to group multiple words into the value

Quoting variables

You can embed variables in each other. However, if you enclose them in double-quotes " ", bash will translate the variable into its value. If you enclose it is single quotes ' ', it will NOT translate it.

Quoting and globs

This does not happen with globbing - globs (eg: *) in variables are not expanded when in single or double quotes.

Shell variables

These are shell variables that are set up when bash starts, eg PPID is bash's parent process. It is a readonly variable.

Variables made with the export command persist across bash restarts. The new bash session inherits the value set in the previous session. Note - this happens only in the bash sessions started by the running shell. So if you open a new terminal and type that variable, it will be empty.

env and export commands

If you want to see all the variables exported to processes that you start in the shell, use env command.

Simple arrays

Bash treats array in a special way. You can access arrays only by its index, that too encloses in . Example - BASH_VERSINFO returns an array with 6 elements (0 - 5).

Functions

You can have functions in bash. There is no type checking, it just works. Functions can have 'local' variables that are viewed and accessed from within the function. Variables defined outside can be accessed inside too. local can be used only inside a function.

Commands can be one of these - builtins, functions, programs or aliases.

Builtins

Commands that come out of the box in bash. eg: builtin pwd.

  • there is a special type builtin which tells how a command will be interpreted by the shell. eg: type ls returns ls is hashed (/bin/ls), type cd => cd is a shell builtin

  • source is also a builtinPrograms - executable files. You can see where the binary is by typing which cd returns /usr/bin/cdAliases - alternate names given to commands.

Pipes and redirects

Basic redirects

Redirect takes the output from the preceding command and sends it to a file you specify. The redirect operator is >.

Basic pipes

Pipe takes output of one command and passes it as input to another command. | is the pipe operator.

File descriptors

In Unix, everything is a file - when you push/pull data to/from things - whether you write to the terminal or an actual file or network interface. This gives a common interface to send data into - this is a sink, an entity that you can push data into. If you want to write to a terminal, you 'open' the file that corresponds to the terminal and get back a file descriptor. It is a number associated with the process that represents that file. You can write to that file descriptor and the OS will send the data to the right place.

Each process gets three file descriptors by default:

  • 0 is standard input

  • 1 is standard output which is linked to the terminal by default. So the output of all commands eg: cat b.txt is sent to the terminal and you see it there.

  • 2 is standard error which is also linked to the terminal by default. So the output of all commands eg: cat some.txt which is an error is sent to the terminal and you see it there.

The pipe operator | sends only the stdout data of the left command to the right command. It does NOT capture data sent to the stderr.

  • So if the left command succeeds, then the data in stdout is passed onto the right command

  • if the left command fails, then the data is in stderr, not stdout. So it is NOT passed to the right command. Instead it is just displayed in the default descriptor of stderr, which is the terminal. The same applies to the redirect operator >. It send only the stdout data to the file. stderr data is not sent to the file.

If you want a specific output to a specific sink you can use the n> syntax where n is the file descriptor id. Let's understand this by running a command that does not exist, eg: blah and throws an error. By default, stderr is sent to the terminal, so you see it in the terminal. If you redirect it to a file, then nothing will be written in the file (because > does not send stderr data to the file)

If you want to send the error to a file, then use 2> as the redirect option. Remember 2 is the file descriptor for standard error. If you want to ignore the error (i.e not show in the terminal), then redirect it to the 'black hole' file /dev/null.

If you use 2>&1 it means stderror data (2 is stderror) must go to wherever stdout was going at that time (1 is stdout) Note: the order of the redirect operator matters.

Standard out vs. standard error

  • pipe operator passes standard output of one command as the standard input of the next command

  • redirect operator sends file descriptor (could be stdout or stderr as we saw above) to a file.

The following three commands produce the same result.

Double redirect >> appends the text to the end of the file. Single > overwrites the file if it exists.

Basic bash scripting

Shell script is a collection of shell commands that can be run non-interactively. Note - in the command below, you must use single quotes ' ' instead of double quotes " ", otherwise you get the error "bash: !/bin/bash": event not found"

The script failed because it is not marked as an executable. To correct this, use chmod to give exec permission to the script.

If you run the script as my_script without the prefix ./, it would fail because the current directory is not in the PATH variable. (this is where bash looks through to find commands)

Startup scripts

When bash starts up, it runs a series of files to set up the environment that you see at the terminal. It follows a sequence of paths depending on various conditions

  • normal vs.** remote. normal - you start with bash command - the default terminal. remote - running under ssh / rsh

  • login vs. non-login. login - you started by logging in with username/password/keys/ non-login - you started with no credentials

  • interactive vs. non-interactive. interactive: you can type things in. non-interactive: running as a script

  • Files loaded in bash

    • /etc/profile, /ets/bash.bashrc, ~/.bash_profile, ~/.bash_login, ~/.bashrc, Environment variable $BASH_ENV, runs bash, then at the end ~/.bash_logout

  • Files loaded in zsh

    • /etc/zshenv, ~/.zshenv, ~/.zprofile, ~/.zshrc, ~/.zlogin, runs bash, then at the end of session ~/.zlogout

If you want to avoid any startup scripts from running, do env -i bash --noprofile --norc

  • --noprofile asks bash not to source system-wide startup files

  • --norc asks bash not to source the user-specific startup files

source builtin

source runs the script within the context of the current shell. So, any variables defined in the current shell are available inside the script. If you don't use source, then those variables will not be available to the script.

Part 2 - Scripting Bash

Page 35 - 73

Command substitution

It allows you to place the standard output of one commands into the script as you manually wrote it there. Use $( ) or the backquotes for this. Note - if you use single quotes ' ', it will ignore the meaning of $ and will print it verbatim. So use double quotes " " It is better to use the $( ) because the backtick option requires backticks to be escaped when you want to nest them

Exit codes

A variable that tells you the result of a command (or function or built-in). It is like the HTTP error code. You can get the exit code using the command $?.

Exit code
Meaning

0

OK. No Error

1

Generic Error (used when there is no specific error code for the command). A grep command that does not see matching text has exit code 1.

126

Cannot execute due to permission issue

127

Command not found (no file found for the command)

128+n

Process killed with a signal 'n'. Eg: 130 = process killed with Ctrl-C (signal 2)

3 - 125

not reserved. Available for everyone. Use exit builtin followed by the code.

Special parameters:

  • $? - exit code of the last executed command

  • $$ - process id of the shell

  • $! - process id of the job placed in background

  • $0 - the name of the shell All params - https://www.gnu.org/software/bash/manual/html_node/Special-Parameters.html

Tests

Bash tests are constructs that allow you to do conditional expressions. Enclosed within square brackets with whitespace on either side (very important). The test utility evaluates the expression and, if it evaluates to true, returns a zero (true) exit status; otherwise it returns 1 (false).

You can compare values with a variable too.

Logical operators

! means no. || means or. && means and.

[[ operator

It tolerates variables that don't exist and treat it as empty string.

Unary operators

  • -z returns true only if the argument is an empty string

  • -a returns true if the argument is a file or directory and it exists

  • -d returns true if the argument is a directory

if statements

if statements consist of a test (enclosed in [[ and ]]), followed by then and the commands to run if that conditions evaluates to true. There can be an optional elif that is executed if the condition evaluates to false. You end the if block with fi

if can be followed by any command. It will compare the exit code of the command to true/false

Loops

for loops

The for loops is like the regular c-style loop. You need to use do and done

You can use the for in construct too.

Bash has while and until too. Use do and done too.

case statements

Case statements are often used to process command-line options. The helper builtin getopts is very useful here.

set command

set is a builtin that shows all the variables and functions set in your environment. You can run is $ set Note - env shows the exported variables only.

File substitution

We saw command substitution operator $(....) which substitutes the output of the process contained in it to the command File substitution operator <() does the same for file, i.e. it substitutes a file containing the output of the process contained within it

Subshells

It is a shell that inherits variables from the parent shell. It starts when you open a parentheses ( and the running is delayed until the parentheses is closed with )

When you change folder when inside a subshell, then that folder change applies only within the subshell. This will help you avoid multiple cd cd - when you are in scripting mode.

Internal field separator (IFS)

This is a variable that specifies the separator character for file names. By default it is (whitespace), and .

Last updated

Was this helpful?