Bash quick start guide=Tom Ryder;Note=Erxin.txt

Bash quick start guide=Tom Ryder;Note=Erxin

# Introduction 
- reference 
https://learning.oreilly.com/library/view/bash-quick-start/9781789538830/

- example code 
 https://github.com/PacktPublishing/Bash-Quick-Start-Guide
 
 
- what is bash 

Bash is not (necessarily) part of Linux. They are separate pieces of software

Bash is not the same thing as SSH. SSH is a service and network protocol for running commands on remote computer

Bash is also not your terminal or TTY. Your terminal is a device for sending information to, and receiving information from, a computer.

Bash is not the same thing as PuTTY, iTerm, or xterm. These are terminal emulators

Bash is not the command line, in the strictest sense. Bash has an interactive mode

- getting bash 

The Bash source code is available at https://www.gnu.org/software/bash/.

- check bash is running 

$ declare -p BASH

printing the value of the SHELL variable:

$ echo "$SHELL"

- swtiching the login shell to bash 

$ bash 

find the location of the bash program:
```
bash$ declare -p BASH
BASH="/usr/local/bin/bash"
```

change Bash to your login shell to that path with the chsh tool:

$ chsh -s /usr/local/bin/bash

    + if get error "/usr/local/bin/bash is an invalid shell"
    
In this case, the invalid shell part likely means that the path given needs to be added to the /etc/shells file

$ cat /etc/shells

- identifyinng the bash version number 

printing the value of the BASH_VERSION variable:

```
bash$ declare -p BASH_VERSION
declare -- BASH_VERSION="4.4.12(1)-release"
```

$ bash --version

- upgrading bash on macOS 

$ brew install bash

- posix shell script features 

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html

running command 

variables 

arithmetic expansion 

control structure 

aliases 

functions 

input and output redirection 

pipes 

argument list 

parameter expansion 

pattern matching 

process management 

compound commands 

reading lines of input data 

formatted strings 

command substitution 

    + calling the grep program, for example, you can select input lines using regular expressions
    
- bash specific features 

named array variables 

performing conditional tests 

extended globs 

regular expression 

local varaibles 

c-style loop 

parameter expansion 

arithmetic expression 

irregular filenames and unusual line separators 

- when to apply bash 

prototyping 

interactive system administration 

automation 

connecting programs together 

filtering and transforming text input 

navigating the unix file system 

basic pattern matching on strings 

portability to unix like systems 

- when to avoid bash 

speed requiring 

fixed or floating point math 

long or complex program 

serialization 

networking programming 

object oriented programming 

functional programming 

concurrent programming 

- getting help 

$ help printf 

$ man printf 

    +  the same command can have more than one implementation. In Bash, see a list with type command 
    
$ type -a printf

    + try help first, example ubuntu output 
```
$ help 
GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu)
These shell commands are defined internally.  Type `help' to see this list.
Type `help name' to find out more about the function `name'.
Use `info bash' to find out more about the shell in general.
Use `man -k' or `info' to find out more about commands not in this list.

A star (*) next to a name means that the command is disabled.

 job_spec [&]                                                            history [-c] [-d offset] [n] or history -anrw [filename] or history >
 (( expression ))                                                        if COMMANDS; then COMMANDS; [ elif COMMANDS; then COMMANDS; ]... [ e>
 . filename [arguments]                                                  jobs [-lnprs] [jobspec ...] or jobs -x command [args]
 :                                                                       kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill ->
 [ arg... ]                                                              let arg [arg ...]
 [[ expression ]]                                                        local [option] name[=value] ...
 alias [-p] [name[=value] ... ]                                          logout [n]
 bg [job_spec ...]                                                       mapfile [-d delim] [-n count] [-O origin] [-s count] [-t] [-u fd] [->
 bind [-lpsvPSVX] [-m keymap] [-f filename] [-q name] [-u name] [-r ke>  popd [-n] [+N | -N]
 break [n]                                                               printf [-v var] format [arguments]
 builtin [shell-builtin [arg ...]]                                       pushd [-n] [+N | -N | dir]
 caller [expr]                                                           pwd [-LP]
 case WORD in [PATTERN [| PATTERN]...) COMMANDS ;;]... esac              read [-ers] [-a array] [-d delim] [-i text] [-n nchars] [-N nchars] >
 cd [-L|[-P [-e]] [-@]] [dir]                                            readarray [-d delim] [-n count] [-O origin] [-s count] [-t] [-u fd] >
 command [-pVv] command [arg ...]                                        readonly [-aAf] [name[=value] ...] or readonly -p
 compgen [-abcdefgjksuv] [-o option] [-A action] [-G globpat] [-W word>  return [n]
 complete [-abcdefgjksuv] [-pr] [-DEI] [-o option] [-A action] [-G glo>  select NAME [in WORDS ... ;] do COMMANDS; done
 compopt [-o|+o option] [-DEI] [name ...]                                set [-abefhkmnptuvxBCHP] [-o option-name] [--] [arg ...]
 continue [n]                                                            shift [n]
 coproc [NAME] command [redirections]                                    shopt [-pqsu] [-o] [optname ...]
 declare [-aAfFgilnrtux] [-p] [name[=value] ...]                         source filename [arguments]
 dirs [-clpv] [+N] [-N]                                                  suspend [-f]
 disown [-h] [-ar] [jobspec ... | pid ...]                               test [expr]
 echo [-neE] [arg ...]                                                   time [-p] pipeline
 enable [-a] [-dnps] [-f filename] [name ...]                            times
 eval [arg ...]                                                          trap [-lp] [[arg] signal_spec ...]
 exec [-cl] [-a name] [command [arguments ...]] [redirection ...]        true
 exit [n]                                                                type [-afptP] name [name ...]
 export [-fn] [name[=value] ...] or export -p                            typeset [-aAfFgilnrtux] [-p] name[=value] ...
 false                                                                   ulimit [-SHabcdefiklmnpqrstuvxPT] [limit]
 fc [-e ename] [-lnr] [first] [last] or fc -s [pat=rep] [command]        umask [-p] [-S] [mode]
 fg [job_spec]                                                           unalias [-a] name [name ...]
 for NAME [in WORDS ... ] ; do COMMANDS; done                            unset [-f] [-v] [-n] [name ...]
 for (( exp1; exp2; exp3 )); do COMMANDS; done                           until COMMANDS; do COMMANDS; done
 function name { COMMANDS ; } or name () { COMMANDS ; }                  variables - Names and meanings of some shell variables
 getopts optstring name [arg]                                            wait [-fn] [id ...]
 hash [-lr] [-p pathname] [-dt] [name ...]                               while COMMANDS; do COMMANDS; done
 help [-dms] [pattern ...]                                               { COMMANDS ; }
```


# Using bash interactively 
- bash manual and gnu readline manual 

```
$ man bash
$ man 3 readline
```

- simple command example 

$ mkdir -p New/bash

- shell metacaracters have special meeting to bash  

many other characters interpreted specially by Bash in some contexts, including {, [, *, and $ etc. 

- quoting, Quoting special characters makes Bash ignore any special meaning they may otherwise have to the shell

- escaping, Using escaping with backslashes, the examples from the previous section

```
$ touch important\ files
$ touch Testfile\<Tom\>.doc
$ touch Review\;Final.doc
$ touch \$\$\$Money.doc
$ echo \\backslash\\
\backslash\

//can't escape newline within a word 
$ echo backslash\
> foo\
> bar
backslashfoobar
```

- single quotes, single quotes can escape a newline in a word

$ echo 'quotes
> foo
> bar'
quotes
foo
bar

two single-quoted strings, It and s today, and pushes them together as one word. The way to do it is to use a backslash outside of the single quote
```
$ echo 'It'\''s today'
It's today
```

- double quote, they perform certain kinds of expansion within them, for shell variables and substitutions

```
$ echo "This is my login shell: $SHELL"
This is my login shell: /bin/bash
```

```
$ echo 'This is my login shell: $SHELL'
This is my login shell: $SHELL
```

    + include a literal dollar sign or backslash in a string by escaping it
    
```
$ echo "Not a variable: \$total"
Not a variable: $total
$ echo "Back\\to\\back\\slashes"
Back\to\back\slashes
```

backtick character is requried to be escaped 
```
$ echo "Backticks: \`\`\`"
Backticks: ```
```

- quote concatenation, put different types of quoting together in the same word, as long as they can never separated by an unquoted space

```
$ echo Hello,\ "$USER"'! Welcome to '"$HOSTNAME"'!'
Hello, bashuser! Welcome to bashdemo!
```

- running commands in sequence 

$ cd ; ls -a ; mkdir New

if one of the commands fails, Bash will still keep running the next command.

- exit values 

$ rmdir ~/nonexistent

check the exist value $? 

$ echo $? 

- stopping a command list on error 

$ cd && rmdir ~/nonexistent && ls

- running command in background with trail &  

$ sleep 10 &



# Essential commands 
- distinguishing command types 

shell builtin commands, they are implemented in the bash binary itself like echo, type and source  

runtime commands, defined at runtime during a Bash session, often by reading startup files 

system commands, commands that can also be run outside of Bash. Examples are grep, ping, and rm.

- builtin commands:

type: Finding what a command is
echo: Printing arguments
printf: Printing formatted arguments
pwd: Printing the current directory
cd: Changing the current directory
set: Viewing and setting shell properties
declare: Managing variables and functions
test, [, [[: Evaluating expressions

- system commands, which are not part of Bash itself:

ls: Listing files for users
mv: Moving and renaming files
cp: Copying files
rm and rmdir: Deleting files and directories
grep: Matching patterns
cut: Extracting columns from data
wc: Counting lines, words, and characters
find: Iterating through a file tree
sort and uniq: Sorting and de-duplicating input

- get help for a command 

$ help type 

The type command, given the name of any command or commands

$ help type 
/bin/true path was returned, even though true is also the name of a shell builtin.

- echo command, emit content to the terminal including variables 

$ echo 'Hello, '"$USER"\!
Hello, bashuser!

-  printf, where echo might struggle, the first argument you provide to it is a format string:

$ printf '%s\n' -n
-n
$ string=-n
$ printf '%s\n' "$string"
-n

- pwd, The pwd Bash builtin prints the current working directory

- tilde paths ~, reference the user home 
$ echo ~

$ echo~root/.ssh 
/root/.ssh 

escape, single-quoting, and double-quoting all work to display ~ instead of reference the home path 
$ echo \~bashuser 
~bashuser 

- cd command, ch to sets the working directory 
$ cd <path> 

- set command, set shell options, and set shell positional parameters.

set prints a list of all of the variables and any functions for the running shell, in alphabetical order. We suggest you use declare instead

set options 
```
$ help set 

-e: Exit immediately if a command exits with a non-zero status. 

-n: Read commands but don't execute them. This is a useful way to check that your Bash script is syntactically correct

-v: Print shell input lines as they are read. This can be good for debugging

-x: Print commands and their arguments as they are executed. 
```

set -x in the shell has the same effect as starting the shell as bash -x, or having a script's shebang line include the option, such as #!/bin/bash -x.

set followed by the option terminator string, --, can be used to set the positional parameters for the shell.

- declare command, only the -p option, declare prints the values of all of the variables in the current shell environment

```
bash$ declare -p
declare -- BASH="/bin/bash"
declare -r BASHOPTS="cdspell:checkhash:checkjobs:checkwinsize:...
declare -ir BASHPID

bash$ declare -p BASH PWD
declare -- BASH="/bin/bash"
declare -x PWD="/home/bashuser"
```

- test, [, and [[ commands 

test file exist 
$ test -e /etc/passwd && echo 'Password file exists!'

alternative way to write this command, called [
$ type -a [

$ [ -e /etc/passwd ] && echo 'Password file exists!'

Bash implements a new version of [ called [[. It's actually not really a command;
$ type [[

[[ keyword that in turn derives its syntax from the [ command.

- system commands 

    + ls command 
    
    $ ls -al 
    
    + getting filenames lists 
    
     use globs or find in this situation. If you only have to deal with one directory, use globs, as the commands tend to be simpler:

$ grep pattern -- *
$ find . -type f -exec grep pattern -- {} \;

- mv command, The mv command moves files or directories to a new directory

last argument is a directory, all of the arguments before it are moved into it

$ mv file1 file2 dir1 path/to/directory

- cp command, used to copy a file or set of files 

$ cp file1 file2 file3 dir

copy a directory and all of the files beneath it:

$ cp -R olddir newdir

- rm and rmdir commands 

You can force this with the standard -R or -r option, but it's safer to use rmdir. This is because rmdir will refuse to delete a directory if it still has files in it;

```
$ mkdir test1 test2
$ touch test1/file test2/file
$ rm -r test1
$ rmdir test2
rmdir: failed to remove 'test2': Directory not empty
```

- grep command match regular expression 

$ grep 'telnet' /etc/services

$ grep '^telnet' /etc/services


$ grep -c telnet /etc/services

```
Usage: grep [OPTION]... PATTERNS [FILE]...
Search for PATTERNS in each FILE.
Example: grep -i 'hello world' menu.h main.c
PATTERNS can contain multiple patterns separated by newlines.

Pattern selection and interpretation:
  -E, --extended-regexp     PATTERNS are extended regular expressions
  -F, --fixed-strings       PATTERNS are strings
  -G, --basic-regexp        PATTERNS are basic regular expressions
  -P, --perl-regexp         PATTERNS are Perl regular expressions
  -e, --regexp=PATTERNS     use PATTERNS for matching
  -f, --file=FILE           take PATTERNS from FILE
  -i, --ignore-case         ignore case distinctions in patterns and data
      --no-ignore-case      do not ignore case distinctions (default)
  -w, --word-regexp         match only whole words
  -x, --line-regexp         match only whole lines
  -z, --null-data           a data line ends in 0 byte, not newline

Miscellaneous:
  -s, --no-messages         suppress error messages
  -v, --invert-match        select non-matching lines
  -V, --version             display version information and exit
      --help                display this help text and exit

Output control:
  -m, --max-count=NUM       stop after NUM selected lines
  -b, --byte-offset         print the byte offset with output lines
  -n, --line-number         print line number with output lines
      --line-buffered       flush output on every line
  -H, --with-filename       print file name with output lines
  -h, --no-filename         suppress the file name prefix on output
      --label=LABEL         use LABEL as the standard input file name prefix
  -o, --only-matching       show only nonempty parts of lines that match
  -q, --quiet, --silent     suppress all normal output
      --binary-files=TYPE   assume that binary files are TYPE;
                            TYPE is 'binary', 'text', or 'without-match'
  -a, --text                equivalent to --binary-files=text
  -I                        equivalent to --binary-files=without-match
  -d, --directories=ACTION  how to handle directories;
                            ACTION is 'read', 'recurse', or 'skip'
  -D, --devices=ACTION      how to handle devices, FIFOs and sockets;
                            ACTION is 'read' or 'skip'
  -r, --recursive           like --directories=recurse
  -R, --dereference-recursive  likewise, but follow all symlinks
      --include=GLOB        search only files that match GLOB (a file pattern)
      --exclude=GLOB        skip files that match GLOB
      --exclude-from=FILE   skip files that match any file pattern from FILE
      --exclude-dir=GLOB    skip directories that match GLOB
  -L, --files-without-match  print only names of FILEs with no selected lines
  -l, --files-with-matches  print only names of FILEs with selected lines
  -c, --count               print only a count of selected lines per FILE
  -T, --initial-tab         make tabs line up (if needed)
  -Z, --null                print 0 byte after FILE name

Context control:
  -B, --before-context=NUM  print NUM lines of leading context
  -A, --after-context=NUM   print NUM lines of trailing context
  -C, --context=NUM         print NUM lines of output context
  -NUM                      same as --context=NUM
      --color[=WHEN],
      --colour[=WHEN]       use markers to highlight the matching strings;
                            WHEN is 'always', 'never', or 'auto'
  -U, --binary              do not strip CR characters at EOL (MSDOS/Windows)

When FILE is '-', read standard input.  With no FILE, read '.' if
recursive, '-' otherwise.  With fewer than two FILEs, assume -h.
Exit status is 0 if any line (or file if -L) is selected, 1 otherwise;
if any error occurs and -q is not given, the exit status is 2.

```

- cut command, select single columns of data from input separated by a single character

the -d specifies the delimiter or separator variable, in this case a colon, and -f specifies the number of the field (or column), starting from 1

$ cut -d: -f1,6 /etc/passwd

- wc, word count command, counts lines, words, and bytes of its input 

$ wc .bashrc

more than one files 
$ wc -c .bashrc .bash_profile

unicode 8 languages 
$ wc -m -c japanese
 6 16 japanese
 
the number of characters (6) is different from the number of bytes (16), as some of the characters are composed of more than one byte in UTF-8

- du command, getting file sizes with wc or du 

The du program can show this information for a file or a directory, but its only portable unit size

- find command, operate recursively on a directory hierarchy 

print filenames 
$ find ~/recipes

explicit with the -print action:

$ find ~/recipes -print

the -name with this glob-style pattern will print the name of any file or directory

To find files modified more than three days ago, you could write:

$ find ~/recipes -mtime +3 -print

can do this with the -prune action, which stops the descent at that point and continues with the rest of the tree

```
Usage: find [-H] [-L] [-P] [`level] [-D debugopts] [path...] [expression]

default path is the current directory; default expression is -print
expression may consist of: operators, options, tests, and actions:
operators (decreasing precedence; -and is implicit where no others are given):
      ( EXPR )   ! EXPR   -not EXPR   EXPR1 -a EXPR2   EXPR1 -and EXPR2
      EXPR1 -o EXPR2   EXPR1 -or EXPR2   EXPR1 , EXPR2
positional options (always true): -daystart -follow -regextype

normal options (always true, specified before other expressions):
      -depth --help -maxdepth LEVELS -mindepth LEVELS -mount -noleaf
      --version -xdev -ignore_readdir_race -noignore_readdir_race
tests (N can be +N or -N or N): -amin N -anewer FILE -atime N -cmin N
      -cnewer FILE -ctime N -empty -false -fstype TYPE -gid N -group NAME
      -ilname PATTERN -iname PATTERN -inum N -iwholename PATTERN -iregex PATTERN
      -links N -lname PATTERN -mmin N -mtime N -name PATTERN -newer FILE
      -nouser -nogroup -path PATTERN -perm [-/]MODE -regex PATTERN
      -readable -writable -executable
      -wholename PATTERN -size N[bcwkMG] -true -type [bcdpflsD] -uid N
      -used N -user NAME -xtype [bcdpfls]      -context CONTEXT

actions: -delete -print0 -printf FORMAT -fprintf FILE FORMAT -print
      -fprint0 FILE -fprint FILE -ls -fls FILE -prune -quit
      -exec COMMAND ; -exec COMMAND {} + -ok COMMAND ;
      -execdir COMMAND ; -execdir COMMAND {} + -okdir COMMAND ;

Valid arguments for -D:
exec, opt, rates, search, stat, time, tree, all, help
Use '-D help' for a description of the options, or see find(1)

Please see also the documentation at http://www.gnu.org/software/findutils/.
```

    + executing commands for each result 

    search string in all files with names ending in .vim in a directory named vim:

$ find vim -type f -name '*.vim' -exec grep -F search -- {} \;

an option terminator (--), and two special arguments:

{}: Replaced with each find result
\;: Terminates the command

    + find and xargs to accomplish something similar to the -exec action of find:

$ find vim -type f -name '*.vim' | xargs grep -F search --

xargs is with the non-standard find -print0 and xargs -0 options that specify null-byte terminators between each file

$ find vim -type f -name '*.vim' -print0 | xargs -0 grep -F search --

- sort and uniq commands 

two forms are the most useful:

$ sort /etc/shells

sort can sort data according to a particular column of its input data

$ sort -t: -k6,6 /etc/passwd

```
Usage: sort [OPTION]... [FILE]...
  or:  sort [OPTION]... --files0-from=F
Write sorted concatenation of all FILE(s) to standard output.

With no FILE, or when FILE is -, read standard input.

Mandatory arguments to long options are mandatory for short options too.
Ordering options:

  -b, --ignore-leading-blanks  ignore leading blanks
  -d, --dictionary-order      consider only blanks and alphanumeric characters
  -f, --ignore-case           fold lower case to upper case characters
  -g, --general-numeric-sort  compare according to general numerical value
  -i, --ignore-nonprinting    consider only printable characters
  -M, --month-sort            compare (unknown) < 'JAN' < ... < 'DEC'
  -h, --human-numeric-sort    compare human readable numbers (e.g., 2K 1G)
  -n, --numeric-sort          compare according to string numerical value
  -R, --random-sort           shuffle, but group identical keys.  See shuf(1)
      --random-source=FILE    get random bytes from FILE
  -r, --reverse               reverse the result of comparisons
      --sort=WORD             sort according to WORD:
                                general-numeric -g, human-numeric -h, month -M,
                                numeric -n, random -R, version -V
  -V, --version-sort          natural sort of (version) numbers within text

Other options:

      --batch-size=NMERGE   merge at most NMERGE inputs at once;
                            for more use temp files
  -c, --check, --check=diagnose-first  check for sorted input; do not sort
  -C, --check=quiet, --check=silent  like -c, but do not report first bad line
      --compress-program=PROG  compress temporaries with PROG;
                              decompress them with PROG -d
      --debug               annotate the part of the line used to sort,
                              and warn about questionable usage to stderr
      --files0-from=F       read input from the files specified by
                            NUL-terminated names in file F;
                            If F is - then read names from standard input
  -k, --key=KEYDEF          sort via a key; KEYDEF gives location and type
  -m, --merge               merge already sorted files; do not sort
  -o, --output=FILE         write result to FILE instead of standard output
  -s, --stable              stabilize sort by disabling last-resort comparison
  -S, --buffer-size=SIZE    use SIZE for main memory buffer
  -t, --field-separator=SEP  use SEP instead of non-blank to blank transition
  -T, --temporary-directory=DIR  use DIR for temporaries, not $TMPDIR or /tmp;
                              multiple options specify multiple directories
      --parallel=N          change the number of sorts run concurrently to N
  -u, --unique              with -c, check for strict ordering;
                              without -c, output only the first of an equal run
  -z, --zero-terminated     line delimiter is NUL, not newline
      --help     display this help and exit
      --version  output version information and exit

KEYDEF is F[.C][OPTS][,F[.C][OPTS]] for start and stop position, where F is a
field number and C a character position in the field; both are origin 1, and
the stop position defaults to the line's end.
```


# Input, Output and Redirection 
- Bash's support for classic shell redirection, specifying a source or a destination
- data-filtering possibilities using the sed and AWK programming languages

```
$ printf 'Hello, terminal!\n'

$ printf 'Hello, file!\n' > myfile
```


HOME can also be expanded for the file destination:

$ printf 'Hello, file!\n' > "$HOME"/myfile

    + only want to allow Bash to create new files, you can set the -C shell option with set:

$ set -C
$ printf 'Third command\n' > myfile

    + overwrite anyway 

$ set -C
$ printf 'Third command\n' >| myfile

- appending to files 

$ printf 'Second command\n' >> file

- created file permissions 

 create a new file using an output redirection, and examine it with ls -l:

$ printf 'Hello, world\n' > myfile 
$ ls -l myfile -rw-r--r-- 1 bashuser bash

first field is file type, normal file is -, directory is d 
next three rw- are the file owner permissions 
next three r-- are the file group permission, a user in the file group 
last three are the permission for the world 

a numeric notation with three octal numbers ranging from 0 to 7:

$ stat -c %a myfile

read and write privileges, so the first value is 4 + 2, or 6
delete is 1 

- choosing permissions for created files 

umask value of the process to find out which of those permissions it should not apply to the created file

other three digits like so:

0022, is a common value for users who are not root. Note the extra zero at the start, a common prefix to signal an octal numbe

an "empty" umask of 0000, Bash creates new files that are readable and writable for everyone:
$ unmask 
$ cat public-content > public

- redirecting errors 

standard error strea's file descriptor number, which is always 2:

$ grep pattern myfile /nonexistent > matches 2> errors

Also note that you can use 2>> to append error output to a file

 using the ampersand (&), syntax to specify 1 (the standard output stream's descriptor):

$ grep pattern myfile /nonexistent > matches 2>&1

- comm tool, which shows lines that differ between files, similar to diff

$ comm myfile1 myfile2 2> /dev/null
$ echo $?
1

- tee command, sending output to more than one place 

$ printf 'Copy this output\n' | tee myfile

$ printf 'Copy this output\n' | tee myfile1 myfile2 myfile3

- redirecting input 

use Bash to redirect input as often as you need to redirect output, because well-designed Unix programs can usually change their input behavior by specifying filenames

    +  tr Unix command is a simple way to translate characters in input to other characters
    
$ tr a-z A-Z
Hello, world!
HELLO, WORLD!

    + from pipe 

$ cat mylines | tr a-z A-Z

    + from file 
    
$ tr a-z A-Z < mylines

we can place the input redirection operator, <, elsewhere on the command line, including at the very start

$ < mylines tr a-z A-Z 

    + perform simultaneously 

$ tr a-z A-Z < mylines > mylines.capitalized 2> mylines.error

- using a long string as input with here documents 

input to cat all lines that are between the <<'EOF' line and the next occurrence of the EOF token as the first word of a new line;

```
#!/bin/bash
case $1 in
    -h|--help)
        cat <<'EOF'
foo command help:
-h, --help: Show this help
-q, --quiet: Run without diagnostics
-v, --verbose: Add extra diagnostics
EOF
        exit 0
        ;;
esac
```

- using pipes 

uppercase letters to lowercase with tr, sorts them, and then prints a count of how often each word is used, sorted by frequency

```
#!/bin/bash

# Convert all capital letters in the input to lowercase
tr A-Z a-z > words.lowercase

# Sort all the lowercase words in order
sort words.lowercase > words.sorted

# Print counts of how many times each word occurs
uniq -c words.sorted > words.frequency

# Sort that list by frequency, descending
sort -k1,1nr words.frequency
```

tr A-Z a-z | sort | uniq -c | sort -k1,1nr

- adding file contents to a stream 

$ cat myfile1 myfile2 myfile3 > myfiles.combined

We could combine them with one pipeline using the - representation for the standard input

- piping ouytput from multiple programs 

a group command, which is delimited with curly brackets, { and }:

$ { date ; hostname ; } | tr a-z A-Z

- filtering programs 

we'll look at the sed and AWK programming language interpreters, both of which are specified by POSIX

- sed stream editor 

read stream, edit, and send the result to output 

delete individual lines with the d command, by prefixing them with the line number
$ sed '1d' text-file 

substitutes a regular expression pattern into a given replacement:

$ sed 's/stream/river/' manual

```
Usage: sed [OPTION]... {script-only-if-no-other-script} [input-file]...

  -n, --quiet, --silent
                 suppress automatic printing of pattern space
      --debug
                 annotate program execution
  -e script, --expression=script
                 add the script to the commands to be executed
  -f script-file, --file=script-file
                 add the contents of script-file to the commands to be executed
  --follow-symlinks
                 follow symlinks when processing in place
  -i[SUFFIX], --in-place[=SUFFIX]
                 edit files in place (makes backup if SUFFIX supplied)
  -l N, --line-length=N
                 specify the desired line-wrap length for the `l' command
  --posix
                 disable all GNU extensions.
  -E, -r, --regexp-extended
                 use extended regular expressions in the script
                 (for portability use POSIX -E).
  -s, --separate
                 consider files as separate rather than as a single,
                 continuous long stream.
      --sandbox
                 operate in sandbox mode (disable e/r/w commands).
  -u, --unbuffered
                 load minimal amounts of data from the input files and flush
                 the output buffers more often
  -z, --null-data
                 separate lines by NUL characters
      --help     display this help and exit
      --version  output version information and exit

If no -e, --expression, -f, or --file option is given, then the first
non-option argument is taken as the sed script to interpret.  All
remaining arguments are names of input files; if no input files are
specified, then the standard input is read.
```
- awk programming language 

AWK program is a set of patterns for each record (normally a line) to match, for which an action is performed for each matching record

$ awk '{ print $2 }' groceries

We can do this with a print command and by specifying the second column with $2

print multiple columns by separating them with commas:

$ awk '{ print $2, $3 }' groceries

NR in the preceding command refers to the record number

As with sed, these examples only scratch the surface; AWK is a small programming language




# Variables and patterns 
- basic form of a variable 

myshell='GNU Bourne-Again shell'

safest to quote all the values in their assignments; it might be unnecessary in some cases

```
myshell='sh'
myvar='GNU'\''s Not Unix!'
myprompt="$USER@$HOST"
```

- listing variables 

$ today='August 9th'
$ printf '%s\n' "$today"
August 9th

$ declare -p today

list all 
$ declare -p

- naming variables 

all legal:

myvar
MYVAR
Myvar
mYVAR
_myvar
my_var
myvar_
my012

- variable name case 

using the ALL_UPPERCASE variable names for your own script variables is that environment variables, such as HOME

variable names such as  myshell and today are in lowercase. 

- clearing variables 

leaving the right side of an assignment to the variable blank:

$ system=

has an empty value:
bash$ declare -p system
declare -- system=""

    + remove varaible 

$ unset -v system
$ declare -p system
bash: declare: system: not found

- environment variables 

shell process after making an assignment. Consider this basic shell session, including an assignment to a variable named realname

bash$ declare -p BASHPID

get a new process ID in BASHPID, and the realname variable does not show as a set:

bash$ bash
bash$ declare -p BASHPID

use environment variables. The POSIX-specified command to do this is called export
```
bash$ REALNAME='Bash User'
bash$ export REALNAME
bash$ declare -p REALNAME
declare -x REALNAME="Bash User"
```

exported to the environment. This export makes the variable available to child processes

- calling programs with environment variables 

sort program uses locale settings to decide the sort order for some strings – specifically, it uses the LANG or LC_COLLATE environment variables

```
$ printf '%s\n' 'alligator' 'Alfred' >words
$ LC_COLLATE=en_US.UTF-8
$ export LC_COLLATE
$ sort words
alligator
Alfred
```
- expanding variables 

$ realname='Bash User'
$ printf '%s\n' "Hello, $realname."

use curly brackets:

$ printf '%s\n' "__Hello, ${realname}__"

    + not exist expanding, by default Bash does not issue any warnings; it just replaces the variable with an empty string
    
$ printf '%s\n' "Hello, $notset."
Hello, .

    + explicit throw error for an empty variable 
    
bash$ set -u
bash$ printf '%s\n' "Hello, $notset."
bash: notset: unbound variable

- reading a value into a variable 

$ read -r myname

Ctrl + D (end-of-file) after pressing Enter after the last preceding line; the entire first line of input will be used as the value for myname

always use the -r option with read, to avoid running into confusing issues with reading backslashed values.

- getting command to output variables 

$ myuser="$(whoami)"
$ printf '%s\n' "$myuser"
bashuser

inner expansion to suppress unwanted expansion:

$ mypwent="$(getent passwd "$(whoami)")"

backticks (`) used for command substitution. Bash still allows this syntax so that older scripts can run, but you should prefer the $(command) syntax to `command`

- parameter expansion forms 

$ prompt for POSIX specified forms 

- specify default values 

use the :- form "${myvar:-default}", where default is the string you want to use instead of the variable's value if it's unset or blank. 


$ cd -- "${MYSCRIPT_DIR:-/usr/local/myscript}"

use the := characters instead of :-, you can additionally assign the default value to the environment variable for the rest of the script

$ cd -- "${MYSCRIPT_DIR:=/usr/local/myscript}"

- string chopping 

The "${myvar#prefix}" form removes one occurrence of prefix from the variable's expansion
```
$ ta='type:json'
$ printf 'Type: %s\n' "${ta#type:}"
Type: json
```

a string from the end of the value instead using % instead of #
```
$ ta='type:json'
$ printf 'Field name: %s\n' "${ta%:*}"
Field name: type
```

escape or quote them:
```
$ text='*WARNING'
$ printf '%s\n' "${text#\*}"
WARNING
```

chop the longest possible match of a pattern with * in it, double the applicable sign: # becomes ##, and % becomes %%.
```
$ path=/usr/local/myscript/bin/myscript
$ printf 'Filename with path removed: %s\n' "${path##*/}"
Filename with path removed: myscript
```

- extracting substrings 

use the ${var:start} or ${var:start:length} form
```
bash$ title='== Password =='
bash$ printf '%s\n' "${title:3}"
Password ==
bash$ printf '%s\n' "${title:3:8}"
Password
```

specifying a negative start value, you can start the counting from the end of the string, starting at -1

```
$ alpha='abcdefghijk'
$ printf '%s\n' "${alpha: -3:2}"
```

- get string length 

${#myvar}" form can be used to expand to the length of a string, counts characters according to your locale, not bytes

```
bash$ alpha='abcdefghijk'
bash$ printf '%u\n' "${#alpha}"
11
```
a short and convenient alternative to counting characters with wc.

- substituting strings 

```
bash$ promise='I'\''ll do it today.'
bash$ printf '%s\n' "${promise/today/tomorrow}"
```
                      
expansions can be done with the "${myvar/pattern/replacement/}" 

replace all instances of the pattern, use two slashes before the first pattern 
```
bash$ promise='Yes, today. I'\''ll do it today.'
bash$ printf '%s\n' "${promise/today/tomorrow}"
Yes, tomorrow. I'll do it today.
bash$ printf '%s\n' "${promise//today/tomorrow}"
Yes, tomorrow. I'll do it tomorrow.
```

- changing case 

all characters uppercase or lowercase using the "${myvar^^}" and "${myvar,,}"

- Combining parameter expansion forms in one statement is not allowed

use a temporary variable instead, to run them one at a time:

#!/bin/bash
temp="${myvar:-foo}"
printf '%s\n' "${temp#.ext}"

- doing math in bash 

$ num_a=3
$ num_b=2
$ printf 'The sum of the two numbers is: %u\n' "$((num_a + num_b))"

- fixed or floating point arithmetic 

$ printf '%.2f\n' "$((3/2))"

- using globs 

A glob by itself will expand to all the filenames that are not prefixed with a dot:
```
$ ls -a
.  ..  .bashrc  april  august  october  september
```

```
$ printf '%s\n' *
april
august
october
september
```

$ printf '%s\n' a*

$ printf '%s\n' ????ber


define valid sets of characters with the [...] syntax:
$ printf '%s\n' *[lr]


inverted with an exclamation mark as the first character in the range:
$ printf '%s\n' *[!lr]

supported [a-z], [0-9], [[:alnum:]]

- configuring globbing behavior 

most useful are dotglob and nullglob.

- including dot file but excluding dot and dot-dot 

exclude . and .. specifically, by requiring a second character that is not a dot
```
$ printf '%s\n' * .[!.]*
```
this would exclude a file named ..bashrc with two leading dots

    + Bash offers an option to deal with this called dotglob, which includes dot files in glob expansion for *, but excludes the . and .. 
    
$ shopt -s dotglob 
$ printf '%s\n' * 


```
shopt: shopt [-pqsu] [-o] [optname ...]
    Set and unset shell options.

    Change the setting of each shell option OPTNAME.  Without any option
    arguments, list each supplied OPTNAME, or all shell options if no
    OPTNAMEs are given, with an indication of whether or not each is set.

    Options:
      -o        restrict OPTNAMEs to those defined for use with `set -o'
      -p        print each shell option with an indication of its status
      -q        suppress output
      -s        enable (set) each OPTNAME
      -u        disable (unset) each OPTNAME
```

- expanding to nothing 

if pattern is not match anything, it expands to itself, unchanged:

$ printf '%s\n' c*

    + bash allow alter this behavior; if we set the nullglob option, c* really will expand to nothing at all
    
```
bash$ shopt -s nullglob
bash$ printf '%s\n' c*
bash$
```
- case insentive globbing 

```
bash$ shopt -s nocaseglob
bash$ printf '%s\n' A*
april
august
```

- extended globbing, Bash's extglob feature allows us to do this. With this shopt option enabled

$ shopt -s extglob 

?(pattern): Match up to one occurrence of the pattern
*(pattern): Match any number of occurrences of the pattern
+(pattern): Match at least one occurrence of the pattern
@(pattern): Match exactly one occurrence of the pattern
!(pattern): Match everything except the pattern



- using array 

$ fruits=('apple' 'banana' 'cherry')
$ printf '%s\n' "${fruits[0]}"
$ printf '%s\n' "${fruits[-1]}"

all of the array elements using @ 
$ printf '%s\n' "${fruits[@]}"

apply parameter expansion operations to every element
$ printf '%s\n' "${fruits[@]}"

- glob expansion arrays 


$ myhomefiles=("$HOME"/*)
$ printf '%s\n' "${myhomefiles[@]}"

- associate arrays, like map or dictionary 

$ declare -A myassocarray
$ myassocarray=([apple]="red" [banana]="yellow" [carrot]="orange")
$ printf '%s\n' "${myassocarray[banana]}"



# Loop and conditionals 
- if 

$ if grep -q bash /etc/shells ; then printf 'Bash is a system shell\n' ; fi

or 

```
if grep -q bash /etc/shells ; then
    printf 'Bash is a system shell\n'
fi

if ! cd /usr/local/myscript ; then
    printf >&2 'Failed to changed directory\n'
    exit 1
fi
```

what is expected after the if keyword is not an expression, but a command to run

- different between [ and [[ 

    + test that can be used to test something, and return an exit status
    
    $ test "$myshell" = 'sh' && printf 'Match!\n'
    
    test -e 'myfile': Whether a file or myfile directory exists
    test -n "$myvar": Whether a myvar variable is set and not empty
    test -d 'mydir': Whether a directory with the name mydir exists
    test -s 'myfile': Whether a file myfile file is not empty
    
    $ man test 
    to get more helps 
    
    + test is the [ command 
    
    $ [ "$myshell" = 'bash' ] && printf 'Match!\n'
    
    ```
    myshell=bash
    if [ "$myshell" = 'bash' ] ; then
        printf 'Match!\n'
    fi
    ```
    
    + doubling it to [[. It's actually a shell keyword, a special part of the Bash syntax
    
    [ and [[ are almost equal but with a few differenciations
    variables are not in double quotes in [[ 
    
    except compare two equal 
    [[ $myshell = "$yourshell" ]]
    it is equal to 
    [[ "$myshell" = "$yourshell" ]]
    
    [[ $myshell = b* ]]
    Because the right side of this expression is unquoted, Bash will check whether the value of the myshell variable starts with b; the rest of it can be any other set of characters, or none at all.

    The [[ keyword also supports a new operator, =~, to test whether strings match regular expressions:
    [[ $myshell =~ 'sh$' ]]
    
    
- when to use bash's (( arithmetic compound command 

[ "$a" -eq "$b" ]: Equal
[ "$a" -ne "$b" ]: Not equal
[ "$a" -lt "$b" ]: Less than
[ "$a" -le "$b" ]: Less than or equal to
[ "$a" -gt "$b" ]: Greater than
[ "$a" -ge "$b" ]: Greater than or equal to

```
if [ "$((bytes / 1000))" -gt "$kbytes" ] ; then
    ...
fi
```

testing arithmetic expressions with (( – without a leading $ sign – including all the terms in one expression
```
if ((bytes / 1000 > kbytes)) ; then
    ...
fi
```

((...)), can contain any arithmetic expression, Variables and Patterns, and it is treated like a command – its exit value reflects the outcome of the expression. An outcome of 0 is treated as false and exits with status 1, and any other outcome is treated as true

```
((2 > 0))  # True; exits 0
((a = 1))  # True; exits 0, and variable a is assigned the value 1
((0 > 3))  # False; exits 1
((0))      # False; exits 1
((a = 0))  # False; exits 1, and variable a is assigned value 0
```

- switch with the case keyword, 

    + don't have to double-quote $command just after the case statement. 
    + The pattern to match has a closing ), but does not require an opening one
    + option is terminated with two semicolons.
    
    
```
case $command in
    help) printf 'Command help:\n...' ;;
    
    //multiple options 
    help|h|usage) printf 'Command help:\n...' ;; 
    
    //glob-matching characters: *, ?, and [...] to test for partial matches
    debug|verb*)
        printf 'Running in verbose mode.\n'
        verbose=1
        ;;
        
    *) printf 'Unknown command\n' ;;
esac
```

- looping over shell words with 
    + for iter in collection; 
    + do is required and needs to be after the semicolon, to specify where the variable and list of words finishes
    + The loop is closed with done, not rof
    + for system in ; do ... ; done is do nothing 

```
#!/bin/bash
for system in bsd linux macosx windows ; do
    printf '%s\n' "${system^^}"
done
```

-  positional parameters, expandable as "$@". We can test this out by assigning the parameters for the current shell

```
set -- bsd linux macosx windows
for system ; do
    printf '%s\n' "${system^^}"
done
```

The for loop is hence the correct choice for iterating over an arbitrary number of arguments on which your script or function is running.

```
for system in "$@" ; do
```

expanding to every value in the array. Don't forget the all-important double quotes

```
#!/bin/bash
systems=(bsd linux macosx windows)
for system in "${systems[@]}" ; do
    printf '%s\n' "${system^^}"
done
```

- skipping an iteration 

The continue keyword skips to the next run of the loop if the directory does not exist.
```
#!/bin/bash
dirs=(/bin /home /mnt /opt /usr)
for dir in "${dirs[@]}" ; do
    [[ -d $dir ]] || continue
    du -s -- "$dir"
done
```

handle the case of an unexpanded glob with a continue condition:
```
#!/bin/bash
for tmp in /tmp/myapp/* ; do
    [[ -e $tmp ]] || continue
    printf >&2 'Warning: file %s still exists\n' "$tmp"
done
```

- ending the loop, The break keyword anywhere within the for loop stops the loop processing.

```
#!/bin/bash
opts=()
for arg ; do
    case $arg in
        --) break ;;
        -*) opts+=($arg) ;;
    esac
done
```

- use when while read -r loops work better "out of the box." instead of for loop to interate through lines. 

- using bash's c-style for loops 

```
#!/bin/bash
for ((i = 1 ; i <= 10 ; i++)) ; do
    printf '%u\n' "$i"
done
```

- while loops 

```
#!/bin/bash
((i = 1))
while ((i < 10)) ; do
    printf '%u\n' "$i"
    ((i++))
done
```

A while loop also behaves much like a for loop in terms of flow control

- infinit loop 

```
while true ; do
    printf 'Infinite loop!\n'
    sleep 1
done
```

the colon builtin, :, used instead of true are equal 

```
while : ; do
    mygameserver --foreground
    sleep 1
done
```

a quick wrapper script, to run a program again even if it crashes or exits; this is often used to keep a game server daemon process running

- reading data line by line

    + Consider a file named fcs

    ```
    #!/bin/bash
    while read -r name ; do
        printf '%s\n' "$name"
        printf 'https://en.wikipedia.org/wiki/%s\n' "${name// /_}"
    done < fcs
    ```
    "${name// /_}" is string replacement, // will replace all the found pattern 

    while loop's test command is read -r name. read is a builtin command that accepts a line from standard input

    + field splitting by while build in support, specify multiple variable names 
    
    ```
    while read -r firstname lastname ; do
        printf '%s\n' "$lastname"
    done < fcs
    ```

    + saving fields into arrays 

    Using the -a option to the read builtin, we can save the separated fields of a read line of input as an array variable

    ```
    #!/bin/bash
    while read -r -a animals ; do
        for animal in "${animals[@]}" ; do
            printf '%s\n' "$animal"
        done
    done < animals-by-letter
    ```

    + choosing the splitting character
    
    a special variable named IFS – the Internal Field Separator – to decide where to split the line it reads
    
    ```
    while IFS=: read -r user pass uid; do
        printf '%s\n' "$uid"
    done < /etc/passwd   
    ```

    the setting for IFS to the colon character, :, occurs just before the read command

    + disable whitespace trimming 

    by default read trim the input line 
    
    Disalbe trim by set IFS to a blank value for the read command:

    $ while IFS= read -r line ; do 
    
    + a while loop can also accept the output of another command as input, using a pipe
    
    ```
    who -T | while read -r username state terminal _ ; do
        printf '%s\n' "$terminal"
    done
    ```
    
    + sub shells problem, a command forked process—changes to variables do not persist after the pipeline is completed.
    
        * use a temp file 
        
        ```
        who > who.out
        while read -r ; do
            ((count++))
        done < who.out
        printf '%u\n' "$count"
        ```
    
        * use a compound command to include all of the code that needs to use the count variable as part of the same subshell
    
        ```
        who | {
            while read -r ; do
                ((count++))
            done
            printf '%u\n' "$count"
        }
        ```
    
        * process substitution:

        ```
        while read -r ; do
            ((count++))
        done < <(who)
        printf '%u\n' "$count"
        ```
        
        Note the < <(who) syntax at the very end of the loop. The first left angle bracket, <, is a standard input redirection
   
        * lastpipe option set with shopt, which makes the last command in a pipeline run in the current shell
        
        ```
        shopt -s lastpipe
        ((count = 0))
        who | while read -r ; do
            ((count++))
        done
        printf '%u\n' "$count"
        ```
        
    + avoiding input problems, some of the program may consume the input stream within the loop such as ssh. 
    
    The -n switch to ssh suppresses this behavior, instructing it not to read any standard input:
    ```
    while read -r hostname ; do
        ssh -n -- "$hostname" uptime
    done < hostnames
    ```
    
    try directing /dev/null into it 
    ```
    while read -r item ; do
        program </dev/null "$item"
    done < items
    ```
    

# Scripts, Functions and Aliases 
- Aliases: Can expand a word into a command line string
    
    $ alias ls='ls --color=auto'
    $ alias ll='ls -l'
    
    turning on the -x option for the shell, which will print the expanded command before executing it, after a + prefix:

    ```
    $ set -x
    $ ll .bashrc
    + ls -l .bashrc
    ```
    
    + Functions: A way to write a command from a set of other commands
    
    + Scripts: Take the form of programs in files on your system
    
    + Shortcoming with aliases 
    
    The commandline argument was only passed to the last command
    
    using quotes in the command definitions can get very confusing
    
    aliases is that invoking them in Bash scripts doesn't work; they are strictly for use in interactive mode. But other shells may allow use aliases in script this is confusing. 
    
- functions
    + define function 

    $ home() { printf '%s\n' "$HOME" ; }

    + arguments 
    
    one possible implementation, put $1 in double quotes to stop any special characters in its value being interpreted by Bash

    $ mkcd() { mkdir -p -- "$1" && cd -- "$1" ; }
    
    "$1", we can get the second positional parameter with "$2", the third with "$3", and so on. This can go beyond "$9" if you want, with curly brackets: "${10}", "${11}", and so on

    + using -- to separate options from filenames 
    
    calls to "$1" with a double dash, --, the option terminator string. This syntax specifies a point beyond which further arguments are meant as argument strings for the program, often filenames, and not as options
    
    Most commands that accept options use this syntax to separate options from filenames or other words, including bash
    
    + getting all the arguments with expand parameter 
    
    Bash has a syntax for this, with the special "$@" expansion parameter:

    $ mkls() { mkdir -- "$@" && ls -dl -- "$@" ; }
    
    changing the shell's own positional parameters with calls to set and printf:
    $ set -- arg1 'argument 2' '-third argument'
    $ printf '%s\n' "$@"
    
    + returning values from functions 
    
    functions, succeed and fail; rather like true and false, they always return 0 and 1, respectively:

    bash$ succeed() { return 0 ; }
    bash$ fail() { return 1 ; }
    
    we can test these using the special $? parameter:

    ```
    $ succeed;echo $?
    0
    $ fail;echo $?
    1
    ```
    
    use return without specifying a number afterward, the function's return value will be the exit value of the last command it ran. This means we could write succeed() and fail() like this instead
    
    ```
    succeed() { true; }
    fail() { false; }
    
    early return be a good way to stop processing a function due to errors 
    ```
    mkcd() {
        if (($# != 1)) ; then
            printf >&2 'Need exactly one argument\n'
            return 1
        fi
        mkdir -- "$1" && cd -- "$1"
    }
    ```
    
    + function input and output, command substitution to include the function's output in a message of its own. 

    $ home() { printf '%s\n' "$HOME" ; }
    $ printf '%s\n' "Your home directory is: $(home)"
    $ home | wc -c
    
    wc still counted the newline character
    
        * read data from standard input as well.
        
        $ grep_un() { grep -F -- "$USER" ; }
        $ getent passwd | grep_un
        
    + function scope, Functions defined during a bash session are only available in that session. 
    
    This means that variable assignments aren't local to functions; they're visible outside it, after the function has run
    
    ```
    $ func1() { foo=bar ; printf '%s\n' "$foo" ; }
    $ func1
    foo
    $ declare -p foo
    declare -- foo="bar"
    ```
    
    declare the variable as being local to the function before you set it:
    ```
    $ func2() { local baz ; baz=quux ; printf '%s\n' "$baz" ; }
    ```
    
    + run a function without affect current shell. run it with a subshell body instead, by using parentheses instead of curly brackets. Consider these two functions, etc1 and etc2
    
    ```
    // influence the shell 
    $ etc1() { cd /etc ; }
    
    // will not influence the shell. The semicolon is not required too.  
    $ etc2() ( cd /etc )
    ```
    
    + reloading function at shell startup 
    
    a good place to put such function declarations is in your ~/.bashrc file. 
    
    we can put aliases, function defintion here. 
    
    declare them with the -f option and redirect to the .bashrc file 
    $ declare -f function_name >> ~/.bashrc 
    
        * reload .bashrc without logout 
    
        $ source .bashrc 
    
- scripts 

    + Sourcing the script means to use the Bash source command to read all of the commands in a script into the current shell session and run them there
    
    $ source hello.bash 
    
    + you will want the script to do its work, and then exit and leave you back at your shell session without changing any. 

    This starts a new Bash subprocess, which runs the commands in the file, and then exits. in three ways 
    ```
    $ bash hello.bash
    Hello, bashuser!
    
    $ cat hello.bash | bash
    Hello, bashuser!
    
    $ bash < hello.bash
    Hello, bashuser!
    ```
    
    + directly run the script with shebang script. shebang as the very first line of the script:

    after add shebang the file extension is not required 
    ```
    #!/bin/bash
    printf 'Hello, %s!\n' "$USER"
    ```
    
    checking the value of the BASH variable:
    
    ```
    $ declare -p BASH
    declare -p BASH="/usr/local/bin/bash"
    ```
    
    use the chmod system program to make the script executable
    ```
    $ chmod +x hello.bash
    ```
    
    The file tool can help you identify scripts and binaries; for example, the ldd program on Debian GNU/Linux turns out to be a Bash script:

    bash$ file /usr/bin/ldd
    Bourne-Again shell script, ASCII text executable
    
    + finding scripts with $PATH 
    
    need to use the special PATH variable to specify where it can find commands such as hello, without needing to specify a full filesystem path
    
    ```
    $ declare -p PATH
    declare -x PATH="/usr/local/bin:/usr/bin:/bin"
    ```
    the -x means it is an environment variable, It's a colon-separated list of directories. the first matching command found is run, and the rest are ignored. We will call these bindirs, short for logical binary directories.
    
    + system bin directory 
    
    write to /usr/local/bin, you might put it there, using su or sudo to get root permissions first
    
    + user bindir, create a new user bindir in our home directory, called ~/bin 
    
    adding the directory to the end of PATH with a variable assignment:

    $ PATH=$PATH:/home/bashuser/bin 
    
    add this to .bash_profile file, or .profile if .bash_profile 
    
    check file in home bin 
    ```
    $ type -a hello
    hello is /home/bashuser/bin/hello
    $ type -t hello
    file
    $ type -p hello
    /home/bashuser/bin/hello
    ```
    
    + arguments to scripts 
    
    You can get the arguments as positional parameters with "$1", "$2", "$3", and so on
    You can dynamically get all of the positional parameters without having to count them with "$@"
    You can count the number of positional parameters if you need to with "$#"
    
- understand sh vs bash 

difference is that /bin/sh may not even be the same program as /bin/bash, and may therefore lack many of the programming features

Some GNU/Linux systems use bash as their /bin/sh implementation (Arch Linux, CentOS), but others do not (Debian GNU/Linux, Ubuntu).

When Bash is called as sh, it changes its behavior slightly to be consistent with the way the /bin/sh program

If you're writing a Bash script, use a bash shebang. This makes clear to both the system and to anyone reading your script what shell it should run in

- using env 

GNU linux almost always available in /bin/bash. On other systems, such as FreeBSD, it might instead be in /usr/local/bin/bash.

a shebang with bash as the argument will find the first bash program in PATH:

```
#!/usr/bin/env bash
```
    
you should specify a fixed path, such as #!/bin/bash, for the correct interpreter either in the packaging process or at build time. 

For casual sharing of programs, using /usr/bin/env might be good enough

- choosing between functions and scripts 

want your command only to be limited to an interactive shell, for example to override interactive behavior, add options to a command, or other shortcuts, use a function.
 
affect a running shell process, such as to change your current shell directory, or to read or set shell variables, use a function.

use a script as a filesystem program in a bindir specified in $PATH. This script will then be callable by the system in any other situation you need, not just in Bash.

- using functions in script 

die function that catches error conditions, prints any given messages, and then exits the script

declare the functions before you call them. It's best to put the functions near the top of the script.

```
die() {
    printf >&2 '%s\n' "$@"
    exit 1
}
tempfile=$(mktemp) || die 'Could not create temporary file'
(( $# > 0 )) || die 'Need at least one argument'
[[ $1 != *:* ]] || die 'Colon not allowed in directory name'
```
 
 

# Best practices 
- quoting correctly 

wrap expansions, such as $myvar, ${myvar[1]}, and ${myvar##*/}, in double quote

    + In a few contexts in Bash, an expansion does not strictly require double quotes

newvar=$myvar
case $myvar in ...
[[ $myvar = "$othervar" ]]

quote expression as much as possible.   

- when you don't want quotes 

an ideal application of the positional parameters in the POSIX shell, or (preferably) array variables in Bash  

```
# POSIX (if necessary)
(
    set -- --location --cookie-jar ~/cookies --cookie ~/cookies
    curl "$@" --head -- http://www.example.org/
    curl "$@" -- http://www.example.net/
)
# Bash (better)
curlopts=(--location --cookie-jar ~/cookies --cookie ~/cookies)
curl "${curlopts[@]}" --head -- http://www.example.org/
curl "${curlopts[@]}" -- http://www.example.net/
```

if quote the position arguments will not work properly 

    + build and store whole command string 
    
```
an example of building all the options and arguments for a grep command line:

#!/bin/bash

# Example starting values
extended=0
fixed=1
patterns=(foo bar)
extra_files=()
# Start with the grep command line
grepcmdline=(grep)
# If the extended or fixed flags are set, add -F or -E
if ((extended)) ; then
    grepcmdline+=(-E)
elif ((fixed)) ; then
    grepcmdline+=(-F)
fi
# Add each pattern to check
for pattern in "${patterns[@]}" ; do
    grepcmdline+=(-e "$pattern")
done
# Add option terminator
grepcmdline+=(--)
# Add files to check
grepcmdline+=(* "${extra_files[@]}")
# Run command!
"${grepcmdline[@]}"
```

- handling filenames starting with dashes 

$ cp "$myvar" destdir 

a file named -alphabet-, this would result in a confusin

use the -- terminator string option, any word after -- is NOT an option 

    + some of the command may not support --, use ./ or fully qualified name 
    
$ myscript /home/bashuser/-alphabet
$ myscript ./-alphabet

- separating output and diagnostics 
    
always write error output to standard error file descriptor, not to output. using the >&2 redirection 

- keeping script brief and simple 
- flexible 
- respecting and applying the user's configuration

sensible defaults if these values are not set, using the :- form of parameter expansion

- allowing script to run without user inmput 

using configuration file to help user reduce the input times    '

- limit the scope of shell state changes 

The working directory
The positional parameters ($1, $2 ... )
Environment variables, especially PATH
Shell-local variables, especially IFS
Shell options with set (such as -x) or shopt (such as dotglob)
Shell resource limits set with ulimit

Keeping the script itself so short that the change doesn't matter
Changing the state back to its original value again
CHange in subshell 
Avoid the change as much as possible 

- avoiding path anit-patterns 

don't save and call absolute path for system command 

- avoiding bash for untrusted user input, Shell script is not a language that was designed with the security of untrusted input in mind. Use python etc.  

- document strings, three documentation methods: writing comments, providing help output, and writing manual pages.

a format such as mandoc: https://mandoc.bsd.lv/

- using temporary files clearly, randomized name, and in a location and pattern that's consistent with the system's settings:

$ mktemp -d

- cleaning up after a script 

```
#!/bin/bash

# Code setting and using tempdir goes here, and then ...

# Remove the directory
if [[ -n $tempdir ]] ; then
    rm -- "$tempdir"/myscript-timestamp
    rmdir -- "$tempdir"
fi
```

- a tool to check shell scripts for problems 

online service and a command-line tool at https://www.shellcheck.net/.




# Additional reference 
- how to write test 
https://www.gasparevitta.com/posts/how-to-test-bash-scripts-bach/