The magic of bash

Personal notes on some bash magic

So, let’s learn some bash

Bash is an old language but by learning it we can leverage it’s amazing benefits. As a developer you can use bash to automate tasks in a server environment, such a CI/CD, without having to create a Node or Python script. It is always awesome to have one more tool in your developer’s toolbelt 😉.

This are my personal study notes on Cameron Nokes course about bash in the egghead platform. If you can, check it out here!. It’s amazing

Commands

I won’t write the ordinary commands such as ls and cd.

• pwd (print working directory): get your current directory path

/home/myuser

• ls -l (list long): gives more information about files in the current directory

  drwxr-xr-x    5 user group 4096 out 18 03:42 public
  -rw-r--r--    1 user group 5453 mar 31  2020 README.md
  drwxr-xr-x    8 user group 4096 mar 31  2020 src
  drwxr-xr-x    2 user group 4096 set 22 13:36 static

The first column indicates weather it’s a file or folder and it’s access rules. If starts with a - is a file, otherwise d stands for directory.

Then we have the number of hard links (see this article), user and the groups that file/folder belongs, the size, date it was created or last modified, and the name.

• cat {{ filename }}: prints the file content in your terminal

  • cat -n {{ filename }}: see content with line numbers

A better way to view big files is:

• less {{ filename }}: also prints the file content but in a better way. We can scroll the content with our mouse, go to the top with g and bottom with shift + g, and search for words using /{{ word }}. To quit, just do :q

• open {{ filename || folder }}: usefull command for opening files and folders.

  • open {{ filename }} -a TextEdit: choose which application to use when opening a file

• echo {{ some string }}: it logs some text in your terminal. We can use it to insert text into a file too.

  • echo 'hello' > file.txt: inserts text into file.txt, overwritting whatever the file had.

  • echo 'hey there' >> file.txt: appends the text in the end of the file

• mkdir -p {{ new folders path }}: creates multiple nested folders, like mkdir -p one/two/three

• mv {{ filename }} {{ target directory }}: moves a file into other directory

  • we can use it to rename files too: mv one.js two.js will rename the file to two.js. It works with folder rename too.

  • to move every file inside a folder into another folder: mv lib/* src/

• cp {{ file to be copied }} {{ target to be copied into }}: copies a file into another place. We can rename it at the same time, like: cp file.js src/file_2.js

  • we can also copy everything inside a folder into another: cp -R src/* lib/. -R stands for recursive

• find: useful for finding files, for example by extension.

  • find images/ -name "*.png" finds all .png inside /images folder. We could also use -iname for case insensitive matching.

  • find . -type d: finds all folders in your current folder

  • find . -type d -name "images": finds all images folder inside your current directory

  • we can also run commands after finding a file. For example: find dist/ -name "*.built.js" -delete will delete the javascript files after finding them

  • we are able to run exec commands to each file matching. For example, for each .png, runs a image optimization service: find images/ -name "*.png" -exec pngquant {} \;. The {} is interpreted by find command to run the command to each file matching. And the \; ends the expression.

• grep: searches for text matching in a specific file

  • grep "npm.config.get" lib/npm.js searches for “npm.config.get” inside the “lib/npm.js” file. It outputs all matching text.

  • we can modify the command above to search many files with: grep "npm.config.get" lib/**/*.js

  • there are multiple flags we can use to make the output more readable

  • grep --color -n -C 1 "npm.config.get" lib/npm.js. The color colorizes the matching text. -n prints the line number. _-C {{ number }}_ stands for context and it prints the number of lines before and after the matching text.

  • grep also supports regex with the -e flag. grep --color -n -e "npm.config.[gs]et" lib/npm.js for matching both get and set files.

• curl: makes a request to some endpoint

  • in it’s basic form, it just make a GET request and returns the reponse, weather is a HMTL, JSON… curl https://example.com

  • to inspect the headers of the response, we make: curl -i https://swapi.co/api/people/2/

  • curl don’t redirect requests like a browser, so you have to type the exact endpoint or you can pass the -iL flag to it redirect the request to the right endpoint

  • to pass some custom header to the request such as an token, we do curl -H "Authorization: Bearer 123" localhost:3000/api/protected

  • to change the request method, we use the -X flag: curl -X POST -H "Content-Type: application/json" -d '{ "title": "Learning bash" }' http://localhost:3000/create

  • we can pass an URL encoded form too: curl -i -X POST --data-urlencode title="Learning bash" --data-urlencode autor="Myself" localhost:3000/create

  • when posting or putting a large amount of data, it can be useful to split visually the data we’re sending. We can do this:

    
      curl -i -X PUT   -d '{ "title": "Learning bash" }'   -H "Content-Type: application/json"   http://localhost:3000/create
    

  • we can also dump the response into a file: curl -iL https://google.com -o google.txt

  • we can also pipe the response output of a JSON for example into a command to format the previous response: curl https://random-api | jsome where jsome is a node module to format JSON.

Executing scripts

After creating a bash file such as script.sh, you need to give it execute permission with chmod u+x script.sh

Variables are referenced in bash with $. You can get command line arguments with it’s number passed after the executing script. So to get the first argument, just reference the $1.

  echo "$1 world"

  # executing
  ./script.sh Hello

To execute a bash command in a line, do $(command):

  echo "Initializing project in directory $(pwd)"
  git init
  npm init -y
  mkdir project
  touch project/index.js
  # anything else you might want to do

If we create an awesome script and want to execute it through the whole computer, we can add it to our $PATH. The $PATH is just a comma separated list of directories your computer looks for executables. You can inspect with echo $PATH

  /home/user/.nvm/versions/node/v10.16.1/bin:/home/user/bin:/usr/local/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/go/bin:/home/user/go/bin

If we run which node to see where our nodejs executable is:

  /home/user/.nvm/versions/node/v10.16.1/bin/node

It give us the same path directory of the outputed $PATH.

To add some file to your $PATH, we can just cp my-script.sh /usr/local/bin/my-script because the path /usr/local/bin already is in our $PATH so everything inside will be executable too.

Variables

To create a variable in bash we do variableName=123. And to reference it we use $variableName.

The variable is only available for that shell session and only in that scope. To make a variable public for any scope in that shell session, we do export variableName.

We can also unset a variable with unset variableName

There are lots of global variables. We can see it ith env command.

A script to create a temporary directory and clone a git repository in it:

  temp=$(mktemp -d) # create a temp directory and store the path inside the temp variable
  git clone --branch $1 $PWD $temp # clone some branch from the current repository into the temp folder
  echo "branch $1 cloned to $temp"
  # run other tasks or tests

To run this script we do ./script.sh new-branch

Functions

It can be seen as small scripts. To pass arguments, it works the same as passing args for scripts in command line. You reference them by their order number. We don’t name the arguments inside the ()

  greet() {
    echo "$1 World"
  }

  greet "Hey"

We can store the function result in a variable like:

  greet() {
    echo "$1 world!"
  }

  greeting=$(greet "Hello")

  echo "The greeting is $greeting"

An function can access global variables and local variables too. To declare local scoped variables, use the local keyword.

  test() {
    local local_var="I am local!"
  }

Exit status

To get the exit status of a bash command, use $?. The response is a number from 0 to 255

  ls nonexist
  echo $?
  # 1

above error of 1 is a general error. The $? always get the error for the previous command.

Conditionals

  if [[ $USER = 'myuser' ]]; then
    echo "true"
  elif [[ 1 -eq 1 ]]; then
    echo "yayy"
  else
    echo "false"
  fi

There are different comparisson syntax:

• for regular comparisson: = or !=

• for number comparisson: 1 -eq 1, or 1 -ne 1. It stands for equal or not equal

• to check if a file exists: [[ -e myfile ]]

• to check if a variable is empty: [[ -z $USER ]]

• to check if something is less then or greater then: [[ $status -lt 200 ]] || [[ $status -gt 299 ]]

We can also use ifs like a ternary style:

  [[ $USER = 'myuser' ]] && echo "yess" || echo "false"

To make a request and get just the status code:

  curl -ILs https://example.org | head -n 1 | cut -d ' ' -f 2

  # make a request with -I to get just the header, L to make redirects and s to be silent and don't show any progress
  # then we pipe the result with the head command to get the header and get just the first line, which is the status code
  # it will return HTTP/2 200
  # then we can cut the response to get just the number 200
  # we used the cut command, -d of delimiter flag to cut between the spaces and -f to get the second part of the cut

Pipes

To show all running processes of the computer: ps ax. It will log a bunch of lines of processes. But if we want to filter just the chrome instances, we could pipe and grep.

  ps ax | grep Chrome

With pipe, the output of the previous command will be the input of the next one