Lecture 1 : Course Overview + The Shell (2020)¶
These are my notes from the video lectures reworked a little. They are not a direct transcript but follow the video content.
The official transcript is available on the course website
Course Overview¶
The class exists to make the computer science students more efficient in their processes on top of making efficient processes for others.
The class will attempt to bring the tools to the students and become more efficient with the tools they know and with new tools.
It consists of 11 lectures (available online on Youtube).
Because the class has limited length, it covers interesting tools and use cases but does not necessarily go in details or cover all use cases.
The shell has more possibilities than Graphical User Interfaces (GUI) because in a GUI you can only do what you have a button for.
They are a lot of different shells and consoles. The most popular in BASH (Born Again Shell) and that’s the one that will be covered in the class. It’s also the one that is provided with Linux Operating Systems.
The Shell : Bases¶
The prompt can be customized (which will be covered in a later lecture). The prompt is often used to run programs and to run programs with arguments.
Different very basic commands are :
date: returns the dateecho: prints its arguments
Arguments are separated by spaces so if you pass one argument that contains a space, for example a folder name with a space in it, you have to escape the space by using a \, otherwise they will be considered as two arguments and two folders will be created.
An important question is :
How does the shell know what to execute when I type a command ?
The answer to this is that some programs are shipped with the computer (or at least with the OS). The shell has a way to search for programs to execute that is based on environmental variables.
Actually, BASH or the “Born Again Shell” is a real language which you can program and that will be covered in the lecture about shell scripting.
Therefore, you can do “for loops”, while loops, conditional statement etc.
Environmental variables contain all sorts of valuable information for the shell such as, what is the user’s name, what is the home directory. They are set automatically when the shell is started.
An important one to answer our question is the variable $PATH.
To run the command echo, the shell searches among the values in $PATH for the command. The shell looks for the command echo in the different paths until it finds it.
To know which path is actually used, we can use the command which.
which echo
It answers the question : “If I were to run a command named echo, I would run …”
which thus provides the path to the program that will be executed.
Paths¶
What is a path ?
A path is a way to name the location of a file on a computer.
In Linux and Mac, all paths start with \ as they start in the same root directory and the directories are separated with a \ in the path.
In Windows, there exists separate file system (C, D, …) and the directories are separated with regular slash /.
Special directories
. is the current directory and .. is the parent directory and you can combine them together.
.\test looks for a directory test in the current directory.
~ is a shortcut for your home directory.
For example : To go up 4 directories, you can use ../../../...
Relative path and absolute paths can be used interchangeably, the shortest is usually the one used.
However, if you write a program that should be runnable from anywhere, using an absolute path is more reliable as it does not depend on the location the program is run from.
Navigate the file system from the command line¶
To navigate the file system you need to see what is in a directory for which you would use :
| Command | Action |
|---|---|
ls |
lists the content of the path that you give it or of the current directory if you don’t give it any arguments. |
cd |
stands for change directory and changes the working directory to the given path |
pwd |
states for print working directory and does exactly that |
N.B. :cd - is a handy way to toggle between directories as it sends you to the last path you were in.
Additional arguments
Functions also take flag and arguments starting with - or --. To discover those arguments, you can nearly always use command name --help.
The result should be read this way.
For example for ls --help:
jason@jason-laptop:~$ ls --help
Utilisation : ls [OPTION]... [FICHIER]...
Afficher des renseignements sur les FICHIERs (du répertoire actuel par défaut).
Trier les entrées alphabétiquement si aucune des options -cftuvSUX ou --sort
ne sont utilisées.
Les arguments obligatoires pour les options longues le sont aussi pour les
options courtes.
-a, --all ne pas ignorer les entrées débutant par .
-A, --almost-all ne pas inclure . ou .. dans la liste
--author avec -l, afficher l'auteur de chaque fichier
-b, --escape afficher les caractères non graphiques avec des
protections selon le style C
--block-size=TAILLE avec -l, dimensionner les tailles selon TAILLE avant
de les afficher. Par exemple, « --block-size=M ».
Consultez le format de TAILLE ci-dessous
-B, --ignore-backups ne pas inclure les entrées se terminant par ~ dans
la liste
-c avec -lt : afficher et trier selon ctime (date de
dernière modification provenant des informations
d'état du fichier) ;
...means 1 or more[]means optional
What takes no argument is called a flag and what takes arguments is called an option.
Let’s look at what -l provides a long format list.
-l utiliser le format long d'affichage
Permissions
So what happens if we type ls -l ?
jason@jason-laptop:~$ ls -l
total 4324
drwxr-xr-x 15 jason jason 4096 May 31 15:31 Arduino
drwxr-xr-x 2 jason jason 4096 Mar 25 10:30 bin
drwxr-xr-x 5 jason jason 4096 Jun 9 12:04 Desktop
drwxr-xr-x 8 jason jason 4096 Aug 17 11:30 Documents
drwxr-xr-x 13 jason jason 4096 Aug 19 08:45 Downloads
How do we read that :
drwxr-xr-x 15 jason jason 4096 May 31 15:31 Arduino
| Value | Meaning |
|---|---|
| d | Directory if not present it’s a file |
| rwx | permissions for the user owner |
| r-x | permissions for the group owner |
| r-x | permissions for everyone else |
| 15 | the number of files in the directory (or 1 if it is a file) |
| jason | the name of the user owner |
| jason | the name of the group owner |
- The
dat the beginning of the line indicates that it is a directory, if it were a file there would be nothing. - Then we have 3 groups of 3 characters which indicate permissions for the different users
- The 1st is for the user owner
- The 3rd is for everyone else
- The 2nd is for the group owner
- After that we have the number of files
The different permissions are :
| Value | Full word | Meaning for files | Meaning for directories |
|---|---|---|---|
r |
read |
Can read the file | Can see what’s in the directory (equivalent to list the content) |
w |
write |
Can modify and save the file | Can rename create or remove files in the directory |
x |
execute |
Can execute the file | Can enter the directory to do anything within the directory |
- |
nothing | no permission | no permission |
Manipulating files and folders from the command line¶
We now know the basic functions to navigate
mv and cp serve to move or copy files from the terminal and obey the following syntax.
rm is used to remove files, it is not applicable to folders as by default rm is not recursive. To delete a folder recursively (including everything in it, files or directories), you can use rm -r but be cautious with that command.
rmdir is $another solution to delete empty folders (and avoid the dangerous rm -r command)
mv INPUT_PATH DESTINATION_PATH enables you to move a file or folder.
| Command | Action |
| :------------- | :------------- |
| mv INPUT_PATH DESTINATION_PATH | Move files or directories to a new location |
|cp INPUT_PATH DESTINATION_PATH |Copy files or directories to a new location |
| rm | used to remove files, it is not applicable to folders as by default rm is not recursive. To delete a folder recursively (including everything in it, files or directories), you can use rm -r but be cautious with that command. |
|rmdir | another solution to delete empty folders (and avoid the dangerous rm -r command) only works with empty directories |
|mkdir | stands for make directory and creates directories |
Caution if you type mkdir My folder you would create 2 directories, one named My and another named folder.
To create a single directory called My folder, you need to escape the space with a \ thus type mkdir My folder.
Learn about the tools¶
To get the manual of a function, you can type man FUNCTION this provides you with clearer description of the function which can be similar to --help but not necessarily.
To quit this mode (and others), you often can type q (for quit).
clear or Ctrl+L clears the terminal and go back to the top.
The power of the shell¶
Chaining programs together is the real power of the shell.
To do that you need to follow streams.
Input and output
To modify this, you can use :
<indicates rewire the input for this program to be the the content of this file>indicates rewire the output for this program to be into this file
The default for the output stream is a print to the console but you can put it in a file.
For example with :
echo Hello\ World > hello.txt
This will create (or modify) the file hello.txt in the working directory as hello.txt is a relative path and write in hello.txt the output of the command echo Hello, so Hello.
Now if we want to see what is in hello.txt we can use the command cat.
And we can also chain inputs and output to it and we get the following :
$ echo Hello\ World
Hello World
$ echo Hello\ World > hello.txt
$ ls
hello.txt
$ cat hello.txt
Hello World
$ cat < hello.txt
Hello World
$ cat < hello.txt > hello2.txt
$ ls
hello2.txt hello.txt
Ok but if we do that again the content of hello2.txt does not change as we replaced it with the same.
$ cat < hello.txt > hello2.txt
$ cat hello2.txt
Hello World
>> we append the file hello2.txt instead of writing it and we can see that it added the content.
$ cat < hello.txt >> hello2.txt
$ cat hello2.txt
Hello World
Hello World
Pipe | and pipelines¶
The character |(called pipe) takes the output of the previous function and pipes it as an input to the next program.
So now we can chain functions together which makes lots of things possible even streaming a video from the net and casting it to your tv with a command line tool.
The root user¶
On Linux and MacOS systems, there is a user called root.
The root user is the superuser on the computer, it can do everything, even if no one else can do something like that.
To use this functionality, you use the program sudo that stands for do as superuser.
In practice, you use it with the following syntax, and the command is run as superuser:
sudo COMMAND
# indicate running as root while the $ in jason@jason-laptop:~/$ indicates that I am not running as root.
To create a shell running as root, the superuser, you can use sudo su, which tells the shell to run the command su as superuser (because of the sudo command).
However, sudo is also a program that is applied only on its argument.
Therefore, sudo echo 500 > test.txt runs echo as superuser but writing in text.txt is done as jason a regular user.
With |, we can do phrase it differently:
We’ll use tee which writes its input to a file while printing to the console.
So now, echo 500 | sudo tee test.txt does what we want, echo is run as regular user and tee is run as superuser.
If you want to see usecases when you need to act as root, like turning on LED corresponding to the Scroll Lock when you receive emails (that is at 43:24), check the extract hereunder.
Additional notes¶
On Linux, you can use xdg-open to open any files with the appropriate tool on your computer.
That is for example the program used to open links for teleconference from my browser such as Zoom, Teams or others that have a native app on Ubuntu.
Exercises¶
Here’s a copy paste of the exercises from the official page of the course
- For this course, you need to be using a Unix shell like Bash or ZSH. If you are on Linux or macOS, you don’t have to do anything special. If you are on Windows, you need to make sure you are not running cmd.exe or PowerShell; you can use Windows Subsystem for Linux or a Linux virtual machine to use Unix-style command-line tools. To make sure you’re running an appropriate shell, you can try the command echo $SHELL. If it says something like /bin/bash or /usr/bin/zsh, that means you’re running the right program.
- Create a new directory called missing under /tmp.
- Look up the touch program. The man program is your friend.
- Use touch to create a new file called semester in missing.
-
Write the following into that file, one line at a time:
The first line might be tricky to get working. It’s helpful to know that # starts a comment in Bash, and ! has a special meaning even within double-quoted (“) strings. Bash treats single-quoted strings (‘) differently: they will do the trick in this case. See the Bash quoting manual page for more information.#!/bin/sh curl --head --silent https://missing.csail.mit.edu -
Try to execute the file, i.e. type the path to the script (./semester) into your shell and press enter. Understand why it doesn’t work by consulting the output of ls (hint: look at the permission bits of the file).
- Run the command by explicitly starting the sh interpreter, and giving it the file semester as the first argument, i.e. sh semester. Why does this work, while ./semester didn’t?
- Look up the chmod program (e.g. use man chmod).
- Use chmod to make it possible to run the command ./semester rather than having to type sh semester. How does your shell know that the file is supposed to be interpreted using sh? See this page on the shebang line for more information.
- Use | and > to write the “last modified” date output by semester into a file called last-modified.txt in your home directory.
- Write a command that reads out your laptop battery’s power level or your desktop machine’s CPU temperature from /sys. Note: if you’re a macOS user, your OS doesn’t have sysfs, so you can skip this exercise.
Solutions¶
-
$ echo $SHELLreturns/bin/bashwhich is the right program (Born Again Shell) -
Create a new directory called missing under /tmp.
cd to go back to the home directory
ls to check that tmp exists
mkdir /tmp/missing to create the folder missing
- Look up the touch program. The man program is your friend.
man touch touch change file timestamps, if the file does not exist it creates it
- Use touch to create a new file called semester in missing.
touch /tmp/missing/semester
At this point, I changed the working directory to be able to use a simpler path with cd /tmp/missing
- Write the following into that file, one line at a time:
#!/bin/sh curl --head --silent https://missing.csail.mit.eduI escaped the spaces to be sure and did not use single-quoted strings to do the test but echo writes everything that follows it in the file so that did not change anything.echo '#!/bin/sh' > semester echo curl\ --head\ --silent https://missing.csail.mit.edu >> semester
However, the backslashes \ used to escape the spaces were not present in the output so they were simply ignored and but the spaces were conserved.
- Try to execute the file, i.e. type the path to the script (./semester) into your shell and press enter. Understand why it doesn’t work by consulting the output of ls (hint: look at the permission bits of the file).
I could not run the files because I did not have the permission to execute it.
-rw-rw-r-- 1 jason jason 60 Aug 24 22:13 semester
There should have been an x for me to be able to execute the file.
- Run the command by explicitly starting the sh interpreter, and giving it the file semester as the first argument, i.e. sh semester. Why does this work, while ./semester didn’t?
sh ./semester works because I am explicitly starting a program (the shell) and giving as an input the content of the file semester.
In this case, sh is only reading the file semester which I have the rights for.
- Look up the chmod program (e.g. use man chmod).
chmod change file mode bits, it allows to change permissions.
- Use chmod to make it possible to run the command ./semester rather than having to type sh semester. How does your shell know that the file is supposed to be interpreted using sh? See this page on the shebang line for more information.
I started with reading the manual with : man chmod but did not of the parameters were supposed to go or meant really.
Using chmod --help
I found that all modes are supposed to take the form of : [ugoa]*([-+=]([rwxXst]*|[ugo]))+|[-+=][0-7]+.
But that was not completely clear either so I went looking for test cases in the online docs on GNU’s website that I accessed in the help of chmod.
Then I understood clearly how to use it.
ugoastands for [user,group,other,all]-+=is there to say what we do to the permissions[rwxXst]is there for read write execute and other stuff that I don’t know about but I have enough to continue.
jason@jason-laptop:/tmp/missing$ chmod -v a+x semester
le mode de 'semester' a été modifié de 0664 (rw-rw-r--) en 0775 (rwxrwxr-x)
jason@jason-laptop:/tmp/missing$ ls -l
total 4
-rwxrwxr-x 1 jason jason 60 Aug 24 22:13 semester
- Use | and > to write the “last modified” date output by semester into a file called last-modified.txt in your home directory.
To do that I started by piping semester with grep that we had seen used during the class, with it, I grabbed the proper attribute.
$ ./semester | grep "last-modified"
last-modified: Fri, 20 Aug 2021 05:12:24 GMT
cut that we had also seen used earlier in the class.
$ ./semester | grep "last-modified" | cut --delimiter=':' -f2
Fri, 20 Aug 2021 05
$ ./semester | grep "last-modified" | cut --delimiter=': ' -f2
cut: le délimiteur doit être un seul caractère
Saisissez « cut --help » pour plus d'informations.
~ as a shortcut.
$ ./semester | grep "last-modified" | cut --delimiter=':' -f2 > ~/last-modified.txt
$ ls
last-modified.txt semester
$ cat last-modified.txt
Fri, 20 Aug 2021 05
- Write a command that reads out your laptop battery’s power level or your desktop machine’s CPU temperature from /sys. Note: if you’re a macOS user, your OS doesn’t have sysfs, so you can skip this exercise.
I started by going to the sys directory and looked for something looking like temperature and found sys/thermal but then I was not sure where to look.
I saw that in there I could control the ventilators and the cooling PID regulation but the temperature itself did not jump to my eye so here I come Google.
A couple second later, here is my answer.
cat /sys/class/thermal/thermal_zone*/temp
64000
20000
60000
63000
64000
I got the temperature in miliCelsius which is fine.
I am not sure which one is the main CPU and I don’t know how many cores I have so I guess I have 5. Actually, it looks like I have 4 cores (which is more logical) and one other thermal zone (for GPU activities ?).
Anyway, the temperature is around 60°C.