Fab Academy 2018

Wk2. Project Management


Work through a git tutorial
Build a personal site in the class archive describing you and your final project

“Getting things done isn’t just about checking off items on a list. Getting things done is a process: it’s a way of thinking that involves planning, execution, iteration, and reflection.” Said the muddleheaded wombat.

Project Management

During the Fab Academy time is going to be one of those scarce assets. In our lecture Neil shared a few very useful tips on project management.

This is organising ones time in accordance to a finite and calculable amount of time as supposed to task completion in order of priority. Don’t work until you finish each task, work back from the available time and figure out how much time you can spend on each task. In other words: Figure out how much time I have. Figure out how much time I can spend on each task. Learn to work to a time schedule and complete what I can according to the allotted time.

In standard development practice, one starts at the beginning and go to the end. This carries the risk of a step getting too complex or hitting a problem that interrupts the flow of development, and possibly stops the project.

With spiral development, you start small and gradually add complexity to a project in a spiral fashion. Every spiral you discover something, but you keep finishing project. In salesman-speak its what they call ABC or Always Be Closing. This means it is more valuable to finish a small phase of development than try to complete the whole massive project in one swoop. Like the tortoise, small manageable steps and adding incremental additions will win the race.

FabAcademy Tools

Documenting as I work is key to the keeping up to speed with the Fab Academy. Documenting is also time consuming and hard work, so it is very good practice to be constantly documenting as you work along. DON’T LEAVE IT TO THE LAST MINUTE!! We are using GitLab to host our static websites. For this there are a number of other tools that will come in handy.

Web development

After exploring the pros and cons of various content management systems, I decided to stick go with Jekyll to design this site. Jekyll is a static site generator which converts Markdown into HTML. This is suuuper useful as Markdown is easy to write in, you don’t have to worry about all those damn anchor tags: <a href="blahblah">annoying tags</a>. Basically, it makes documenting much, much easier. Here is a simple Markdown cheatsheet.

What are the benefits of using static sites?

I am using Atom which is a flexible and hackable text editor. As you can see below, it allows be to work on my project while accessing a terminal window below. It’s very customisable with hundreds of package add-ons and user friendly.

Atom

Basic Jekyll

As you can see, my site is super basic for the time being as I’m building a barebones site without using a custom Jekyll theme. It is pretty simple to create a basic Jekyll site using no templates. This is what I have done, Tania Rascia has a great step to step tutorial.

Git On Uppa

This week we learned all about version control systems (VCS) for tracking changes in computer files and coordinating a project among multiple people. Why do we use version control? Because whether in building a website or working in teams to create complex software, this allows for:

Git is a powerful VCS because of the way it thinks about data: like a series of snapshots of a miniature filesystem. This filesystem is stored locally in a designated folder, which helps for working offline. Also everything in Git is check-summed before it is stored, which means every change or modification to a file is registered. So it’s virtually impossible to lose information in transit or get file corruption without Git being able to detect it.

Git basicaly works something like this:

Git

Git is thought of for working using terminal commands, but there are also various GUI systems you can use. The basic Git workflow goes something like this:

GitLab

GitLab is an Git repository manager with a wiki, issue tracking, continuous integration. This is what we will be using to host our website, files and documentation.

As students we each have a GitLab account, in which we create a repository and configure Git to work with a local repository on our computer. This should all be visible on my website

SSH (secure shell protocol)

Using an SSH key enables us FabAcademy students to communicate and use our GitLab repositories in a secure and encrypted way. We do this by generating a key.

An SSH key consists of a pair of files. One is the private key, which should never be shared with anyone. The other is the public key. The other file is a public key which allows you to configure my GitLab repo so that it knows its me pushing and pulling files.

To generate SSH keys in macOS, I entered the following command in the Terminal window:

ssh-keygen -t rsa

This gives me somehthing like this:

Your identification has been saved in /Users/myname/.ssh/id_rsa.
Your public key has been saved in /Users/myname/.ssh/id_rsa.pub.
The key fingerprint is:
ae:89:72:0b:85:da:5a:f4:7c:1f:c2:43:fd:c6:44:38 myname@mymac.local
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|         .       |
|        E .      |
|   .   . o       |
|  o . . S .      |
| + + o . +       |
|. + o = o +      |
| o...o * o       |
|.  oo.o .        |
+-----------------+

It then prompts for a password that is optional. Basically what this tells me is my private key is saved to the id_rsa file in the .ssh directory in my home directory. This is a hidden folder. Don’t share it with anyone.

Then copy the public part of the SSH key with:

pbcopy < ~/.ssh/id_rsa.pub

Copy it to GitLab. This now means that git can call my remote repository in GitLab and work it with my local repository on my computer.

Uploading on GitLab using the Terminal is pretty straight forward. Here is how I did it on the using the following commands:

cd ~/Desktop/fabacademy

then:

git status

then:

git add .

then:

git commit -m "add a message here"

then:

git push

And here’s my terminal working the Git:

Bash

Git here requests my SSH secure key before pushing.

Bash

Since we have to make sure our repository remain under 100MB, I make sure to check the size of my .git folder using:

$ du -h .git

Running my Jekyll site on GitLab

Gitlab offers a continuous integration (CI) which is a common practice in software development, basically allows for use continuous delivery and continuous deployment of your site with every git commit.

This basically works using something called Docker. GitLab CI allows to start a virtual server, install software on it, execute it for performing tasks, then it stores results and publishes them on the web.

This may sound very complicated, but to use CI we need to add a .gitlab-ci.yml file to our site root directory which basically holds the instructions for this process. This means when you git push to your repository, GitLab will look for the .gitlab-ci.yml file and start jobs on Runners according to the contents of the file, for that commit. It seems more complicated that it really is. Basically, I followed this tutorial. I basically used the same .gitlab-ci.yml file as found in the Jekyll pages repository on gitlab.

Image Compression with ImageMagick

These are some super fast command-line tools for batch edit images and videos. This helps especially to convert file formats and compress images to make the easy to post on our site without having to get into the hassle of individually opening each file in Photoshop, VLC or whichever editing program.

ImageMagick is a command line tool used to create, edit, compose, or convert bitmap images from the terminal. First you have to install ImageMagick, here I am using Homebrew:

$ brew install imagemagick

Be aware that ImageMagick requires a bunch of dependencies, so if you are wanting to convert crazy files into more crazy files, study well what libraries you need to do so. Being open source it also has a good resources and community of users.

Resizing

$ mogrify -resize 1280x -path ~/Desktop/test ~/Desktop/inkscape.png

This defines the x-axis width to resize to 1280 pixels (which will adjust the height proportionally) and then where to find the image.

Converting

$ convert *.JPG -resize 1024 new_name.png

This converts all the .JPG files. The * says ignore the name and just choose all the .JPG files, resize them to 1024 pixels width and title them new_name.png. If there are more they will be indexed.

Image quality

The -quality value for .png works the other way round compared to .jpg See here.

.png => -quality 0 (min compression, max quality) , -quality 100 (max compression, min quality)

.jpg => -quality 0 (max compression, min quality) , -quality 100 (min compression, max quality)

So now:

convert *.JPG -resize 1024 -quality 80 1.jpg

Video compression with FFmpeg

FFmpeg is an amazing multi-media conversion tool that allows you to decode, encode, transcode, mux, demux, stream, filter and play pretty much anything that humans and machines have created. Check their documentation here. Install via Homebrew (for MacOs users) in the Terminal.

$ brew options ffmpeg
$ brew install ffmpeg --with-libvpx

The libvpx library is used to convert Mp4 to WebM which is another more powerful encoder that allows for much better video compression with less loss of quality. However, it only seems to be supported by Mozilla and Chrome browsers.

Ready to rock n roll? Lets do a basic compression:

$ ffmpeg -i example.mov example.mp4 -hide_banner

Now there are all sorts of parameters we can set in the conversion. Check this out:

$ ffmpeg -i example.mov -f mp4 -vf scale=640:360 -vcodec libx264
  -crf 22 -preset fast -profile:v baseline -level 3.0 -acodec aac
  example.mp4 -hide_banner

Here are some resources on how to encode MP4 videos using the standard H.264 encoder with FFmpeg. Also huge shout out to the BugCodeMaster for sharing some quality FFmpeg documentation. If all these different video formats give you brain flatulence, let rip with ease here. This is also another good guide on encoding video for web. Although Neil uses the avconv video converter (which branched off from FFmpeg) the commands a basically the same.

Also see here for converting a movie into a gif, using both ffmpeg and ImageMagik in one command:

$ ffmpeg -i example.mov -vf scale=800:-1 -r 10 -f image2pipe -vcodec ppm - | convert -delay 5 -loop 0 - gif:- | convert - output.gif

Here is a fun gif I made this way:

My Final Project

Please browse this website to read more about me and refer to my final project page page that describes at length my project idea.

But here are some initial sketches:

concept