2. Project management

Research

For this weeks assigment we needed to upload our personal information on github and to create a own website on it. Being a complete novice on the subject i needed to learn the basics of all the programs necessary to be able to create my own site. First i like to explain about the different programs and methods i needed to be able to make the site and a small overview of what they do. My OS system is currently windows 10 and for this weeks assigment i installed a Ubuntu terminal on my laptop to be able to use the commands. I would consider myself as a novice when it comes to computer set ups. Along the way i learned to work with different programs but when it comes to settings and preferences of the operations system there is a lot of improvement possible. Here a small summary of the different programs and languages used.

Git: Git is a distributed version control system which enables online collaboration between designers, programmers and developers. Basicly it enable to work and share information between a group of people. You have a online repositary and a local repository. You can have acces to the remote repository by dragging the information to your local one. You can adjust information and share it to the online repository based in our case on gitlab.

git environment where i will be posting my work

ssh To be able to link your local account to gitlab you need to have ssh. This provides a secure connection between your computer and remote repository. To connect you have to fill in a private or public key. This program enables you to have a secure and encrypted connection.

texteditor: In the text editor you can work ofline on the project. here you can adjust files, upload files, or write text. By using Markdown symbols you can prepare the files in such a way that when you share the information on the online repository it gives a website feeling. Further the text editor can be used to share links, upload pictures, highlight sentences, etc. For now i choose to work with sublime text editor which have some good functions for gitlab use.

Markdown: Markdown convert text by using symbols and translate the text in a ready html file. Using markdown symbols is a simplified way for writing in html. Git will convert text written with markdown symbols and safed as .MD automaticly as a html file for online viewing.

css: Where markdown focus on text, css is foccused on the look of the site. It decided colors, location of the file and it is used to alter the looks of a document online. It enables you to create a stunning look of your online repository. I will not be using this for the time being since i would like to put my focus on mastering the other skills.

terminal: The terminal shows a simple display where you can acces your computer by writing the commands. It gives a good overview of the files on the computer and allows you to alter each individual file. The terminal we use to install some requererd programs but also combine your local repository with the online repository. In order to create a safe connection with gitlab we need to connect git with your computer with a ssh key, which recognize the remote user and send information encrypted to gitlab.

imagemagick: This is a tool to downsize your pictures before uploading it to the git server. Uncompressed pictures requere lots of date. To ensure a smooth running interface it is important to downsize the files. Further it has options to resize the picture in the desired size. To change pictures by imagemagick u have to use the terminal and write the changes you would like to commit.

preparations

One of the first thing requered was installing a ssh key on your computer. I followed simple instructions to open my terminal to create and copy the ssh key to my gitlab account. I was able to create a ssh key on windows 10 command prompt but not able to copy. It did not recognize the commands. My instructor recommanded installing a ubuntu terminal on my computer to work better with the different commands.

To install the ubuntu server it requere me to take a few steps to do so. First i had to open my taskbar and open windows powershell as administrator. That brought me to a terminal where i had to copy and paste

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

After pressing enter i had to comfirm my command and restart my computer. The next step was installing the ubuntu server on my laptop. This could be done by downloading the ubuntu app in windows store. Once installed you have acces to the ubuntu server. Next step is to set up a username and password and then you are ready to install the git username and the ssh key.

command for git username

git config –-global user.name “YOUR_USERNAME”

command for using your email for uploading git files

git config -–global user.email “myemailadress@gitlab.org”

In Command promp i had tried before to install the ssh key and it was present in the terminal using the command:

sh-keygen -t rsa -C "$your_email"

Now all i had to do was copy the ssh file in my gitlab file on https://gitlab.fabcloud.org/profile/keys

Being in a new environment not understanding how it had to be done i received some help getting the ssh key copied to my gitlab account. With the following command i could see my ssh key and copied this to my account.

cat ~/.ssh/id_rsa.pub

Next step is cloning your repository from gitlab in to your local repository. A file under the name roomkes/fabacademy2019 was used to send the repository to.

git clone http://git@gitlab.fabcloud.org:academany/fabacademy/2019/labs/waag/students/rutger-oomkes.git fabacademy2019

Chosing a Editor

Now the local and online repository are connected you can work local in documenting your projects. A text editor enables you to write on your local repository. Since my lack of knowledge regarding text editors my instructor adviced using sublime text 3. A text editor which have a more friendly interface and can recognize Markdown documents. I might change the editor later in the fabacademy when i get more acquainted to editing. For now this program seems good enough to start documenting.

workflow preview on sublime text editor

Terminal

Lots of the commands i have to perform to use git and to control my images i make use of the the terminal. Also is the terminal a good place for controlling settings and documents. Here is a list of common used terminal commands.

Terminal Commands
ls -la	List of files in directory with detailed information
ls -a	List of files including hidden files
pwd	Shows current location on the drive
cd	This command direct you to a directory. Just using cd returns to home directory. With variation you can move up and down your directories
rm (file)	Removes file
mv (file)	Move a file
mkdir	Make a directory
rmdir	remove a directory
sudo	Making changes with admistrator rights
help	Shows all possible commands in the directory. This you can specify when you want to know about one specific option
touch (name)	Make a new directory

playing in terminal using different commands

Git

When you change anything in the editor, these changes will be visibible in the terminal. When you want to share changes with the online repository you need to use the terminal by using different git commands. to get used to GIT envirement i watched several tutorials about GIT

basic keycomands https://www.youtube.com/watch?v=HVsySz-h9r4 overview invirement < https://www.youtube.com/watch?v=LgY8XuoM8aU&t=428s> common mistakes https://www.youtube.com/watch?v=FdZecVxzJbk&t=105s

Here is a list of most commonly used commands for git
git status	Show the status of the different files. tracked/untracked
git -dif	shows changes made inside the file
git add (file)	Ads a file to the commit stage
git add .	Add all files to commit stage
git commit -m “message”	Commiting all the changes to staging area. Add messages with it to be able to check your updates
Git push	pushing your local repository with the remote repository. Always made sure there has not been changes in the remote. This result in a conflict
gitt pull	Update local repository with remote repository
gitt log	Shows changes made with the file
git –help	Shows different commands possible in git environment
git add –help	Shows the different commands in this particular command option
pwd	Shows origin directory
git init[project name]	Create new repository
git rm (file)	remove file from directory
git reset (file)	Retrack a file from the staging area

When you make changes in your local repository you want to share them with the remote repository. This is the moment you starting using these commands in the teminal. When i add for example a new chapter to my fab academy course with the use of a texteditor (sublime) i like to share this to gitlab. First thing after saving the file i go to my local repository in the terminal and use the git status command. Now you see all the tracked and untracked files. Now you can make a selecting in which files you like to update. Using the command gitt add . you ad all the tracked and untracked files to the commiting stage. If you now press git status all files should be green, meaning the files are now tracked. With the use of git commit -m “message” you put all the files in the staging area. The message is used to clarify your commits. When realising a file shouldnot be in the staging area you can use the command git reset (file) to remove that partulair file from the staging area. Last step is to push your files to the remote repository. this you do with the command git push. When you are working alone on a project these procedure is good. When working with others you first should apply the git pull command. This way you sync the local with the remote repository. After that you can push the changes.

Common procedure when pushing local repository to remote

Mkdocs

Mkdocs is a static site generator well suited for documentation. On the fabacademy this site generator is already installed. When i make alterations in my sublime text editor and upload it to the git repository it will shown in the mkdocs static website. I can therefore choose to create a extra page by simple adding a extra folder in my docs. I added a extra folder called contact, which should be visible on my website. http://fabacademy.org/2019/labs/waag/students/rutger-oomkes/ In the text editor you can adjust the looks of the mkdocs interface. In the directory mkdocs.yml. To use certain themes you can upload mkdocs themes in the terminal. You need also to install python pip to use the mkdocs themes. https://www.mkdocs.org/

For now i am happy with mkdocs to show my work. It is a simple template which automatic change when i add new folders in my repository. At one point i might change to a more appealing format.

image magick

For good documentation using pictures is essential. With terminal commands you can adjust, identify, quality for picture and resize pictures to your repository.

Here is a list of most commonly used commands for imagemagick
adjust extension	convert rutger.jpg rutgernew.png
resize	convert rutgernew.png -resize 50% rutgernew1.png
quality	convert rutgernew1.png -quality 85% rutgernew2.png
identify	convert rutgernew1.png -quality 85% rutgernew2.png
convert, resize, adjus	convert -quality 85% -resize 60% rutger.jpg rutgertest.png
convert all	magick mogrify -format png *.jpg

As a test i used the command on http://academy.cba.mit.edu/classes/computer_design/image.html my own excisting files in images. Made a mess of my image folder. Most of the commands are for making the pictures more suitable for website content. All the pictures are uploaded on my gitlab repository.

A few weeks later i came to the realization that my png pictures were to big. I was number two in filesize on gitlab. Second place sounds appealing but in this case it is not. Your suposed to upload small files on gitlab to keep the website running smoothly. After some trouble seeking i found a way to convert all my images without to much qualityloss with this formula.

magick mogrify -strip -interlace Plane -resize 665x332 -quality 85% *.png

This convert all images at the same time. It will replace the original files When you just want to update one picture you should type.

convert -strip -interlace Plane -resize 665x332-quality 85% source.jpg result.jp

source: [Stack overflow]https://stackoverflow.com/questions/7261855/recommendation-for-compressing-jpg-files-with-imagemagick After bit more research i found that -strip is better not to be used. It removes the copyright from the image. Here is the new formula:

mogrify -interlace Plane -resize 665x332 -quality 85% *.png

Having changed my terminal from ubuntu to windows bash the command i need to use changed. instead of convert i need to use magick magick mogrify -format jpg *.png

Markdown

Still have to learn all the possible markdown symbols and shortcut. So far i put my focus on the most common use of Markdown. In case of documenting your work i like to highlight a few used commands.

• Highlight code: you can place 3 ` in front and after sentences, this result in a grey indicated block. (often used in code)
• Size title: You can use # to decide which size title you like to see, One # is the biggest size, ## sligly smaller, 6# smallest title
• Create a Link: When you have a website you like to link to you can put the url between <url>, this result in a clickable link
• A link button using a specific word/sentence to a website can be created by `[specific word](url from specific website)
• bold or italic text: When you like to emphasize something, you can add *emphasize* around the text for italic and to **emphasize** in bold
• Adding pictures: ![text for image](directory file where image is placed )

Adding pictures from computer to local repository

One struggle i encountered was adding new pictures to my repository. First i tried dragging files in my text editor file. It turned out that when i do it like this i have no permission to gitt add (file) to commit fase. I had to give permission manually. A way that seems to work for now is to upload the picture in my gitlab environment and by using gitt pull dragging it in my local repository. Works for now but keen on finding a more efficient way for doing this.

Update on using my terminal

I kept having issues with ubuntu terminal installed on windows 10. Often when i work on the terminal i kept recieving permission denied messages. This was frustrating since i like to continue on my new assigment and it did not alowed me to push my work in to gitlab. I really wanted to fix these recuring issues and start searching for a perminant solution. In my limited knowlegde i asumed the problem was with a ubuntu terminal in a windows workplace. I searched if it was possible to use a windows terminal to push my git project further. I found a good tutorial on youtube how to install git bash and set up the new settings. https://www.youtube.com/watch?v=SzOa1TFltaY&index=4&list=PLdYQMTciVWO-J8Qj3G6YVwUmQ8vkkgTZd. Following the different tutorials i was able to create a new terminal setting. To get acces to my sublime documents i had to go in many subdirectories. I was affraid to have to do these steps everytime i open the new git bash terminal and decided to replace the whole directory in a more accesable folder. When i go into the terminal it indicate Master and using git status i could see my repository. I did all the steps before when installing ubuntu to set up my account.

git config --global user.name "Roomkes"

 git config --global user.email "email@gmail.com"

Hoping to be able to push now i tried the git add . command.

$ git add .
fatal: Unable to create 'C:/Users/Ruter Oomkes/Desktop/fabacademy2019/rutger-oomkes/.git/index.lock': File exists.

Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.

After search the error i found online the solution.

rm -f .git/index.lock

This solved the issue and i could commit changes. I recieved the folowing error trying to push.

$ git push
The authenticity of host 'gitlab.fabcloud.org (13.59.248.79)' can't be established.
ECDSA key fingerprint is SHA256:Rphnjz211wPLjTJcyAip0xozt0hOoFgT9hl5eZfxzd0.
Are you sure you want to continue connecting (yes/no)? y
Please type 'yes' or 'no': yes
Warning: Permanently added 'gitlab.fabcloud.org,13.59.248.79' (ECDSA) to the list of known hosts.
git@gitlab.fabcloud.org: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

In response to this i recreated a new ssh key. After this i used the clip command to copy the ssh key.

$ clip < ~/.ssh/id_rsa.pub

This i pasted again on my ssh key settings in gitlab acount. Now i was able to push my local repository in to my git lab account. I hope i now achieved a more stable git environment. I do receive a message everytime i push.

$ git add docs/assignments/week03.md
warning: LF will be replaced by CRLF in docs/assignments/week03.md.
The file will have its original line endings in your working directory

I looked on line what are the consequences of the comment and it seemed not to be important. https://stackoverflow.com/questions/5834014/lf-will-be-replaced-by-crlf-in-git-what-is-that-and-is-it-importantThere is a way to hide these comments but i first like to verify that it is indeed not relevant for me. This was the possible command to hide this comment.

git config --global core.autocrlf false

I really hope these changes will make the work flow better and that i don’t have to spend lots of time on problem solving. Lets see how it will go. Otherwise i keep updating the process here.

Trying to clean my repository

In the first weeks of the fab academy i have been uploading large png files to my git repository. In result of this i ended up with a way to big repository. After some troubleshooting i decided to clean my git account with the use of bfg repo cleaner. https://rtyley.github.io/bfg-repo-cleaner/ This program is used to clean unwated or big files from the repository. It gives a clear overview in how to archieve this. As mentioned earlier i am by far a expert in using git or working in the terminal, so this simple tutorial turned out quite a challenge. I mostly was worried to erase my complete repository or delete all my commits in the past.

First step was to clone my repository which i did with the following command.

$ git clone --mirror git://example.com/some-big-repo.git

For the git i used my link in gitlab. After this i had to install java runtime environment to be able to use the bfg repo cleaner. When i tried to use the following command it did not recognized my command.

$ java -jar bfg.jar --strip-blobs-bigger-than 100M some-big-repo.git

Trying to check where the mistake was i tried different directories to work with.

$ cd some-big-repo.git

I went to rutger-oomkes.git directory thinking this was the right step. Using command:

$ git reflog expire --expire=now --all && git gc --prune=now --aggressive

After which i used git push to my remote repository.

Curious to see what effect this had on the size of the repository i checked on gitlab my account. It did change indeed. Only the opposite way. My new size was almost twice as big as the previous version. I my lack of skills in git and terminal i posted my mirror clone and my local repository to gitlab. Now having my history posted twice on gitlab…

I the flow of trying to clean the repository my size doubled again to a a shoking 850 mb. In despiration of how to fix tis i recieved some help of my instructor. After the review of Neil Gershenfeld where he explain how to get rid of the history in the repository we managed to get rid of the history. https://vimeo.com/320588564

In the terminal you can get rid of the history. I explain the command used for doing this. When you are in your masterbranch in the terminal you can type

ls -a .git

This shows you the hidden folders including the history. Even when removing files in your repository the will remain in the history. Resulting in enlarging the repository. To remove history you clone your remote server repository. For this you direct to the remote server.

cd site (repository)
git clone --bare sit server

Neil explaining how to erase history

When go to the log of the server you see all the files stored in the history. To remove the history you remove the whole git directory.

rm -rf .git

Now you create a new git repository.

git init

Now when you check:

git log

You see a empty directory. On the server it still has all the history. Now with command. Change the server master to your specific server

git push --set-upstream ../server master

When this command is exuceted you recieve a warningand the command is not done. By using –force you overwrite the warning and are able to push.

git push --force --set-upstream ../server master

This way you able to push and get rid of the hidden directory. Not your files. Now the repository is cleaned from the history and only contains your last push. With this procedure you are able to downsize your repository from old files.

What i learned (and failed) on this weeks assignment

This weeks assignment was very learning full in teaching me the essentials for good documentations and use of all the different programs to share your local repository with the gitlab repository. I have now a understanding of the purpose of the different facets in documenting. This being said i realize i still have big steps to make to master all the different skills. Most hard in this week assignment i found using git in the terminal. Both were alien to me before i started out and i spend allot of time figuring out problems that occurred. From files that were not recognized to permission denied. Working on windows 10 with a ubuntu server also resulted in not being able to use imported pictures in the local repository. One of the reasons for this that i haven’t optimized the settings between ubuntu and windows 10. I will continue to work on this the coming weeks to create a site more appealing to the eye.

This weeks assigment keeps me occupied during the course since this is a interface i have to keep working with the whole fab academy. I changed my terminal in the mean time to git bash which made my work much more efficient. I did mess around in the beginning making a huge mesh of my repository resulting in double files, weird directories, huge sized git repository. Now my repository size is modest again and i will try my best of keeping it this way.Till the next problem accurs.. Well.. Thats life.

The latest update