Project Management¶
- Build a personal site describing you and your final project
- Work through a git tutorial
- Upload it to the class archive
Setup¶
I’m already familiar with website development since I was working as a Data Scientist before entering the Fab Academy. I know HTML/CSS basics (though it’s been a while), markdown syntax and I daily work with git and GitLab.
This week I plan to setup my documentation to ease my workflow for the next six months to come.
GitLab and SSH keys¶
When I received my credentials to access all Fab Academy tools, a website was already configured. To work locally, I needed to clone the repository by creating a SSH key and adding it to my new account. GitLab offers a detailed example of this process here.
# Create a new SSH key
ssh-keygen -t ed25519 -C "fabacademy"
# Copy to clipbpard
xclip -sel clip < ~/.ssh/id_ed25519_fab.pub
As I already have a personal GitLab account, I configured my ~/.ssh/config file to include my new repository (which you can find here).
# Personal Account Identity
Host gitlab.com
Hostname gitlab.com
PreferredAuthentications publickey
IdentityFile ~/.ssh/id_ed25519
# Fab Academy Account Identity
Host gitlab.fabcloud.org
Hostname gitlab.fabcloud.org
PreferredAuthentications publickey
IdentityFile ~/.ssh/id_ed25519_fab
I then verified that I was connected to the right server with ssh -T git@gitlab.fabcloud.org and accepted to add it as a known host.
After everything was set, I cloned the repository on my laptop.
git clone git@gitlab.fabcloud.org:academany/fabacademy/2022/labs/sorbonne/students/chloe-laurent.git
Mkdocs and virtual environments¶
I decided to use the provided MkDocs as a static site generator: I like to keep my documentation simple, and I know I can customize it with custom CSS and JS. Moreover, there are a lot of new python plugins that I want to explore.
After opening my newly cloned folder, I created a python virtual environment, a good practice that proved itself very valuable over the time.
# Create a virtual environment
python3 -m venv .venv-fab
# Activate
source .venv-fab/bin/activate
# Verify
which python
>>> $REPOSITORY/.venv-fab/bin/python
Warning
Don’t forget to add your newly created virtual environment into your .gitignore file, otherwise you’ll end up with a git instance way too heavy.
Once you have activated your virtual environment, you can install mkdocs and as many plugins as you want without interfering with your main python instance (and believe me, you don’t want to mess up your main python, especially on a linux OS!).
You can now build you website locally and start completing your documentation! You will automatically see your changes on your browser (127.0.0.1:8000).
mkdocs serve
>>> INFO - Building documentation...
>>> INFO - Cleaning site directory
>>> INFO - Documentation built in 0.19 seconds
>>> INFO - [19:28:36] Serving on http://127.0.0.1:8000/2022/labs/sorbonne/students/chloe-laurent/
Writing the documentation¶
I will try to keep the mantra document as you go alive during the all Fab Academy. This week it’s quite easy because I’m only working on the computer, but I know that it will become harder when using the machines in the lab. I have with me all the time a little notebook where I note everything. I hope this will help me!
Markdown¶
I will use mainly markdown, so I made myself a cheatsheet where I’ll keep some useful features.
Visual Studio Code¶
I used Visual Studio Code as an IDE for a while now and I’m very satisfied with its UI and practicality. It’s open source and available on all platforms if you want to give it a try. I love that there are a lot of extensions in there and that you can write your own if you want.
Here’s a peek of how my screen looks like when I’m working: 
Custom theme¶
I really wanted to pimp my website, even if I used a static site generator. MkDocs is very reliable but not that fancy, so I twisted it a little to obtain a result that I like. I use as a base template Material for MkDocs.
You can change a lot with only the mkdocs.yml file, such as the title, the color palette, the fonts…
Extensions¶
The Material theme was already configured on my website when I cloned it. I completed the basic informations on mkdocs.yml and added some plugins and markdown extensions that I found on the Material documentation.
These are extensions that I didn’t know and found very useful:
-
pymdownx.keysAdd keycaps on your doc Ctrl+Alt+T
-
pymdownx.tasklist- Add task lists to your assignments
-
attr_listAdd HTML/CSS tags directly to markdown syntax:
{: .width=50%} -
pymdownx.highlightandpymdownx.superfencesAdd highlights on code blocks
Custom stylesheets (CSS and JS)¶
You can also write HTML directly in your markdown files, but not only. It’s possible to write custom CSS and JS to customize your website even more.
To help me build a beautiful website, I used the Fab Academy 2022 color palette to overwrite default colors of mkdocs.
I also added a radius to the border of images, and a gallery on the home page.
img {
border-radius: 6px;
}
.gallery {
display: flex;
flex-direction: row;
}
.gallery-link {
width: 50%;
margin: 3px 4px;
}
.gallery-link a {
width: 100%;
height: 100%;
display: block;
text-align: center;
color: #f04260;
}
.gallery-link div {
display: flex;
flex-direction: column;
justify-content: center;
align-items: center;
text-align: center;
}
I found a useful zoom feature hack, it allows to scale and outscale an image.
/* CSS */
.zoom {
transition: transform ease-in-out 0.5s;
cursor: zoom-in;
}
.image-zoom-large {
transform: scale(1.5);
cursor: zoom-out;
position: relative;
}
// JS
document.querySelectorAll('.zoom').forEach(item => {
item.addEventListener('click', function () {
this.classList.toggle('image-zoom-large');
})
});
For the final touch, I added the Fab Academy 2022 assets for the favicon and the home button. Ready for six months of documentation!
You can now see the differences between the original website and my version:
Storage optimization¶
Note
Before pushing my work to the cloud, I would like to write some scripts to optimize the size of my repository. Ideally, I want to automate the process and execute it just before the deployement.
I can write a python script, but maybe there is a more efficient way ? Here are some articles I found during my research:
What is the size of my GitLab repository?¶
I started by checking my repository size with the use of git-sizer --verbose and obtained the following output:
Processing blobs: 103
Processing trees: 98
Processing commits: 39
Matching commits to trees: 39
Processing annotated tags: 0
Processing references: 3
| Name | Value | Level of concern |
| ---------------------------- | --------- | ------------------------------ |
| Overall repository size | | |
| * Commits | | |
| * Count | 39 | |
| * Total size | 9.08 KiB | |
| * Trees | | |
| * Count | 98 | |
| * Total size | 16.8 KiB | |
| * Total tree entries | 456 | |
| * Blobs | | |
| * Count | 103 | |
| * Total size | 587 KiB | |
| * Annotated tags | | |
| * Count | 0 | |
| * References | | |
| * Count | 3 | |
Nothing to worry about for now, but I can easily imagine how this could look like after a few weeks of documenting Fab Academy: large images and blob files all over the place. I will keep this command in my CLI cheatsheet.
Dealing with large files¶
I would like to use Git LFS to track my most large files (3D files for example). However, this seems to be not applicable for GitLab pages, so I won’t be using this for images and videos.
Warning
This must be done before committing the files to the repository.
Luckily, I have not commited anything large for now (mainly .md files). I can configure my storage rules from scratch without having to migrate old files.
-
Install git-lfs
- On your local machine:
sudo apt-get install git-lfs - On your repository:
git lfs install --local
- On your local machine:
-
Track your large files
git lfs track "$FILENAME"git add .gitattributes
-
Check which files are tracked
git lfs ls-files
After these steps, I have nothing more to worry about, all my files in .gitattributes will be tracked with Git LFS.
Success
I tested it during the Computer Aided Design week and it worked perfectly. My files are stored with Git-LFS and are accessibles in the website.
Pruning a git tree¶
Warning
This can damage the repository, so I’ll just put a link of the procedure here as a reminder, I won’t do it as it is quite unnecessary for the moment.
Image and video compression¶
Images¶
When I look at this week images folder (with ncdu), I can see that they take a lot of place, especially the photo I took of my notebook:
1,7 MiB [ 65,4%] notebook.jpg
440,0 KiB [ 16,7%] screen_peek.png
192,0 KiB [ 7,3%] website-before.png
160,0 KiB [ 6,1%] website-after.png
64,0 KiB [ 2,4%] fab-color-palette.png
52,0 KiB [ 2,0%] notebook-size.png
This is completely useless, because if I take a closer look to my generated website, I can see that my images are reduced to a sixth of their sizes (in the case of JPG) !
By using python and OpenCV, I wrote a script to resize automatically images that are way too big for this usage.
def resize_image(img_path: str, width: int) -> object:
'''
Resize images bigger than 800px width
'''
image = cv2.imread(img_path, cv2.IMREAD_UNCHANGED)
(img_height, img_width) = image.shape[:2]
if img_width > width:
ratio = width / float(img_width)
dim = (width, int(img_height * ratio))
resized = cv2.resize(image, dim, interpolation=cv2.INTER_AREA)
return resized
return image
"""
Python script to optimize images and video sizes for web
"""
from typing import List
import argparse
import os
import cv2
def parse_arguments():
"""
Argument parser
"""
parser = argparse.ArgumentParser(
description="Usage: python compression.py --path $FOLDER_PATH"
)
parser.add_argument("--path", type=str, help="Path to the folder.")
parser.add_argument("--width", type=int,
default=800, help="Maximum px width")
parser.add_argument("--quality", type=int,
default=70, help="JPEG quality")
return parser.parse_known_args()
def list_images(path) -> List[str]:
"""
Returns a list of JPG and PNG images
in the folder directory and subdirectories
"""
img_list = list()
for root, _, files in os.walk(path):
for file in files:
if ".svg" in file or ".mp4" in file:
print(f"File is not JPG or PNG: {file}")
elif "_web.jpg" in file:
print(f"File already optimized for web: {file}")
else:
img_list.append(os.path.join(root, file))
return img_list
def resize_image(img_path: str, width: int) -> object:
'''
Resize images bigger than 800px width
'''
image = cv2.imread(img_path, cv2.IMREAD_UNCHANGED)
(img_height, img_width) = image.shape[:2]
if img_width > width:
ratio = width / float(img_width)
dim = (width, int(img_height * ratio))
resized = cv2.resize(image, dim, interpolation=cv2.INTER_AREA)
print(f"File resized for web: {img_path}")
return resized
print(f"File not resized: {img_path}")
return image
def compress_images():
"""
Main function
Resizes images and rewrites the folder
"""
args, _ = parse_arguments()
img_list = list_images(args.path)
for img in img_list:
web_img = resize_image(img, args.width)
path_in, _ = os.path.splitext(img)
path_out = f"{path_in}_web.jpg"
cv2.imwrite(
path_out, web_img, [cv2.IMWRITE_JPEG_QUALITY, args.quality]
)
os.remove(img)
if __name__ == "__main__":
compress_images()
I chose a default width size of 800px (thanks to that article), but it can be changed to match a specific case.
# Blog images
python compression.py --path ./docs/images/week02/
# Home gallery
python compression.py --path ./docs/images/gallery/ --width=400
We can see the size was reduced to an acceptable web size, without loosing ratio or quality.
52,0 KiB [ 20,6%] website-before_web.jpg
52,0 KiB [ 20,6%] screen_peek_web.jpg
40,0 KiB [ 15,9%] fab-color-palette_web.jpg
40,0 KiB [ 15,9%] website-after_web.jpg
40,0 KiB [ 15,9%] notebook_web.jpg
24,0 KiB [ 9,5%] notebook-size_web.jpg
Videos¶
I will use ffmpeg to compress my videos later on.
Success
During the Electonic production week, I made some explanatory videos and needed to compress them (they were unnecessary long and loudy).
I used this command to compress them without the sound:
And this one to cut the ten first seconds of it:
To insert a video into my documentation:
Thumbnails¶
For the thumbnails on the home page, I need to keep the same ratio. I can use gThumb to resize my gallery as I want. The ideal ratio is 388*218.
CI/CD¶
I am now ready to push my work to the class archive with a simple git push after committed all my modifications.
Locking support detected on remote "origin". Consider enabling it with:
$ git config lfs.https://gitlab.fabcloud.org/academany/fabacademy/2022/labs/sorbonne/students/chloe-laurent.git/info/lfs.locksverify true
Enumerating objects: 105, done.
Counting objects: 100% (105/105), done.
Delta compression using up to 8 threads
Compressing objects: 100% (79/79), done.
Writing objects: 100% (91/91), 255.36 KiB | 18.24 MiB/s, done.
Total 91 (delta 26), reused 1 (delta 0)
To gitlab.fabcloud.org:academany/fabacademy/2022/labs/sorbonne/students/chloe-laurent.git
5b3835a..49777d1 master -> master
The pipeline deployed the website without any issue.
Note
That’s it for this week, I’m glad I did my documentation along with my modifications, it allowed me to keep track of everything.