project management

The first assignment was setting up a personal site for documenting this and all future assignments, as well as sketching out the tentative final project - literally and figuratively. This post will be split in two parts accordingly.

website setup

A part of this assignment was also to get familiar with git. Since I have worked extensively with git in the past, this was no problem. However I had not used GitLab before, so I spent some time getting familiar with the hierarchical organisation principles and the CI/CD subsystems GitLab provides.

cms choice

For managing the site, I chose to deploy a version of a content management system I built as part of my bachelor thesis - mmmfs. mmmfs is designed as a highly customizable multimedia wiki-like system for personal documentation and computing. In its full implementation, it can embed any kind of content (images, videos, graphs, interactive widgets) from anywhere inside the system as well as from external sources in very flexible ways. However to keep the deployment and repo size small and comply with the statically-served content policy, I chose to deploy it differently from the main installation on my personal website. Rather than converting and serving content on demand, in this setup the system is run as part of the GitLab CI service whenever the content is pushed to the repository, and compiles every ‘fileder’ in the system into a static HTML file.

This mode of static content generation used to be a feature of mmmfs, but was sacrificed for the sake of in-system editing as part of my bachelor thesis. I decided to restore it for the FabAcademy, because I wanted to use the same system here that I use for the main page. I prefer to use mmmfs here because it will allow me to share content between this repository and my main website in the future, and it means that I can keep my visual identity consistent between the two contexts. I will also be able to keep customizing it and add capabilities like inline 3d model previews etc as I go along documenting.

technical setup

For continuous delivery for my own website, I had already dockerized the dependencies for running the ‘live’ server of my blog with the following Dockerfile:

FROM nickblah/lua:5.3-luarocks-stretch

RUN echo "deb http://ppa.launchpad.net/jonathonf/tup/ubuntu xenial main" \
      >/etc/apt/sources.list.d/tup.list
RUN apt-get update && \
    apt-get install -y --allow-unauthenticated \
      build-essential m4 tup sassc \
      libmarkdown2-dev libsqlite3-dev libssl-dev
RUN luarocks install discount DISCOUNT_INCDIR=/usr/include/x86_64-linux-gnu
RUN luarocks install sqlite3 && \
    luarocks install moonscript && \
    luarocks install http && \
    luarocks install lua-cjson 2.1.0-1

COPY . /code
WORKDIR /code
RUN tup init && tup generate --config tup.docker.config build-static.sh && ./build-static.sh

EXPOSE 8000
ENTRYPOINT ["moon", "build/server.moon", "fs", "0.0.0.0", "8000"]

While the EXPOSE and ENTRYPOINT directives here are superfluous for the static site workflow, I chose to re-use the image as-is so that I would only have to maintain one docker image for this project, rather than dealing with two slightly different variants. Thankfully the GitLab CI configuration turned out to be very flexible in this regard.

The docker image usually is rebuilt on my own server whenever I push to the git repository and deployed there (you can read the blog post documenting my simple homebuilt git-ci setup here), but not published publicly online. For GitLab to find my docker image, I had to manually publish it on dockerhub as s0lll0s/mmm.

Once the image was available, I could move on to configuring GitLab’s CI system using the .gitlab-ci.yml file. Using the documentation I figured out that by overriding the entrypoint with an empty command I could get the shell-command injection working as it was intended. Compared to the default .gitlab-ci.yml using marked, I could drop the before_script part, since all my dependencies come pre-installed in the docker image.

In the pages part on the other hand there are a few more things to do:

image:
  name: s0lll0s/mmm
  entrypoint: [""]

pages:
  script:
  - cd /code
  - "patch -p0 <$CI_PROJECT_DIR/static/layout/'text$patch.patch'"
  - moon build/render_all.moon fs:$CI_PROJECT_DIR public /2020/labs/opendot/students/sol-bekic
  - cp -r public $CI_PROJECT_DIR/public
  artifacts:
    paths:
    - public
  only:
  - master

First the shell has to change into the /code directory, where my CMS code is located (compare the Dockerfile above). In theory I could set the LUA_PATH and LUA_CPATH appropriately and run the code directly from inside the site repository, but it was a bit easier to set it up this way. After this, I apply a patch file to change some details about the page layout that are currently hardcoded for my main website. This is what the patch file looks like currently:

--- mmm/mmmfs/layout.moon 2020-01-30 12:03:40.601460375 +0100
+++ mmm/mmmfs/layout.fab.moon 2020-01-30 12:58:20.264865069 +0100
@@ -36,7 +36,7 @@
       h1 {
         navigate_to '', logo
         span {
-          span 'mmm', class: 'bold'
+          span 'fab', class: 'bold'
           '​'
           '.s‑ol.nu'
         }
@@ -51,9 +51,7 @@
     }
     aside {
-      navigate_to '/about', 'about me'
-      navigate_to '/portfolio', 'portfolio'
-      navigate_to '/games', 'games'
-      navigate_to '/projects', 'other'
+      navigate_to '/', 'about me'
+      navigate_to '/log', 'log'
       a {
         href: 'mailto:s%20[removethis]%20[at]%20s-ol.nu'
         'contact'
@@ -78,8 +76,6 @@
     iconlink 'https://github.com/s-ol', 'https://cdn.jsdelivr.net/npm/simple-icons@latest/icons/github.svg', 'github'
     iconlink 'https://merveilles.town/@s_ol', 'https://cdn.jsdelivr.net/npm/simple-icons@latest/icons/mastodon.svg', 'mastodon'
     iconlink 'https://twitter.com/S0lll0s', 'https://cdn.jsdelivr.net/npm/simple-icons@latest/icons/twitter.svg', 'twitter'
-    iconlink 'https://webring.xxiivv.com/#random', 'https://webring.xxiivv.com/icon.black.svg', 'webring',
-      { height: '1.3em', 'margin-left': '.3em', 'margin-top': '-0.12em' }
   }
 }

@@ -104,8 +100,8 @@

     <meta property=\"og:title\" content=#{e title} />
     <meta property=\"og:type\"  content=\"website\" />
-    <meta property=\"og:url\"   content=\"https://mmm.s-ol.nu#{@path}/\" />
-    <meta property=\"og:site_name\" content=\"mmm\" />"
+    <meta property=\"og:url\"   content=\"https://fab.s-ol.nu#{@path}/\" />
+    <meta property=\"og:site_name\" content=\"sol bekic - fab academy 2020\" />"

     if desc = @get 'description: text/plain'
       meta ..= "

This again is a quick solution to allow me to change things about the CMS between my main website and this project log respectively, without fully diverting the two instances into different projects. In the future I would like to integrate the website layout into the ‘content’ of the CMS itself, so that the system and its looks are completely separate and nothing is hardcoded, but you can read more about that in my thesis linked above.

One thing that took me a while to realize was that my site was being served as a sub-site deep in fabacademy.org, so that all of my page URLs start with /2020/labs/opendot/students/sol-bekic, whereas on my own site everything is at the root (/). To cope with this I had to revisit a couple of places in mmmfs‘ code, since absolute links are used almost everywhere: for links between pages, inclusion of assets like images and videos, and for the sitewide stylesheet. My solution was to add a configuration flag for the static site export that simply prepends the prefix string to all URLs produced in the system. You can see me passing the prefix as an option to the build script in the .gitlab-ci.yml above.

repository layout

I added three top-level Fileders:

• static, a hidden fileder that contains the stylesheet, some web runtime components (like the syntax highlighting plugin) and the patch file mentioned above,
• about, containing the short about section,
• log, containg the weekly logs (such as this one you are currently reading).

The about page and this log post are composed in Markdown, but with mmmfs nearly any format can theoretically be supported, and all supported formats and filetypes can be mixed and matched freely across and inside pages. It is also possible to describe pages using code that is executed on demand and can compose pages based on other data stored in the system. This is how the main page is realized: because the about section is so short, I wanted to show it on the main page rather than on its own, and to also preview the weekly log index alongside it. The main page Facet has the type text/moonscript -> fn -> mmm/dom (read as: A MoonScript file that, when parsed, yields a function that, when executed, returns a HTML DOM snippet), and looks like this:

import div from require 'mmm.dom'

-- delegate to the two child fileders
=>
  div for child in *@children
    continue if child\get 'hidden: bool'
    div (child\get 'mmm/dom'), class: 'well'

As for all pages, when mmmfs renders this page, it attempts a conversion to text/html. A conversion path is searched for using (all combinations of) the supported individual conversions. Here is an excerpt of the graph of possible conversions mmmfs currently enumerates for this specific facet:

The cheapest path (by cumulative cost) that is found is the intended one: first the MoonScript is parsed, then the resulting function is executed, and finally the resulting DOM snippet is embedded in the HTML layout template. When the function (the => and everything indented past it in the snippet above) is executed, it loops over all children of its Fileder (the main directory of this repository). It skips all Fileders that have a hidden Facet that evaluates to false when converted to a boolean value (that is currently only the static fileder) and embeds all other fileders in <div>s with the class well. The embedded divs are all grouped together in another <div> due to a technical limitation that only allows passing single elements as mmm/dom objects currently.

image optimization

For image optimization, I tried to use MozJPEG first, but found that the workflow for lossy conversion was a bit less straightforward than I assumed. I then found jpegoptim to be easier to use.

I will add all images as high-quality files locally with a hq: facet name, which I have excluded in the repository .gitignore file, so they will not be committed. I can then duplicate these files and compress them before pushing the compressed version like so:

$ cp "hq: image$jpeg.jpg" "image$jpeg.jpg"
$ jpegoptim -sS100 "image$jpeg.jpg"

The option -s instructs jpegoptim to strip all metadata, and -S 100 tells it to bring the file down as close as possible to a 100 kilobyte size as possible. For the image below this has worked out pretty well, but I will keep checking the image quality and adjusting settings as necessary as I keep adding pictures in the upcoming weeks.

final project sketch