ipng/ipng.ch
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add Jekyll to Hugo article
All checks were successful
continuous-integration/drone/push Build is passing
Details

Browse Source
This commit is contained in:
Pim van Pelt
2024-08-12 22:36:17 +02:00
parent ee4534c23a
commit f201aeb596
4 changed files with 461 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

448
content/articles/2024-08-12-jekyll-hugo.md Normal file
View File

@ -0,0 +1,448 @@
---
date: "2024-08-12T09:01:23Z"
title: 'Case Study: From Jekyll to Hugo'
---
# Introduction
{{< image width="16em" float="right" src="/assets/jekyll-hugo/before.png" alt="ipng.nl before" >}}
In the _before-days_, I had a very modest perrsonal website runnong on [[ipng.nl](https://ipng.nl)]
and [[ipng.ch](https://ipng.ch/)]. Over the years I've had quite a few different designs, and
although one of them was hosted (on Google Sites) for a brief moment, they were mostly very much web
1.0, "The 90s called, they wanted their website back!" style.
The site didn't have much other than a little blurb on a few open source projects of mine, and a
gallery hosted on PicasaWeb [which Google subsequently turned down], and a mostly empty Blogger
page. Would you imagine that I hand-typed the XHTML and CSS for this website, where the menu at the
top (thinks like `Home` - `Resume` - `History` - `Articles`) would just have a HTML page which
meticulously linked to the other HTML pages. It was the way of the world, in the 1990s.
## Jekyll
{{< image width="9em" float="right" src="/assets/jekyll-hugo/jekyll-logo.png" alt="Jekyll" >}}
My buddy Michal suggested in May of 2021 that, if I was going to write all of the HTML skeleton by
hand, I may as well switch to a static website generator. He's fluent in Ruby, and suggested I take
a look at [[Jekyll](https://jekyllrb.com/)], a static site generator. It takes text written in
your favorite markup language and uses layouts to create a static website. You can tweak the site’s
look and feel, URLs, the data displayed on the page, and more.
I immediately fell in love! As an experiment, I moved [[IPng.ch](https://ipng.ch)] to a new
webserver, and kept my personal website on [[IPng.nl](https://ipng.nl)]. I had always wanted to
write a little bit more about technology, and since I was working on an interesting project [[Linux
Control Plane]({{< ref 2021-08-12-vpp-1 >}})] in VPP, I thought it'd be nice to write a little bit
about it, but certainly not while hand-crafting all of the HTML exoskeleton. I just wanted to write
Markdown, and this is precisely the _raison d'&ecirc;tre_ of Jekyll!
Since April 2021, I wrote in total 67 articles with Jekyll. Some of them proved to become quite
popular, and (_humblebrag_) my website is widely considered one of the best resources for Vector
Packet Processing, with my [[VPP]({{< ref 2021-09-21-vpp-7 >}})] series, [[MPLS]({{< ref
2023-05-07-vpp-mpls-1 >}})] series and a few others like the [[Mastodon]({{< ref
2022-11-20-mastodon-1 >}})] series being amongst some of the top visited articles, with ~2.5-3K
monthly unique visitors.
## The catalyst
There were two distinct events that lead up to this. Firstly, I started a side project called [[Free
IX](https://free-ix.ch/)], which I also created in Jekyll. When I did that, I branched the
[[IPng.ch](https://ipng.ch)] site, but the build faild with Ruby errors. My buddy Antonios fixed
those, and we were underway. Secondly, later on I attempted to upgrade the IPng website to the same
fixes that Antonios had provided for Free-IX, and all hell broke loose (luckily, only in staging
environment). I spent several hours pulling my hear out re-assembling the dependencies, downgrading
Jekyll, pulling new `gems`, downgrading `ruby`. Finally, I got it to work again, only to see after
my first production build, that the build immediately failed because the Docker container that does
the build no longer liked what I had put in the `Gemfile` and `_config.yml`. It was something to do
with `sass-embedded` gem, and I spent waaaay too long fixing this incredibly frustrating breakage.
## Hugo
{{< image width="9em" float="right" src="/assets/jekyll-hugo/hugo-logo-wide.svg" alt="Hugo" >}}
When I made my roadtrip from Zurich to the North Cape with my buddy Paul, we took extensive notes on
our daily travels, and put them on [[2022roadtripnose](https://2022roadtripnose.weirdnet.nl/)]
website. At the time, I was looking for a photo caroussel for Jekyll, and while I found a few, none
of them really worked in the way I wanted them to. I stumbled across [[Hugo](https://gohugo.io)],
which says on its website that it is one of the most popular open-source static site generators.
With its amazing speed and flexibility, Hugo makes building websites fun again. So I dabbled a bit
and liked what I saw. I used the [[notrack](https://github.com/gevhaz/hugo-theme-notrack)] theme from
GitHub user `@gevhaz`, as they had made a really nice gallery widget (called a `shortcode` in Hugo).
The main reason for me to move to Hugo is that it is a **standalone Go** program, with no runtime or
build time dependencies. The Hugo [[GitHub](https://github.com/gohugoio/hugo)] delivers ready to go
build artifacts, tests amd releases regularly, and has a vibrant user community.
### Migrating
I have only a few strong requirements if I am to move my website:
1. The site's URL namespace MUST be *identical* (not just similar) to Jekyll. I do not want to
lose my precious ranking on popular search engines.
1. MUST be built in a CI/CD tool like Drone or Jenkins, and autodeploy
1. Code MUST be _hermetic_, not pulling in external dependencies, neither in the build system (eg.
Hugo itself) nor the website (eg. dependencies, themes, etc).
1. Theme MUST support images, videos and SHOULD support asciinema.
1. Theme SHOULD try to look very similar to the current Jekyll `minima` theme.
#### Attempt 1: Auto import ❌
With that in mind, I notice that Hugo has a site _importer_, that can import a site from Jekyll! I
run it, but it produces completely broken code, and Hugo doesn't even want to compile the site. This
turns out to be a _theme_ issue, so I take Hugo's advice and install the recommended them. The site
comes up, but is pretty screwed up. I now realize that the `hugo import jekyll` imports the markdown
as-is, and only rewrites the _frontmatter_ (the little blurb of YAML metadata at the top of each
file). Two notable problems:
**1. images** - I make liberal use of Markdown images, which in Jekyll can be decorated with CSS
styling, like so:
```
![Alt](/path/to/image){: style="width:200px; float: right; margin: 1em;"}
```
**2. post_url** - Another widely used feature is cross-linking my own articles, using Jekyll
template expansion, like so:
```
.. Remember in my [[VPP Babel]({% post_url 2024-03-06-vpp-babel-1 %})] ..
```
I do some grepping, and have 246 such Jekyll template expansions, and 272 images OK, that's a dud.
#### Attempt 2: Skeleton ✅
I decide to do this one step at a time. First, I create a completely new website `hugo new site
hugo.ipng.ch`, download the `notrack` theme, and add only the front page `index.md` from the
original IPng site. OK, that renders.
Now comes a fun part: going over the theme's SCSS to adjust it to look and feel similar to the
Jekyll `minima` theme. I change a bunch of stuff in the skeleton of the website theme.
First, I take a look at the site media breakpoints, to feel correct for desktop screen, tablet
screen and iPhone/Android screens. Then, I inspect the font family, size and H1/H2/H3...
magnifications, also scaling them with media size. Finally I notice the footer, which in `notrack`
spans the whole width of the browser. I change it to be as wide as the header and main page.
I go one by one on the site's main pages and, just as on the Jekyll site, I make them into menu
items at the top of the page. The [[Services]({{< ref services >}})] page serves as my proof of
concept, as it has both the `image` and the `post_url` pattern in Jekyll. It references six articles
and has two images which float on the right side of the canvas. If I can figure out how to rewrite
these to fit the Hugo variants of the same pattern, I should be home free.
### Hugo: image
The idiomatic way in `notrack` is an `image` shortcode. I hope you know where to find the curly
braces on your keyboard - because geez, Hugo templating sure does like them!
```
<figure class="image-shortcode{{ with .Get "class" }} {{ . }}{{ end }}
{{- with .Get "wide" }}{{- if eq . "true" }} wide{{ end -}}{{ end -}}
{{- with .Get "frame" }}{{- if eq . "true" }} frame{{ end -}}{{ end -}}
{{- with .Get "float" }} {{ . }}{{ end -}}"
style="
{{- with .Get "width" }}width: {{ . }};{{ end -}}
{{- with .Get "height" }}height: {{ . }};{{ end -}}">
{{- if .Get "link" -}}
<a href="{{ .Get "link" }}"{{ with .Get "target" }} target="{{ . }}"{{ end -}}
{{- with .Get "rel" }} rel="{{ . }}"{{ end }}>
{{- end }}
<img src="{{ .Get "src" | relURL }}"
{{- if or (.Get "alt") (.Get "caption") }}
alt="{{ with .Get "alt" }}{{ replace . "'" "&#39;" }}{{ else -}}
{{- .Get "caption" | markdownify| plainify }}{{ end }}"
{{- end -}}
/> <!-- Closing img tag -->
{{- if .Get "link" }}</a>{{ end -}}
{{- if or (or (.Get "title") (.Get "caption")) (.Get "attr") -}}
<figcaption>
{{ with (.Get "title") -}}
<h4>{{ . }}</h4>
{{- end -}}
{{- if or (.Get "caption") (.Get "attr") -}}<p>
{{- .Get "caption" | markdownify -}}
{{- with .Get "attrlink" }}
<a href="{{ . }}">
{{- end -}}
{{- .Get "attr" | markdownify -}}
{{- if .Get "attrlink" }}</a>{{ end }}</p>
{{- end }}
</figcaption>
{{- end }}
</figure>
```
From the top - Hugo creates a figure with a certain set of classes, the default `image-shortcode`
but also classes for `frame`, `wide` and `float` to further decorate the image. Then it applies
direct styling for `width` and `height`, optionally inserts a link (something I had missed out on in
Jekyll), then inlines the `<img>` tag with an `alt` or (markdown based!) `caption`. It then reuses
the `caption` or `title` or `attr` variables to assemble a `<figcaption>` block. I absolutely love it!
I've rather consistently placed my images by themselves, on a single line, and they all have at
least one style (be it `width`, or `float`), so it's really straight forward to rewrite this with a
little bit of Python:
```
def convert_image(line):
p = re.compile(r'^!\[(.+)\]\((.+)\){:\s*(.*)}')
m = p.match(line)
if not m:
return False
alt=m.group(1)
src=m.group(2)
style=m.group(3)
image_line = "{{</* image "
if sm := re.search(r'width:\s*(\d+px)', style):
image_line += f'width="{sm.group(1)}" '
if sm := re.search(r'float:\s*(\w+)', style):
image_line += f'float="{sm.group(1)}" '
image_line += f'src="{src}" alt="{alt}" */>}}}}'
print(image_line)
return True
with open(sys.argv[1], "r", encoding="utf-8") as file_handle:
for line in file_handle.readlines():
if not convert_image(line):
print(line.rstrip())
```
### Hugo: ref
In Hugo, the idiomatic way to reference another document in the corpus is with the builtin `ref`
shortcode, requiring a single argument: the path to a content document, with or without a file
extension, with or without an anchor. Paths without a leading / are first resolved relative to the
current page, then to the remainder of the site. This is super cool, because I can essentially
reference any file by just its name!
```
for fn in $(find content/ -name \*.md); do
sed -i -r 's/{%[ ]?post_url (.*)[ ]?%}/{{</* ref \1 */>}}/' $fn
done
```
And with that, the converted markdown from Jekyll renders perfectly in Hugo. Of course, other sites
may use other templating commands, but for [[IPng.ch](https://ipng.ch)], these were the only two
special cases.
### Hugo: URL redirects
It is a hard requirement for me to keep the same URLs that I had from Jekyll. Luckily, this is a
trivial matter for Hugo, as it supports URL aliases in the _frontmatter_. Jekyll will add a file
extension to the article _slugs_, while Hugo uses only the directly and serves an `index.html` from
it. Also, the default for Hugo is to put content in a different directory.
The first change I make is to the main `hugo.toml` config file:
```
[permalinks]
articles = "/s/articles/:year/:month/:day/:slug"
```
That solves the main directory problem (I chose `s/articles/` in Jekyll). Then, adding the URL
redirect is a simple matter of looking up which filename Jekyll ultimately used, and adding a little
frontmatter at the top of each article, for example my [[VPP #1]({{< ref 2024-08-12-jekyll-hugo
>}})] article would get this addition:
```
---
date: "2021-08-12T11:17:54Z"
title: VPP Linux CP - Part1
aliases:
- /s/articles/2021/08/12/vpp-1.html
---
```
Hugo by default renders it in `/s/articles/2021/08/12/vpp-linux-cp-part1/index.html` but the
addition of the `alias` makes it also generate a drop-in placeholder HTML page that offers a
permanent redirect (cleverly setting `noindex` for web crawlers and offering the `canonical` link
for the new place, aka a permanent redirect:
```
$ curl https://ipng.ch/s/articles/2021/08/12/vpp-1.html
<!DOCTYPE html>
<html lang="en-us">
<head>
<title>https://ipng.ch/s/articles/2021/08/12/vpp-linux-cp-part1/</title>
<link rel="canonical" href="https://ipng.ch/s/articles/2021/08/12/vpp-linux-cp-part1/">
<meta name="robots" content="noindex">
<meta charset="utf-8">
<meta http-equiv="refresh" content="0; url=https://ipng.ch/s/articles/2021/08/12/vpp-linux-cp-part1/">
</head>
</html>
```
### Hugo: Asciinema
One thing that I always wanted to add is the ability to inline [[Asciinema](https://asciinema.org)]
screen recordings. First, I take a look at what is needed to serve Asciinema: One Javascript file,
and one CSS file, followed by a named `<div>` which invokes the Javascript. Armed with that
knowledge, I dive into the `shortcode` language a little bit:
```
$ cat themes/hugo-theme-ipng/layouts/shortcodes/asciinema.html
<div id='{{ .Get "src" | replaceRE "[[:^alnum:]]" "" }}'></div>
<script>
AsciinemaPlayer.create("{{ .Get "src" }}",
document.getElementById('{{ .Get "src" | replaceRE "[[:^alnum:]]" "" }}'));
</script>
```
This file creates the `id` of the `<div>` by means of stripping all non-alphanumeric characters from
the `src` argument of the _shortcode_. So if I were to create an `{{</* asciinema
src='/casts/my.cast' */>}}`, the resulting DIV will be uniquely called `castsmycast`. This way, I
can add multiple screencasts in the same document, which is dope
But, as I now know, I need to load some CSS and JS so that the `AsciinemaPlayer` class becomes
available. For this, I use a feature in Hugo, which allows for `params` to be set in the
frontmatter, for example in the [[VPP OSPF]({{< ref 2024-06-22-vpp-ospf-2 >}})] article:
```
---
date: "2024-06-22T09:17:54Z"
title: VPP with loopback-only OSPFv3 - Part 2
aliases:
- /s/articles/2024/06/22/vpp-ospf-2.html
params:
asciinema: true
---
```
The presence of that `params.asciinema` can be used in any page, including the HTML skeleton of the
theme, like so:
```
$ cat themes/hugo-theme-ipng/layouts/partials/head.html
<head>
...
{{ if eq .Params.asciinema true -}}
<link rel="stylesheet" type="text/css" href="{{ "css/asciinema-player.css" | relURL }}" />
<script src="{{ "js/asciinema-player.min.js" | relURL }}"></script>
{{- end }}
</head>
```
Now all that's left for me to do is drop the two Asciinema player files in their respective theme
directories, and for each article that wants to use an Asciinema, set the `param` and it'll ship the
CSS and Javascript to the browser. I think I'm going to have a good relationship with Hugo :)
### Gitea: Large File Support
One mistake I made with the old Jekyll based website, is that I checked in all of the images and
binary files directly into Git. This bloats the repository and is otherwise completely unnecessary.
For this new repository, I enable [[Git LFS](https://git-lfs.com/)], which is available for OpenBSD
(packages), Debian (apt) and MacOS (homebrew). Turning this on is very simple:
```
$ brew install git-lfs
$ cd ipng.ch
$ git lfs install
$ for i in gz png gif jpg jpeg tgz zip; do \\
git track "*.$i" \\
git lfs import --everything --include "*.$i" \\
done
$ git push --force --all
```
The `force` push rewrites the history of the repo to reference the binary blobs in LFS instead of
directly in the repo. As a result, the size of the repository greatly shrinks, and handling it
becomes easier once it grows. A really nice feature!
### Gitea: CI/CD with Drone
At IPng, I run a [[Gitea](https://gitea.io)] server, which is one of the coolest pieces of open
source that I use on a daily basis. There's a very clean integration of a continuous integration
tool called [[Drone](https://drone.io/)] and these two tools are literally made for each other.
Drone can be enabled for any Git repo in Gitea, and upon the presence of a `.drone.yml` file, upon
repository events, called _triggers_. It can then run a sequence of steps, hermetically in a Docker
container called a _drone-runner_, which first checks out the repository at the latest commit, and
then does whatever I'd like with it. I'd like to build a Hugo site, please!
As it turns out, there is a [[Drone Hugo](https://plugins.drone.io/plugins/hugo)] readily available,
but it seems to be very outdated. Luckily, this being open source and all, I can download the source
on [[GitHub](https://github.com/drone-plugins/drone-hugo)], and in the `Dockerfile`, bump the Alpine
version, the Go version and build the latest Hugo release, which is 0.130.1 at the moment. I really
do need this version, because the `params` feature was introduced in 0.123 and the upstream package
is still for 0.77 -- which is about four years old. Ouch!
I build a docker image and upload it to my private repo at IPng, hosted as well on Gitea, by the
way. As I said, it really is a great piece of kit! In case anybody else would like to give it a
whirl, ping me on Mastodon or e-mail and I'll upload one to public Docker Hub as well.
### Putting it all together
With Drone activated for this repo, and the Drone Hugo plugin built with a new version, I can submit
the following file to the root directory of the `ipng.ch` repository:
```
$ cat .drone.yml
kind: pipeline
name: default
steps:
- name: git-lfs
image: alpine/git
commands:
- git lfs install
- git lfs pull
- name: build
image: git.ipng.ch/ipng/drone-hugo:release-0.130.0
settings:
hugo_version: 0.130.0
extended: true
- name: rsync
image: drillster/drone-rsync
settings:
user: drone
key:
from_secret: drone_sshkey
hosts:
- nginx0.chrma0.net.ipng.ch
- nginx0.chplo0.net.ipng.ch
- nginx0.nlams1.net.ipng.ch
- nginx0.nlams2.net.ipng.ch
port: 22
args: '-6u --delete-after'
source: public/
target: /var/www/ipng.ch/
recursive: true
secrets: [ drone_sshkey ]
image_pull_secrets:
- git_ipng_ch_docker
```
The file is relatively self-explanatory. Before my first step runs, Drone already checks out the
repo in the current working directory of the docker container. I then install package `alpine/git`
and run the `git lfs install` and `git lfs pull` commands to resolve the LFS symlinks into actual
files by pulling those objects that are referenced (and, notably, not all historical versions of any
binary file ever added to the repo).
Then, I run a step called `build` which invokes the Hugo Drone package that I created before.
Finally, I run a step called `rsync` which uses package `drillster/drone-rsync` to rsync-over-ssh
the files to the four NGINX servers running at IPng: two in Amsterdam, one in Geneva and one in
Zurich.
One really cool feature is the use of so called _Drone Secrets_ which are references to locked
secrets such as the SSH key, and, notably, the Docker Repository credentials, because Gitea at IPng
does not run a public docker repo. Using secrets is nifty, because it allows to safely check in the
`.drone.yml` configuration file without leaking any specifics.
### NGINX and SSL
Now that the website is automatically built and rsync'd to the webservers upon every `git merge`,
all that's left for me to do is arm the webservers with SSL certificates. I actually wrote a whole
story about specifically that, as for `*.ipng.ch` and `*.ipng.nl` and a bunch of others,
periodically there is a background task that retrieves multiple wildcard certificates with Let's
Encrypt, and distributes them to any server that needs them (like the NGINX cluster, or the Postfix
cluster). I wrote about the [[Frontends]({{< ref 2023-03-17-ipng-frontends >}})], the spiffy
[[DNS-01]({{< ref 2023-03-24-lego-dns01.md >}})] certificate subsystem, and the internal network
called [[IPng Site Local]({{< ref 2023-03-11-mpls-core >}})] each in their own articles, so I won't
repeat that information here.
## The Results
The results are really cool, as I'll demonstrate in this video. I can just submit and merge this
change, and it'll automatically kick off a build and push. Take a look!