Migrating this blog to Nix / NixOS
This blog used to live on GitHub Pages. Which is… fine? It’s nice that it’s free, it’s nice that it builds and deploys automatically. But you get no server-side logs, so posting here feels a bit like shouting into a void. Which, admittedly, it probably is anyway, but it would be nice to at least be able to see the void I’m shouting into.
I’ve had an old VPS (a DigitalOcean droplet) lying around for about 10 years now, so I figured that’d be a natural place to put it. But the thing was running Ubuntu 16.04 🫣. I have been using NixOS as my daily driver for some time now, and have completely fallen in love with it, so I thought this was as good an opportunity as any to nix things up a bit.
dev shells are awesome
One really cool thing about the nix package manager – which is cross-platform, you don’t need to run NixOS – is dev shells.
You can think of a dev shell as a bit like a python virtualenv, except for configuration, system packages, and environment variables too.
Or you can think of it like a docker container, but without containerization.
It’s pretty magical.
Jekyll is a bit of a pain to run, and its dependencies sort of get smeared all over your computer.
It’s written in Ruby which means you get Ruby, Ruby gems, gem versions, the bundler, conflicting system Ruby versions, and lots of sudo sprinkled around.
A DevShell just makes the whole problem go away.
I created a flake.nix file in the repo root:
{
description = "Jekyll dev environment for ojensen.net";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
outputs = { self, nixpkgs }:
let
systems = [ "x86_64-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin" ];
forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f system);
in
{
devShells = forAllSystems (system:
let
pkgs = nixpkgs.legacyPackages.${system};
rubyEnv = pkgs.ruby.withPackages (ps: with ps; [
jekyll
webrick
kramdown-parser-gfm
jekyll-paginate
jekyll-gist
jemoji
]);
in
{
default = pkgs.mkShell {
packages = [ rubyEnv ];
};
});
};
}
and now when I cd into the directory, I run nix develop and I have everything I need on my path, without installing anything system-wide. Exiting the shell brings me right back to my “normal” system with none of that stuff installed.
Adding a Shell Hook
The dev shell flake can run arbitrary setup when the shell activates:
default = pkgs.mkShell {
packages = [ rubyEnv ];
shellHook = ''
export PORT="''${PORT:-4000}"
export IP="''${IP:-127.0.0.1}"
echo "Jekyll dev shell ready."
echo "Run: jekyll serve --watch (http://localhost:4000)"
echo "Or: ./serve"
'';
};
It’s nice to be reminded of relevant invocations.
I did this for local development a little while back, and it’s been so nice. Not using dev shells feels so… python in the pre-venv era.
Seriously, if you take one thing from this post, it’s: install the nix package manager, and use dev shells.
NixOS on DigitalOcean
I absolutely love declarative system management, and discovering I could do that for system packages and configuration was incredible. NixOS is fantastic as a daily-driver for a lot of reasons – my favorites are:
- dev shells obviously, but you can get those anywhere you can install
nix. - it completely removes any update anxiety, because you can just boot previous generations if you want to.
- you can do things that Are Simply Not Done in the Linux world, like try a new DE for a week to see if you like it, with trivial cleanup afterwards.
- you can add programs to your current shell without installing them system-wide, a sort of “i just need this for now, not forever” which is actually really nice.
Maybe someday I’ll write a post about NixOS in general because I won’t shut up about it to my friends. But for now, suffice to say that if I’m going to be reimaging a machine, I might as well put NixOS on it.
Installing
DigitalOcean does not offer a NixOS image to build droplets off of. Normally, working around this problem is really easy:
- spin up an ubuntu droplet
- write a basic NixOS flake (system definition) on your laptop
- use nixos-anywhere which kexecs a NixOS installer out of whatever Linux is currently running, partitions the disk, and installs your flake, all over SSH (or in layman’s terms: Magic!)
BUT.
So one thing you should be aware of that I didn’t think about was: new systems get utterly hammered with brute-force SSH traffic when they first go online.
This means that connecting to your system via SSH is pretty unreliable unless you put up a firewall blocking SSH traffic from IPs that are not yours.
It’s also not a bad idea to have your SSH daemon listen on another port (e.g. 2222) which you can leave open to the world for when you are not at your typical IP.
This… did not occur to me right away. And it turns out that having a large fraction of your SSH connection attempts time out makes nixos-anywhere not work very well.
So if you’re smart, firewall off port 22 and use nixos-anywhere.
If you’re dumb like me, reconfigure your flake to output a built image:
web-image = nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
modules = [
"${nixpkgs}/nixos/modules/virtualisation/digital-ocean-image.nix"
./hosts/web/base.nix
];
};
and then build it via
nix build .#nixosConfigurations.web-image.config.system.build.digitalOceanImage
Then upload the resulting .qcow2 file (you’ll need to gunzip it first) as a CustomImage in DigitalOcean, install from that, then delete the CustomImage again.
Voila, you have a fresh NixOS droplet!
Structuring the config
So I had a handful of old projects lying around on that old Xenial box – mostly a few html-only websites served from my home directory, one rust binary that I ran via a tmux shell, that sort of thing. One might as well drink this thing to the very dregs – let’s get those set up on our new NixOS box before worrying about the blog.
hosts/web/
default.nix # users, ssh, nix settings, packages
nginx.nix # nginx service, ACME, logging, catch-all vhost
sites/
ojensen.nix # this blog
pastebin.nix # a rust thing from 2018
tocker.nix # a tool for aligning a pendulum clock
...
Each nix file in sites/ basically contributes an nginx vhost, and then some.
This configuration lives alongside hosts/human, which defines my laptop setup.
Neat!
The flake pulls in default.nix, and that imports the rest:
{ pkgs, ... }:
{
imports = [
./nginx.nix
./sites/ojensen.nix
./sites/pastebin.nix
./sites/tocker.nix
];
# ... users, ssh, etc
}
A simple HTML website
This stuff is surprisingly easy. Here’s a whole site, tocker:
{ ... }:
{
fileSystems."/srv/www/tocker" = {
device = "/home/ojensen/tocker";
fsType = "none";
options = [ "bind" "ro" ];
};
services.nginx.virtualHosts."tocker.ojensen.net" = {
enableACME = true;
forceSSL = true;
root = "/srv/www/tocker";
extraConfig = ''
access_log /var/log/nginx/tocker.access.log combined;
error_page 403 /error403.html;
error_page 404 /index.html;
'';
};
}
It binds the folder in my home directory to a reasonable place, and then serves nginx over it, automatically sorting out certificates.
I realize I’ve never written about Tocker before, but basically it’s a tool that helps you align a pendulum clock that maybe doesn’t have a straight edge, or whose mechanisms are *ahem* antique enough that ticks are not particularly regular – it listens on your microphone and averages the time difference between “tick/tocks” and “tock/ticks” so you can tilt the clock until they’re as even as you can get them.
It’s not great, but it works well enough. You can see the result here.
The process was pretty much the same for all of my other little html projects.
A rust binary last compiled in 2018
So, one thing to be aware of with NixOS is you can’t just… run binaries that were built for other linuxes. It’s… complicated.
Anyway, that old rust binary is a fun little pastebin clone, designed to be accessed and operated via curl.
And refactoring this thing to compile on modern rust with modern dependencies is… a project for another day.
But we can still make this work.
And as long as we’re here, let’s fix the whole “manually run the binary in a tmux session” thing.
So does the thing run? No. Basically there is no /lib64/ld-linux-x86-64.so.2 on NixOS.
Dynamically linked binaries can be painful, but this is Rust: everything but glibc is statically linked, so there’s very little to work around and the fix is just a one-liner programs.nix-ld.enable = true;.
Somewhat surprisingly, this 8-year-old binary then runs successfully against a 2026 glibc, which is a pretty nice showcase of glibc’s symbol versioning practice.
Anyway, here’s the config
{pkgs, ...}: {
programs.nix-ld.enable = true;
# systemd supervising
systemd.services.pastebin = {
description = "pastebin";
wantedBy = ["multi-user.target"];
serviceConfig = {
ExecStart = "/home/ojensen/pastebin-actix/pastebin";
WorkingDirectory = "/home/ojensen/pastebin-actix";
User = "ojensen";
Restart = "always";
RestartSec = 5;
ProtectSystem = "strict";
ProtectHome = "read-only";
ReadWritePaths = ["/home/ojensen/pastebin-actix/uploads"];
PrivateTmp = true;
PrivateDevices = true;
NoNewPrivileges = true;
RestrictAddressFamilies = ["AF_INET"];
RestrictNamespaces = true;
LockPersonality = true;
};
};
fileSystems."/srv/www/pastebin" = {
device = "/home/ojensen/pastebin-actix";
fsType = "none";
options = ["bind" "ro"];
};
services.nginx.virtualHosts."0x3c.net" = {
enableACME = true;
forceSSL = true;
extraConfig = ''
access_log /var/log/nginx/0x3c.access.log combined;
add_header 'Access-Control-Allow-Origin' '*';
'';
locations = {
"/".proxyPass = "http://127.0.0.1:8080/";
};
};
}
The systemd stuff is more complicated than it needs to be, mainly because this binary reads files in its pwd, and I haven’t looked at the code in 8 years, and it’d sure be nice if the blast radius were contained if someone gets it to misbehave. Ideally ProtectHome would be true, but the thing lives in my home directory, so… yeah someday we’ll refactor and clean this up.
But hey, it’s running.
Migrating ojensen.net
OK. Yak shaving: done. Let’s get to the whole reason we’re doing this. And with all that setup done, it comes together really nicely.
A Jekyll website is just a bunch of HTML files, so the webserver part is easy.
But we’ll want a way to deploy it, so we’ll create a user called deploy and define its permissions and key.
We can do this all in the site’s nix file:
In hosts/web/sites/ojensen.nix:
{pkgs, ...}: {
users.users.deploy = {
isSystemUser = true;
group = "deploy";
shell = pkgs.bashInteractive;
home = "/var/lib/deploy";
createHome = true;
openssh.authorizedKeys.keys = [
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPYftc2UmLbydC8GRBEovKeQXKXUsexr/hbhGXeZkD9R blog_deploy"
];
};
users.groups.deploy = {};
systemd.tmpfiles.rules = [
"d /srv/www/blog 0755 deploy deploy -"
];
services.nginx.virtualHosts."ojensen.net" = {
enableACME = true;
forceSSL = true;
root = "/srv/www/blog";
locations."/".extraConfig = ''
try_files $uri $uri.html $uri/ =404;
'';
extraConfig = ''
access_log /var/log/nginx/ojensen.access.log combined;
error_page 404 /404.html;
'';
};
}
Originally I’d planned to use a github action to build and deploy this thing so it would get deployed on every push like before.
It was straight-forward enough – just build with nix develop --command jekyll build (using the dev shell) and rsync it to the droplet.
But it occurred to me: I don’t actually like that. I’d like deploys to be an explicit action.
Easy enough: we’ll just build a deploy command into the blog repo’s dev shell flake:
let
pkgs = nixpkgs.legacyPackages.${system};
rubyEnv = pkgs.ruby.withPackages (ps: with ps; [
jekyll webrick kramdown-parser-gfm
jekyll-paginate jekyll-gist jemoji
]);
deploy = pkgs.writeShellScriptBin "deploy" ''
set -euo pipefail
cd "$(${pkgs.git}/bin/git rev-parse --show-toplevel)"
if ! ${pkgs.git}/bin/git diff --quiet HEAD; then
echo "Working tree is dirty:"
${pkgs.git}/bin/git status --short
echo
read -r -p "Deploy uncommitted changes? [y/N] " reply
case "$reply" in
[yY]|[yY][eE][sS]) ;;
*) echo "Aborted."; exit 1 ;;
esac
fi
${rubyEnv}/bin/jekyll build
${pkgs.rsync}/bin/rsync -avz --delete \
-e "${pkgs.openssh}/bin/ssh -i $HOME/.ssh/blog_deploy -o IdentitiesOnly=yes" \
_site/ deploy@0x3c.net:/srv/www/blog/
echo "deployed to https://ojensen.net/"
'';
in
{
default = pkgs.mkShell {
packages = [ rubyEnv deploy ];
};
});
Now, when I’m in the ojensen.net dev shell, I can just run deploy to deploy.
Magic!
System Maintenance
So… this is a weird thing to get excited about. But it’s also really cool.
This DigitalOcean droplet has 1gb of RAM and a teensy weensy little CPU, which is… not a lot. But a neat thing is that since the system definition is part of my Flake that lives on my computer, I can build/run updates there, and then just push the closure to the droplet over SSH.
Basically you just throw a --target-host option into your rebuild command and you’re off to the races:
nixos-rebuild switch --flake .#web --target-host root@0x3c.net
In real life, I have a whole rebuild script which makes a few paved roads for me:
rebuild # apply changes to my laptop
rebuild -u # apply changes and install updates to my laptop
rebuild web # apply changes to the droplet
rebuild -u web # you get the idea
In all cases, all the “work” is done locally, and with the web targets, the closure is then pushed over ssh.
Again: Magic!
CISO of Regrello. PhD from and previously Adj. Assoc. Professor at the Universitty of Texas at Austin. Passionate about security and privacy.