Nix Flake Architecture in Practice

Starting fresh

Some options for trying to combat the issues of using one’s current environment

1. Use environment variables like $VIM, $VIMRUNTIME, & $XDG_{CONFIG,STATE,DATA}_HOME or some sort of

   $ vim -Nu <(cat << EOF
   filetype off
   set rtp+=~/.vim/bundle/$MY_PLUGIN
   filetype plugin indent on
   syntax enable
   EOF)

2. A full VM like QEMU or VirtualBox (ala Vagant) with just Vim
3. Containers (e.g. Docker, Podman) using some Linux image

RUN ["apt-get", "update"]
RUN ["apt-get", "install", "-y", "vim"]

Enter our lighter, reproducible Option N: Nix

Nix + Nixpkgs is a FOSS pairing for making “reproducible, declarative, reliable” software. For our use case, Nix can fetch us a clean copy of Vim & any minimal plugins we depend on for testing, handle our local changes, & offer distribution options for our new plugin.

Echoing the most basic vim-hello-world, a Vim plugin

Since this isn’t a Vim plugin tutorial per se, here’s the most basic echo that can be a stand-in for any script or program.

$ cd $PROJECT
$ mkdir -p plugin
$ echo "echo 'Hello, world!'" >> plugin/hello_world.vim

Beginning with a flake.nix

With flakes installed, it’s time to create our $PROJECT’s flake.nix

$ cd $PROJECT
$ nix flake init
$ cat flake.nix
{
  description = "A very basic flake";

  outputs = { self, nixpkgs }: {

    packages.x86_64-linux.hello = nixpkgs.legacyPackages.x86_64-linux.hello;

    packages.x86_64-linux.default = self.packages.x86_64-linux.hello;

  };
}

This base can get us started, but it’s in need of tweaking for our projects.

Supporting forAll architectures

Vim (& Neovim) support on a lot of platforms, & we should also support those users (plus Nix will require us to specify these platforms as we can see in the nix flake init output). There are a lot of ways to handle multiple hardware architecture support in Nix, but we will be writing our own utility functions to avoid introducing unnecessary dependencies. When you add to inputs, downstream users will necessarily need to pull our dependencies even if it’s just for a single function so be careful & audit your inputs. It’s never worth it to pull in a Nix utility library for what is effectively a for loop over our supported systems. Here’s one way to write that loop:

# …assumes nixpkgs is in scope such as a flake.nix’s outputs
forAllSystems = nixpkgs.lib.genAttrs nixpkgs.lib.systems.flakeExposed;

nixpkgsFor = forAllSystems (system: import nixpkgs {
  inherit system;
  config = { };
  overlays = [ ];
});

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
  };
}

Our plugin

Many languages have handy helpers in Nixpkgs to use existing community tooling as well such as ocamlPackages.buildDuneProject or rustPlatform.buildRustPackage. In the style of many public Vim plugins, our plugin will merely sit in repository’s root directory—no runtimepath switching, no additional build steps. For Vim, we can pull one of those helper in vimUtils.buildVimPlugin (formerly known as vimUtils.buildVimPluginFrom2Nix). This function will behave similarly to how many Vim package managers behave—which is perfect for our basic hello_world. We will add our first package, & we’ll use the name default for it.

{
  description = "A Hello World demo plugin for Vim";

  # not default, but `inputs` pun (`@`) of our input records argument
  outputs = { self, nixpkgs, ... }@inputs:
    let
      forAllSystems = nixpkgs.lib.genAttrs nixpkgs.lib.platforms.unix;

      nixpkgsFor = forAllSystems (system: import nixpkgs {
        inherit system;
        config = { };
        overlays = [ ];
      });
    in
    {
      packages = forAllSystems (system:
        let pkgs = nixpkgsFor."${system}"; in {
          default = pkgs.vimUtils.buildVimPlugin {
             name = "hello-world";
             namePrefix = "vim-";
             src = ./.;
           };
        });
    };
}

Our modified Vim editor

Since we are working from that clean-slate plan, we need to make clean Vim package. We will call this vim-mod to remind ourselves that it’s not the one on the system but the one we have modified. Let’s add it to the packages (leaving out sections we’ve seen to keep it short):

{
  outputs = { self, nixpkgs, ... }@inputs:
    let
      # …
    in
    {
      packages = forAllSystems (system:
        let pkgs = nixpkgsFor."${system}"; in {
          vim-mod = pkgs.vim_configurable.customize {
            vimrcConfig = {
              packages.myPlugin = with pkgs.vimPlugins; {
                start = [
                  # list plugins we need to be compatible with such as:
                  # ale
                  # powerline
                  # …but also include our plugin
                  self.packages."${system}".default
                ];
              };
              customRC = ''
                " A VimL RC file for our default settings for testing

                syntax on

                " Maybe we need flags for our plugin too here
              '';
            };
          };
        });
    };
}

Inspecting our output

By running nix-build with our directory as the entry followed with a # & our package name, vim-mod, we’ll instruct the Nix to build just that. Keep in mind that since our modified Vim has our plugin as an input, it will be built as well. After it’s built we can execute our modified Vim.

$ nix build .#vim-mod
$ tree ./result
result/
├── bin
│   ├── eview -> /nix/store/$HASH1-vim-bin/bin/eview
│   ├── evim -> /nix/store/$HASH1-vim-bin/bin/evim
│   ├── ex -> /nix/store/$HASH1-vim-bin/bin/ex
│   ├── gview -> /nix/store/$HASH1-vim-bin/bin/gview
│   ├── gvim -> /nix/store/$HASH1-vim-bin/bin/gvim
│   ├── gvimdiff -> /nix/store/$HASH1-vim-bin/bin/gvimdiff
│   ├── gvimtutor -> /nix/store/$HASH2-vim-full-9.0.1562/bin/gvimtutor
│   ├── rgview -> /nix/store/$HASH1-vim-bin/bin/rgview
│   ├── rgvim -> /nix/store/$HASH1-vim-bin/bin/rgvim
│   ├── rview -> /nix/store/$HASH1-vim-bin/bin/rview
│   ├── rvim -> /nix/store/$HASH1-vim-bin/bin/rvim
│   ├── vi -> /nix/store/$HASH1-vim-bin/bin/vi
│   ├── view -> /nix/store/$HASH1-vim-bin/bin/view
│   ├── vim -> /nix/store/$HASH1-vim-bin/bin/vim
│   ├── vimdiff -> /nix/store/$HASH1-vim-bin/bin/vimdiff
│   ├── vimtutor -> /nix/store/$HASH2-vim-full-9.0.1562/bin/vimtutor
│   └── xxd -> /nix/store/$HASH2-vim-full-9.0.1562/bin/xxd
└── share -> /nix/store/$HASH2-vim-full-9.0.1562/share
$ result/bin/vim
result/bin/vim
Hello, world!
Press ENTER or type command to continue

Nix’s checks

checks can be used to set up automated tests, linters, formatters, or really anything we want to assert about our code base. While we can implement as many checks as we want, for this demonstration we only set up vint, a Vim Script linter.

{
  outputs = { self, nixpkgs, ... }@inputs:
    let
      # …
    in
    {
      checks = forAllSystems (system:
        let pkgs = nixpkgsFor.${system}; in {
          vint = pkgs.runCommand "vint-lint"
            {
              src = ./.;
              nativeBuildInputs = with pkgs; [ vim-vint ];
            } ''
              set -euo pipefail
              vint --style-problem $src/**/*.vim | tee $out
            '';
        });
    };
}

$ nix flake check

Uses for devShells

A lot of folks introduce Nix flakes via the devShells more akin to a replacement for the asdf version manager, a way to get specific versions of tooling into your environment or $PATH. That can be a good place to start, however, when a project is fully beflaked with Nix, there is less need to run anything in the mutable environment of devShells rather than the immutable environment of Nix derivations. Where devShell usage can help is is when working with tools outside your project’s environment. Let’s say we are using our system’s version of (Neo)Vim to edit plugin because it’s been fully modfied for our needs, & we’d like more immediate feedback about our linting errors from vint. In this scenario we would want our editor to have access to vint for a plugin to execute displaying warnings automatically. Because we don’t want version conflicts between nix flake check & the version of vint our editor is using, we can get put the exact same version onto our path for our editor to hook into.

{
  outputs = { self, nixpkgs, ... }@inputs:
    {
      # …

      devShells = forAllSystems (system:
        let pkgs = nixpkgsFor.${system}; in {
          default = pkgs.mkShell {
            buildInputs = with pkgs; [
              vint
            ];
          };
        });
    };
};

$ nix develop

Distributing an app via overlays

Our vim-hello-world is working as expected, but we want others to be able to use it too! Overlays are an a easy way for other Nix enjoyers to have our software directly from the flake.nix in our project root, often fetched from a repository, which will modify their nixpkgs. To make using our plugin as painless as possible, all we need to do is append our plugin to the user’s Nixpkgs’s vimPlugins attrset which they can use in their personal Vim setup. You can use a similar method to append or override any sofware in the user’s environment.

{
  outputs = { self, nixpkgs, ... }@inputs:
    let
      # …
    in
    {
      overlays = {
        # final: the output of Nixpkgs after being lazily evaluated
        # prev: the input of previous Nixpkgs
        vimPlugins = final: prev: {
          vimPlugins = prev.vimPlugins // {
            ${name} = self.packages.${prev.system}.vimPlugins.${name};
          };
        };
      };
    };
}

Now users can import our Flake into their own via Input!

{
  input = {
    # …
    vim-hello-world = {
      url = "protocol://repository_or_tarball";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { self, nixpkgs, ... }@inputs: {
    # …wherever you do your overlays…
    overlays = [
      inputs.vim-hello-world.overlays.vimPlugins
    ];
  };
}

Distributing as a tarball

Unfortunately, not everyone is using Nix or Nix+flakes for their text editor management so we’ll need an alternative (besides many would prefer upstreamed packages in Nixpkgs to overlays). While it may be common to just use the whole version control repository for editor plugins (and in many cases this is simpler/recommended for maintenance), it may not be the best experience for your users. A basic, non-trivial plugin repository is likely going to contain files the user will never interact with such as settings files, CI & other build scripts, images & other media for HTML documentation, source files for compilation to Lua in the case of Neovim, as well as the entire repositories history. Rather than telling users “just go install Fennel, append its compiler to your $PATH, & add this build step command to your plugin manager’s config & it should work”, we could instead build it, strip it, tree-shake it, minify it, etc. for the user. To do this, it would be time to break out into a ‘real’ Nix derivation—which is out of scope since it’s not specific to flakes, but the basic idea would be…

{
  outputs = { self, nixpkgs, ... }@inputs:
    let
      # …
    in
    {
      packages = forAllSystems (system:
        let pkgs = nixpkgsFor."${system}"; in {
          # The ‘kind’ of derivation can vary depending on language &
          # complexity, but all boil down to mkDerivation. The
          # `trivial-builders` can get you pretty far in many cases
          default = pkgs.mkDerivation {
            pname = "vim-hello-world";
            version = "0.0.1";
            src = ./.;
            # …& add some buildPhase, etc.
          };
        });
    };
}

We could build our package like before locally or in CI, tar the ./result folder (following links!), & push a build artifact to wherever you plan to host tarballs for users.

{
 description = "A Hello World demo plugin for Vim";

 outputs = { self, nixpkgs, ... }@inputs:
   let
     forAllSystems = nixpkgs.lib.genAttrs nixpkgs.lib.platforms.unix;

     nixpkgsFor = forAllSystems (system: import nixpkgs {
       inherit system;
       config = { };
       overlays = [ ];
     });
   in
   {
     overlays = {
       vimPlugins = final: prev: {
         vimPlugins = prev.vimPlugins // { ${name} = self.packages.${prev.system}.${name}; };
       };
     };

     packages = forAllSystems (system:
       let pkgs = nixpkgsFor."${system}"; in {
         default = pkgs.vimUtils.buildVimPlugin {
           name = "hello-world";
           namePrefix = "vim-";
           src = ./.;
           meta.license = pkgs.lib.licenses.isc;
         };

         vim-mod = pkgs.vim_configurable.customize {
           name = "vim";
           vimrcConfig = {
             packages.myplugins = with pkgs.vimPlugins; {
               start = [
                 # list plugins we need to be compatible with such as:
                 # ale
                 # powerline
                 # …but also include our plugin
                 self.packages."${system}".default
               ];
               # optional plugins to not be automatically loaded
               opt = [ ];
             };
             customRC = ''
               " A VimL RC file for our default settings for testing

               syntax on

               " Maybe we need flags for our plugin too here
             '';
           };
         };
       });

     apps = forAllSystems (system:
       let pkgs = nixpkgsFor.${system}; in {
         default = self.apps.${system}.vim-mod;

         vim-mod = {
           type = "app";
           program = "${self.packages.${system}.vim-mod}/bin/vim";
         };
       });

     checks = forAllSystems (system:
       let pkgs = nixpkgsFor.${system}; in {
         vint = pkgs.runCommand "vint-lint"
           {
             src = ./.;
             nativeBuildInputs = with pkgs; [ vim-vint ];
           } ''
             set -euo pipefail
             vint --style-problem $src/**/*.vim | tee $out
           '';
       });

     devShells = forAllSystems (system:
       let pkgs = nixpkgsFor.${system}; in {
         default = pkgs.mkShell {
           buildInputs = with pkgs; [
             vint
           ];
         };
       });
   };
}

$ nix repl
Welcome to Nix 2.17.0. Type :? for help.

nix-repl> :l flake.nix

Exercises for reader if you would like to continue hacking on Vim plugins

• Set up nix flake check with a Vim testing framework like vader.vim.
• Use various Nixpkgs pinned version to test older versions of Vim
• Set up a Neovim configuration + nvim-mod to test our vim-hello-world VimL plugin in both editors
• Try a lua/ folder & write a basic print("Hello, world!") from Neovim
• Build a Neovim plugin in a different language (could be Haskell, OCaml, Rust, JavaScript, Python, et al.) or try a compile-to-Lua options, like Fennel or LunarML or MoonScript, to build Lua.