Posted on May 18, 2021 by Nick Sigulya

Nix: reproducible build

Creating a reproducible build of our Haskell project using Nix

Today we continue our series of posts about Nix and the way we use it at Typeable.

You can find our first post about Nix basics here.

As we like Haskell very much and often use it for development, we’ll use it for our sample application, but you don’t need to have any knowledge of Haskell. After some small fixes, the code shown in the examples can also be used to build projects in other languages.

You can find the complete code for this post in this Github repository.

Problem

Building and CI are some of the biggest problems in software development. Support for build infrastructure often devours an incredible amount of resources. We will use Nix and try to make the situation at least a little more tolerable, even if it’s not possible to amend it. Nix allows us to achieve the reproducibility of builds in our projects, portability between different OSs, unification of components written in different languages, and so on.

Our application

So, let’s start with the application we want to build. In our case, it will be a simple program in Haskell displaying the message “Hello, world”.

Our Main.hs:

module Main where

main :: IO ()
main = putStrLn "Hello, World!"

To build a project without Nix, we use stack utility (for more details on the utility see here). The description of our project for stack requires stack.yaml file containing the list of our packages and the resolver. The latter is a stable slice of Hackage, the package base for Haskell which guarantees that all packages are built and get on with each other (NB: such slices are very much needed in other languages): ).

stack.yaml:

resolver: lts-17.11

packages:
- hello-world

You can find the recipe for building the specific package in hello-world.cabal:

cabal-version:      2.4
name:               hello-world
version:            1.0
synopsis:           Hello World
license:            MIT
license-file:       LICENSE
author:             Nick

executable hello-world
    main-is:          Main.hs
    build-depends:    base >= 4 && < 5
    hs-source-dirs:   src
    default-language: Haskell2010
    ghc-options:      -Wall -O2

Our Nix code will use these two files to find the information on what exactly and how it should build. By way of an experiment, we can check if our code really runs and works:

$ stack run hello-world
Hello, World!

Come to the dar^Wnix side, we have cookies!

Though Stack by itself is a great tool used to build projects in Haskell, it lacks many capabilities. There is the library haskell.nix developed by IOHK to build Haskell programs for Nix. This is what we’re going to use here. To begin with, let’s make sure that our project is built using Nix.

Haskell.nix allows using several lines to convert all data on our project build from .cabal-files and stack.yaml into derivationfor Nix.

nix/stackyaml.nix:

{
  # We import the latest version of haskell.nix from GitHub and use it to initialize Nixpkgs.
  haskellNix ? import (builtins.fetchTarball "https://github.com/input-output-hk/haskell.nix/archive/b0d03596f974131ab64d718b98e16f0be052d852.tar.gz") {}
  # Here we use the latest stable version of Nixpkgs. Version 21.05 is coming soon :)
, nixpkgsSrc ? haskellNix.sources.nixpkgs-2009
, nixpkgsArgs ? haskellNix.nixpkgsArgs
, pkgs ? import nixpkgsSrc nixpkgsArgs
}:

let
  # We create a project based on stack. Function cabalProject is available for Cabal projects.
  project = pkgs.haskell-nix.stackProject {
    name = "hello-world";

    # Derivation with the project source code.
    # To build the project, function cleanGit copies only the files available in our git-repository.
    src = pkgs.haskell-nix.haskellLib.cleanGit {
      name = "hello-world";

      # src parameter must refer to the root directory containing stack.yaml.
      src = ../.;

      # keepGitDir keeps .git directory when creating the build.
      # This can be of use, for example, to insert the commit hash into the code.
      keepGitDir = true;
    };

    # You can indicate the build parameters in the modules parameter either for all modules at once or for each module individually.
    modules = [{
      # doCheck is responsible for running unit tests when building the project, including the ones contained in all dependencies.
      # Here we want to avoid this, which is why the best thing is to define this parameter as false and enable it only for the required
      # packages.
      doCheck = false;

      # Let’s add -Werror flag for our Hello World.
      packages.hello-world.components.exes.hello-world.ghcOptions = [ "-Werror" ];
    }];
  };

# To the outside of this file we set out the project -- our project, and pkgs -- nixpkgs slice we’ll use further.
in { inherit project; inherit pkgs; }

Let’s check whether it’s now possible to build our project using Nix. To do this, it’s enough to use nix build command. As usual, the symbolic link result containing the resulting build will be created in the current directory.

$ nix build project.hello-world.components.exes
$ ./result/bin/hello-world
Hello, World!

Great! The little magic shown above has provided a completely reproducible build of our project, including all system dependencies. Let’s keep going!

Dockerfile? What Dockerfile?

It’s 2021 and many companies use Docker to deploy and launch services. Typeable is not an exception here. nixpkgs include very convenient tools named dockerTools designed to build containers. For more details on their capabilities open the link. I’m just going to show how we pack our code in containers using these tools. You can see the complete code in the file nix/docker.nix.

For a start, we’ll need the source container where we’ll place everything we need. Nix allows building a container completely from scratch without any unneeded components, but this approach is not always convenient. Sometimes, especially in abnormal situations, you have to get into the container manually using the command line. That’s why we use CentOS here.

sourceImage = dockerTools.pullImage {
  imageName = "centos";
  imageDigest = "sha256:e4ca2ed0202e76be184e75fb26d14bf974193579039d5573fb2348664deef76e";
  sha256 = "1j6nplfs6999qmbhjkaxwjgdij7yf31y991sna7x4cxzf77k74v3";
  finalImageTag = "7";
  finalImageName = "centos";
};

For those who’ve ever worked with Docker everything is obvious here. We tell Nix what image from the public Docker Registry we’d like to use and specify that from now on we’ll refer to it as sourceImage.

The image itself is built using the function buildImage available in the dockerTools. This function has quite a few parameters and it’s often easier to write a wrapper for it, which is exactly what we’re going to do:

makeDockerImage = name: revision: packages: entryPoint:
  dockerTools.buildImage {
    name = name;
    tag = revision;
    fromImage = sourceImage;
    contents = (with pkgs; [ bashInteractive coreutils htop strace vim ]) ++ packages;
    config.Cmd = entryPoint;
  };

Our function makeDockerImage accepts four parameters: the container name, its version (at Typeable, we usually use a commit hash from git as the tag), the packages we’d like to enable and the entry point used at the container startup. Inside it we refer to the CentOS image as the basis (fromImage), plus add some utilities which can be extremely useful in emergencies.

Finally, let’s create the image with our marvellous application.

hello-world = project.hello-world.components.exes.hello-world;
helloImage = makeDockerImage "hello"
  (if imageTag == null then "undefined" else imageTag)
  [ hello-world ]
  [ "${hello-world}/bin/hello-world"
  ];

As the first step, we create an alias for the package we need so that we don’t need to write project.hello-world... everywhere. After that, we create the image of the container with hello-world package by calling the function makeDockerImage created previously. Parameter imageTag passed from the outside will be indicated as the tag; if nothing was passed, “undefined” will be indicated.

Let’s check the build:

$ nix build --argstr imageTag 1.0 helloImage
[4 built, 0.0 MiB DL]

 $ ls -l result
lrwxrwxrwx 1 user users 69 May 11 13:12 result -> /nix/store/56qqhiwahyi46g6mf355fjr1g6mcab0b-docker-image-hello.tar.gz

In a couple of minutes or even sooner we’ll get the symbolic link result leading to our finished image. Check again whether everything works as intended.

$ docker load < result
76241b8b0c76: Loading layer [==================================================>]  285.9MB/285.9MB
Loaded image: hello:1.0
$ docker run hello:1.0
Hello, World!

Conclusion

The result is that we’ve managed to create a reproducible build of our Haskell project using a comparatively small amount of code. The same can be done for projects in other languages after replacing haskell.nix with something else: nixpkgs has built-in tools for C/C++, Python, Node and other popular languages.

In the next post, I’ll tell you about the frequent issues occurring when you work with Nix. Stay tuned!

You may also find interesting our previous posts about stackage2nix tool:

Want to know more?
Get in touch with us!
Contact Us

Privacy policy

Last updated: 1 September 2021

Typeable OU ("us", "we", or "our") operates https://typeable.io (the "Site"). This page informs you of our policies regarding the collection, use and disclosure of Personal Information we receive from users of the Site.

We use your Personal Information only for providing and improving the Site. By using the Site, you agree to the collection and use of information in accordance with this policy.

Information Collection And Use

While using our Site, we may ask you to provide us with certain personally identifiable information that can be used to contact or identify you. Personally identifiable information may include, but is not limited to your name ("Personal Information").

Log Data

Like many site operators, we collect information that your browser sends whenever you visit our Site ("Log Data").

This Log Data may include information such as your computer's Internet Protocol ("IP") address, browser type, browser version, the pages of our Site that you visit, the time and date of your visit, the time spent on those pages and other statistics.

In addition, we may use third party services such as Google Analytics that collect, monitor and analyze this ...

Cookies

Cookies are files with small amount of data, which may include an anonymous unique identifier. Cookies are sent to your browser from a web site and stored on your computer's hard drive.

Like many sites, we use "cookies" to collect information. You can instruct your browser to refuse all cookies or to indicate when a cookie is being sent. However, if you do not accept cookies, you may not be able to use some portions of our Site.

Security

The security of your Personal Information is important to us, so we don't store any personal information and use third-party GDPR-compliant services to store contact data supplied with a "Contact Us" form and job applications data, suplied via "Careers" page.

Changes To This Privacy Policy

This Privacy Policy is effective as of @@privacePolicyDate​ and will remain in effect except with respect to any changes in its provisions in the future, which will be in effect immediately after being posted on this page.

We reserve the right to update or change our Privacy Policy at any time and you should check this Privacy Policy periodically. Your continued use of the Service after we post any modifications to the Privacy Policy on this page will constitute your acknowledgment of the modifications and your consent to abide and be bound by the modified Privacy Policy.

If we make any material changes to this Privacy Policy, we will notify you either through the email address you have provided us, or by placing a prominent notice on our website.

Contact Us

If you have any questions about this Privacy Policy, please contact us.