Building Singularity containers locally

Preparing your computer

Singularity runs natively on Linux, if you have a root on your Linux machine, you can install Singularity natively.

On a Windows machine, we recommend to install a Linux Virtual Machine, in which we can then install Singularity for Linux. This is detailed below.

Similarly, on MacOS, Singularity website  provides information how to install a Linux Virtual Machine using Vagrant.

Installing Linux VM on a Windows machine

In order to make the Linux VM simple, we will be using a few command line tools. Namely GIT Bash, Virtual Box and Vagrant.

The whole process is nicely described at the Sylabs page, but, we'll outline it below as well with a few specifics. So, follow their recipe along with our details below.

1. Download and install the prerequisites.

Namely, GIT Bash, which is a part of Git for Windows , Virtual Box and Vagrant. All installs are GUI based and quite straightforward.

Once installed, right click on the desktop and you should see Git Bash here ..., click on that and a command shell window opens. This is a bash shell (with MINGW Linux running underneath). We will be using it to do the rest of the VM work.

2. Set up Linux Ubuntu Virtual Machine

We can do this as any Windows user in Git Bash, but, if you want to sync a Windows folder to your Linux VM, you need to start the Git Bash as an administrator. Go to the search bar at the bottom left WIndows pane, type Git Bash, right click it and choose "Run as Administrator". Then run the first few commands (the $ means command prompt):

$ pwd
/c/Users/Martin

Notice that we are in your User's directory, on drive C:. It is preferable to organize files and directories in certain fashion, so, we want to put all our container files to some different directory, e.g.:

$ mkdir -p /c/work/containers/vm_singularity
$ cd /c/work/containers/vm_singularity

Now we are ready to prepare the VM. First we add, initialize and download the base image:

$ vagrant init sylabs/singularity-3.1-centos-7-64 --box-version 20190228.0.0

Search for the most recent Singularity Vagrant files at Sylabs Vagrant repository and change the name/version accordingly.

This prepares a file called Vagrantfile, which defines the Ubuntu VM setup.

If we want to sync folders with our Windows host, edit Vagrantfile and add the following to it, right before the last line which says end:

  config.vm.synced_folder "vmfiles", "/vagrant", type: "smb"

This says Vagrant to make directory /vagrantfile in the Ubuntu VM visible as vmfiles directory in the current directory (/c/work/containers/vagrant_ubuntu14/vmfiles) on your Windows host. We also need to create this directory:

$ mkdir vmfiles

We also need to make sure that the Virtual Box Guest Additions are installed and up to date (guest additions are extra packages for the VM guest - Ubuntu in our case). This can be done by installing a Vagrant plugin:

$ vagrant plugin install vagrant-vbguest

Now we are ready to start the Ubuntu VM:

$ vagrant up

If you set up the folder sync, you will be prompted for your Windows username and password.

If your Guest Additions are not present or out of sync with the Virtual Box, you'll see a lot of messages but the VM will eventually start up. If you get any errors, make sure to update Virtual Box and Vagrant to the latest versions.

Now we can ssh to the VM, and see Singularity:

$ vagrant ssh
$ which singularity
/usr/local/bin/singularity

That should be it, you should be able to call Singularity command either as an user (singularity ) or as a root (sudo singularity).

To shut down the VM when done, exit from the ssh session halt the VM by:

$ exit
$ vagrant halt

Building a Singularity container

Singularity website offers a host of tutorials on how to use Singularity and how to build containers. We will here focus on building an Ubuntu Linux based container, because in our experience most of harder-to-install packages are developed on Ubuntu and thus the easiest to deploy on Ubuntu.

There are two basic components of building a Singularity container, the first is to run the singularity command to provision the container and to bootstrap it (install the OS and required programs). The second is to provide an information what OS to install and how. This can be done either through a definition file, or through importing other container definition, such as Docker. Different ways to import a container are described in detail on Singularity documentation.

In our simple demonstration here, we will build the container based on a bootstrap definition file. The container we will build should serve as a good base for Ubuntu based application deployment. For more details on how to include more complicated pieces such as GPU and MPI support or specific Python based programs, and for some tips and tricks, see the CHPC Singularity containers github page.

One important detail on deploying locally built container at CHPC machines is to create mount points to CHPC file spaces, so that we can use our home directory and scratch file systems in the container. To do that, put the following at the end of the %post section of the bootstrap definiton file:

mkdir /uufs /scratch

 The second command to create the container is:

sudo singularity build ubuntu.sif ubuntu.def

There will be a lot of output as the OS and programs are being installed. Once that is done, the container is ready.

To check that the container is working we can shell into it:

singularity shell -s /bin/bash ubuntu.sif

If one needs to install additional packages, we need to build the "sandbox" container which allows writing, and then shell to the container as sudo with write permissions (-w) on the container:

sudo singularity build --sandbox development ubuntu.def
sudo singularity shell -w -s /bin/bash development

After interactively installing packages, copy/paste the installation commands to the %post section of the definition file (ubuntu.def in this case), so that they can be automatically included in the future container build. In general, when installing new application in a container, we will iterate between running the install commands interactively and putting them to the bootstrap definition file, until the automatic build works correctly.

Once everything in the container works, we can build the production sif container file by:

sudo singularity build ubuntu_production.sif development/

Deploying Singularity container at CHPC clusters

Once we have a container built to our liking, we need to move it to the CHPC systems. One can use their favorite scp program (WinSCP, CyberDuck) - if the folder sync between the host and VM guest were set up, or scp the container directly from the Linux VM (or native Linux machine) you are running:

scp ubuntu_production.sif u0123456@kingspeak.chpc.utah.edu:~/my_containers

To test the container, we ssh to a CHPC machine and run:

ml singularity
singularity shell ~/my_containers/ubuntu_production.sif