Spack package manager

Setting up Spack in the user space

Spack has many design goals, one of which is to be usable both by systems administrators and by users. Users can thus use CHPC installed Spack to build their own programs, stored in their own user space (e.g. a home directory). Spack installs all the applications that it built in a repository. CHPC maintains such a repository of the programs that we install. Our users can use this repository, where they don't have write permissions, as an “upstream” to tell Spack to use the applications, thus reducing the storage need for their own repository in their own user space, and also save time in not having to build the dependencies that we have already built.

First create user setting files and user Spack directories, by running the following script: /uufs/chpc.utah.edu/sys/installdir/spack/user/setup-user-spack.sh. This needs to be done only once.

The steps taken in this script are detailed in our GitHub page. Once this is done, load the Spack module, module load spack. This needs to be done every time one wants to use Spack.

Building a program with Spack in the user space

Most program have dependencies – programs or libraries that are necessary to build the program. Because we have set up Spack to use the CHPC repository as an upstream, many of the common dependencies have been installed. The spack spec command, which asks Spack to generate a list of dependencies needed to build a program, can show what dependencies are available and which ones are missing. For example, for the program Octave – an open source replacement for Matlab, we can:

$ spack spec -I octave%gcc@8.5.0 target=nehalem 
Input spec --------------------------------- 
octave arch=linux-None-nehalem 
Concretized -------------------------------- 
[-]octave@6.2.0%gcc@8.5.0~arpack~curl~fftw~fltk~fontconfig~freetype~gl2ps~glpk~gnuplot~hdf5~jdk~llvm~magick~opengl~qhull~qrupdate~qscintilla~qt+readline~suitesparse~zlib arch=linux-rocky8-nehalem
[^] ^intel-oneapi-mkl@2021.4.0%gcc@8.5.0~ilp64+shared threads=none arch=linux-rocky8-nehalem 
[^] ^cpio@2.13%gcc@8.5.0 arch=linux-rocky8-nehalem
[^] ^pcre@8.44%gcc@8.5.0~jit+multibyte+utf arch=linux-rocky8-nehalem
[^] ^pkgconf@1.7.3%gcc@8.5.0 arch=linux-rocky8-nehalem
[^] ^readline@8.0%gcc@8.5.0 arch=linux-rocky8-nehalem
[^] ^ncurses@6.2%gcc@8.5.0~symlinks+termlib arch=linux-rocky8-nehalem

The [^] denotes packages used from upstream, [-] packages that are missing in the upstream, [+] denotes a package installed in the user repository.

Now we can build the new program which will then store the build in $HOME/spack/local/builds:

spack install octave%gcc@8.5.0 target=nehalem 

Notice that we told Spack to buid the program for the CPU target nehalem. That is the lowest common denominator for all the CPUs that CHPC clusters and workstations use. We will describe building optimized executables for specific CPUs in a future article.

We have set up Spack to not generate environment module files for programs automatically – that would result in a large number of modules for all the dependencies - therefore we need to generate the module after installation. The first step is to find the right Octave version among all the available builds – there will be some from the CHPC repository as well:

$ spack find -pl octave 
... 
ahjj6xh octave@6.2.0 
/uufs/chpc.utah.edu/common/home/u0123456/spack/local/builds/linux-rocky8-nehalem/gcc-8.5.0/octave-6.2.0-ahjj6xhs46mk4zycdq7irpnwhyojxp46 
... 

Look for the one with the path in your home. Overall, the spack find command is good for querying what programs are installed with Spack. Notice the unique string, ahjj6xh in this case, at the start of the line, called the hash. The hash uniquely identifies the installed package. Another unique identification would be through the complete specification of the build, though this can get long. You can see it by adding -dv options to the spack find command. Using the hash to build the module file:

 $ spack module lmod refresh /ahjj6xh 

We approve to proceed and the module file gets generated.

To make the new Octave module active, we need to first load the gcc/8.5.0 compiler module that was used to build it, and then also add the module path with the user built programs:

module load gcc/8.5.0 
module use $HOME/spack/local/modules/linux-rocky8-x86_64/Compiler/linux-rocky8-nehalem/gcc/8.5.0 

Now we should be able to see the new Octave module with:

$ module avail 
... 
/uufs/chpc.utah.edu/common/home/u0123456/spack/local/modules/linux-rocky8-x86_64/Compiler/linux-rocky8-nehalem/gcc/8.5.0
 octave/6.2.0 
… 

Note: be aware of the disk space that the Spack built packages take, especially if you have the default 50 GB disk quota. You can use the command mydiskquota to display current usage. To see how much is used by spack, run du -hs $HOME/spack/local.