Simple package installation¶

Installing the default version of a package is simple. This will install the latest version of the mpileaks package and all of its dependencies:

$ spack install mpileaks

Custom versions & configurations¶

Spack allows installation to be customized. Users can specify the version, build compiler, compile-time options, and cross-compile platform, all on the command line.

# Install a particular version by appending @
$ spack install mpileaks@1.1.2
# Specify a compiler (and its version), with %
$ spack install mpileaks@1.1.2 %gcc@4.7.3
# Add special compile-time options by name
$ spack install mpileaks@1.1.2 %gcc@4.7.3 debug=True
# Add special boolean compile-time options with +
$ spack install mpileaks@1.1.2 %gcc@4.7.3 +debug
# Add compiler flags using the conventional names
$ spack install mpileaks@1.1.2 %gcc@4.7.3 cppflags="-O3 -floop-block"
# Cross-compile for a different micro-architecture with target=
$ spack install mpileaks@1.1.2 target=icelake

Users can specify as many or few options as they care about. Spack will fill in the unspecified values with sensible defaults. The two listed syntaxes for variants are identical when the value is boolean.

Customize dependencies¶

Spack allows dependencies of a particular installation to be customized extensively. Suppose that hdf5 depends on openmpi and indirectly on hwloc. Using ^, users can add custom configurations for the dependencies:

# Install hdf5 and link it with specific versions of openmpi and hwloc
$ spack install hdf5@1.10.1 %gcc@4.7.3 +debug ^openmpi+cuda fabrics=auto ^hwloc+gl

Non-destructive installs¶

Spack installs every unique package/dependency configuration into its own prefix, so new installs will not break existing ones.

Packages can peacefully coexist¶

Spack avoids library misconfiguration by using RPATH to link dependencies. When a user links a library or runs a program, it is tied to the dependencies it was built with, so there is no need to manipulate LD_LIBRARY_PATH at runtime.

Creating packages is easy¶

To create a new packages, all Spack needs is a URL for the source archive. The spack create command will create a boilerplate package file, and the package authors can fill in specific build steps in pure Python.

For example, this command:

$ spack create https://ftp.osuosl.org/pub/blfs/conglomeration/libelf/libelf-0.8.13.tar.gz

creates a simple python file:

from spack.package import *
class Libelf(AutotoolsPackage):


    """FIXME: Put a proper description of your package here."""


    # FIXME: Add a proper url for your package's homepage here.


    homepage = "https://www.example.com"


    url = "https://ftp.osuosl.org/pub/blfs/conglomeration/libelf/libelf-0.8.13.tar.gz"


    # FIXME: Add a list of GitHub accounts to


    # notify when the package is updated.


    # maintainers("github_user1", "github_user2")


    version("0.8.13", sha256="591a9b4ec81c1f2042a97aa60564e0cb79d041c52faa7416acb38bc95bd2c76d")


    # FIXME: Add dependencies if required.


    # depends_on("foo")


    def configure_args(self):


        # FIXME: Add arguments other than --prefix


        # FIXME: If not needed delete this function


        args = []


        return args

It doesn't take much python coding to get from there to a working package:

from spack.package import *
class Libelf(AutotoolsPackage):


    """libelf lets you read, modify or create ELF object files in an


    architecture-independent way. The library takes care of size


    and endian issues, e.g. you can process a file for SPARC


    processors on an Intel-based system. Note: libelf is no longer


    maintained and packages that depend on libelf should migrate to


    elfutils."""


    # The original homepage no longer exists, but the tar file is


    # archived at fossies.org.


    # homepage = "http://www.mr511.de/software/english.html"


    homepage = "https://directory.fsf.org/wiki/Libelf"


    urls = [


        "https://fossies.org/linux/misc/old/libelf-0.8.13.tar.gz",


        "https://ftp.osuosl.org/pub/blfs/conglomeration/libelf/libelf-0.8.13.tar.gz",


    ]


    version("0.8.13", sha256="591a9b4ec81c1f2042a97aa60564e0cb79d041c52faa7416acb38bc95bd2c76d")


    provides("elf@0")


    # configure: error: neither int nor long is 32-bit


    depends_on("automake", when="platform=darwin", type="build")


    depends_on("autoconf", when="platform=darwin", type="build")


    depends_on("libtool", when="platform=darwin", type="build")


    depends_on("m4", when="platform=darwin", type="build")


    @property


    def force_autoreconf(self):


        return self.spec.satisfies("platform=darwin")


    def configure_args(self):


        args = ["--enable-shared", "--disable-debug"]


        # config.sub: invalid option -apple-darwin21.6.0


        if self.spec.satisfies("platform=darwin target=aarch64:"):


            args.append("--build=aarch64-apple-darwin")


        return args


    def install(self, spec, prefix):


        make("install", parallel=False)


    def flag_handler(self, name, flags):


        if name == "cflags":


            if self.spec.satisfies("%clang@16:"):


                flags.append("-Wno-error=implicit-int")


                flags.append("-Wno-error=implicit-function-declaration")


        return (flags, None, None)

Spack also provides wrapper functions around common commands like configure, make, and cmake to make writing packages simple.

System Prerequisites¶

Spack has the following minimum system requirements, which are assumed to be present on the machine where Spack is run:

System prerequisites for Spack¶

These requirements can be easily installed on most modern Linux systems; on macOS, the Command Line Tools package is required, and a full XCode suite may be necessary for some packages such as Qt and apple-gl. Spack is designed to run on HPC platforms like Cray. Not all packages should be expected to work on all platforms.

A build matrix showing which packages are working on which systems is shown below.

Installation¶

Getting Spack is easy. You can clone it from the github repository using this command:

$ git clone -c feature.manyFiles=true https://github.com/spack/spack.git

This will create a directory called spack.

Shell support¶

Once you have cloned Spack, we recommend sourcing the appropriate script for your shell:

# For bash/zsh/sh
$ . spack/share/spack/setup-env.sh
# For tcsh/csh
$ source spack/share/spack/setup-env.csh
# For fish
$ . spack/share/spack/setup-env.fish

That's it! You're ready to use Spack.

Sourcing these files will put the spack command in your PATH, set up your MODULEPATH to use Spack's packages, and add other useful shell integration for certain commands, environments, and modules. For bash and zsh, it also sets up tab completion.

In order to know which directory to add to your MODULEPATH, these scripts query the spack command. On shared filesystems, this can be a bit slow, especially if you log in frequently. If you don't use modules, or want to set MODULEPATH manually instead, you can set the SPACK_SKIP_MODULES environment variable to skip this step and speed up sourcing the file.

If you do not want to use Spack's shell support, you can always just run the spack command directly from spack/bin/spack.

When the spack command is executed it searches for an appropriate Python interpreter to use, which can be explicitly overridden by setting the SPACK_PYTHON environment variable. When sourcing the appropriate shell setup script, SPACK_PYTHON will be set to the interpreter found at sourcing time, ensuring future invocations of the spack command will continue to use the same consistent python version regardless of changes in the environment.

Bootstrapping clingo¶

Spack uses clingo under the hood to resolve optimal versions and variants of dependencies when installing a package. Since clingo itself is a binary, Spack has to install it on initial use, which is called bootstrapping.

Spack provides two ways of bootstrapping clingo: from pre-built binaries (default), or from sources. The fastest way to get started is to bootstrap from pre-built binaries.

The first time you concretize a spec, Spack will bootstrap automatically:

$ spack spec zlib
==> Bootstrapping clingo from pre-built binaries
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64-gcc-10.2.1-clingo-bootstrap-spack-ba5ijauisd3uuixtmactc36vps7yfsrl.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64/gcc-10.2.1/clingo-bootstrap-spack/linux-centos7-x86_64-gcc-10.2.1-clingo-bootstrap-spack-ba5ijauisd3uuixtmactc36vps7yfsrl.spack
==> Installing "clingo-bootstrap@spack%gcc@10.2.1~docs~ipo+python+static_libstdcpp build_type=Release arch=linux-centos7-x86_64" from a buildcache
==> Bootstrapping patchelf from pre-built binaries
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64/gcc-10.2.1/patchelf-0.16.1/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spack
==> Installing "patchelf@0.16.1%gcc@10.2.1 ldflags="-static-libstdc++ -static-libgcc"  build_system=autotools arch=linux-centos7-x86_64" from a buildcache
Input spec
--------------------------------
zlib
Concretized
--------------------------------
zlib@1.2.13%gcc@9.4.0+optimize+pic+shared build_system=makefile arch=linux-ubuntu20.04-icelake

If for security concerns you cannot bootstrap clingo from pre-built binaries, you have to disable fetching the binaries we generated with Github Actions.

$ spack bootstrap disable github-actions-v0.4
==> "github-actions-v0.4" is now disabled and will not be used for bootstrapping
$ spack bootstrap disable github-actions-v0.3
==> "github-actions-v0.3" is now disabled and will not be used for bootstrapping

You can verify that the new settings are effective with:

$ spack bootstrap list
Name: github-actions-v0.5 ENABLED


  Type: buildcache


  Info: 


    url: https://mirror.spack.io/bootstrap/github-actions/v0.5


    homepage: https://github.com/spack/spack-bootstrap-mirrors


    releases: https://github.com/spack/spack-bootstrap-mirrors/releases


  Description: 


    Buildcache generated from a public workflow using Github Actions.


    The sha256 checksum of binaries is checked before installation.


    
Name: github-actions-v0.4 ENABLED


  Type: buildcache


  Info: 


    url: https://mirror.spack.io/bootstrap/github-actions/v0.4


    homepage: https://github.com/spack/spack-bootstrap-mirrors


    releases: https://github.com/spack/spack-bootstrap-mirrors/releases


  Description: 


    Buildcache generated from a public workflow using Github Actions.


    The sha256 checksum of binaries is checked before installation.


    
Name: spack-install ENABLED


  Type: install


  Info: 


    url: https://mirror.spack.io


  Description: 


    Specs built from sources downloaded from the Spack public mirror.

NOTE:

Spack will build the required software on the first request to concretize a spec:

$ spack spec zlib
[+] /usr (external bison-3.0.4-wu5pgjchxzemk5ya2l3ddqug2d7jv6eb)
[+] /usr (external cmake-3.19.4-a4kmcfzxxy45mzku4ipmj5kdiiz5a57b)
[+] /usr (external python-3.6.9-x4fou4iqqlh5ydwddx3pvfcwznfrqztv)
==> Installing re2c-1.2.1-e3x6nxtk3ahgd63ykgy44mpuva6jhtdt
[ ... ]
zlib@1.2.11%gcc@10.1.0+optimize+pic+shared arch=linux-ubuntu18.04-broadwell

The Bootstrap Store¶

All the tools Spack needs for its own functioning are installed in a separate store, which lives under the ${HOME}/.spack directory. The software installed there can be queried with:

$ spack -b find
-- linux-ubuntu18.04-x86_64 / gcc@10.1.0 ------------------------
clingo-bootstrap@spack  python@3.6.9  re2c@1.2.1

In case it's needed the bootstrap store can also be cleaned with:

$ spack clean -b
==> Removing bootstrapped software and configuration in "/home/spack/.spack/bootstrap"

Check Installation¶

With Spack installed, you should be able to run some basic Spack commands. For example:

$ spack spec netcdf-c target=x86_64 os=SUSE

In theory, Spack doesn't need any additional installation; just download and run! But in real life, additional steps are usually required before Spack can work in a practical sense. Read on...

Clean Environment¶

Many packages' installs can be broken by changing environment variables. For example, a package might pick up the wrong build-time dependencies (most of them not specified) depending on the setting of PATH. GCC seems to be particularly vulnerable to these issues.

Therefore, it is recommended that Spack users run with a clean environment, especially for PATH. Only software that comes with the system, or that you know you wish to use with Spack, should be included. This procedure will avoid many strange build errors.

Optional: Alternate Prefix¶

You may want to run Spack out of a prefix other than the git repository you cloned. The spack clone command provides this functionality. To install spack in a new directory, simply type:

$ spack clone /my/favorite/prefix

This will install a new spack script in /my/favorite/prefix/bin, which you can use just like you would the regular spack script. Each copy of spack installs packages into its own $PREFIX/opt directory.

Compiler configuration¶

Spack has the ability to build packages with multiple compilers and compiler versions. Compilers can be made available to Spack by specifying them manually in compilers.yaml, or automatically by running spack compiler find, but for convenience Spack will automatically detect compilers the first time it needs them.

spack compilers¶

You can see which compilers are available to Spack by running spack compilers or spack compiler list:

$ spack compilers
==> Available compilers
-- gcc ---------------------------------------------------------


    gcc@4.9.0  gcc@4.8.0  gcc@4.7.0  gcc@4.6.2  gcc@4.4.7


    gcc@4.8.2  gcc@4.7.1  gcc@4.6.3  gcc@4.6.1  gcc@4.1.2
-- intel -------------------------------------------------------


    intel@15.0.0  intel@14.0.0  intel@13.0.0  intel@12.1.0  intel@10.0


    intel@14.0.3  intel@13.1.1  intel@12.1.5  intel@12.0.4  intel@9.1


    intel@14.0.2  intel@13.1.0  intel@12.1.3  intel@11.1


    intel@14.0.1  intel@13.0.1  intel@12.1.2  intel@10.1
-- clang -------------------------------------------------------


    clang@3.4  clang@3.3  clang@3.2  clang@3.1
-- pgi ---------------------------------------------------------


    pgi@14.3-0   pgi@13.2-0  pgi@12.1-0   pgi@10.9-0  pgi@8.0-1


    pgi@13.10-0  pgi@13.1-1  pgi@11.10-0  pgi@10.2-0  pgi@7.1-3


    pgi@13.6-0   pgi@12.8-0  pgi@11.1-0   pgi@9.0-4   pgi@7.0-6

Any of these compilers can be used to build Spack packages. More on how this is done is in Specs & dependencies.

spack compiler add¶

An alias for spack compiler find.

spack compiler find¶

Lists the compilers currently available to Spack. If you do not see a compiler in this list, but you want to use it with Spack, you can simply run spack compiler find with the path to where the compiler is installed. For example:

$ spack compiler find /usr/local/tools/ic-13.0.079
==> Added 1 new compiler to ~/.spack/linux/compilers.yaml


    intel@13.0.079

Or you can run spack compiler find with no arguments to force auto-detection. This is useful if you do not know where compilers are installed, but you know that new compilers have been added to your PATH. For example, you might load a module, like this:

$ module load gcc/4.9.0
$ spack compiler find
==> Added 1 new compiler to ~/.spack/linux/compilers.yaml


    gcc@4.9.0

This loads the environment module for gcc-4.9.0 to add it to PATH, and then it adds the compiler to Spack.

NOTE:

spack compiler info¶

If you want to see specifics on a particular compiler, you can run spack compiler info on it:

$ spack compiler info intel@15
intel@15.0.0:


  paths:


    cc  = /usr/local/bin/icc-15.0.090


    cxx = /usr/local/bin/icpc-15.0.090


    f77 = /usr/local/bin/ifort-15.0.090


    fc  = /usr/local/bin/ifort-15.0.090


  modules = []


  operating_system = centos6
...

This shows which C, C++, and Fortran compilers were detected by Spack. Notice also that we didn't have to be too specific about the version. We just said intel@15, and information about the only matching Intel compiler was displayed.

Manual compiler configuration¶

If auto-detection fails, you can manually configure a compiler by editing your ~/.spack/<platform>/compilers.yaml file. You can do this by running spack config edit compilers, which will open the file in your favorite editor.

Each compiler configuration in the file looks like this:

compilers:
- compiler:


    modules: []


    operating_system: centos6


    paths:


      cc: /usr/local/bin/icc-15.0.024-beta


      cxx: /usr/local/bin/icpc-15.0.024-beta


      f77: /usr/local/bin/ifort-15.0.024-beta


      fc: /usr/local/bin/ifort-15.0.024-beta


    spec: intel@15.0.0

For compilers that do not support Fortran (like clang), put None for f77 and fc:

compilers:
- compiler:


    modules: []


    operating_system: centos6


    paths:


      cc: /usr/bin/clang


      cxx: /usr/bin/clang++


      f77: None


      fc: None


    spec: clang@3.3svn

Once you save the file, the configured compilers will show up in the list displayed by spack compilers.

You can also add compiler flags to manually configured compilers. These flags should be specified in the flags section of the compiler specification. The valid flags are cflags, cxxflags, fflags, cppflags, ldflags, and ldlibs. For example:

compilers:
- compiler:


    modules: []


    operating_system: centos6


    paths:


      cc: /usr/bin/gcc


      cxx: /usr/bin/g++


      f77: /usr/bin/gfortran


      fc: /usr/bin/gfortran


    flags:


      cflags: -O3 -fPIC


      cxxflags: -O3 -fPIC


      cppflags: -O3 -fPIC


    spec: gcc@4.7.2

These flags will be treated by spack as if they were entered from the command line each time this compiler is used. The compiler wrappers then inject those flags into the compiler command. Compiler flags entered from the command line will be discussed in more detail in the following section.

Some compilers also require additional environment configuration. Examples include Intels oneAPI and AMDs AOCC compiler suites, which have custom scripts for loading environment variables and setting paths. These variables should be specified in the environment section of the compiler specification. The operations available to modify the environment are set, unset, prepend_path, append_path, and remove_path. For example:

compilers:
- compiler:


    modules: []


    operating_system: centos6


    paths:


      cc: /opt/intel/oneapi/compiler/latest/linux/bin/icx


      cxx: /opt/intel/oneapi/compiler/latest/linux/bin/icpx


      f77: /opt/intel/oneapi/compiler/latest/linux/bin/ifx


      fc: /opt/intel/oneapi/compiler/latest/linux/bin/ifx


    spec: oneapi@latest


    environment:


      set:


        MKL_ROOT: "/path/to/mkl/root"


      unset: # A list of environment variables to unset


        - CC


      prepend_path: # Similar for append|remove_path


        LD_LIBRARY_PATH: /ld/paths/added/by/setvars/sh

Build Your Own Compiler¶

If you are particular about which compiler/version you use, you might wish to have Spack build it for you. For example:

$ spack install gcc@4.9.3

Once that has finished, you will need to add it to your compilers.yaml file. You can then set Spack to use it by default by adding the following to your packages.yaml file:

packages:


  all:


    compiler: [gcc@4.9.3]

Compilers Requiring Modules¶

Many installed compilers will work regardless of the environment they are called with. However, some installed compilers require $LD_LIBRARY_PATH or other environment variables to be set in order to run; this is typical for Intel and other proprietary compilers.

In such a case, you should tell Spack which module(s) to load in order to run the chosen compiler (If the compiler does not come with a module file, you might consider making one by hand). Spack will load this module into the environment ONLY when the compiler is run, and NOT in general for a package's install() method. See, for example, this compilers.yaml file:

compilers:
- compiler:


    modules: [other/comp/gcc-5.3-sp3]


    operating_system: SuSE11


    paths:


      cc: /usr/local/other/SLES11.3/gcc/5.3.0/bin/gcc


      cxx: /usr/local/other/SLES11.3/gcc/5.3.0/bin/g++


      f77: /usr/local/other/SLES11.3/gcc/5.3.0/bin/gfortran


      fc: /usr/local/other/SLES11.3/gcc/5.3.0/bin/gfortran


    spec: gcc@5.3.0

Some compilers require special environment settings to be loaded not just to run, but also to execute the code they build, breaking packages that need to execute code they just compiled. If it's not possible or practical to use a better compiler, you'll need to ensure that environment settings are preserved for compilers like this (i.e., you'll need to load the module or source the compiler's shell script).

By default, Spack tries to ensure that builds are reproducible by cleaning the environment before building. If this interferes with your compiler settings, you CAN use spack install --dirty as a workaround. Note that this MAY interfere with package builds.

Licensed Compilers¶

Some proprietary compilers require licensing to use. If you need to use a licensed compiler (eg, PGI), the process is similar to a mix of build your own, plus modules:

Mixed Toolchains¶

Modern compilers typically come with related compilers for C, C++ and Fortran bundled together. When possible, results are best if the same compiler is used for all languages.

In some cases, this is not possible. For example, starting with macOS El Capitan (10.11), many packages no longer build with GCC, but XCode provides no Fortran compilers. The user is therefore forced to use a mixed toolchain: XCode-provided Clang for C/C++ and GNU gfortran for Fortran.

$ xcode-select --install

xcode-select: error: command line tools are already installed, use "Software Update" to install updates

$ xcode-select -p

/Applications/Xcode.app/Contents/Developer

/Library/Developer/CommandLineTools

$ sudo xcodebuild -license accept

compilers:
- compiler:


  ...


  paths:


    cc: /usr/bin/clang


    cxx: /usr/bin/clang++


    f77: /path/to/bin/gfortran


    fc: /path/to/bin/gfortran


  spec: apple-clang@11.0.0

Compiler Verification¶

You can verify that your compilers are configured properly by installing a simple package. For example:

$ spack install zlib%gcc@5.3.0

Vendor-Specific Compiler Configuration¶

With Spack, things usually "just work" with GCC. Not so for other compilers. This section provides details on how to get specific compilers working.

Intel Compilers¶

Intel compilers are unusual because a single Intel compiler version can emulate multiple GCC versions. In order to provide this functionality, the Intel compiler needs GCC to be installed. Therefore, the following steps are necessary to successfully use Intel compilers:

Intel compilers may therefore be configured in one of two ways with Spack: using modules, or using compiler flags.

Configuration with Modules¶

One can control which GCC is seen by the Intel compiler with modules. A module must be loaded both for the Intel Compiler (so it will run) and GCC (so the compiler can find the intended GCC). The following configuration in compilers.yaml illustrates this technique:

compilers:
- compiler:


    modules: [gcc-4.9.3, intel-15.0.24]


    operating_system: centos7


    paths:


      cc: /opt/intel-15.0.24/bin/icc-15.0.24-beta


      cxx: /opt/intel-15.0.24/bin/icpc-15.0.24-beta


      f77: /opt/intel-15.0.24/bin/ifort-15.0.24-beta


      fc: /opt/intel-15.0.24/bin/ifort-15.0.24-beta


    spec: intel@15.0.24.4.9.3

NOTE:

Command Line Configuration¶

One can also control which GCC is seen by the Intel compiler by adding flags to the icc command:

$ spack location --install-dir gcc
~/spack/opt/spack/linux-centos7-x86_64/gcc-4.9.3-iy4rw...

compilers:
- compiler:


    modules: [intel-15.0.24]


    operating_system: centos7


    paths:


      cc: /opt/intel-15.0.24/bin/icc-15.0.24-beta


      cxx: /opt/intel-15.0.24/bin/icpc-15.0.24-beta


      f77: /opt/intel-15.0.24/bin/ifort-15.0.24-beta


      fc: /opt/intel-15.0.24/bin/ifort-15.0.24-beta


    flags:


      cflags: -gcc-name ~/spack/opt/spack/linux-centos7-x86_64/gcc-4.9.3-iy4rw.../bin/gcc


      cxxflags: -gxx-name ~/spack/opt/spack/linux-centos7-x86_64/gcc-4.9.3-iy4rw.../bin/g++


      fflags: -gcc-name ~/spack/opt/spack/linux-centos7-x86_64/gcc-4.9.3-iy4rw.../bin/gcc


    spec: intel@15.0.24.4.9.3

PGI¶

PGI comes with two sets of compilers for C++ and Fortran, distinguishable by their names. "Old" compilers:

cc:  /soft/pgi/15.10/linux86-64/15.10/bin/pgcc
cxx: /soft/pgi/15.10/linux86-64/15.10/bin/pgCC
f77: /soft/pgi/15.10/linux86-64/15.10/bin/pgf77
fc:  /soft/pgi/15.10/linux86-64/15.10/bin/pgf90

"New" compilers:

cc:  /soft/pgi/15.10/linux86-64/15.10/bin/pgcc
cxx: /soft/pgi/15.10/linux86-64/15.10/bin/pgc++
f77: /soft/pgi/15.10/linux86-64/15.10/bin/pgfortran
fc:  /soft/pgi/15.10/linux86-64/15.10/bin/pgfortran

Older installations of PGI contains just the old compilers; whereas newer installations contain the old and the new. The new compiler is considered preferable, as some packages (hdf) will not build with the old compiler.

When auto-detecting a PGI compiler, there are cases where Spack will find the old compilers, when you really want it to find the new compilers. It is best to check this compilers.yaml; and if the old compilers are being used, change pgf77 and pgf90 to pgfortran.

Other issues:

$ spack install openmpi%pgi ^libpciaccess%gcc

NOTE:

NAG¶

The Numerical Algorithms Group provides a licensed Fortran compiler. Like Clang, this requires you to set up a Mixed Toolchains. It is recommended to use GCC for your C/C++ compilers.

The NAG Fortran compilers are a bit more strict than other compilers, and many packages will fail to install with error messages like:

Error: mpi_comm_spawn_multiple_f90.f90: Argument 3 to MPI_COMM_SPAWN_MULTIPLE has data type DOUBLE PRECISION in reference from MPI_COMM_SPAWN_MULTIPLEN and CHARACTER in reference from MPI_COMM_SPAWN_MULTIPLEA

In order to convince the NAG compiler not to be too picky about calling conventions, you can use FFLAGS=-mismatch and FCFLAGS=-mismatch. This can be done through the command line:

$ spack install openmpi fflags="-mismatch"

Or it can be set permanently in your compilers.yaml:

- compiler:


 modules: []


 operating_system: centos6


 paths:


   cc: /soft/spack/opt/spack/linux-x86_64/gcc-5.3.0/gcc-6.1.0-q2zosj3igepi3pjnqt74bwazmptr5gpj/bin/gcc


   cxx: /soft/spack/opt/spack/linux-x86_64/gcc-5.3.0/gcc-6.1.0-q2zosj3igepi3pjnqt74bwazmptr5gpj/bin/g++


   f77: /soft/spack/opt/spack/linux-x86_64/gcc-4.4.7/nag-6.1-jt3h5hwt5myezgqguhfsan52zcskqene/bin/nagfor


   fc: /soft/spack/opt/spack/linux-x86_64/gcc-4.4.7/nag-6.1-jt3h5hwt5myezgqguhfsan52zcskqene/bin/nagfor


 flags:


   fflags: -mismatch


 spec: nag@6.1

System Packages¶

Once compilers are configured, one needs to determine which pre-installed system packages, if any, to use in builds. This is configured in the file ~/.spack/packages.yaml. For example, to use an OpenMPI installed in /opt/local, one would use:

packages:


    openmpi:


        externals:


        - spec: openmpi@1.10.1


          prefix: /opt/local


        buildable: False

In general, Spack is easier to use and more reliable if it builds all of its own dependencies. However, there are several packages for which one commonly needs to use system versions:

MPI¶

On supercomputers, sysadmins have already built MPI versions that take into account the specifics of that computer's hardware. Unless you know how they were built and can choose the correct Spack variants, you are unlikely to get a working MPI from Spack. Instead, use an appropriate pre-installed MPI.

If you choose a pre-installed MPI, you should consider using the pre-installed compiler used to build that MPI; see above on compilers.yaml.

OpenSSL¶

The openssl package underlies much of modern security in a modern OS; an attacker can easily "pwn" any computer on which they can modify SSL. Therefore, any openssl used on a system should be created in a "trusted environment" --- for example, that of the OS vendor.

OpenSSL is also updated by the OS vendor from time to time, in response to security problems discovered in the wider community. It is in everyone's best interest to use any newly updated versions as soon as they come out. Modern Linux installations have standard procedures for security updates without user involvement.

Spack running at user-level is not a trusted environment, nor do Spack users generally keep up-to-date on the latest security holes in SSL. For these reasons, a Spack-installed OpenSSL should likely not be trusted.

As long as the system-provided SSL works, you can use it instead. One can check if it works by trying to download an https://. For example:

$ curl -O https://github.com/ImageMagick/ImageMagick/archive/7.0.2-7.tar.gz

To tell Spack to use the system-supplied OpenSSL, first determine what version you have:

$ openssl version
OpenSSL 1.0.2g  1 Mar 2016

Then add the following to ~/.spack/packages.yaml:

packages:


    openssl:


        externals:


        - spec: openssl@1.0.2g


          prefix: /usr


        buildable: False

BLAS / LAPACK¶

The recommended way to use system-supplied BLAS / LAPACK packages is to add the following to packages.yaml:

packages:


    netlib-lapack:


        externals:


        - spec: netlib-lapack@3.6.1


          prefix: /usr


        buildable: False


    all:


        providers:


            blas: [netlib-lapack]


            lapack: [netlib-lapack]

NOTE:

Git¶

Some Spack packages use git to download, which might not work on some computers. For example, the following error was encountered on a Macintosh during spack install julia@master:

==> Cloning git repository:


  https://github.com/JuliaLang/julia.git


  on branch master
Cloning into 'julia'...
fatal: unable to access 'https://github.com/JuliaLang/julia.git/':


    SSL certificate problem: unable to get local issuer certificate

This problem is related to OpenSSL, and in some cases might be solved by installing a new version of git and openssl:

If this doesn't work, it is also possible to disable checking of SSL certificates by using:

$ spack --insecure install

Using --insecure makes Spack disable SSL checking when fetching from websites and from git.

WARNING:

certificate for security (no verification).

Utilities Configuration¶

Although Spack does not need installation per se, it does rely on other packages to be available on its host system. If those packages are out of date or missing, then Spack will not work. Sometimes, an appeal to the system's package manager can fix such problems. If not, the solution is have Spack install the required packages, and then have Spack use them.

For example, if curl doesn't work, one could use the following steps to provide Spack a working curl:

$ spack install curl
$ spack load curl

or alternately:

$ spack module tcl loads curl >>~/.bashrc

or if environment modules don't work:

$ export PATH=`spack location --install-dir curl`/bin:$PATH

External commands are used by Spack in two places: within core Spack, and in the package recipes. The bootstrapping procedure for these two cases is somewhat different, and is treated separately below.

Core Spack Utilities¶

Core Spack uses the following packages, mainly to download and unpack source code: curl, env, git, go, hg, svn, tar, unzip, patch

As long as the user's environment is set up to successfully run these programs from outside of Spack, they should work inside of Spack as well. They can generally be activated as in the curl example above; or some systems might already have an appropriate hand-built environment module that may be loaded. Either way works.

A few notes on specific programs in this list:

cURL, git, Mercurial, etc.¶

Spack depends on cURL to download tarballs, the format that most Spack-installed packages come in. Your system's cURL should always be able to download unencrypted http://. However, the cURL on some systems has problems with SSL-enabled https:// URLs, due to outdated / insecure versions of OpenSSL on those systems. This will prevent Spack from installing any software requiring https:// until a new cURL has been installed, using the technique above.

WARNING:

Some packages use source code control systems as their download method: git, hg, svn and occasionally go. If you had to install a new curl, then chances are the system-supplied version of these other programs will also not work, because they also rely on OpenSSL. Once curl has been installed, you can similarly install the others.

Package Utilities¶

Spack may also encounter bootstrapping problems inside a package's install() method. In this case, Spack will normally be running inside a sanitized build environment. This includes all of the package's dependencies, but none of the environment Spack inherited from the user: if you load a module or modify $PATH before launching Spack, it will have no effect.

In this case, you will likely need to use the --dirty flag when running spack install, causing Spack to not sanitize the build environment. You are now responsible for making sure that environment does not do strange things to Spack or its installs.

Another way to get Spack to use its own version of something is to add that something to a package that needs it. For example:

depends_on('binutils', type='build')

This is considered best practice for some common build dependencies, such as autotools (if the autoreconf command is needed) and cmake --- cmake especially, because different packages require a different version of CMake.

binutils¶

Sometimes, strange error messages can happen while building a package. For example, ld might crash. Or one receives a message like:

ld: final link failed: Nonrepresentable section on output

or:

ld: .../_fftpackmodule.o: unrecognized relocation (0x2a) in section `.text'

These problems are often caused by an outdated binutils on your system. Unlike CMake or Autotools, adding depends_on('binutils') to every package is not considered a best practice because every package written in C/C++/Fortran would need it. A potential workaround is to load a recent binutils into your environment and use the --dirty flag.

GPG Signing¶

spack gpg¶

Spack has support for signing and verifying packages using GPG keys. A separate keyring is used for Spack, so any keys available in the user's home directory are not used.

spack gpg init¶

When Spack is first installed, its keyring is empty. Keys stored in var/spack/gpg are the default keys for a Spack installation. These keys may be imported by running spack gpg init. This will import the default keys into the keyring as trusted keys.

Trusting keys¶

Additional keys may be added to the keyring using spack gpg trust <keyfile>. Once a key is trusted, packages signed by the owner of they key may be installed.

Creating keys¶

You may also create your own key so that you may sign your own packages using spack gpg create <name> <email>. By default, the key has no expiration, but it may be set with the --expires <date> flag (see the gnupg2 documentation for accepted date formats). It is also recommended to add a comment as to the use of the key using the --comment <comment> flag. The public half of the key can also be exported for sharing with others so that they may use packages you have signed using the --export <keyfile> flag. Secret keys may also be later exported using the spack gpg export <location> [<key>...] command.

NOTE:

$ sudo apt-get install rng-tools
$ sudo rngd -r /dev/urandom

$ sudo yum install haveged
$ sudo chkconfig haveged on

Here is an example of creating a key. Note that we provide a name for the key first (which we can use to reference the key later) and an email address:

$ spack gpg create dinosaur dinosaur@thedinosaurthings.com

If you want to export the key as you create it:

$ spack gpg create --export key.pub dinosaur dinosaur@thedinosaurthings.com

Or the private key:

$ spack gpg create --export-secret key.priv dinosaur dinosaur@thedinosaurthings.com

You can include both --export and --export-secret, each with an output file of choice, to export both.

Listing keys¶

In order to list the keys available in the keyring, the spack gpg list command will list trusted keys with the --trusted flag and keys available for signing using --signing. If you would like to remove keys from your keyring, spack gpg untrust <keyid>. Key IDs can be email addresses, names, or (best) fingerprints. Here is an example of listing the key that we just created:

gpgconf: socketdir is '/run/user/1000/gnupg'
/home/spackuser/spack/opt/spack/gpg/pubring.kbx
----------------------------------------------------------
pub   rsa4096 2021-03-25 [SC]


      60D2685DAB647AD4DB54125961E09BB6F2A0ADCB
uid           [ultimate] dinosaur (GPG created for Spack) <dinosaur@thedinosaurthings.com>

Note that the name "dinosaur" can be seen under the uid, which is the unique id. We might need this reference if we want to export or otherwise reference the key.

Signing and Verifying Packages¶

In order to sign a package, spack gpg sign <file> should be used. By default, the signature will be written to <file>.asc, but that may be changed by using the --output <file> flag. If there is only one signing key available, it will be used, but if there is more than one, the key to use must be specified using the --key <keyid> flag. The --clearsign flag may also be used to create a signed file which contains the contents, but it is not recommended. Signed packages may be verified by using spack gpg verify <file>.

Exporting Keys¶

You likely might want to export a public key, and that looks like this. Let's use the previous example and ask spack to export the key with uid "dinosaur." We will provide an output location (typically a *.pub file) and the name of the key.

$ spack gpg export dinosaur.pub dinosaur

You can then look at the created file, dinosaur.pub, to see the exported key. If you want to include the private key, then just add --secret:

$ spack gpg export --secret dinosaur.priv dinosaur

This will write the private key to the file dinosaur.priv.

WARNING:

Spack on Cray¶

Spack differs slightly when used on a Cray system. The architecture spec can differentiate between the front-end and back-end processor and operating system. For example, on Edison at NERSC, the back-end target processor is "Ivy Bridge", so you can specify to use the back-end this way:

$ spack install zlib target=ivybridge

You can also use the operating system to build against the back-end:

$ spack install zlib os=CNL10

Notice that the name includes both the operating system name and the major version number concatenated together.

Alternatively, if you want to build something for the front-end, you can specify the front-end target processor. The processor for a login node on Edison is "Sandy bridge" so we specify on the command line like so:

$ spack install zlib target=sandybridge

And the front-end operating system is:

$ spack install zlib os=SuSE11

Cray compiler detection¶

Spack can detect compilers using two methods. For the front-end, we treat everything the same. The difference lies in back-end compiler detection. Back-end compiler detection is made via the Tcl module avail command. Once it detects the compiler it writes the appropriate PrgEnv and compiler module name to compilers.yaml and sets the paths to each compiler with Cray's compiler wrapper names (i.e. cc, CC, ftn). During build time, Spack will load the correct PrgEnv and compiler module and will call appropriate wrapper.

The compilers.yaml config file will also differ. There is a modules section that is filled with the compiler's Programming Environment and module name. On other systems, this field is empty []:

- compiler:


    modules:


      - PrgEnv-intel


      - intel/15.0.109

As mentioned earlier, the compiler paths will look different on a Cray system. Since most compilers are invoked using cc, CC and ftn, the paths for each compiler are replaced with their respective Cray compiler wrapper names:

paths:


  cc: cc


  cxx: CC


  f77: ftn


  fc: ftn

As opposed to an explicit path to the compiler executable. This allows Spack to call the Cray compiler wrappers during build time.

For more on compiler configuration, check out Compiler configuration.

Spack sets the default Cray link type to dynamic, to better match other other platforms. Individual packages can enable static linking (which is the default outside of Spack on cray systems) using the -static flag.

Setting defaults and using Cray modules¶

If you want to use default compilers for each PrgEnv and also be able to load cray external modules, you will need to set up a packages.yaml.

Here's an example of an external configuration for cray modules:

packages:


  mpich:


    externals:


    - spec: "mpich@7.3.1%gcc@5.2.0 arch=cray_xc-haswell-CNL10"


      modules:


      - cray-mpich


    - spec: "mpich@7.3.1%intel@16.0.0.109 arch=cray_xc-haswell-CNL10"


      modules:


      - cray-mpich


  all:


    providers:


      mpi: [mpich]

This tells Spack that for whatever package that depends on mpi, load the cray-mpich module into the environment. You can then be able to use whatever environment variables, libraries, etc, that are brought into the environment via module load.

NOTE:

You can set the default compiler that Spack can use for each compiler type. If you want to use the Cray defaults, then set them under all: in packages.yaml. In the compiler field, set the compiler specs in your order of preference. Whenever you build with that compiler type, Spack will concretize to that version.

Here is an example of a full packages.yaml used at NERSC

packages:


  mpich:


    externals:


    - spec: "mpich@7.3.1%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge"


      modules:


      - cray-mpich


    - spec: "mpich@7.3.1%intel@16.0.0.109 arch=cray_xc-SuSE11-ivybridge"


      modules:


      - cray-mpich


    buildable: False


  netcdf:


    externals:


    - spec: "netcdf@4.3.3.1%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge"


      modules:


      - cray-netcdf


    - spec: "netcdf@4.3.3.1%intel@16.0.0.109 arch=cray_xc-CNL10-ivybridge"


      modules:


      - cray-netcdf


    buildable: False


  hdf5:


    externals:


    - spec: "hdf5@1.8.14%gcc@5.2.0 arch=cray_xc-CNL10-ivybridge"


      modules:


      - cray-hdf5


    - spec: "hdf5@1.8.14%intel@16.0.0.109 arch=cray_xc-CNL10-ivybridge"


      modules:


      - cray-hdf5


    buildable: False


  all:


    compiler: [gcc@5.2.0, intel@16.0.0.109]


    providers:


      mpi: [mpich]

Here we tell spack that whenever we want to build with gcc use version 5.2.0 or if we want to build with intel compilers, use version 16.0.0.109. We add a spec for each compiler type for each cray modules. This ensures that for each compiler on our system we can use that external module.

For more on external packages check out the section External Packages.

Using Linux containers on Cray machines¶

Spack uses environment variables particular to the Cray programming environment to determine which systems are Cray platforms. These environment variables may be propagated into containers that are not using the Cray programming environment.

To ensure that Spack does not autodetect the Cray programming environment, unset the environment variable MODULEPATH. This will cause Spack to treat a linux container on a Cray system as a base linux distro.

Spack On Windows¶

Windows support for Spack is currently under development. While this work is still in an early stage, it is currently possible to set up Spack and perform a few operations on Windows. This section will guide you through the steps needed to install Spack and start running it on a fresh Windows machine.

Step 1: Install prerequisites¶

To use Spack on Windows, you will need the following packages:

Required: * Microsoft Visual Studio * Python * Git

Optional: * Intel Fortran (needed for some packages)

NOTE:

Microsoft Visual Studio¶

Microsoft Visual Studio provides the only Windows C/C++ compiler that is currently supported by Spack.

We require several specific components to be included in the Visual Studio installation. One is the C/C++ toolset, which can be selected as "Desktop development with C++" or "C++ build tools," depending on installation type (Professional, Build Tools, etc.) The other required component is "C++ CMake tools for Windows," which can be selected from among the optional packages. This provides CMake and Ninja for use during Spack configuration.

If you already have Visual Studio installed, you can make sure these components are installed by rerunning the installer. Next to your installation, select "Modify" and look at the "Installation details" pane on the right.

Intel Fortran¶

For Fortran-based packages on Windows, we strongly recommend Intel's oneAPI Fortran compilers. The suite is free to download from Intel's website, located at https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/fortran-compiler.html. The executable of choice for Spack will be Intel's Beta Compiler, ifx, which supports the classic compiler's (ifort's) frontend and runtime libraries by using LLVM.

Python¶

As Spack is a Python-based package, an installation of Python will be needed to run it. Python 3 can be downloaded and installed from the Windows Store, and will be automatically added to your PATH in this case.

NOTE:

Git¶

A bash console and GUI can be downloaded from https://git-scm.com/downloads. If you are unfamiliar with Git, there are a myriad of resources online to help guide you through checking out repositories and switching development branches.

When given the option of adjusting your PATH, choose the Git from the command line and also from 3rd-party software option. This will automatically update your PATH variable to include the git command.

Spack support on Windows is currently dependent on installing the Git for Windows project as the project providing Git support on Windows. This is additionally the recommended method for installing Git on Windows, a link to which can be found above. Spack requires the utilities vendored by this project.

utilities vendored by this project.

Step 2: Install and setup Spack¶

We are now ready to get the Spack environment set up on our machine. We begin by using Git to clone the Spack repo, hosted at https://github.com/spack/spack.git into a desired directory, for our purposes today, called spack_install.

In order to install Spack with Windows support, run the following one liner in a Windows CMD prompt.

git clone https://github.com/spack/spack.git

NOTE:

Step 3: Run and configure Spack¶

To use Spack, run bin\spack_cmd.bat (you may need to Run as Administrator) from the top-level spack directory. This will provide a Windows command prompt with an environment properly set up with Spack and its prerequisites. If you receive a warning message that Python is not in your PATH (which may happen if you installed Python from the website and not the Windows Store) add the location of the Python executable to your PATH now. You can permanently add Python to your PATH variable by using the Edit the system environment variables utility in Windows Control Panel.

NOTE:

To configure Spack, first run the following command inside the Spack console:

spack compiler find

This creates a .staging directory in our Spack prefix, along with a windows subdirectory containing a compilers.yaml file. On a fresh Windows install with the above packages installed, this command should only detect Microsoft Visual Studio and the Intel Fortran compiler will be integrated within the first version of MSVC present in the compilers.yaml output.

Spack provides a default config.yaml file for Windows that it will use unless overridden. This file is located at etc\spack\defaults\windows\config.yaml. You can read more on how to do this and write your own configuration files in the Configuration Files section of our documentation. If you do this, pay particular attention to the build_stage block of the file as this specifies the directory that will temporarily hold the source code for the packages to be installed. This path name must be sufficiently short for compliance with cmd, otherwise you will see build errors during installation (particularly with CMake) tied to long path names.

To allow Spack use of external tools and dependencies already on your system, the external pieces of software must be described in the packages.yaml file. There are two methods to populate this file:

The first and easiest choice is to use Spack to find installation on your system. In the Spack terminal, run the following commands:

spack external find cmake
spack external find ninja

The spack external find <name> will find executables on your system with the same name given. The command will store the items found in packages.yaml in the .staging\ directory.

Assuming that the command found CMake and Ninja executables in the previous step, continue to Step 4. If no executables were found, we may need to manually direct spack towards the CMake and Ninja installations we set up with Visual Studio. Therefore, your packages.yaml file will look something like this, with possibly slight variants in the paths to CMake and Ninja:

packages:


  cmake:


    externals:


    - spec: cmake@3.19


      prefix: 'c:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake'


    buildable: False


  ninja:


    externals:


    - spec: ninja@1.8.2


      prefix: 'c:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja'


    buildable: False

You can also use an separate installation of CMake if you have one and prefer to use it. If you don't have a path to Ninja analogous to the above, then you can obtain it by running the Visual Studio Installer and following the instructions at the start of this section. Also note that .yaml files use spaces for indentation and not tabs, so ensure that this is the case when editing one directly.

NOTE:

Step 4: Use Spack¶

Once the configuration is complete, it is time to give the installation a test. Install a basic package though the Spack console via:

spack install cpuinfo

If in the previous step, you did not have CMake or Ninja installed, running the command above should bootstrap both packages

Windows Compatible Packages¶

Not all spack packages currently have Windows support. Some are inherently incompatible with the platform, and others simply have yet to be ported. To view the current set of packages with Windows support, the list command should be used via spack list -t windows. If there's a package you'd like to install on Windows but is not in that list, feel free to reach out to request the port or contribute the port yourself.

NOTE:

For developers¶

The intent is to provide a Windows installer that will automatically set up Python, Git, and Spack, instead of requiring the user to do so manually. Instructions for creating the installer are at https://github.com/spack/spack/blob/develop/lib/spack/spack/cmd/installer/README.md

Alternatively a pre-built copy of the Windows installer is available as an artifact of Spack's Windows CI available at each run of the CI on develop or any PR.

Listing available packages¶

To install software with Spack, you need to know what software is available. You can see a list of available package names at the packages.spack.io website, or using the spack list command.

spack list¶

The spack list command prints out a list of all of the packages Spack can install:

$ spack list
3dtk                                      py-googleapis-common-protos
3proxy                                    py-googledrivedownloader
7zip                                      py-gosam
abacus                                    py-gpaw
abduco                                    py-gpustat
abi-compliance-checker                    py-gputil
abi-dumper                                py-gpy
abinit                                    py-gpyopt
abseil-cpp                                py-gpytorch
abyss                                     py-gql
...

There are thousands of them, so we've truncated the output above, but you can find a full list here. Packages are listed by name in alphabetical order. A pattern to match with no wildcards, * or ?, will be treated as though it started and ended with *, so util is equivalent to *util*. All patterns will be treated as case-insensitive. You can also add the -d to search the description of the package in addition to the name. Some examples:

All packages whose names contain "sql":

$ spack list sql
mysql              py-agate-sql                     py-mysqlclient  py-sqlalchemy-migrate  r-rpostgresql  sqlitebrowser
mysql-connector-c  py-aiosqlite                     py-mysqldb1     py-sqlalchemy-stubs    r-rsqlite
mysqlpp            py-azure-mgmt-sql                py-pygresql     py-sqlalchemy-utils    r-sqldf
perl-dbd-mysql     py-azure-mgmt-sqlvirtualmachine  py-pymysql      py-sqlitedict          sqlcipher
perl-dbd-sqlite    py-flask-sqlalchemy              py-pysqlite3    py-sqlparse            sqlite
postgresql         py-mysql-connector-python        py-sqlalchemy   r-rmysql               sqlite-jdbc

All packages whose names or descriptions contain documentation:

$ spack list --search-description documentation
asciidoc-py3       perl-bioperl          py-interface-meta       py-sphinx-tabs               r-spam
byacc              perl-db-file          py-markdown             py-sphinxautomodapi          r-stanheaders
compositeproto     perl-io-prompt        py-mkdocs               py-sphinxcontrib-websupport  r-units
damageproto        py-alabaster          py-mkdocs-material      r-downlit                    r-uwot
double-conversion  py-astropy-helpers    py-mkdocstrings         r-lifecycle                  sowing
doxygen            py-dask-sphinx-theme  py-myst-parser          r-modeltools                 texinfo
gflags             py-docutils           py-param                r-pkgdown                    totalview
gtk-doc            py-epydoc             py-pdoc3                r-quadprog                   xorg-docs
libxfixes          py-exhale             py-python-docs-theme    r-rcpp                       xorg-sgml-doctools
libxpresent        py-fastai             py-recommonmark         r-rdpack
man-db             py-ford               py-sphinx               r-rinside
ntl                py-furo               py-sphinx-immaterial    r-roxygen2
oommf              py-griffe             py-sphinx-multiversion  r-satellite

spack info¶

To get more information on a particular package from spack list, use spack info. Just supply the name of a package:

$ spack info --all mpich
AutotoolsPackage:   mpich
Description:


    MPICH is a high performance and widely portable implementation of the


    Message Passing Interface (MPI) standard.
Homepage: https://www.mpich.org
Maintainers: @raffenet @yfguo
Externally Detectable: 


    True (version, variants)
Tags: 


    detectable  e4s
Preferred version:  


    4.1.2      https://www.mpich.org/static/downloads/4.1.2/mpich-4.1.2.tar.gz
Safe versions:  


    develop    [git] https://github.com/pmodels/mpich.git


    4.1.2      https://www.mpich.org/static/downloads/4.1.2/mpich-4.1.2.tar.gz


    4.1.1      https://www.mpich.org/static/downloads/4.1.1/mpich-4.1.1.tar.gz


    4.1        https://www.mpich.org/static/downloads/4.1/mpich-4.1.tar.gz


    4.0.3      https://www.mpich.org/static/downloads/4.0.3/mpich-4.0.3.tar.gz


    4.0.2      https://www.mpich.org/static/downloads/4.0.2/mpich-4.0.2.tar.gz


    4.0.1      https://www.mpich.org/static/downloads/4.0.1/mpich-4.0.1.tar.gz


    4.0        https://www.mpich.org/static/downloads/4.0/mpich-4.0.tar.gz


    3.4.3      https://www.mpich.org/static/downloads/3.4.3/mpich-3.4.3.tar.gz


    3.4.2      https://www.mpich.org/static/downloads/3.4.2/mpich-3.4.2.tar.gz


    3.4.1      https://www.mpich.org/static/downloads/3.4.1/mpich-3.4.1.tar.gz


    3.4        https://www.mpich.org/static/downloads/3.4/mpich-3.4.tar.gz


    3.3.2      https://www.mpich.org/static/downloads/3.3.2/mpich-3.3.2.tar.gz


    3.3.1      https://www.mpich.org/static/downloads/3.3.1/mpich-3.3.1.tar.gz


    3.3        https://www.mpich.org/static/downloads/3.3/mpich-3.3.tar.gz


    3.2.1      https://www.mpich.org/static/downloads/3.2.1/mpich-3.2.1.tar.gz


    3.2        https://www.mpich.org/static/downloads/3.2/mpich-3.2.tar.gz


    3.1.4      https://www.mpich.org/static/downloads/3.1.4/mpich-3.1.4.tar.gz


    3.1.3      https://www.mpich.org/static/downloads/3.1.3/mpich-3.1.3.tar.gz


    3.1.2      https://www.mpich.org/static/downloads/3.1.2/mpich-3.1.2.tar.gz


    3.1.1      https://www.mpich.org/static/downloads/3.1.1/mpich-3.1.1.tar.gz


    3.1        https://www.mpich.org/static/downloads/3.1/mpich-3.1.tar.gz


    3.0.4      https://www.mpich.org/static/downloads/3.0.4/mpich-3.0.4.tar.gz
Deprecated versions:  


    None
Variants:


    argobots [false]                false, true


        Enable Argobots support


    build_system [autotools]        autotools


        Build systems supported by the package


    cuda [false]                    false, true


        Build with CUDA


    device [ch4]                    ch3, ch4


        Abstract Device Interface (ADI)


        implementation. The ch4 device is in experimental state for versions


        before 3.4.


    fortran [true]                  false, true


        Enable Fortran support


    hwloc [true]                    false, true


        Use external hwloc package


    hydra [true]                    false, true


        Build the hydra process manager


    libxml2 [true]                  false, true


        Use libxml2 for XML support instead of the custom minimalistic implementation


    netmod [ofi]                    mxm, ofi, tcp, ucx


        Network module. Only single netmod builds are


        supported. For ch3 device configurations, this presumes the


        ch3:nemesis communication channel. ch3:sock is not supported by this


        spack package at this time.


    pci [true]                      false, true


        Support analyzing devices on PCI bus


    pmi [pmi]                       cray, off, pmi, pmi2, pmix


        PMI interface.


    rocm [false]                    false, true


        Enable ROCm support


    romio [true]                    false, true


        Enable ROMIO MPI I/O implementation


    slurm [false]                   false, true


        Enable SLURM support


    verbs [false]                   false, true


        Build support for OpenFabrics verbs.


    wrapperrpath [true]             false, true


        Enable wrapper rpath


    when +cuda


      cuda_arch [none]              none, 10, 11, 12, 13, 20, 21, 30, 32, 35, 37, 50, 52, 53, 60, 61, 62, 70, 72, 75,


                                    80, 86, 87, 89, 90


          CUDA architecture


    when +rocm


      amdgpu_target [none]          none, gfx1010, gfx1011, gfx1012, gfx1013, gfx1030, gfx1031, gfx1032, gfx1033,


                                    gfx1034, gfx1035, gfx1036, gfx1100, gfx1101, gfx1102, gfx1103, gfx701, gfx801,


                                    gfx802, gfx803, gfx900, gfx900:xnack-, gfx902, gfx904, gfx906, gfx906:xnack-,


                                    gfx908, gfx908:xnack-, gfx909, gfx90a, gfx90a:xnack+, gfx90a:xnack-, gfx90c,


                                    gfx940


          AMD GPU architecture


    when @3.3: device=ch4 netmod=ucx


      hcoll [false]                 false, true


          Enable support for Mellanox HCOLL accelerated collective operations library


    when @3.4:


      datatype-engine [auto]        auto, dataloop, yaksa


          controls the datatype engine to use


    when @4: device=ch4


      vci [false]                   false, true


          Enable multiple VCI (virtual communication interface) critical sections to improve performance of


          applications that do heavy concurrent MPIcommunications. Set MPIR_CVAR_CH4_NUM_VCIS=<N> to enable multiple


          vcis at runtime.
Installation Phases:


    autoreconf    configure    build    install
Build Dependencies:


    argobots  cray-pmi   gmake      hip           libfabric     libxml2      mxm        python  yaksa


    autoconf  cuda       gnuconfig  hsa-rocr-dev  libpciaccess  llvm-amdgpu  pkgconfig  slurm


    automake  findutils  hcoll      hwloc         libtool       m4           pmix       ucx
Link Dependencies:


    argobots  cuda   hip           hwloc      libpciaccess  llvm-amdgpu  pmix   ucx


    cray-pmi  hcoll  hsa-rocr-dev  libfabric  libxml2       mxm          slurm  yaksa
Run Dependencies:


    None
Virtual Packages: 


    mpich provides mpi@:4.0


    mpich@:3.2 provides mpi@:3.1


    mpich@:3.1 provides mpi@:3.0


    mpich@:1.2 provides mpi@:2.2


    mpich@:1.1 provides mpi@:2.1


    mpich@:1.0 provides mpi@:2.0
Available Build Phase Test Methods:


    None
Available Install Phase Test Methods:


    None
Stand-Alone/Smoke Test Methods:


    Mpi.test_mpi_hello  Mpich.test_cpi  Mpich.test_finalized  Mpich.test_manyrma  Mpich.test_sendrecv
Licenses: 


    None

Most of the information is self-explanatory. The safe versions are versions that Spack knows the checksum for, and it will use the checksum to verify that these versions download without errors or viruses.

Dependencies and virtual dependencies are described in more detail later.

spack versions¶

To see more available versions of a package, run spack versions. For example:

$ spack versions libelf


  0.8.13

There are two sections in the output. Safe versions are versions for which Spack has a checksum on file. It can verify that these versions are downloaded correctly.

In many cases, Spack can also show you what versions are available out on the web---these are remote versions. Spack gets this information by scraping it directly from package web pages. Depending on the package and how its releases are organized, Spack may or may not be able to find remote versions.

Installing and uninstalling¶

spack install¶

spack install will install any package shown by spack list. For example, To install the latest version of the mpileaks package, you might type this:

$ spack install mpileaks

If mpileaks depends on other packages, Spack will install the dependencies first. It then fetches the mpileaks tarball, expands it, verifies that it was downloaded without errors, builds it, and installs it in its own directory under $SPACK_ROOT/opt. You'll see a number of messages from Spack, a lot of build output, and a message that the package is installed.

$ spack install mpileaks
... dependency build output ...
==> Installing mpileaks-1.0-ph7pbnhl334wuhogmugriohcwempqry2
==> No binary for mpileaks-1.0-ph7pbnhl334wuhogmugriohcwempqry2 found: installing from source
==> mpileaks: Executing phase: 'autoreconf'
==> mpileaks: Executing phase: 'configure'
==> mpileaks: Executing phase: 'build'
==> mpileaks: Executing phase: 'install'
[+] ~/spack/opt/linux-rhel7-broadwell/gcc-8.1.0/mpileaks-1.0-ph7pbnhl334wuhogmugriohcwempqry2

The last line, with the [+], indicates where the package is installed.

Add the Spack debug option (one or more times) -- spack -d install mpileaks -- to get additional (and even more verbose) output.

Building a specific version¶

Spack can also build specific versions of a package. To do this, just add @ after the package name, followed by a version:

$ spack install mpich@3.0.4

Any number of versions of the same package can be installed at once without interfering with each other. This is good for multi-user sites, as installing a version that one user needs will not disrupt existing installations for other users.

In addition to different versions, Spack can customize the compiler, compile-time options (variants), compiler flags, and platform (for cross compiles) of an installation. Spack is unique in that it can also configure the dependencies a package is built with. For example, two configurations of the same version of a package, one built with boost 1.39.0, and the other version built with version 1.43.0, can coexist.

This can all be done on the command line using the spec syntax. Spack calls the descriptor used to refer to a particular package configuration a spec. In the commands above, mpileaks and mpileaks@3.0.4 are both valid specs. We'll talk more about how you can use them to customize an installation in Specs & dependencies.

Reusing installed dependencies¶

By default, when you run spack install, Spack tries hard to reuse existing installations as dependencies, either from a local store or from remote buildcaches if configured. This minimizes unwanted rebuilds of common dependencies, in particular if you update Spack frequently.

In case you want the latest versions and configurations to be installed instead, you can add the --fresh option:

$ spack install --fresh mpich

Reusing installations in this mode is "accidental", and happening only if there's a match between existing installations and what Spack would have installed anyhow.

You can use the spack spec -I mpich command to see what will be reused and what will be built before you install.

You can configure Spack to use the --fresh behavior by default in concretizer.yaml:

concretizer:


  reuse: false

spack uninstall¶

To uninstall a package, type spack uninstall <package>. This will ask the user for confirmation before completely removing the directory in which the package was installed.

$ spack uninstall mpich

If there are still installed packages that depend on the package to be uninstalled, spack will refuse to uninstall it.

To uninstall a package and every package that depends on it, you may give the --dependents option.

$ spack uninstall --dependents mpich

will display a list of all the packages that depend on mpich and, upon confirmation, will uninstall them in the right order.

A command like

$ spack uninstall mpich

may be ambiguous if multiple mpich configurations are installed. For example, if both mpich@3.0.2 and mpich@3.1 are installed, mpich could refer to either one. Because it cannot determine which one to uninstall, Spack will ask you either to provide a version number to remove the ambiguity or use the --all option to uninstall all of the matching packages.

You may force uninstall a package with the --force option

$ spack uninstall --force mpich

but you risk breaking other installed packages. In general, it is safer to remove dependent packages before removing their dependencies or use the --dependents option.

Garbage collection¶

When Spack builds software from sources, if often installs tools that are needed just to build or test other software. These are not necessary at runtime. To support cases where removing these tools can be a benefit Spack provides the spack gc ("garbage collector") command, which will uninstall all unneeded packages:

$ spack find
==> 24 installed packages
-- linux-ubuntu18.04-broadwell / gcc@9.0.1 ----------------------
autoconf@2.69    findutils@4.6.0  libiconv@1.16        libszip@2.1.1  m4@1.4.18    openjpeg@2.3.1  pkgconf@1.6.3  util-macros@1.19.1
automake@1.16.1  gdbm@1.18.1      libpciaccess@0.13.5  libtool@2.4.6  mpich@3.3.2  openssl@1.1.1d  readline@8.0   xz@5.2.4
cmake@3.16.1     hdf5@1.10.5      libsigsegv@2.12      libxml2@2.9.9  ncurses@6.1  perl@5.30.0     texinfo@6.5    zlib@1.2.11
$ spack gc
==> The following packages will be uninstalled:


    -- linux-ubuntu18.04-broadwell / gcc@9.0.1 ----------------------


    vn47edz autoconf@2.69    6m3f2qn findutils@4.6.0  ubl6bgk libtool@2.4.6  pksawhz openssl@1.1.1d  urdw22a readline@8.0


    ki6nfw5 automake@1.16.1  fklde6b gdbm@1.18.1      b6pswuo m4@1.4.18      k3s2csy perl@5.30.0     lp5ya3t texinfo@6.5


    ylvgsov cmake@3.16.1     5omotir libsigsegv@2.12  leuzbbh ncurses@6.1    5vmfbrq pkgconf@1.6.3   5bmv4tg util-macros@1.19.1
==> Do you want to proceed? [y/N] y
[ ... ]
$ spack find
==> 9 installed packages
-- linux-ubuntu18.04-broadwell / gcc@9.0.1 ----------------------
hdf5@1.10.5  libiconv@1.16  libpciaccess@0.13.5  libszip@2.1.1  libxml2@2.9.9  mpich@3.3.2  openjpeg@2.3.1  xz@5.2.4  zlib@1.2.11

In the example above Spack went through all the packages in the package database and removed everything that is not either:

You can check Viewing more metadata to see how to query for explicitly installed packages or Dependency types for a more thorough treatment of dependency types.

Marking packages explicit or implicit¶

By default, Spack will mark packages a user installs as explicitly installed, while all of its dependencies will be marked as implicitly installed. Packages can be marked manually as explicitly or implicitly installed by using spack mark. This can be used in combination with spack gc to clean up packages that are no longer required.

$ spack install m4
==> 29005: Installing libsigsegv
[...]
==> 29005: Installing m4
[...]
$ spack install m4 ^libsigsegv@2.11
==> 39798: Installing libsigsegv
[...]
==> 39798: Installing m4
[...]
$ spack find -d
==> 4 installed packages
-- linux-fedora32-haswell / gcc@10.1.1 --------------------------
libsigsegv@2.11
libsigsegv@2.12
m4@1.4.18


    libsigsegv@2.12
m4@1.4.18


    libsigsegv@2.11
$ spack gc
==> There are no unused specs. Spack's store is clean.
$ spack mark -i m4 ^libsigsegv@2.11
==> m4@1.4.18 : marking the package implicit
$ spack gc
==> The following packages will be uninstalled:


    -- linux-fedora32-haswell / gcc@10.1.1 --------------------------


    5fj7p2o libsigsegv@2.11  c6ensc6 m4@1.4.18
==> Do you want to proceed? [y/N]

In the example above, we ended up with two versions of m4 since they depend on different versions of libsigsegv. spack gc will not remove any of the packages since both versions of m4 have been installed explicitly and both versions of libsigsegv are required by the m4 packages.

spack mark can also be used to implement upgrade workflows. The following example demonstrates how the spack mark and spack gc can be used to only keep the current version of a package installed.

When updating Spack via git pull, new versions for either libsigsegv or m4 might be introduced. This will cause Spack to install duplicates. Since we only want to keep one version, we mark everything as implicitly installed before updating Spack. If there is no new version for either of the packages, spack install will simply mark them as explicitly installed and spack gc will not remove them.

$ spack install m4
==> 62843: Installing libsigsegv
[...]
==> 62843: Installing m4
[...]
$ spack mark -i -a
==> m4@1.4.18 : marking the package implicit
$ git pull
[...]
$ spack install m4
[...]
==> m4@1.4.18 : marking the package explicit
[...]
$ spack gc
==> There are no unused specs. Spack's store is clean.

When using this workflow for installations that contain more packages, care has to be taken to either only mark selected packages or issue spack install for all packages that should be kept.

You can check Viewing more metadata to see how to query for explicitly or implicitly installed packages.

Non-Downloadable Tarballs¶

The tarballs for some packages cannot be automatically downloaded by Spack. This could be for a number of reasons:

To install these packages, one must create a mirror and manually add the tarballs in question to it (see Mirrors (mirrors.yaml)):

$ mkdir ~/.spack/manual_mirror

mirrors:


  manual: file://~/.spack/manual_mirror

$ ls -l manual_mirror/galahad
-rw-------. 1 me me 11657206 Jun 21 19:25 galahad-2.60003.tar.gz

$ spack install galahad

Seeing installed packages¶

We know that spack list shows you the names of available packages, but how do you figure out which are already installed?

spack find¶

spack find shows the specs of installed packages. A spec is like a name, but it has a version, compiler, architecture, and build options associated with it. In spack, you can have many installations of the same package with different specs.

Running spack find with no arguments lists installed packages:

$ spack find
==> 74 installed packages.
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
ImageMagick@6.8.9-10  libdwarf@20130729  py-dateutil@2.4.0
adept-utils@1.0       libdwarf@20130729  py-ipython@2.3.1
atk@2.14.0            libelf@0.8.12      py-matplotlib@1.4.2
boost@1.55.0          libelf@0.8.13      py-nose@1.3.4
bzip2@1.0.6           libffi@3.1         py-numpy@1.9.1
cairo@1.14.0          libmng@2.0.2       py-pygments@2.0.1
callpath@1.0.2        libpng@1.6.16      py-pyparsing@2.0.3
cmake@3.0.2           libtiff@4.0.3      py-pyside@1.2.2
dbus@1.8.6            libtool@2.4.2      py-pytz@2014.10
dbus@1.9.0            libxcb@1.11        py-setuptools@11.3.1
dyninst@8.1.2         libxml2@2.9.2      py-six@1.9.0
fontconfig@2.11.1     libxml2@2.9.2      python@2.7.8
freetype@2.5.3        llvm@3.0           qhull@1.0
gdk-pixbuf@2.31.2     memaxes@0.5        qt@4.8.6
glib@2.42.1           mesa@8.0.5         qt@5.4.0
graphlib@2.0.0        mpich@3.0.4        readline@6.3
gtkplus@2.24.25       mpileaks@1.0       sqlite@3.8.5
harfbuzz@0.9.37       mrnet@4.1.0        stat@2.1.0
hdf5@1.8.13           ncurses@5.9        tcl@8.6.3
icu@54.1              netcdf@4.3.3       tk@src
jpeg@9a               openssl@1.0.1h     vtk@6.1.0
launchmon@1.0.1       pango@1.36.8       xcb-proto@1.11
lcms@2.6              pixman@0.32.6      xz@5.2.0
libdrm@2.4.33         py-dateutil@2.4.0  zlib@1.2.8
-- linux-debian7-x86_64 / gcc@4.9.2 --------------------------------
libelf@0.8.10  mpich@3.0.4

Packages are divided into groups according to their architecture and compiler. Within each group, Spack tries to keep the view simple, and only shows the version of installed packages.

Viewing more metadata¶

spack find can filter the package list based on the package name, spec, or a number of properties of their installation status. For example, missing dependencies of a spec can be shown with --missing, deprecated packages can be included with --deprecated, packages which were explicitly installed with spack install <package> can be singled out with --explicit and those which have been pulled in only as dependencies with --implicit.

In some cases, there may be different configurations of the same version of a package installed. For example, there are two installations of libdwarf@20130729 above. We can look at them in more detail using spack find --deps, and by asking only to show libdwarf packages:

$ spack find --deps libdwarf
==> 2 installed packages.
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------


    libdwarf@20130729-d9b90962


        ^libelf@0.8.12


    libdwarf@20130729-b52fac98


        ^libelf@0.8.13

Now we see that the two instances of libdwarf depend on different versions of libelf: 0.8.12 and 0.8.13. This view can become complicated for packages with many dependencies. If you just want to know whether two packages' dependencies differ, you can use spack find --long:

$ spack find --long libdwarf
==> 2 installed packages.
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
libdwarf@20130729-d9b90962  libdwarf@20130729-b52fac98

Now the libdwarf installs have hashes after their names. These are hashes over all of the dependencies of each package. If the hashes are the same, then the packages have the same dependency configuration.

If you want to know the path where each package is installed, you can use spack find --paths:

$ spack find --paths
==> 74 installed packages.
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------


    ImageMagick@6.8.9-10  ~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/ImageMagick@6.8.9-10-4df950dd


    adept-utils@1.0       ~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/adept-utils@1.0-5adef8da


    atk@2.14.0            ~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/atk@2.14.0-3d09ac09


    boost@1.55.0          ~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/boost@1.55.0


    bzip2@1.0.6           ~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/bzip2@1.0.6


    cairo@1.14.0          ~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/cairo@1.14.0-fcc2ab44


    callpath@1.0.2        ~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/callpath@1.0.2-5dce4318
...

You can restrict your search to a particular package by supplying its name:

$ spack find --paths libelf
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------


    libelf@0.8.11  ~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/libelf@0.8.11


    libelf@0.8.12  ~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/libelf@0.8.12


    libelf@0.8.13  ~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/libelf@0.8.13

Spec queries¶

spack find actually does a lot more than this. You can use specs to query for specific configurations and builds of each package. If you want to find only libelf versions greater than version 0.8.12, you could say:

$ spack find libelf@0.8.12:
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------


    libelf@0.8.12  libelf@0.8.13

Finding just the versions of libdwarf built with a particular version of libelf would look like this:

$ spack find --long libdwarf ^libelf@0.8.12
==> 1 installed packages.
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
libdwarf@20130729-d9b90962

We can also search for packages that have a certain attribute. For example, spack find libdwarf +debug will show only installations of libdwarf with the 'debug' compile-time option enabled.

The full spec syntax is discussed in detail in Specs & dependencies.

Machine-readable output¶

If you only want to see very specific things about installed packages, Spack has some options for you. spack find --format can be used to output only specific fields:

$ spack find --format "{name}-{version}-{hash}"
autoconf-2.69-icynozk7ti6h4ezzgonqe6jgw5f3ulx4
automake-1.16.1-o5v3tc77kesgonxjbmeqlwfmb5qzj7zy
bzip2-1.0.6-syohzw57v2jfag5du2x4bowziw3m5p67
bzip2-1.0.8-zjny4jwfyvzbx6vii3uuekoxmtu6eyuj
cmake-3.15.1-7cf6onn52gywnddbmgp7qkil4hdoxpcb
...

or:

$ spack find --format "{hash:7}"
icynozk
o5v3tc7
syohzw5
zjny4jw
7cf6onn
...

This uses the same syntax as described in documentation for format() -- you can use any of the options there. This is useful for passing metadata about packages to other command-line tools.

Alternately, if you want something even more machine readable, you can output each spec as JSON records using spack find --json. This will output metadata on specs and all dependencies as json:

$ spack find --json sqlite@3.28.0
[


 {


  "name": "sqlite",


  "hash": "3ws7bsihwbn44ghf6ep4s6h4y2o6eznv",


  "version": "3.28.0",


  "arch": {


   "platform": "darwin",


   "platform_os": "mojave",


   "target": "x86_64"


  },


  "compiler": {


   "name": "apple-clang",


   "version": "10.0.0"


  },


  "namespace": "builtin",


  "parameters": {


   "fts": true,


   "functions": false,


   "cflags": [],


   "cppflags": [],


   "cxxflags": [],


   "fflags": [],


   "ldflags": [],


   "ldlibs": []


  },


  "dependencies": {


   "readline": {


    "hash": "722dzmgymxyxd6ovjvh4742kcetkqtfs",


    "type": [


     "build",


     "link"


    ]


   }


  }


 },


 ...
]

You can use this with tools like jq to quickly create JSON records structured the way you want:

$ spack find --json sqlite@3.28.0 | jq -C '.[] | { name, version, hash }'
{


  "name": "sqlite",


  "version": "3.28.0",


  "hash": "3ws7bsihwbn44ghf6ep4s6h4y2o6eznv"
}
{


  "name": "readline",


  "version": "7.0",


  "hash": "722dzmgymxyxd6ovjvh4742kcetkqtfs"
}
{


  "name": "ncurses",


  "version": "6.1",


  "hash": "zvaa4lhlhilypw5quj3akyd3apbq5gap"
}

spack diff¶

It's often the case that you have two versions of a spec that you need to disambiguate. Let's say that we've installed two variants of zlib, one with and one without the optimize variant:

$ spack install zlib
$ spack install zlib -optimize

When we do spack find we see the two versions.

$ spack find zlib
==> 2 installed packages
-- linux-ubuntu20.04-skylake / gcc@9.3.0 ------------------------
zlib@1.2.11  zlib@1.2.11

Let's now say that we want to uninstall zlib. We run the command, and hit a problem real quickly since we have two!

$ spack uninstall zlib
==> Error: zlib matches multiple packages:


    -- linux-ubuntu20.04-skylake / gcc@9.3.0 ------------------------


    efzjziy zlib@1.2.11  sl7m27m zlib@1.2.11
==> Error: You can either:


    a) use a more specific spec, or


    b) specify the spec by its hash (e.g. `spack uninstall /hash`), or


    c) use `spack uninstall --all` to uninstall ALL matching specs.

Oh no! We can see from the above that we have two different versions of zlib installed, and the only difference between the two is the hash. This is a good use case for spack diff, which can easily show us the "diff" or set difference between properties for two packages. Let's try it out. Since the only difference we see in the spack find view is the hash, let's use spack diff to look for more detail. We will provide the two hashes:

$ spack diff /efzjziy /sl7m27m
==> Warning: This interface is subject to change.
--- zlib@1.2.11efzjziyc3dmb5h5u5azsthgbgog5mj7g
+++ zlib@1.2.11sl7m27mzkbejtkrajigj3a3m37ygv4u2
@@ variant_value @@
-  zlib optimize False
+  zlib optimize True

The output is colored, and written in the style of a git diff. This means that you can copy and paste it into a GitHub markdown as a code block with language "diff" and it will render nicely! Here is an example:

```diff
--- zlib@1.2.11/efzjziyc3dmb5h5u5azsthgbgog5mj7g
+++ zlib@1.2.11/sl7m27mzkbejtkrajigj3a3m37ygv4u2
@@ variant_value @@
-  zlib optimize False
+  zlib optimize True
```

Awesome! Now let's read the diff. It tells us that our first zlib was built with ~optimize (False) and the second was built with +optimize (True). You can't see it in the docs here, but the output above is also colored based on the content being an addition (+) or subtraction (-).

This is a small example, but you will be able to see differences for any attributes on the installation spec. Running spack diff A B means we'll see which spec attributes are on B but not on A (green) and which are on A but not on B (red). Here is another example with an additional difference type, version:

$ spack diff python@2.7.8 python@3.8.11
==> Warning: This interface is subject to change.
--- python@2.7.8/tsxdi6gl4lihp25qrm4d6nys3nypufbf
+++ python@3.8.11/yjtseru4nbpllbaxb46q7wfkyxbuvzxx
@@ variant_value @@
-  python patches a8c52415a8b03c0e5f28b5d52ae498f7a7e602007db2b9554df28cd5685839b8
+  python patches 0d98e93189bc278fbc37a50ed7f183bd8aaf249a8e1670a465f0db6bb4f8cf87
@@ version @@
-  openssl 1.0.2u
+  openssl 1.1.1k
-  python 2.7.8
+  python 3.8.11

Let's say that we were only interested in one kind of attribute above, version. We can ask the command to only output this attribute. To do this, you'd add the --attribute for attribute parameter, which defaults to all. Here is how you would filter to show just versions:

$ spack diff --attribute version python@2.7.8 python@3.8.11
==> Warning: This interface is subject to change.
--- python@2.7.8/tsxdi6gl4lihp25qrm4d6nys3nypufbf
+++ python@3.8.11/yjtseru4nbpllbaxb46q7wfkyxbuvzxx
@@ version @@
-  openssl 1.0.2u
+  openssl 1.1.1k
-  python 2.7.8
+  python 3.8.11

And you can add as many attributes as you'd like with multiple --attribute arguments (for lots of attributes, you can use -a for short). Finally, if you want to view the data as json (and possibly pipe into an output file) just add --json:

$ spack diff --json python@2.7.8 python@3.8.11

This data will be much longer because along with the differences for A vs. B and B vs. A, the JSON output also showsthe intersection.

Using installed packages¶

There are several different ways to use Spack packages once you have installed them. As you've seen, spack packages are installed into long paths with hashes, and you need a way to get them into your path. The easiest way is to use spack load, which is described in the next section.

Some more advanced ways to use Spack packages include:

spack load / unload¶

If you have shell support enabled you can use the spack load command to quickly get a package on your PATH.

For example this will add the mpich package built with gcc to your path:

$ spack install mpich %gcc@4.4.7
# ... wait for install ...
$ spack load mpich %gcc@4.4.7
$ which mpicc
~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/mpich@3.0.4/bin/mpicc

These commands will add appropriate directories to your PATH and MANPATH according to the prefix inspections defined in your modules configuration. When you no longer want to use a package, you can type unload or unuse similarly:

$ spack unload mpich %gcc@4.4.7

Ambiguous specs¶

If a spec used with load/unload or is ambiguous (i.e. more than one installed package matches it), then Spack will warn you:

$ spack load libelf
==> Error: libelf matches multiple packages.
Matching packages:


  qmm4kso libelf@0.8.13%gcc@4.4.7 arch=linux-debian7-x86_64


  cd2u6jt libelf@0.8.13%intel@15.0.0 arch=linux-debian7-x86_64
Use a more specific spec

You can either type the spack load command again with a fully qualified argument, or you can add just enough extra constraints to identify one package. For example, above, the key differentiator is that one libelf is built with the Intel compiler, while the other used gcc. You could therefore just type:

$ spack load libelf %intel

To identify just the one built with the Intel compiler. If you want to be very specific, you can load it by its hash. For example, to load the first libelf above, you would run:

$ spack load /qmm4kso

To see which packages that you have loaded to your environment you would use spack find --loaded.

$ spack find --loaded
==> 2 installed packages
-- linux-debian7 / gcc@4.4.7 ------------------------------------
libelf@0.8.13
-- linux-debian7 / intel@15.0.0 ---------------------------------
libelf@0.8.13

You can also use spack load --list to get the same output, but it does not have the full set of query options that spack find offers.

We'll learn more about Spack's spec syntax in the next section.

Specs & dependencies¶

We know that spack install, spack uninstall, and other commands take a package name with an optional version specifier. In Spack, that descriptor is called a spec. Spack uses specs to refer to a particular build configuration (or configurations) of a package. Specs are more than a package name and a version; you can use them to specify the compiler, compiler version, architecture, compile options, and dependency options for a build. In this section, we'll go over the full syntax of specs.

Here is an example of a much longer spec than we've seen thus far:

mpileaks @1.2:1.4 %gcc@4.7.5 +debug -qt target=x86_64 ^callpath @1.1 %gcc@4.7.2

If provided to spack install, this will install the mpileaks library at some version between 1.2 and 1.4 (inclusive), built using gcc at version 4.7.5 for a generic x86_64 architecture, with debug options enabled, and without Qt support. Additionally, it says to link it with the callpath library (which it depends on), and to build callpath with gcc 4.7.2. Most specs will not be as complicated as this one, but this is a good example of what is possible with specs.

More formally, a spec consists of the following pieces:

There are two things to notice here. The first is that specs are recursively defined. That is, each dependency after ^ is a spec itself. The second is that everything is optional except for the initial package name identifier. Users can be as vague or as specific as they want about the details of building packages, and this makes spack good for beginners and experts alike.

To really understand what's going on above, we need to think about how software is structured. An executable or a library (these are generally the artifacts produced by building software) depends on other libraries in order to run. We can represent the relationship between a package and its dependencies as a graph. Here is the full dependency graph for mpileaks: [graph]

Each box above is a package and each arrow represents a dependency on some other package. For example, we say that the package mpileaks depends on callpath and mpich. mpileaks also depends indirectly on dyninst, libdwarf, and libelf, in that these libraries are dependencies of callpath. To install mpileaks, Spack has to build all of these packages. Dependency graphs in Spack have to be acyclic, and the depends on relationship is directional, so this is a directed, acyclic graph or DAG.

The package name identifier in the spec is the root of some dependency DAG, and the DAG itself is implicit. Spack knows the precise dependencies among packages, but users do not need to know the full DAG structure. Each ^ in the full spec refers to some dependency of the root package. Spack will raise an error if you supply a name after ^ that the root does not actually depend on (e.g. mpileaks ^emacs@23.3).

Spack further simplifies things by only allowing one configuration of each package within any single build. Above, both mpileaks and callpath depend on mpich, but mpich appears only once in the DAG. You cannot build an mpileaks version that depends on one version of mpich and on a callpath version that depends on some other version of mpich. In general, such a configuration would likely behave unexpectedly at runtime, and Spack enforces this to ensure a consistent runtime environment.

The point of specs is to abstract this full DAG from Spack users. If a user does not care about the DAG at all, she can refer to mpileaks by simply writing mpileaks. If she knows that mpileaks indirectly uses dyninst and she wants a particular version of dyninst, then she can refer to mpileaks ^dyninst@8.1. Spack will fill in the rest when it parses the spec; the user only needs to know package names and minimal details about their relationship.

When spack prints out specs, it sorts package names alphabetically to normalize the way they are displayed, but users do not need to worry about this when they write specs. The only restriction on the order of dependencies within a spec is that they appear after the root package. For example, these two specs represent exactly the same configuration:

mpileaks ^callpath@1.0 ^libelf@0.8.3
mpileaks ^libelf@0.8.3 ^callpath@1.0

You can put all the same modifiers on dependency specs that you would put on the root spec. That is, you can specify their versions, compilers, variants, and architectures just like any other spec. Specifiers are associated with the nearest package name to their left. For example, above, @1.1 and %gcc@4.7.2 associates with the callpath package, while @1.2:1.4, %gcc@4.7.5, +debug, -qt, and target=haswell os=CNL10 all associate with the mpileaks package.

In the diagram above, mpileaks depends on mpich with an unspecified version, but packages can depend on other packages with constraints by adding more specifiers. For example, mpileaks could depend on mpich@1.2: if it can only build with version 1.2 or higher of mpich.

Below are more details about the specifiers that you can add to specs.

Version specifier¶

A version specifier pkg@<specifier> comes after a package name and starts with @. It can be something abstract that matches multiple known versions, or a specific version. During concretization, Spack will pick the optimal version within the spec's constraints according to policies set for the particular Spack installation.

The version specifier can be a specific version, such as @=1.0.0 or @=1.2a7. Or, it can be a range of versions, such as @1.0:1.5. Version ranges are inclusive, so this example includes both 1.0 and any 1.5.x version. Version ranges can be unbounded, e.g. @:3 means any version up to and including 3. This would include 3.4 and 3.4.2. Similarly, @4.2: means any version above and including 4.2. As a short-hand, @3 is equivalent to the range @3:3 and includes any version with major version 3.

Notice that you can distinguish between the specific version @=3.2 and the range @3.2. This is useful for packages that follow a versioning scheme that omits the zero patch version number: 3.2, 3.2.1, 3.2.2, etc. In general it is preferable to use the range syntax @3.2, since ranges also match versions with one-off suffixes, such as 3.2-custom.

A version specifier can also be a list of ranges and specific versions, separated by commas. For example, @1.0:1.5,=1.7.1 matches any version in the range 1.0:1.5 and the specific version 1.7.1.

For packages with a git attribute, git references may be specified instead of a numerical version i.e. branches, tags and commits. Spack will stage and build based off the git reference provided. Acceptable syntaxes for this are:

 # commit hashes
foo@abcdef1234abcdef1234abcdef1234abcdef1234    # 40 character hashes are automatically treated as git commits
foo@git.abcdef1234abcdef1234abcdef1234abcdef1234


 # branches and tags
foo@git.develop # use the develop branch
foo@git.0.19 # use the 0.19 tag

Spack always needs to associate a Spack version with the git reference, which is used for version comparison. This Spack version is heuristically taken from the closest valid git tag among ancestors of the git ref.

Once a Spack version is associated with a git ref, it always printed with the git ref. For example, if the commit @git.abcdefg is tagged 0.19, then the spec will be shown as @git.abcdefg=0.19.

If the git ref is not exactly a tag, then the distance to the nearest tag is also part of the resolved version. @git.abcdefg=0.19.git.8 means that the commit is 8 commits away from the 0.19 tag.

In cases where Spack cannot resolve a sensible version from a git ref, users can specify the Spack version to use for the git ref. This is done by appending = and the Spack version to the git ref. For example:

foo@git.my_ref=3.2 # use the my_ref tag or branch, but treat it as version 3.2 for version comparisons
foo@git.abcdef1234abcdef1234abcdef1234abcdef1234=develop # use the given commit, but treat it as develop for version comparisons

Details about how versions are compared and how Spack determines if one version is less than another are discussed in the developer guide.

Compiler specifier¶

A compiler specifier comes somewhere after a package name and starts with %. It tells Spack what compiler(s) a particular package should be built with. After the % should come the name of some registered Spack compiler. This might include gcc, or intel, but the specific compilers available depend on the site. You can run spack compilers to get a list; more on this below.

The compiler spec can be followed by an optional compiler version. A compiler version specifier looks exactly like a package version specifier. Version specifiers will associate with the nearest package name or compiler specifier to their left in the spec.

If the compiler spec is omitted, Spack will choose a default compiler based on site policies.

Variants¶

Variants are named options associated with a particular package. They are optional, as each package must provide default values for each variant it makes available. Variants can be specified using a flexible parameter syntax name=<value>. For example, spack install mercury debug=True will install mercury built with debug flags. The names of particular variants available for a package depend on what was provided by the package author. spack info <package> will provide information on what build variants are available.

For compatibility with earlier versions, variants which happen to be boolean in nature can be specified by a syntax that represents turning options on and off. For example, in the previous spec we could have supplied mercury +debug with the same effect of enabling the debug compile time option for the libelf package.

Depending on the package a variant may have any default value. For mercury here, debug is False by default, and we turned it on with debug=True or +debug. If a variant is True by default you can turn it off by either adding -name or ~name to the spec.

There are two syntaxes here because, depending on context, ~ and - may mean different things. In most shells, the following will result in the shell performing home directory substitution:

mpileaks ~debug   # shell may try to substitute this!
mpileaks~debug    # use this instead

If there is a user called debug, the ~ will be incorrectly expanded. In this situation, you would want to write libelf -debug. However, - can be ambiguous when included after a package name without spaces:

mpileaks-debug     # wrong!
mpileaks -debug    # right

Spack allows the - character to be part of package names, so the above will be interpreted as a request for the mpileaks-debug package, not a request for mpileaks built without debug options. In this scenario, you should write mpileaks~debug to avoid ambiguity.

When spack normalizes specs, it prints them out with no spaces boolean variants using the backwards compatibility syntax and uses only ~ for disabled boolean variants. The - and spaces on the command line are provided for convenience and legibility.

Spack allows variants to propagate their value to the package's dependency by using ++, --, and ~~ for boolean variants. For example, for a debug variant:

mpileaks ++debug   # enabled debug will be propagated to dependencies
mpileaks +debug    # only mpileaks will have debug enabled

To propagate the value of non-boolean variants Spack uses name==value. For example, for the stackstart variant:

mpileaks stackstart==4   # variant will be propagated to dependencies
mpileaks stackstart=4    # only mpileaks will have this variant value

Compiler Flags¶

Compiler flags are specified using the same syntax as non-boolean variants, but fulfill a different purpose. While the function of a variant is set by the package, compiler flags are used by the compiler wrappers to inject flags into the compile line of the build. Additionally, compiler flags can be inherited by dependencies by using ==. spack install libdwarf cppflags=="-g" will install both libdwarf and libelf with the -g flag injected into their compile line.

NOTE:

Notice that the value of the compiler flags must be quoted if it contains any spaces. Any of cppflags=-O3, cppflags="-O3", cppflags='-O3', and cppflags="-O3 -fPIC" are acceptable, but cppflags=-O3 -fPIC is not. Additionally, if the value of the compiler flags is not the last thing on the line, it must be followed by a space. The command spack install libelf cppflags="-O3"%intel will be interpreted as an attempt to set cppflags="-O3%intel".

The six compiler flags are injected in the order of implicit make commands in GNU Autotools. If all flags are set, the order is $cppflags $cflags|$cxxflags $ldflags <command> $ldlibs for C and C++ and $fflags $cppflags $ldflags <command> $ldlibs for Fortran.

Compiler environment variables and additional RPATHs¶

Sometimes compilers require setting special environment variables to operate correctly. Spack handles these cases by allowing custom environment modifications in the environment attribute of the compiler configuration section. See also the Environment Modifications section of the configuration files docs for more information.

It is also possible to specify additional RPATHs that the compiler will add to all executables generated by that compiler. This is useful for forcing certain compilers to RPATH their own runtime libraries, so that executables will run without the need to set LD_LIBRARY_PATH.

compilers:


  - compiler:


      spec: gcc@4.9.3


      paths:


        cc: /opt/gcc/bin/gcc


        c++: /opt/gcc/bin/g++


        f77: /opt/gcc/bin/gfortran


        fc: /opt/gcc/bin/gfortran


      environment:


        unset:


          - BAD_VARIABLE


        set:


          GOOD_VARIABLE_NUM: 1


          GOOD_VARIABLE_STR: good


        prepend_path:


          PATH: /path/to/binutils


        append_path:


          LD_LIBRARY_PATH: /opt/gcc/lib


      extra_rpaths:


      - /path/to/some/compiler/runtime/directory


      - /path/to/some/other/compiler/runtime/directory

Architecture specifiers¶

Each node in the dependency graph of a spec has an architecture attribute. This attribute is a triplet of platform, operating system and processor. You can specify the elements either separately, by using the reserved keywords platform, os and target:

$ spack install libelf platform=linux
$ spack install libelf os=ubuntu18.04
$ spack install libelf target=broadwell

or together by using the reserved keyword arch:

$ spack install libelf arch=cray-CNL10-haswell

Normally users don't have to bother specifying the architecture if they are installing software for their current host, as in that case the values will be detected automatically. If you need fine-grained control over which packages use which targets (or over all packages' default target), see Package Preferences.

Support for specific microarchitectures¶

Spack knows how to detect and optimize for many specific microarchitectures (including recent Intel, AMD and IBM chips) and encodes this information in the target portion of the architecture specification. A complete list of the microarchitectures known to Spack can be obtained in the following way:

$ spack arch --known-targets
Generic architectures (families)


    aarch64  armv8.1a  armv8.3a  armv8.5a  ppc    ppc64le  riscv64  sparc64  x86_64     x86_64_v3


    arm      armv8.2a  armv8.4a  armv9.0a  ppc64  ppcle    sparc    x86      x86_64_v2  x86_64_v4
GenuineIntel - x86


    i686  pentium2  pentium3  pentium4  prescott
GenuineIntel - x86_64


    nocona  nehalem   sandybridge  haswell    skylake  cannonlake      cascadelake  sapphirerapids


    core2   westmere  ivybridge    broadwell  mic_knl  skylake_avx512  icelake
AuthenticAMD - x86_64


    k10  bulldozer  piledriver  zen  steamroller  zen2  zen3  excavator  zen4
IBM - ppc64


    power7  power8  power9  power10
IBM - ppc64le


    power8le  power9le  power10le
Cavium - aarch64


    thunderx2
Fujitsu - aarch64


    a64fx
ARM - aarch64


    cortex_a72  neoverse_n1  neoverse_v1  neoverse_v2
Apple - aarch64


    m1  m2
SiFive - riscv64


    u74mc

When a spec is installed Spack matches the compiler being used with the microarchitecture being targeted to inject appropriate optimization flags at compile time. Giving a command such as the following:

$ spack install zlib%gcc@9.0.1 target=icelake

will produce compilation lines similar to:

$ /usr/bin/gcc-9 -march=icelake-client -mtune=icelake-client -c ztest10532.c
$ /usr/bin/gcc-9 -march=icelake-client -mtune=icelake-client -c -fPIC -O2 ztest10532.
...

where the flags -march=icelake-client -mtune=icelake-client are injected by Spack based on the requested target and compiler.

If Spack knows that the requested compiler can't optimize for the current target or can't build binaries for that target at all, it will exit with a meaningful error message:

$ spack install zlib%gcc@5.5.0 target=icelake
==> Error: cannot produce optimized binary for micro-architecture "icelake" with gcc@5.5.0 [supported compiler versions are 8:]

When instead an old compiler is selected on a recent enough microarchitecture but there is no explicit target specification, Spack will optimize for the best match it can find instead of failing:

$ spack arch
linux-ubuntu18.04-broadwell
$ spack spec zlib%gcc@4.8
Input spec
--------------------------------
zlib%gcc@4.8
Concretized
--------------------------------
zlib@1.2.11%gcc@4.8+optimize+pic+shared arch=linux-ubuntu18.04-haswell
$ spack spec zlib%gcc@9.0.1
Input spec
--------------------------------
zlib%gcc@9.0.1
Concretized
--------------------------------
zlib@1.2.11%gcc@9.0.1+optimize+pic+shared arch=linux-ubuntu18.04-broadwell

In the snippet above, for instance, the microarchitecture was demoted to haswell when compiling with gcc@4.8 since support to optimize for broadwell starts from gcc@4.9:.

Finally, if Spack has no information to match compiler and target, it will proceed with the installation but avoid injecting any microarchitecture specific flags.

WARNING:

Virtual dependencies¶

The dependency graph for mpileaks we saw above wasn't quite accurate. mpileaks uses MPI, which is an interface that has many different implementations. Above, we showed mpileaks and callpath depending on mpich, which is one particular implementation of MPI. However, we could build either with another implementation, such as openmpi or mvapich.

Spack represents interfaces like this using virtual dependencies. The real dependency DAG for mpileaks looks like this: [graph]

Notice that mpich has now been replaced with mpi. There is no real MPI package, but some packages provide the MPI interface, and these packages can be substituted in for mpi when mpileaks is built.

You can see what virtual packages a particular package provides by getting info on it:

$ spack info --virtuals mpich
AutotoolsPackage:   mpich
Description:


    MPICH is a high performance and widely portable implementation of the


    Message Passing Interface (MPI) standard.
Homepage: https://www.mpich.org
Preferred version:  


    4.1.2      https://www.mpich.org/static/downloads/4.1.2/mpich-4.1.2.tar.gz
Safe versions:  


    develop    [git] https://github.com/pmodels/mpich.git


    4.1.2      https://www.mpich.org/static/downloads/4.1.2/mpich-4.1.2.tar.gz


    4.1.1      https://www.mpich.org/static/downloads/4.1.1/mpich-4.1.1.tar.gz


    4.1        https://www.mpich.org/static/downloads/4.1/mpich-4.1.tar.gz


    4.0.3      https://www.mpich.org/static/downloads/4.0.3/mpich-4.0.3.tar.gz


    4.0.2      https://www.mpich.org/static/downloads/4.0.2/mpich-4.0.2.tar.gz


    4.0.1      https://www.mpich.org/static/downloads/4.0.1/mpich-4.0.1.tar.gz


    4.0        https://www.mpich.org/static/downloads/4.0/mpich-4.0.tar.gz


    3.4.3      https://www.mpich.org/static/downloads/3.4.3/mpich-3.4.3.tar.gz


    3.4.2      https://www.mpich.org/static/downloads/3.4.2/mpich-3.4.2.tar.gz


    3.4.1      https://www.mpich.org/static/downloads/3.4.1/mpich-3.4.1.tar.gz


    3.4        https://www.mpich.org/static/downloads/3.4/mpich-3.4.tar.gz


    3.3.2      https://www.mpich.org/static/downloads/3.3.2/mpich-3.3.2.tar.gz


    3.3.1      https://www.mpich.org/static/downloads/3.3.1/mpich-3.3.1.tar.gz


    3.3        https://www.mpich.org/static/downloads/3.3/mpich-3.3.tar.gz


    3.2.1      https://www.mpich.org/static/downloads/3.2.1/mpich-3.2.1.tar.gz


    3.2        https://www.mpich.org/static/downloads/3.2/mpich-3.2.tar.gz


    3.1.4      https://www.mpich.org/static/downloads/3.1.4/mpich-3.1.4.tar.gz


    3.1.3      https://www.mpich.org/static/downloads/3.1.3/mpich-3.1.3.tar.gz


    3.1.2      https://www.mpich.org/static/downloads/3.1.2/mpich-3.1.2.tar.gz


    3.1.1      https://www.mpich.org/static/downloads/3.1.1/mpich-3.1.1.tar.gz


    3.1        https://www.mpich.org/static/downloads/3.1/mpich-3.1.tar.gz


    3.0.4      https://www.mpich.org/static/downloads/3.0.4/mpich-3.0.4.tar.gz
Deprecated versions:  


    None
Variants:


    argobots [false]                false, true


        Enable Argobots support


    build_system [autotools]        autotools


        Build systems supported by the package


    cuda [false]                    false, true


        Build with CUDA


    device [ch4]                    ch3, ch4


        Abstract Device Interface (ADI)


        implementation. The ch4 device is in experimental state for versions


        before 3.4.


    fortran [true]                  false, true


        Enable Fortran support


    hwloc [true]                    false, true


        Use external hwloc package


    hydra [true]                    false, true


        Build the hydra process manager


    libxml2 [true]                  false, true


        Use libxml2 for XML support instead of the custom minimalistic implementation


    netmod [ofi]                    mxm, ofi, tcp, ucx


        Network module. Only single netmod builds are


        supported. For ch3 device configurations, this presumes the


        ch3:nemesis communication channel. ch3:sock is not supported by this


        spack package at this time.


    pci [true]                      false, true


        Support analyzing devices on PCI bus


    pmi [pmi]                       cray, off, pmi, pmi2, pmix


        PMI interface.


    rocm [false]                    false, true


        Enable ROCm support


    romio [true]                    false, true


        Enable ROMIO MPI I/O implementation


    slurm [false]                   false, true


        Enable SLURM support


    verbs [false]                   false, true


        Build support for OpenFabrics verbs.


    wrapperrpath [true]             false, true


        Enable wrapper rpath


    when +cuda


      cuda_arch [none]              none, 10, 11, 12, 13, 20, 21, 30, 32, 35, 37, 50, 52, 53, 60, 61, 62, 70, 72, 75,


                                    80, 86, 87, 89, 90


          CUDA architecture


    when +rocm


      amdgpu_target [none]          none, gfx1010, gfx1011, gfx1012, gfx1013, gfx1030, gfx1031, gfx1032, gfx1033,


                                    gfx1034, gfx1035, gfx1036, gfx1100, gfx1101, gfx1102, gfx1103, gfx701, gfx801,


                                    gfx802, gfx803, gfx900, gfx900:xnack-, gfx902, gfx904, gfx906, gfx906:xnack-,


                                    gfx908, gfx908:xnack-, gfx909, gfx90a, gfx90a:xnack+, gfx90a:xnack-, gfx90c,


                                    gfx940


          AMD GPU architecture


    when @3.3: device=ch4 netmod=ucx


      hcoll [false]                 false, true


          Enable support for Mellanox HCOLL accelerated collective operations library


    when @3.4:


      datatype-engine [auto]        auto, dataloop, yaksa


          controls the datatype engine to use


    when @4: device=ch4


      vci [false]                   false, true


          Enable multiple VCI (virtual communication interface) critical sections to improve performance of


          applications that do heavy concurrent MPIcommunications. Set MPIR_CVAR_CH4_NUM_VCIS=<N> to enable multiple


          vcis at runtime.
Build Dependencies:


    argobots  cray-pmi   gmake      hip           libfabric     libxml2      mxm        python  yaksa


    autoconf  cuda       gnuconfig  hsa-rocr-dev  libpciaccess  llvm-amdgpu  pkgconfig  slurm


    automake  findutils  hcoll      hwloc         libtool       m4           pmix       ucx
Link Dependencies:


    argobots  cuda   hip           hwloc      libpciaccess  llvm-amdgpu  pmix   ucx


    cray-pmi  hcoll  hsa-rocr-dev  libfabric  libxml2       mxm          slurm  yaksa
Run Dependencies:


    None
Virtual Packages: 


    mpich provides mpi@:4.0


    mpich@:3.2 provides mpi@:3.1


    mpich@:3.1 provides mpi@:3.0


    mpich@:1.2 provides mpi@:2.2


    mpich@:1.1 provides mpi@:2.1


    mpich@:1.0 provides mpi@:2.0
Licenses: 


    None

Spack is unique in that its virtual packages can be versioned, just like regular packages. A particular version of a package may provide a particular version of a virtual package, and we can see above that mpich versions 1 and above provide all mpi interface versions up to 1, and mpich versions 3 and above provide mpi versions up to 3. A package can depend on a particular version of a virtual package, e.g. if an application needs MPI-2 functions, it can depend on mpi@2: to indicate that it needs some implementation that provides MPI-2 functions.

Constraining virtual packages¶

When installing a package that depends on a virtual package, you can opt to specify the particular provider you want to use, or you can let Spack pick. For example, if you just type this:

$ spack install mpileaks

Then spack will pick a provider for you according to site policies. If you really want a particular version, say mpich, then you could run this instead:

$ spack install mpileaks ^mpich

This forces spack to use some version of mpich for its implementation. As always, you can be even more specific and require a particular mpich version:

$ spack install mpileaks ^mpich@3

The mpileaks package in particular only needs MPI-1 commands, so any MPI implementation will do. If another package depends on mpi@2 and you try to give it an insufficient MPI implementation (e.g., one that provides only mpi@:1), then Spack will raise an error. Likewise, if you try to plug in some package that doesn't provide MPI, Spack will raise an error.

Explicit binding of virtual dependencies¶

There are packages that provide more than just one virtual dependency. When interacting with them, users might want to utilize just a subset of what they could provide, and use other providers for virtuals they need.

It is possible to be more explicit and tell Spack which dependency should provide which virtual, using a special syntax:

$ spack spec strumpack ^[virtuals=mpi] intel-parallel-studio+mkl ^[virtuals=lapack] openblas

Concretizing the spec above produces the following DAG:

where intel-parallel-studio could provide mpi, lapack, and blas but is used only for the former. The lapack and blas dependencies are satisfied by openblas.

Specifying Specs by Hash¶

Complicated specs can become cumbersome to enter on the command line, especially when many of the qualifications are necessary to distinguish between similar installs. To avoid this, when referencing an existing spec, Spack allows you to reference specs by their hash. We previously discussed the spec hash that Spack computes. In place of a spec in any command, substitute /<hash> where <hash> is any amount from the beginning of a spec hash.

For example, lets say that you accidentally installed two different mvapich2 installations. If you want to uninstall one of them but don't know what the difference is, you can run:

$ spack find --long mvapich2
==> 2 installed packages.
-- linux-centos7-x86_64 / gcc@6.3.0 ----------
qmt35td mvapich2@2.2%gcc
er3die3 mvapich2@2.2%gcc

You can then uninstall the latter installation using:

$ spack uninstall /er3die3

Or, if you want to build with a specific installation as a dependency, you can use:

$ spack install trilinos ^/er3die3

If the given spec hash is sufficiently long as to be unique, Spack will replace the reference with the spec to which it refers. Otherwise, it will prompt for a more qualified hash.

Note that this will not work to reinstall a dependency uninstalled by spack uninstall --force.

spack providers¶

You can see what packages provide a particular virtual package using spack providers. If you wanted to see what packages provide mpi, you would just run:

$ spack providers mpi
cray-mpich     intel-mpi              mpich@:1.0  mpich@:3.2     mpt     mvapich        mvapich2-gdr  openmpi@1.6.5
cray-mvapich2  intel-oneapi-mpi       mpich@:1.1  mpich          mpt@1:  mvapich2       mvapich2x     openmpi@1.7.5:
fujitsu-mpi    intel-parallel-studio  mpich@:1.2  mpilander      mpt@3:  mvapich2@2.1:  nvhpc         openmpi@2.0.0:
hpcx-mpi       mpi-serial             mpich@:3.1  mpitrampoline  msmpi   mvapich2@2.3:  openmpi       spectrum-mpi

And if you only wanted to see packages that provide MPI-2, you would add a version specifier to the spec:

$ spack providers mpi@2
hpcx-mpi               mpi-serial  mpich@:3.1  mpt      mvapich2       mvapich2x      openmpi@1.7.5:
intel-mpi              mpich@:1.0  mpich@:3.2  mpt@3:   mvapich2@2.1:  nvhpc          openmpi@2.0.0:
intel-oneapi-mpi       mpich@:1.1  mpich       msmpi    mvapich2@2.3:  openmpi        spectrum-mpi
intel-parallel-studio  mpich@:1.2  mpilander   mvapich  mvapich2-gdr   openmpi@1.6.5

Notice that the package versions that provide insufficient MPI versions are now filtered out.

Deprecating insecure packages¶

spack deprecate allows for the removal of insecure packages with minimal impact to their dependents.

WARNING:

The spack deprecate command will remove one package and replace it with another by replacing the deprecated package's prefix with a link to the deprecator package's prefix.

WARNING:

Spack tracks concrete deprecated specs and ensures that no future packages concretize to a deprecated spec.

The first spec given to the spack deprecate command is the package to deprecate. It is an abstract spec that must describe a single installed package. The second spec argument is the deprecator spec. By default it must be an abstract spec that describes a single installed package, but with the -i/--install-deprecator it can be any abstract spec that Spack will install and then use as the deprecator. The -I/--no-install-deprecator option will ensure the default behavior.

By default, spack deprecate will deprecate all dependencies of the deprecated spec, replacing each by the dependency of the same name in the deprecator spec. The -d/--dependencies option will ensure the default, while the -D/--no-dependencies option will deprecate only the root of the deprecate spec in favor of the root of the deprecator spec.

spack deprecate can use symbolic links or hard links. The default behavior is symbolic links, but the -l/--link-type flag can take options hard or soft.

Verifying installations¶

The spack verify command can be used to verify the validity of Spack-installed packages any time after installation.

At installation time, Spack creates a manifest of every file in the installation prefix. For links, Spack tracks the mode, ownership, and destination. For directories, Spack tracks the mode, and ownership. For files, Spack tracks the mode, ownership, modification time, hash, and size. The Spack verify command will check, for every file in each package, whether any of those attributes have changed. It will also check for newly added files or deleted files from the installation prefix. Spack can either check all installed packages using the -a,--all or accept specs listed on the command line to verify.

The spack verify command can also verify for individual files that they haven't been altered since installation time. If the given file is not in a Spack installation prefix, Spack will report that it is not owned by any package. To check individual files instead of specs, use the -f,--files option.

Spack installation manifests are part of the tarball signed by Spack for binary package distribution. When installed from a binary package, Spack uses the packaged installation manifest instead of creating one at install time.

The spack verify command also accepts the -l,--local option to check only local packages (as opposed to those used transparently from upstream spack instances) and the -j,--json option to output machine-readable json data for any errors.

Extensions & Python support¶

Spack's installation model assumes that each package will live in its own install prefix. However, certain packages are typically installed within the directory hierarchy of other packages. For example, Python packages are typically installed in the $prefix/lib/python-2.7/site-packages directory.

In Spack, installation prefixes are immutable, so this type of installation is not directly supported. However, it is possible to create views that allow you to merge install prefixes of multiple packages into a single new prefix. Views are a convenient way to get a more traditional filesystem structure. Using extensions, you can ensure that Python packages always share the same prefix in the view as Python itself. Suppose you have Python installed like so:

$ spack find python
==> 1 installed packages.
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
python@2.7.8

spack extensions¶

You can find extensions for your Python installation like this:

$ spack extensions python
==> python@2.7.8%gcc@4.4.7 arch=linux-debian7-x86_64-703c7a96
==> 36 extensions:
geos          py-ipython     py-pexpect    py-pyside            py-sip
py-basemap    py-libxml2     py-pil        py-pytz              py-six
py-biopython  py-mako        py-pmw        py-rpy2              py-sympy
py-cython     py-matplotlib  py-pychecker  py-scientificpython  py-virtualenv
py-dateutil   py-mpi4py      py-pygments   py-scikit-learn
py-epydoc     py-mx          py-pylint     py-scipy
py-gnuplot    py-nose        py-pyparsing  py-setuptools
py-h5py       py-numpy       py-pyqt       py-shiboken
==> 12 installed:
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------
py-dateutil@2.4.0    py-nose@1.3.4       py-pyside@1.2.2
py-dateutil@2.4.0    py-numpy@1.9.1      py-pytz@2014.10
py-ipython@2.3.1     py-pygments@2.0.1   py-setuptools@11.3.1
py-matplotlib@1.4.2  py-pyparsing@2.0.3  py-six@1.9.0

The extensions are a subset of what's returned by spack list, and they are packages like any other. They are installed into their own prefixes, and you can see this with spack find --paths:

$ spack find --paths py-numpy
==> 1 installed packages.
-- linux-debian7-x86_64 / gcc@4.4.7 --------------------------------


    py-numpy@1.9.1  ~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/py-numpy@1.9.1-66733244

However, even though this package is installed, you cannot use it directly when you run python:

$ spack load python
$ python
Python 2.7.8 (default, Feb 17 2015, 01:35:25)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):


  File "<stdin>", line 1, in <module>
ImportError: No module named numpy
>>>

Using Extensions in Environments¶

The recommended way of working with extensions such as py-numpy above is through Environments. For example, the following creates an environment in the current working directory with a filesystem view in the ./view directory:

$ spack env create --with-view view --dir .
$ spack -e . add py-numpy
$ spack -e . concretize
$ spack -e . install

We recommend environments for two reasons. Firstly, environments can be activated (requires Shell support):

$ spack env activate .

which sets all the right environment variables such as PATH and PYTHONPATH. This ensures that

$ python
>>> import numpy

works. Secondly, even without shell support, the view ensures that Python can locate its extensions:

$ ./view/bin/python
>>> import numpy

See Environments (spack.yaml) for a more in-depth description of Spack environments and customizations to views.

Using spack load¶

A more traditional way of using Spack and extensions is spack load (requires Shell support). This will add the extension to PYTHONPATH in your current shell, and Python itself will be available in the PATH:

$ spack load py-numpy
$ python
>>> import numpy

The loaded packages can be checked using spack find --loaded

Loading Extensions via Modules¶

Apart from spack env activate and spack load, you can load numpy through your environment modules (using environment-modules or lmod). This will also add the extension to the PYTHONPATH in your current shell.

$ module load <name of numpy module>

If you do not know the name of the specific numpy module you wish to load, you can use the spack module tcl|lmod loads command to get the name of the module from the Spack spec.

Filesystem requirements¶

By default, Spack needs to be run from a filesystem that supports flock locking semantics. Nearly all local filesystems and recent versions of NFS support this, but parallel filesystems or NFS volumes may be configured without flock support enabled. You can determine how your filesystems are mounted with mount. The output for a Lustre filesystem might look like this:

$ mount | grep lscratch
mds1-lnet0@o2ib100:/lsd on /p/lscratchd type lustre (rw,nosuid,lazystatfs,flock)
mds2-lnet0@o2ib100:/lse on /p/lscratche type lustre (rw,nosuid,lazystatfs,flock)

Note the flock option on both Lustre mounts.

If you do not see this or a similar option for your filesystem, you have a few options. First, you can move your Spack installation to a filesystem that supports locking. Second, you could ask your system administrator to enable flock for your filesystem.

If none of those work, you can disable locking in one of two ways:

WARNING:

This issue typically manifests with the error below:

$ ./spack find
Traceback (most recent call last):
File "./spack", line 176, in <module>


  main()
File "./spack", line 154,' in main


  return_val = command(parser, args)
File "./spack/lib/spack/spack/cmd/find.py", line 170, in find


  specs = set(spack.installed_db.query(\**q_args))
File "./spack/lib/spack/spack/database.py", line 551, in query


  with self.read_transaction():
File "./spack/lib/spack/spack/database.py", line 598, in __enter__


  if self._enter() and self._acquire_fn:
File "./spack/lib/spack/spack/database.py", line 608, in _enter


  return self._db.lock.acquire_read(self._timeout)
File "./spack/lib/spack/llnl/util/lock.py", line 103, in acquire_read


  self._lock(fcntl.LOCK_SH, timeout)   # can raise LockError.
File "./spack/lib/spack/llnl/util/lock.py", line 64, in _lock


  fcntl.lockf(self._fd, op | fcntl.LOCK_NB)
IOError: [Errno 38] Function not implemented

A nicer error message is TBD in future versions of Spack.

Troubleshooting¶

The spack audit command:

$ spack audit -h
usage: spack audit [-h] SUBCOMMAND ...
audit configuration files, packages, etc.
positional arguments:


  SUBCOMMAND


    configs       audit configuration files


    externals     check external detection in packages


    packages-https


                  check https in packages


    packages      audit package recipes


    list          list available checks and exits
options:


  -h, --help      show this help message and exit

can be used to detect a number of configuration issues. This command detects configuration settings which might not be strictly wrong but are not likely to be useful outside of special cases.

It can also be used to detect dependency issues with packages - for example cases where a package constrains a dependency with a variant that doesn't exist (in this case Spack could report the problem ahead of time but automatically performing the check would slow down most runs of Spack).

A detailed list of the checks currently implemented for each subcommand can be printed with:

$ spack -v audit list
generic:


  Generic checks relying on global variables
configs:


  Sanity checks on compilers.yaml


    1. Report compilers with the same spec and two different definitions


  Sanity checks on packages.yaml


    1. Search for duplicate specs declared as externals


    2. Search package preferences deprecated in v0.21 (and slated for removal in v0.22)


    3. Warns if variant preferences have mismatched types or names.
packages:


  Sanity checks on specs used in directives


    1. Ensure stand-alone test method is not included in build-time callbacks


    2. Ensure that patches fetched from GitHub and GitLab have stable sha256


    hashes.


    3. Report unknown or wrong variants in directives for this package


    4. Report unknown dependencies and wrong variants for dependencies


    5. Ensures that variant defaults are present and parsable from cli


    6. Ensures that all variants have a description.


    7. Report if version constraints used in directives are not satisfiable


    8. Reports named specs in the 'when=' attribute of a directive.


    Note that 'conflicts' is the only directive allowing that.


    


  Sanity checks on reserved attributes of packages


    1. Ensure that packages don't override reserved names


  Sanity checks on properties a package should maintain


    1. Ensure package names are lowercase and consistent


    2. Ensure that package objects are pickleable


    3. Ensure that all packages can unparse and that unparsed code is valid Python


    4. Ensure all versions in a package can produce a fetcher


    5. Ensure the package has a docstring and no fixmes


    6. Ensure no packages use md5 checksums


    7. Ensure that methods modifying the build environment are ported to builder classes.
packages-https:


  Sanity checks on https checks of package urls, etc.


    1. Check for correctness of links
externals:


  Sanity checks for external software detection


    1. Test drive external detection for packages

Depending on the use case, users might run the appropriate subcommands to obtain diagnostics. Issues, if found, are reported to stdout:

% spack audit packages lammps
PKG-DIRECTIVES: 1 issue found
1. lammps: wrong variant in "conflicts" directive


    the variant 'adios' does not exist


    in /home/spack/spack/var/spack/repos/builtin/packages/lammps/package.py

Getting Help¶

spack help¶

If you don't find what you need here, the help subcommand will print out out a list of all of spack's options and subcommands:

$ spack help
usage: spack [-hkV] [--color {always,never,auto}] COMMAND ...
A flexible package manager that supports multiple versions,
configurations, platforms, and compilers.
These are common spack commands:
query packages:


  list                  list and search available packages


  info                  get detailed information on a particular package


  find                  list and search installed packages
build packages:


  install               build and install packages


  uninstall             remove installed packages


  gc                    remove specs that are now no longer needed


  spec                  show what would be installed, given a spec
configuration:


  external              manage external packages in Spack configuration
environments:


  env                   manage virtual environments


  view                  project packages to a compact naming scheme on the filesystem
create packages:


  create                create a new package file


  edit                  open package files in $EDITOR
system:


  arch                  print architecture information about this machine


  audit                 audit configuration files, packages, etc.


  compilers             list available compilers
user environment:


  load                  add package to the user environment


  module                generate/manage module files


  unload                remove package from the user environment
options:


  --color {always,never,auto}


                        when to colorize output (default: auto)


  -V, --version         show version number and exit


  -h, --help            show this help message and exit


  -k, --insecure        do not check ssl certificates when downloading
more help:


  spack help --all       list all commands and options


  spack help <command>   help on a specific command


  spack help --spec      help on the package specification syntax


  spack docs             open https://spack.rtfd.io/ in a browser

Adding an argument, e.g. spack help <subcommand>, will print out usage information for a particular subcommand:

$ spack help install
usage: spack install [-hnvyU] [--only {package,dependencies}] [-u UNTIL] [-j JOBS] [--overwrite] [--fail-fast]


                     [--keep-prefix] [--keep-stage] [--dont-restage]


                     [--use-cache | --no-cache | --cache-only | --use-buildcache [{auto,only,never},][package:{auto,only,never},][dependencies:{auto,only,never}]]


                     [--include-build-deps] [--no-check-signature] [--show-log-on-error] [--source] [--deprecated]


                     [--fake] [--only-concrete] [--add | --no-add] [-f SPEC_YAML_FILE] [--clean | --dirty]


                     [--test {root,all}] [--log-format {junit,cdash}] [--log-file LOG_FILE] [--help-cdash] [--reuse]


                     [--reuse-deps]


                     ...
build and install packages
positional arguments:


  spec                  package spec
options:


  --add                 (with environment) add spec to the environment as a root


  --cache-only          only install package from binary mirrors


  --clean               unset harmful variables in the build environment (default)


  --deprecated          fetch deprecated versions without warning


  --dirty               preserve user environment in spack's build environment (danger!)


  --dont-restage        if a partial install is detected, don't delete prior state


  --fail-fast           stop all builds if any build fails (default is best effort)


  --fake                fake install for debug purposes


  --help-cdash          show usage instructions for CDash reporting


  --include-build-deps  include build deps when installing from cache, useful for CI pipeline troubleshooting


  --keep-prefix         don't remove the install prefix if installation fails


  --keep-stage          don't remove the build stage if installation succeeds


  --log-file LOG_FILE   filename for the log file


  --log-format {junit,cdash}


                        format to be used for log files


  --no-add              (with environment) do not add spec to the environment as a root


  --no-cache            do not check for pre-built Spack packages in mirrors


  --no-check-signature  do not check signatures of binary packages


  --only {package,dependencies}


                        select the mode of installation


                        


                        default is to install the package along with all its dependencies. alternatively, one can decide to install only the package or only the dependencies


  --only-concrete       (with environment) only install already concretized specs


  --overwrite           reinstall an existing spec, even if it has dependents


  --show-log-on-error   print full build log to stderr if build fails


  --source              install source files in prefix


  --test {root,all}     run tests on only root packages or all packages


  --use-buildcache [{auto,only,never},][package:{auto,only,never},][dependencies:{auto,only,never}]


                        select the mode of buildcache for the 'package' and 'dependencies'


                        


                        default: package:auto,dependencies:auto


                        


                        - `auto` behaves like --use-cache


                        - `only` behaves like --cache-only


                        - `never` behaves like --no-cache


  --use-cache           check for pre-built Spack packages in mirrors (default)


  -f SPEC_YAML_FILE, --file SPEC_YAML_FILE


                        read specs to install from .yaml files


  -h, --help            show this help message and exit


  -j JOBS, --jobs JOBS  explicitly set number of parallel jobs


  -n, --no-checksum     do not use checksums to verify downloaded files (unsafe)


  -u UNTIL, --until UNTIL


                        phase to stop after when installing (default None)


  -v, --verbose         display verbose build output while installing


  -y, --yes-to-all      assume "yes" is the answer to every confirmation request
concretizer arguments:


  --reuse               reuse installed packages/buildcaches when possible


  --reuse-deps          reuse installed dependencies only


  -U, --fresh           do not reuse installed deps; build newest configuration

Alternately, you can use spack --help in place of spack help, or spack <subcommand> --help to get help on a particular subcommand.

Setup¶

First, let's create a new environment. We'll assume that Spack is already set up correctly, and that you've already sourced the setup script for your shell. To create a new environment, simply run:

$ spack env create myenv

Here, myenv can be anything you want to name your environment. Next, we can add a list of packages we would like to install into our environment. Let's say we want a newer version of Bash than the one that comes with macOS, and we want a few Python libraries. We can run:

$ spack -e myenv add bash@5 python py-numpy py-scipy py-matplotlib

Each package can be listed on a separate line, or combined into a single line like we did above. Notice that we're explicitly asking for Bash 5 here. You can use any spec you would normally use on the command line with other Spack commands.

Next, we want to manually configure a couple of things:

$ spack -e myenv config edit

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:


  # add package specs to the `specs` list


  specs: [bash@5, python, py-numpy, py-scipy, py-matplotlib]


  view: true

You can see the packages we added earlier in the specs: section. If you ever want to add more packages, you can either use spack add or manually edit this file.

We also need to change the concretizer:unify option. By default, Spack concretizes each spec separately, allowing multiple versions of the same package to coexist. Since we want a single consistent environment, we want to concretize all of the specs together.

Here is what your spack.yaml looks like with this new setting:

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:


  # add package specs to the `specs` list


  specs: [bash@5, python, py-numpy, py-scipy, py-matplotlib]


  view: true


  concretizer:


    unify: true

Symlink location¶

Spack symlinks all installations to /Users/me/spack/var/spack/environments/myenv/.spack-env/view, which is the default when view: true. You can actually change this to any directory you want. For example, Homebrew uses /usr/local, while Conda uses /Users/me/anaconda. In order to access files in these locations, you need to update PATH and other environment variables to point to them. Activating the Spack environment does this automatically, but you can also manually set them in your .bashrc.

WARNING:

for directory in .spack bin contrib include lib man share
do


    sudo mkdir -p /usr/local/$directory


    sudo chown $(id -un):$(id -gn) /usr/local/$directory
done

Installation¶

To actually concretize the environment, run:

$ spack -e myenv concretize

This will tell you which if any packages are already installed, and alert you to any conflicting specs.

To actually install these packages and symlink them to your view: directory, simply run:

$ spack -e myenv install
$ spack env activate myenv

Now, when you type which python3, it should find the one you just installed.

In order to change the default shell to our newer Bash installation, we first need to add it to this list of acceptable shells. Run:

$ sudo vim /etc/shells

and add the absolute path to your bash executable. Then run:

$ chsh -s /path/to/bash

Now, when you log out and log back in, echo $SHELL should point to the newer version of Bash.

Updating Installed Packages¶

Let's say you upgraded to a new version of macOS, or a new version of Python was released, and you want to rebuild your entire software stack. To do this, simply run the following commands:

$ spack env activate myenv
$ spack concretize --fresh --force
$ spack install

The --fresh flag tells Spack to use the latest version of every package where possible instead of trying to optimize for reuse of existing installed packages.

The --force flag in addition tells Spack to overwrite its previous concretization decisions, allowing you to choose a new version of Python. If any of the new packages like Bash are already installed, spack install won't re-install them, it will keep the symlinks in place.

Updating & Cleaning Up Old Packages¶

If you're looking to mimic the behavior of Homebrew, you may also want to clean up out-of-date packages from your environment after an upgrade. To upgrade your entire software stack within an environment and clean up old package versions, simply run the following commands:

$ spack env activate myenv
$ spack mark -i --all
$ spack concretize --fresh --force
$ spack install
$ spack gc

Running spack mark -i --all tells Spack to mark all of the existing packages within an environment as "implicitly" installed. This tells spack's garbage collection system that these packages should be cleaned up.

Don't worry however, this will not remove your entire environment. Running spack install will reexamine your spack environment after a fresh concretization and will re-mark any packages that should remain installed as "explicitly" installed.

Note: if you use multiple spack environments you should re-run spack install in each of your environments prior to running spack gc to prevent spack from uninstalling any shared packages that are no longer required by the environment you just upgraded.

Uninstallation¶

If you decide that Spack isn't right for you, uninstallation is simple. Just run:

$ spack env activate myenv
$ spack uninstall --all

This will uninstall all packages in your environment and remove the symlinks.

Why does Spack pick particular versions and variants?¶

This question comes up in a variety of forms:

The short answer is that Spack always picks an optimal configuration based on a complex set of criteria[1]. These criteria are more nuanced than always choosing the latest versions or default variants.

NOTE:

The following set of criteria (from lowest to highest precedence) explain common cases where concretization output may seem surprising at first.

packages:


  foo:


    version: [1.0, 1.1]


    variants: ~mpi

concretizer:


  reuse: dependencies  # other options are 'true' and 'false'

packages:


  foo:


    require:


    - "@1.2: +mpi"

Requirements and constraints restrict the set of possible solutions, while reuse behavior and preferences influence what an optimal solution looks like.

YAML Format¶

Spack configuration files are written in YAML. We chose YAML because it's human readable, but also versatile in that it supports dictionaries, lists, and nested sections. For more details on the format, see yaml.org and libyaml. Here is an example config.yaml file:

config:


  install_tree: $spack/opt/spack


  build_stage:


    - $tempdir/$user/spack-stage


    - ~/.spack/stage

Each Spack configuration file is nested under a top-level section corresponding to its name. So, config.yaml starts with config:, mirrors.yaml starts with mirrors:, etc.

Configuration Scopes¶

Spack pulls configuration data from files in several directories. There are six configuration scopes. From lowest to highest:

Each configuration directory may contain several configuration files, such as config.yaml, compilers.yaml, or mirrors.yaml. When configurations conflict, settings from higher-precedence scopes override lower-precedence settings.

Commands that modify scopes (e.g., spack compilers, spack repo, etc.) take a --scope=<name> parameter that you can use to control which scope is modified. By default, they modify the highest-precedence scope.

Custom scopes¶

In addition to the defaults, system, site, and user scopes, you may add configuration scopes directly on the command line with the --config-scope argument, or -C for short.

For example, the following adds two configuration scopes, named scopea and scopeb, to a spack spec command:

$ spack -C ~/myscopes/scopea -C ~/myscopes/scopeb spec ncurses

Custom scopes come after the spack command and before the subcommand, and they specify a single path to a directory full of configuration files. You can add the same configuration files to that directory that you can add to any other scope (config.yaml, packages.yaml, etc.).

If multiple scopes are provided:

Example: scopes for release and development¶

Suppose that you need to support simultaneous building of release and development versions of mypackage, where mypackage -> A -> B. You could create The following files:

~/myscopes/release/packages.yaml

packages:


    mypackage:


        version: [1.7]


    A:


        version: [2.3]


    B:


        version: [0.8]

~/myscopes/develop/packages.yaml

packages:


    mypackage:


        version: [develop]


    A:


        version: [develop]


    B:


        version: [develop]

You can switch between release and develop configurations using configuration arguments. You would type spack -C ~/myscopes/release when you want to build the designated release versions of mypackage, A, and B, and you would type spack -C ~/myscopes/develop when you want to build all of these packages at the develop version.

Example: swapping MPI providers¶

Suppose that you need to build two software packages, packagea and packageb. packagea is Python 2-based and packageb is Python 3-based. packagea only builds with OpenMPI and packageb only builds with MPICH. You can create different configuration scopes for use with packagea and packageb:

~/myscopes/packgea/packages.yaml

packages:


    python:


        version: [2.7.11]


    all:


        providers:


            mpi: [openmpi]

~/myscopes/packageb/packages.yaml

packages:


    python:


        version: [3.5.2]


    all:


        providers:


            mpi: [mpich]

Platform-specific Scopes¶

For each scope above (excluding environment scopes), there can also be platform-specific settings. For example, on most platforms, GCC is the preferred compiler. However, on macOS (darwin), Clang often works for more packages, and is set as the default compiler. This configuration is set in $(prefix)/etc/spack/defaults/darwin/packages.yaml. It will take precedence over settings in the defaults scope, but can still be overridden by settings in system, system/darwin, site, site/darwin, user, user/darwin, custom, or custom/darwin. So, the full scope precedence is:

You can get the name to use for <platform> by running spack arch --platform. The system config scope has a <platform> section for sites at which /etc is mounted on multiple heterogeneous machines.

Scope Precedence¶

When spack queries for configuration parameters, it searches in higher-precedence scopes first. So, settings in a higher-precedence file can override those with the same key in a lower-precedence one. For list-valued settings, Spack prepends higher-precedence settings to lower-precedence settings. Completely ignoring higher-level configuration options is supported with the :: notation for keys (see Overriding entire sections below).

There are also special notations for string concatenation and precendense override. Using the +: notation can be used to force prepending strings or lists. For lists, this is identical to the default behavior. Using the -: works similarly, but for appending values. String Concatenation

Simple keys¶

Let's look at an example of overriding a single key in a Spack file. If your configurations look like this:

$(prefix)/etc/spack/defaults/config.yaml

config:


  install_tree: $spack/opt/spack


  build_stage:


    - $tempdir/$user/spack-stage


    - ~/.spack/stage

~/.spack/config.yaml

config:


  install_tree: /some/other/directory

Spack will only override install_tree in the config section, and will take the site preferences for other settings. You can see the final, combined configuration with the spack config get <configtype> command:

$ spack config get config
config:


  install_tree: /some/other/directory


  build_stage:


    - $tempdir/$user/spack-stage


    - ~/.spack/stage

String Concatenation¶

Above, the user config.yaml completely overrides specific settings in the default config.yaml. Sometimes, it is useful to add a suffix/prefix to a path or name. To do this, you can use the -: notation for append string concatenation at the end of a key in a configuration file. For example:

~/.spack/config.yaml

config:


  install_tree-: /my/custom/suffix/

Spack will then append to the lower-precedence configuration under the install_tree-: section:

$ spack config get config
config:


  install_tree: /some/other/directory/my/custom/suffix


  build_stage:


    - $tempdir/$user/spack-stage


    - ~/.spack/stage

Similarly, +: can be used to prepend to a path or name:

~/.spack/config.yaml

config:


  install_tree+: /my/custom/suffix/

Overriding entire sections¶

Above, the user config.yaml only overrides specific settings in the default config.yaml. Sometimes, it is useful to completely override lower-precedence settings. To do this, you can use two colons at the end of a key in a configuration file. For example:

~/.spack/config.yaml

config::


  install_tree: /some/other/directory

Spack will ignore all lower-precedence configuration under the config:: section:

$ spack config get config
config:


  install_tree: /some/other/directory

List-valued settings¶

Let's revisit the config.yaml example one more time. The build_stage setting's value is an ordered list of directories:

$(prefix)/etc/spack/defaults/config.yaml

build_stage:


  - $tempdir/$user/spack-stage


  - ~/.spack/stage

Suppose the user configuration adds its own list of build_stage paths:

~/.spack/config.yaml

build_stage:


  - /lustre-scratch/$user/spack


  - ~/mystage

Spack will first look at the paths in the defaults config.yaml, then the paths in the user's ~/.spack/config.yaml. The list in the higher-precedence scope is prepended to the defaults. spack config get config shows the result:

$ spack config get config
config:


  install_tree: /some/other/directory


  build_stage:


    - /lustre-scratch/$user/spack


    - ~/mystage


    - $tempdir/$user/spack-stage


    - ~/.spack/stage

As in Overriding entire sections, the higher-precedence scope can completely override the lower-precedence scope using ::. So if the user config looked like this:

~/.spack/config.yaml

build_stage::


  - /lustre-scratch/$user/spack


  - ~/mystage

The merged configuration would look like this:

$ spack config get config
config:


  install_tree: /some/other/directory


  build_stage:


    - /lustre-scratch/$user/spack


    - ~/mystage

Config File Variables¶

Spack understands several variables which can be used in config file paths wherever they appear. There are three sets of these variables: Spack-specific variables, environment variables, and user path variables. Spack-specific variables and environment variables are both indicated by prefixing the variable name with $. User path variables are indicated at the start of the path with ~ or ~user.

Spack-specific variables¶

Spack understands over a dozen special variables. These are:

Note that, as with shell variables, you can write these as $varname or with braces to distinguish the variable from surrounding characters: ${varname}. Their names are also case insensitive, meaning that $SPACK works just as well as $spack. These special variables are substituted first, so any environment variables with the same name will not be used.

Environment variables¶

After Spack-specific variables are evaluated, environment variables are expanded. These are formatted like Spack-specific variables, e.g., ${varname}. You can use this to insert environment variables in your Spack configuration.

User home directories¶

Spack performs Unix-style tilde expansion on paths in configuration files. This means that tilde (~) will expand to the current user's home directory, and ~user will expand to a specified user's home directory. The ~ must appear at the beginning of the path, or Spack will not expand it.

Environment Modifications¶

Spack allows to prescribe custom environment modifications in a few places within its configuration files. Every time these modifications are allowed they are specified as a dictionary, like in the following example:

environment:


  set:


    LICENSE_FILE: '/path/to/license'


  unset:


  - CPATH


  - LIBRARY_PATH


  append_path:


    PATH: '/new/bin/dir'

The possible actions that are permitted are set, unset, append_path, prepend_path and finally remove_path. They all require a dictionary of variable names mapped to the values used for the modification. The only exception is unset that requires just a list of variable names. No particular order is ensured on the execution of each of these modifications.

Seeing Spack's Configuration¶

With so many scopes overriding each other, it can sometimes be difficult to understand what Spack's final configuration looks like.

Spack provides two useful ways to view the final "merged" version of any configuration file: spack config get and spack config blame.

spack config get¶

spack config get shows a fully merged configuration file, taking into account all scopes. For example, to see the fully merged config.yaml, you can type:

$ spack config get config
config:


  debug: false


  checksum: true


  verify_ssl: true


  dirty: false


  build_jobs: 8


  install_tree: $spack/opt/spack


  template_dirs:


  - $spack/templates


  directory_layout: {architecture}/{compiler.name}-{compiler.version}/{name}-{version}-{hash}


  build_stage:


  - $tempdir/$user/spack-stage


  - ~/.spack/stage


  - $spack/var/spack/stage


  source_cache: $spack/var/spack/cache


  misc_cache: ~/.spack/cache


  locks: true

Likewise, this will show the fully merged packages.yaml:

$ spack config get packages

You can use this in conjunction with the -C / --config-scope argument to see how your scope will affect Spack's configuration:

$ spack -C /path/to/my/scope config get packages

spack config blame¶

spack config blame functions much like spack config get, but it shows exactly which configuration file each preference came from. If you do not know why Spack is behaving a certain way, this can help you track down the problem:

$ spack --insecure -C ./my-scope -C ./my-scope-2 config blame config
==> Warning: You asked for --insecure. Will NOT check SSL certificates.
---                                                   config:
_builtin                                                debug: False
/home/myuser/spack/etc/spack/defaults/config.yaml:72    checksum: True
command_line                                            verify_ssl: False
./my-scope-2/config.yaml:2                              dirty: False
_builtin                                                build_jobs: 8
./my-scope/config.yaml:2                                install_tree: /path/to/some/tree
/home/myuser/spack/etc/spack/defaults/config.yaml:23    template_dirs:
/home/myuser/spack/etc/spack/defaults/config.yaml:24    - $spack/templates
/home/myuser/spack/etc/spack/defaults/config.yaml:28    directory_layout: {architecture}/{compiler.name}-{compiler.version}/{name}-{version}-{hash}
/home/myuser/spack/etc/spack/defaults/config.yaml:49    build_stage:
/home/myuser/spack/etc/spack/defaults/config.yaml:50    - $tempdir/$user/spack-stage
/home/myuser/spack/etc/spack/defaults/config.yaml:51    - ~/.spack/stage
/home/myuser/spack/etc/spack/defaults/config.yaml:52    - $spack/var/spack/stage
/home/myuser/spack/etc/spack/defaults/config.yaml:57    source_cache: $spack/var/spack/cache
/home/myuser/spack/etc/spack/defaults/config.yaml:62    misc_cache: ~/.spack/cache
/home/myuser/spack/etc/spack/defaults/config.yaml:86    locks: True

You can see above that the build_jobs and debug settings are built in and are not overridden by a configuration file. The verify_ssl setting comes from the --insecure option on the command line. dirty and install_tree come from the custom scopes ./my-scope and ./my-scope-2, and all other configuration options come from the default configuration files that ship with Spack.

Overriding Local Configuration¶

Spack's system and user scopes provide ways for administrators and users to set global defaults for all Spack instances, but for use cases where one wants a clean Spack installation, these scopes can be undesirable. For example, users may want to opt out of global system configuration, or they may want to ignore their own home directory settings when running in a continuous integration environment.

Spack also, by default, keeps various caches and user data in ~/.spack, but users may want to override these locations.

Spack provides three environment variables that allow you to override or opt out of configuration locations:

And one that allows you to move the default cache location:

With these settings, if you want to isolate Spack in a CI environment, you can do this:

export SPACK_DISABLE_LOCAL_CONFIG=true
export SPACK_USER_CACHE_PATH=/tmp/spack

install_tree:root¶

The location where Spack will install packages and their dependencies. Default is $spack/opt/spack.

install_hash_length and install_path_scheme¶

The default Spack installation path can be very long and can create problems for scripts with hardcoded shebangs. Additionally, when using the Intel compiler, and if there is also a long list of dependencies, the compiler may segfault. If you see the following:

: internal error: ** The compiler has encountered an unexpected problem.
** Segmentation violation signal raised. **
Access violation or stack overflow. Please contact Intel Support for assistance.

it may be because variables containing dependency specs may be too long. There are two parameters to help with long path names. Firstly, the install_hash_length parameter can set the length of the hash in the installation path from 1 to 32. The default path uses the full 32 characters.

Secondly, it is also possible to modify the entire installation scheme. By default Spack uses {architecture}/{compiler.name}-{compiler.version}/{name}-{version}-{hash} where the tokens that are available for use in this directive are the same as those understood by the format() method. Using this parameter it is possible to use a different package layout or reduce the depth of the installation paths. For example

config:


  install_path_scheme: '{name}/{version}/{hash:7}'

would install packages into sub-directories using only the package name, version and a hash length of 7 characters.

When using either parameter to set the hash length it only affects the representation of the hash in the installation directory. You should be aware that the smaller the hash length the more likely naming conflicts will occur. These parameters are independent of those used to configure module names.

WARNING:

build_stage¶

Spack is designed to run out of a user home directory, and on many systems the home directory is a (slow) network file system. On most systems, building in a temporary file system is faster. Usually, there is also more space available in the temporary location than in the home directory. If the username is not already in the path, Spack will append the value of $user to the selected build_stage path.

WARNING:

By default, Spack's build_stage is configured like this:

build_stage:


 - $tempdir/$user/spack-stage


 - ~/.spack/stage

This can be an ordered list of paths that Spack should search when trying to find a temporary directory for the build stage. The list is searched in order, and Spack will use the first directory to which it has write access.

Specifying ~/.spack/stage first will ensure each user builds in their home directory. The historic Spack stage path $spack/var/spack/stage will build directly inside the Spack instance. See Config File Variables for more on $tempdir and $spack.

When Spack builds a package, it creates a temporary directory within the build_stage. After the package is successfully installed, Spack deletes the temporary directory it used to build. Unsuccessful builds are not deleted, but you can manually purge them with spack clean --stage.

NOTE:

source_cache¶

Location to cache downloaded tarballs and repositories. By default these are stored in $spack/var/spack/cache. These are stored indefinitely by default. Can be purged with spack clean --downloads.

misc_cache¶

Temporary directory to store long-lived cache files, such as indices of packages available in repositories. Defaults to ~/.spack/cache. Can be purged with spack clean --misc-cache.

verify_ssl¶

When set to true (default) Spack will verify certificates of remote hosts when making ssl connections. Set to false to disable, and tools like curl will use their --insecure options. Disabling this can expose you to attacks. Use at your own risk.

checksum¶

When set to true, Spack verifies downloaded source code using a checksum, and will refuse to build packages that it cannot verify. Set to false to disable these checks. Disabling this can expose you to attacks. Use at your own risk.

locks¶

When set to true, concurrent instances of Spack will use locks to avoid modifying the install tree, database file, etc. If false, Spack will disable all locking, but you must not run concurrent instances of Spack. For file systems that don't support locking, you should set this to false and run one Spack at a time, but otherwise we recommend enabling locks.

dirty¶

By default, Spack unsets variables in your environment that can change the way packages build. This includes LD_LIBRARY_PATH, CPATH, LIBRARY_PATH, DYLD_LIBRARY_PATH, and others.

By default, builds are clean, but on some machines, compilers and other tools may need custom LD_LIBRARY_PATH settings to run. You can set dirty to true to skip the cleaning step and make all builds "dirty" by default. Be aware that this will reduce the reproducibility of builds.

build_jobs¶

Unless overridden in a package or on the command line, Spack builds all packages in parallel. The default parallelism is equal to the number of cores available to the process, up to 16 (the default of build_jobs). For a build system that uses Makefiles, this spack install runs:

If you work on a shared login node or have a strict ulimit, it may be necessary to set the default to a lower value. By setting build_jobs to 4, for example, commands like spack install will run make -j4 instead of hogging every core. To build all software in serial, set build_jobs to 1.

Note that specifying the number of jobs on the command line always takes priority, so that spack install -j<n> always runs make -j<n>, even when that exceeds the number of cores available.

ccache¶

When set to true Spack will use ccache to cache compiles. This is useful specifically in two cases: (1) when using spack dev-build, and (2) when building the same package with many different variants. The default is false.

When enabled, Spack will look inside your PATH for a ccache executable and stop if it is not found. Some systems come with ccache, but it can also be installed using spack install ccache. ccache comes with reasonable defaults for cache size and location. (See the Configuration settings section of man ccache to learn more about the default settings and how to change them). Please note that we currently disable ccache's hash_dir feature to avoid an issue with the stage directory (see https://github.com/spack/spack/pull/3761#issuecomment-294352232).

shared_linking:type¶

Control whether Spack embeds RPATH or RUNPATH attributes in ELF binaries so that they can find their dependencies. Has no effect on macOS. Two options are allowed:

RPATH search paths have higher precedence than LD_LIBRARY_PATH and ld.so will search for libraries in transitive RPATHs of parent objects.

RUNPATH search paths have lower precedence than LD_LIBRARY_PATH, and ld.so will ONLY search for dependencies in the RUNPATH of the loading object.

DO NOT MIX the two options within the same install tree.

shared_linking:bind¶

This is an experimental option that controls whether Spack embeds absolute paths to needed shared libraries in ELF executables and shared libraries on Linux. Setting this option to true has two advantages:

In the current implementation, Spack sets the soname (shared object name) of libraries to their install path upon installation. This has two implications:

It is also worth noting that:

NOTE:

class Example(Package):


   non_bindable_shared_objects = ["libinterface.so"]

install_status¶

When set to true, Spack will show information about its current progress as well as the current and total package numbers. Progress is shown both in the terminal title and inline. Setting it to false will not show any progress information.

To work properly, this requires your terminal to reset its title after Spack has finished its work, otherwise Spack's status information will remain in the terminal's title indefinitely. Most terminals should already be set up this way and clear Spack's status information.

aliases¶

Aliases can be used to define new Spack commands. They can be either shortcuts for longer commands or include specific arguments for convenience. For instance, if users want to use spack install's -v argument all the time, they can create a new alias called inst that will always call install -v:

aliases:


  inst: install -v

External Packages¶

Spack can be configured to use externally-installed packages rather than building its own packages. This may be desirable if machines ship with system packages, such as a customized MPI that should be used instead of Spack building its own MPI.

External packages are configured through the packages.yaml file. Here's an example of an external configuration:

packages:


  openmpi:


    externals:


    - spec: "openmpi@1.4.3%gcc@4.4.7 arch=linux-debian7-x86_64"


      prefix: /opt/openmpi-1.4.3


    - spec: "openmpi@1.4.3%gcc@4.4.7 arch=linux-debian7-x86_64+debug"


      prefix: /opt/openmpi-1.4.3-debug


    - spec: "openmpi@1.6.5%intel@10.1 arch=linux-debian7-x86_64"


      prefix: /opt/openmpi-1.6.5-intel

This example lists three installations of OpenMPI, one built with GCC, one built with GCC and debug information, and another built with Intel. If Spack is asked to build a package that uses one of these MPIs as a dependency, it will use the pre-installed OpenMPI in the given directory. Note that the specified path is the top-level install prefix, not the bin subdirectory.

packages.yaml can also be used to specify modules to load instead of the installation prefixes. The following example says that module CMake/3.7.2 provides cmake version 3.7.2.

cmake:


  externals:


  - spec: cmake@3.7.2


    modules:


    - CMake/3.7.2

Each packages.yaml begins with a packages: attribute, followed by a list of package names. To specify externals, add an externals: attribute under the package name, which lists externals. Each external should specify a spec: string that should be as well-defined as reasonably possible. If a package lacks a spec component, such as missing a compiler or package version, then Spack will guess the missing component based on its most-favored packages, and it may guess incorrectly.

Each package version and compiler listed in an external should have entries in Spack's packages and compiler configuration, even though the package and compiler may not ever be built.

Prevent packages from being built from sources¶

Adding an external spec in packages.yaml allows Spack to use an external location, but it does not prevent Spack from building packages from sources. In the above example, Spack might choose for many valid reasons to start building and linking with the latest version of OpenMPI rather than continue using the pre-installed OpenMPI versions.

To prevent this, the packages.yaml configuration also allows packages to be flagged as non-buildable. The previous example could be modified to be:

packages:


  openmpi:


    externals:


    - spec: "openmpi@1.4.3%gcc@4.4.7 arch=linux-debian7-x86_64"


      prefix: /opt/openmpi-1.4.3


    - spec: "openmpi@1.4.3%gcc@4.4.7 arch=linux-debian7-x86_64+debug"


      prefix: /opt/openmpi-1.4.3-debug


    - spec: "openmpi@1.6.5%intel@10.1 arch=linux-debian7-x86_64"


      prefix: /opt/openmpi-1.6.5-intel


    buildable: False

The addition of the buildable flag tells Spack that it should never build its own version of OpenMPI from sources, and it will instead always rely on a pre-built OpenMPI.

NOTE:

If an external module is specified as not buildable, then Spack will load the external module into the build environment which can be used for linking.

The buildable does not need to be paired with external packages. It could also be used alone to forbid packages that may be buggy or otherwise undesirable.

Non-buildable virtual packages¶

Virtual packages in Spack can also be specified as not buildable, and external implementations can be provided. In the example above, OpenMPI is configured as not buildable, but Spack will often prefer other MPI implementations over the externally available OpenMPI. Spack can be configured with every MPI provider not buildable individually, but more conveniently:

packages:


  mpi:


    buildable: False


  openmpi:


    externals:


    - spec: "openmpi@1.4.3%gcc@4.4.7 arch=linux-debian7-x86_64"


      prefix: /opt/openmpi-1.4.3


    - spec: "openmpi@1.4.3%gcc@4.4.7 arch=linux-debian7-x86_64+debug"


      prefix: /opt/openmpi-1.4.3-debug


    - spec: "openmpi@1.6.5%intel@10.1 arch=linux-debian7-x86_64"


      prefix: /opt/openmpi-1.6.5-intel

Spack can then use any of the listed external implementations of MPI to satisfy a dependency, and will choose depending on the compiler and architecture.

In cases where the concretizer is configured to reuse specs, and other mpi providers (available via stores or buildcaches) are not wanted, Spack can be configured to require specs matching only the available externals:

packages:


  mpi:


    buildable: False


    require:


    - one_of: [


        "openmpi@1.4.3%gcc@4.4.7 arch=linux-debian7-x86_64",


        "openmpi@1.4.3%gcc@4.4.7 arch=linux-debian7-x86_64+debug",


        "openmpi@1.6.5%intel@10.1 arch=linux-debian7-x86_64"


      ]


  openmpi:


    externals:


    - spec: "openmpi@1.4.3%gcc@4.4.7 arch=linux-debian7-x86_64"


      prefix: /opt/openmpi-1.4.3


    - spec: "openmpi@1.4.3%gcc@4.4.7 arch=linux-debian7-x86_64+debug"


      prefix: /opt/openmpi-1.4.3-debug


    - spec: "openmpi@1.6.5%intel@10.1 arch=linux-debian7-x86_64"


      prefix: /opt/openmpi-1.6.5-intel

This configuration prevents any spec using MPI and originating from stores or buildcaches to be reused, unless it matches the requirements under packages:mpi:require. For more information on requirements see Package Requirements.

Automatically Find External Packages¶

You can run the spack external find command to search for system-provided packages and add them to packages.yaml. After running this command your packages.yaml may include new entries:

packages:


  cmake:


    externals:


    - spec: cmake@3.17.2


      prefix: /usr

Generally this is useful for detecting a small set of commonly-used packages; for now this is generally limited to finding build-only dependencies. Specific limitations include:

Package Requirements¶

Spack can be configured to always use certain compilers, package versions, and variants during concretization through package requirements.

Package requirements are useful when you find yourself repeatedly specifying the same constraints on the command line, and wish that Spack respects these constraints whether you mention them explicitly or not. Another use case is specifying constraints that should apply to all root specs in an environment, without having to repeat the constraint everywhere.

Apart from that, requirements config is more flexible than constraints on the command line, because it can specify constraints on packages when they occur as a dependency. In contrast, on the command line it is not possible to specify constraints on dependencies while also keeping those dependencies optional.

SEE ALSO:

Requirements syntax¶

The package requirements configuration is specified in packages.yaml, keyed by package name and expressed using the Spec syntax. In the simplest case you can specify attributes that you always want the package to have by providing a single spec string to require:

packages:


  libfabric:


    require: "@1.13.2"

In the above example, libfabric will always build with version 1.13.2. If you need to compose multiple configuration scopes require accepts a list of strings:

packages:


  libfabric:


    require:


    - "@1.13.2"


    - "%gcc"

In this case libfabric will always build with version 1.13.2 and using GCC as a compiler.

For more complex use cases, require accepts also a list of objects. These objects must have either a any_of or a one_of field, containing a list of spec strings, and they can optionally have a when and a message attribute:

packages:


  openmpi:


    require:


    - any_of: ["@4.1.5", "%gcc"]


      message: "in this example only 4.1.5 can build with other compilers"

any_of is a list of specs. One of those specs must be satisfied and it is also allowed for the concretized spec to match more than one. In the above example, that means you could build openmpi@4.1.5%gcc, openmpi@4.1.5%clang or openmpi@3.9%gcc, but not openmpi@3.9%clang.

If a custom message is provided, and the requirement is not satisfiable, Spack will print the custom error message:

$ spack spec openmpi@3.9%clang
==> Error: in this example only 4.1.5 can build with other compilers

We could express a similar requirement using the when attribute:

packages:


  openmpi:


    require:


    - any_of: ["%gcc"]


      when: "@:4.1.4"


      message: "in this example only 4.1.5 can build with other compilers"

In the example above, if the version turns out to be 4.1.4 or less, we require the compiler to be GCC. For readability, Spack also allows a spec key accepting a string when there is only a single constraint:

packages:


  openmpi:


    require:


    - spec: "%gcc"


      when: "@:4.1.4"


      message: "in this example only 4.1.5 can build with other compilers"

This code snippet and the one before it are semantically equivalent.

Finally, instead of any_of you can use one_of which also takes a list of specs. The final concretized spec must match one and only one of them:

packages:


  mpich:


    require:


    - one_of: ["+cuda", "+rocm"]

In the example above, that means you could build mpich+cuda or mpich+rocm but not mpich+cuda+rocm.

NOTE:

NOTE:

Setting default requirements¶

You can also set default requirements for all packages under all like this:

packages:


  all:


    require: '%clang'

which means every spec will be required to use clang as a compiler.

Note that in this case all represents a default set of requirements - if there are specific package requirements, then the default requirements under all are disregarded. For example, with a configuration like this:

packages:


  all:


    require: '%clang'


  cmake:


    require: '%gcc'

Spack requires cmake to use gcc and all other nodes (including cmake dependencies) to use clang.

Setting requirements on virtual specs¶

A requirement on a virtual spec applies whenever that virtual is present in the DAG. This can be useful for fixing which virtual provider you want to use:

packages:


  mpi:


    require: 'mvapich2 %gcc'

With the configuration above the only allowed mpi provider is mvapich2 %gcc.

Requirements on the virtual spec and on the specific provider are both applied, if present. For instance with a configuration like:

packages:


  mpi:


    require: 'mvapich2 %gcc'


  mvapich2:


    require: '~cuda'

you will use mvapich2~cuda %gcc as an mpi provider.

Package Preferences¶

In some cases package requirements can be too strong, and package preferences are the better option. Package preferences do not impose constraints on packages for particular versions or variants values, they rather only set defaults. The concretizer is free to change them if it must, due to other constraints, and also prefers reusing installed packages over building new ones that are a better match for preferences.

SEE ALSO:

Most package preferences (compilers, target and providers) can only be set globally under the all section of packages.yaml:

packages:


  all:


    compiler: [gcc@12.2.0, clang@12:, oneapi@2023:]


    target: [x86_64_v3]


    providers:


      mpi: [mvapich2, mpich, openmpi]

These preferences override Spack's default and effectively reorder priorities when looking for the best compiler, target or virtual package provider. Each preference takes an ordered list of spec constraints, with earlier entries in the list being preferred over later entries.

In the example above all packages prefer to be compiled with gcc@12.2.0, to target the x86_64_v3 microarchitecture and to use mvapich2 if they depend on mpi.

The variants and version preferences can be set under package specific sections of the packages.yaml file:

packages:


  opencv:


    variants: +debug


  gperftools:


    version: [2.2, 2.4, 2.3]

In this case, the preference for opencv is to build with debug options, while gperftools prefers version 2.2 over 2.4.

Any preference can be overwritten on the command line if explicitly requested.

Preferences cannot overcome explicit constraints, as they only set a preferred ordering among homogeneous attribute values. Going back to the example, if gperftools@2.3: was requested, then Spack will install version 2.4 since the most preferred version 2.2 is prohibited by the version constraint.

Package Permissions¶

Spack can be configured to assign permissions to the files installed by a package.

In the packages.yaml file under permissions, the attributes read, write, and group control the package permissions. These attributes can be set per-package, or for all packages under all. If permissions are set under all and for a specific package, the package-specific settings take precedence.

The read and write attributes take one of user, group, and world.

packages:


  all:


    permissions:


      write: group


      group: spack


  my_app:


    permissions:


      read: group


      group: my_team

The permissions settings describe the broadest level of access to installations of the specified packages. The execute permissions of the file are set to the same level as read permissions for those files that are executable. The default setting for read is world, and for write is user. In the example above, installations of my_app will be installed with user and group permissions but no world permissions, and owned by the group my_team. All other packages will be installed with user and group write privileges, and world read privileges. Those packages will be owned by the group spack.

The group attribute assigns a Unix-style group to a package. All files installed by the package will be owned by the assigned group, and the sticky group bit will be set on the install prefix and all directories inside the install prefix. This will ensure that even manually placed files within the install prefix are owned by the assigned group. If no group is assigned, Spack will allow the OS default behavior to go as expected.

Assigning Package Attributes¶

You can assign class-level attributes in the configuration:

packages:


  mpileaks:


    package_attributes:


      # Override existing attributes


      url: http://www.somewhereelse.com/mpileaks-1.0.tar.gz


      # ... or add new ones


      x: 1

Attributes set this way will be accessible to any method executed in the package.py file (e.g. the install() method). Values for these attributes may be any value parseable by yaml.

These can only be applied to specific packages, not "all" or virtual packages.

Reuse already installed packages¶

The reuse attribute controls whether Spack will prefer to use installed packages (true), or whether it will do a "fresh" installation and prefer the latest settings from package.py files and packages.yaml (false). You can use:

% spack install --reuse <spec>

to enable reuse for a single installation, and you can use:

spack install --fresh <spec>

to do a fresh install if reuse is enabled by default. reuse: dependencies is the default.

SEE ALSO:

Selection of the target microarchitectures¶

The options under the targets attribute control which targets are considered during a solve. Currently the options in this section are only configurable from the concretizer.yaml file and there are no corresponding command line arguments to enable them for a single solve.

The granularity option can take two possible values: microarchitectures and generic. If set to:

concretizer:


  targets:


    granularity: microarchitectures

Spack will consider all the microarchitectures known to archspec to label nodes for compatibility. If instead the option is set to:

concretizer:


  targets:


    granularity: generic

Spack will consider only generic microarchitectures. For instance, when running on an Haswell node, Spack will consider haswell as the best target in the former case and x86_64_v3 as the best target in the latter case.

The host_compatible option is a Boolean option that determines whether or not the microarchitectures considered during the solve are constrained to be compatible with the host Spack is currently running on. For instance, if this option is set to true, a user cannot concretize for target=icelake while running on an Haswell node.

Duplicate nodes¶

The duplicates attribute controls whether the DAG can contain multiple configurations of the same package. This is mainly relevant for build dependencies, which may have their version pinned by some nodes, and thus be required at different versions by different nodes in the same DAG.

The strategy option controls how the solver deals with duplicates. If the value is none, then a single configuration per package is allowed in the DAG. This means, for instance, that only a single cmake or a single py-setuptools version is allowed. The result would be a slightly faster concretization, at the expense of making a few specs unsolvable.

If the value is minimal Spack will allow packages tagged as build-tools to have duplicates. This allows, for instance, to concretize specs whose nodes require different, and incompatible, ranges of some build tool. For instance, in the figure below the latest py-shapely requires a newer py-setuptools, while py-numpy still needs an older version:

Up to Spack v0.20 duplicates:strategy:none was the default (and only) behavior. From Spack v0.21 the default behavior is duplicates:strategy:minimal.

Using Environments¶

Here we follow a typical use case of creating, concretizing, installing and loading an environment.

Creating a managed Environment¶

An environment is created by:

$ spack env create myenv

Spack then creates the directory var/spack/environments/myenv.

NOTE:

In the var/spack/environments/myenv directory, Spack creates the file spack.yaml and the hidden directory .spack-env.

Spack stores metadata in the .spack-env directory. User interaction will occur through the spack.yaml file and the Spack commands that affect it. When the environment is concretized, Spack will create a file spack.lock with the concrete information for the environment.

In addition to being the default location for the view associated with an Environment, the .spack-env directory also contains:

Spack Environments can also be created from either a manifest file (usually but not necessarily named, spack.yaml) or a lockfile. To create an Environment from a manifest:

$ spack env create myenv spack.yaml

To create an Environment from a spack.lock lockfile:

$ spack env create myenv spack.lock

Either of these commands can also take a full path to the initialization file.

A Spack Environment created from a spack.yaml manifest is guaranteed to have the same root specs as the original Environment, but may concretize differently. A Spack Environment created from a spack.lock lockfile is guaranteed to have the same concrete specs as the original Environment. Either may obviously then differ as the user modifies it.

Activating an Environment¶

To activate an environment, use the following command:

$ spack env activate myenv

By default, the spack env activate will load the view associated with the Environment into the user environment. The -v, --with-view argument ensures this behavior, and the -V, --without-view argument activates the environment without changing the user environment variables.

The -p option to the spack env activate command modifies the user's prompt to begin with the environment name in brackets.

$ spack env activate -p myenv
[myenv] $ ...

To deactivate an environment, use the command:

$ spack env deactivate

or the shortcut alias

$ despacktivate

If the environment was activated with its view, deactivating the environment will remove the view from the user environment.

Anonymous Environments¶

Any directory can be treated as an environment if it contains a file spack.yaml. To load an anonymous environment, use:

$ spack env activate -d /path/to/directory

Anonymous specs can be created in place using the command:

$ spack env create -d .

In this case Spack simply creates a spack.yaml file in the requested directory.

Environment Sensitive Commands¶

Spack commands are environment sensitive. For example, the find command shows only the specs in the active Environment if an Environment has been activated. Similarly, the install and uninstall commands act on the active environment.

$ spack find
==> 0 installed packages
$ spack install zlib@1.2.11
==> Installing zlib-1.2.11-q6cqrdto4iktfg6qyqcc5u4vmfmwb7iv
==> No binary for zlib-1.2.11-q6cqrdto4iktfg6qyqcc5u4vmfmwb7iv found: installing from source
==> zlib: Executing phase: 'install'
[+] ~/spack/opt/spack/linux-rhel7-broadwell/gcc-8.1.0/zlib-1.2.11-q6cqrdto4iktfg6qyqcc5u4vmfmwb7iv
$ spack env activate myenv
$ spack find
==> In environment myenv
==> No root specs
==> 0 installed packages
$ spack install zlib@1.2.8
==> Installing zlib-1.2.8-yfc7epf57nsfn2gn4notccaiyxha6z7x
==> No binary for zlib-1.2.8-yfc7epf57nsfn2gn4notccaiyxha6z7x found: installing from source
==> zlib: Executing phase: 'install'
[+] ~/spack/opt/spack/linux-rhel7-broadwell/gcc-8.1.0/zlib-1.2.8-yfc7epf57nsfn2gn4notccaiyxha6z7x
==> Updating view at ~/spack/var/spack/environments/myenv/.spack-env/view
$ spack find
==> In environment myenv
==> Root specs
zlib@1.2.8
==> 1 installed package
-- linux-rhel7-broadwell / gcc@8.1.0 ----------------------------
zlib@1.2.8
$ despacktivate
$ spack find
==> 2 installed packages
-- linux-rhel7-broadwell / gcc@8.1.0 ----------------------------
zlib@1.2.8  zlib@1.2.11

Note that when we installed the abstract spec zlib@1.2.8, it was presented as a root of the Environment. All explicitly installed packages will be listed as roots of the Environment.

All of the Spack commands that act on the list of installed specs are Environment-sensitive in this way, including install, uninstall, find, extensions, and more. In the Configuring Environments section we will discuss Environment-sensitive commands further.

Adding Abstract Specs¶

An abstract spec is the user-specified spec before Spack has applied any defaults or dependency information.

Users can add abstract specs to an Environment using the spack add command. The most important component of an Environment is a list of abstract specs.

Adding a spec adds to the manifest (the spack.yaml file), which is used to define the roots of the Environment, but does not affect the concrete specs in the lockfile, nor does it install the spec.

The spack add command is environment aware. It adds to the currently active environment. All environment aware commands can also be called using the spack -e flag to specify the environment.

$ spack env activate myenv
$ spack add mpileaks

or

$ spack -e myenv add python

Concretizing¶

Once some user specs have been added to an environment, they can be concretized. There are at the moment three different modes of operation to concretize an environment, which are explained in details in Spec concretization. Regardless of which mode of operation has been chosen, the following command will ensure all the root specs are concretized according to the constraints that are prescribed in the configuration:

[myenv]$ spack concretize

In the case of specs that are not concretized together, the command above will concretize only the specs that were added and not yet concretized. Forcing a re-concretization of all the specs can be done instead with this command:

[myenv]$ spack concretize -f

When the -f flag is not used to reconcretize all specs, Spack guarantees that already concretized specs are unchanged in the environment.

The concretize command does not install any packages. For packages that have already been installed outside of the environment, the process of adding the spec and concretizing is identical to installing the spec assuming it concretizes to the exact spec that was installed outside of the environment.

The spack find command can show concretized specs separately from installed specs using the -c (--concretized) flag.

[myenv]$ spack add zlib
[myenv]$ spack concretize
[myenv]$ spack find -c
==> In environment myenv
==> Root specs
zlib
==> Concretized roots
-- linux-rhel7-x86_64 / gcc@4.9.3 -------------------------------
zlib@1.2.11
==> 0 installed packages

Installing an Environment¶

In addition to installing individual specs into an Environment, one can install the entire Environment at once using the command

[myenv]$ spack install

If the Environment has been concretized, Spack will install the concretized specs. Otherwise, spack install will first concretize the Environment and then install the concretized specs.

NOTE:

[myenv]$ spack install & spack install & spack install & spack install

As it installs, spack install creates symbolic links in the logs/ directory in the Environment, allowing for easy inspection of build logs related to that environment. The spack install command also stores a Spack repo containing the package.py file used at install time for each package in the repos/ directory in the Environment.

The --no-add option can be used in a concrete environment to tell spack to install specs already present in the environment but not to add any new root specs to the environment. For root specs provided to spack install on the command line, --no-add is the default, while for dependency specs on the other hand, it is optional. In other words, if there is an unambiguous match in the active concrete environment for a root spec provided to spack install on the command line, spack does not require you to specify the --no-add option to prevent the spec from being added again. At the same time, a spec that already exists in the environment, but only as a dependency, will be added to the environment as a root spec without the --no-add option.

Developing Packages in a Spack Environment¶

The spack develop command allows one to develop Spack packages in an environment. It requires a spec containing a concrete version, and will configure Spack to install the package from local source. By default, it will also clone the package to a subdirectory in the environment. This package will have a special variant dev_path set, and Spack will ensure the package and its dependents are rebuilt any time the environment is installed if the package's local source code has been modified. Spack ensures that all instances of a developed package in the environment are concretized to match the version (and other constraints) passed as the spec argument to the spack develop command.

For packages with git attributes, git branches, tags, and commits can also be used as valid concrete versions (see Version specifier). This means that for a package foo, spack develop foo@git.main will clone the main branch of the package, and spack install will install from that git clone if foo is in the environment. Further development on foo can be tested by reinstalling the environment, and eventually committed and pushed to the upstream git repo.

Loading¶

Once an environment has been installed, the following creates a load script for it:

$ spack env loads -r

This creates a file called loads in the environment directory. Sourcing that file in Bash will make the environment available to the user; and can be included in .bashrc files, etc. The loads file may also be copied out of the environment, renamed, etc.

Configuring Environments¶

A variety of Spack behaviors are changed through Spack configuration files, covered in more detail in the Configuration Files section.

Spack Environments provide an additional level of configuration scope between the custom scope and the user scope discussed in the configuration documentation.

There are two ways to include configuration information in a Spack Environment:

Many Spack commands also affect configuration information in files automatically. Those commands take a --scope argument, and the environment can be specified by env:NAME (to affect environment foo, set --scope env:foo). These commands will automatically manipulate configuration inline in the spack.yaml file.

Inline configurations¶

Inline Environment-scope configuration is done using the same yaml format as standard Spack configuration scopes, covered in the Configuration Files section. Each section is contained under a top-level yaml object with it's name. For example, a spack.yaml manifest file containing some package preference configuration (as in a packages.yaml file) could contain:

spack:


  ...


  packages:


    all:


      compiler: [intel]


  ...

This configuration sets the default compiler for all packages to intel.

Included configurations¶

Spack environments allow an include heading in their yaml schema. This heading pulls in external configuration files and applies them to the Environment.

spack:


  include:


  - relative/path/to/config.yaml


  - https://github.com/path/to/raw/config/compilers.yaml


  - /absolute/path/to/packages.yaml

Environments can include files or URLs. File paths can be relative or absolute. URLs include the path to the text for individual files or can be the path to a directory containing configuration files.

Configuration precedence¶

Inline configurations take precedence over included configurations, so you don't have to change shared configuration files to make small changes to an individual environment. Included configurations listed earlier will have higher precedence, as the included configs are applied in reverse order.

Manually Editing the Specs List¶

The list of abstract/root specs in the Environment is maintained in the spack.yaml manifest under the heading specs.

spack:


    specs:


      - ncview


      - netcdf


      - nco


      - py-sphinx

Appending to this list in the yaml is identical to using the spack add command from the command line. However, there is more power available from the yaml file.

Spec concretization¶

An environment can be concretized in three different modes and the behavior active under any environment is determined by the concretizer:unify configuration option.

The default mode is to unify all specs:

spack:


    specs:


      - hdf5+mpi


      - zlib@1.2.8


    concretizer:


      unify: true

This means that any package in the environment corresponds to a single concrete spec. In the above example, when hdf5 depends down the line of zlib, it is required to take zlib@1.2.8 instead of a newer version. This mode of concretization is particularly useful when environment views are used: if every package occurs in only one flavor, it is usually possible to merge all install directories into a view.

A downside of unified concretization is that it can be overly strict. For example, a concretization error would happen when both hdf5+mpi and hdf5~mpi are specified in an environment.

The second mode is to unify when possible: this makes concretization of root specs more independendent. Instead of requiring reuse of dependencies across different root specs, it is only maximized:

spack:


    specs:


      - hdf5~mpi


      - hdf5+mpi


      - zlib@1.2.8


    concretizer:


      unify: when_possible

This means that both hdf5 installations will use zlib@1.2.8 as a dependency even if newer versions of that library are available.

The third mode of operation is to concretize root specs entirely independently by disabling unified concretization:

spack:


    specs:


      - hdf5~mpi


      - hdf5+mpi


      - zlib@1.2.8


    concretizer:


      unify: false

In this example hdf5 is concretized separately, and does not consider zlib@1.2.8 as a constraint or preference. Instead, it will take the latest possible version.

The last two concretization options are typically useful for system administrators and user support groups providing a large software stack for their HPC center.

NOTE:

Spec Matrices¶

Entries in the specs list can be individual abstract specs or a spec matrix.

A spec matrix is a yaml object containing multiple lists of specs, and evaluates to the cross-product of those specs. Spec matrices also contain an excludes directive, which eliminates certain combinations from the evaluated result.

The following two Environment manifests are identical:

spack:


  specs:


    - zlib %gcc@7.1.0


    - zlib %gcc@4.9.3


    - libelf %gcc@7.1.0


    - libelf %gcc@4.9.3


    - libdwarf %gcc@7.1.0


    - cmake
spack:


  specs:


    - matrix:


        - [zlib, libelf, libdwarf]


        - ['%gcc@7.1.0', '%gcc@4.9.3']


      exclude:


        - libdwarf%gcc@4.9.3


    - cmake

Spec matrices can be used to install swaths of software across various toolchains.

Spec List References¶

The last type of possible entry in the specs list is a reference.

The Spack Environment manifest yaml schema contains an additional heading definitions. Under definitions is an array of yaml objects. Each object has one or two fields. The one required field is a name, and the optional field is a when clause.

The named field is a spec list. The spec list uses the same syntax as the specs entry. Each entry in the spec list can be a spec, a spec matrix, or a reference to an earlier named list. References are specified using the $ sigil, and are "splatted" into place (i.e. the elements of the referent are at the same level as the elements listed separately). As an example, the following two manifest files are identical.

spack:


  definitions:


    - first: [libelf, libdwarf]


    - compilers: ['%gcc', '%intel']


    - second:


        - $first


        - matrix:


            - [zlib]


            - [$compilers]


  specs:


    - $second


    - cmake
spack:


  specs:


    - libelf


    - libdwarf


    - zlib%gcc


    - zlib%intel


    - cmake

NOTE:

In short files like the example, it may be easier to simply list the included specs. However for more complicated examples involving many packages across many toolchains, separately factored lists make Environments substantially more manageable.

Additionally, the -l option to the spack add command allows one to add to named lists in the definitions section of the manifest file directly from the command line.

The when directive can be used to conditionally add specs to a named list. The when directive takes a string of Python code referring to a restricted set of variables, and evaluates to a boolean. The specs listed are appended to the named list if the when string evaluates to True. In the following snippet, the named list compilers is ['%gcc', '%clang', '%intel'] on x86_64 systems and ['%gcc', '%clang'] on all other systems.

spack:


  definitions:


    - compilers: ['%gcc', '%clang']


    - when: arch.satisfies('x86_64:')


      compilers: ['%intel']

NOTE:

The valid variables for a when clause are:

SpecLists as Constraints¶

Dependencies and compilers in Spack can be both packages in an environment and constraints on other packages. References to SpecLists allow a shorthand to treat packages in a list as either a compiler or a dependency using the $% or $^ syntax respectively.

For example, the following environment has three root packages: gcc@8.1.0, mvapich2@2.3.1 %gcc@8.1.0, and hdf5+mpi %gcc@8.1.0 ^mvapich2@2.3.1.

spack:


  definitions:


  - compilers: [gcc@8.1.0]


  - mpis: [mvapich2@2.3.1]


  - packages: [hdf5+mpi]


  specs:


  - $compilers


  - matrix:


    - [$mpis]


    - [$%compilers]


  - matrix:


    - [$packages]


    - [$^mpis]


    - [$%compilers]

This allows for a much-needed reduction in redundancy between packages and constraints.

Filesystem Views¶

Spack Environments can define filesystem views, which provide a direct access point for software similar to the directory hierarchy that might exist under /usr/local. Filesystem views are updated every time the environment is written out to the lock file spack.lock, so the concrete environment and the view are always compatible. The files of the view's installed packages are brought into the view by symbolic or hard links, referencing the original Spack installation, or by copy.

Configuration in spack.yaml¶

The Spack Environment manifest file has a top-level keyword view. Each entry under that heading is a view descriptor, headed by a name. Any number of views may be defined under the view heading. The view descriptor contains the root of the view, and optionally the projections for the view, select and exclude lists for the view and link information via link and link_type.

For example, in the following manifest file snippet we define a view named mpis, rooted at /path/to/view in which all projections use the package name, version, and compiler name to determine the path for a given package. This view selects all packages that depend on MPI, and excludes those built with the PGI compiler at version 18.5. The root specs with their (transitive) link and run type dependencies will be put in the view due to the link: all option, and the files in the view will be symlinks to the spack install directories.

spack:


  ...


  view:


    mpis:


      root: /path/to/view


      select: [^mpi]


      exclude: ['%pgi@18.5']


      projections:


        all: '{name}/{version}-{compiler.name}'


      link: all


      link_type: symlink

The default for the select and exclude values is to select everything and exclude nothing. The default projection is the default view projection ({}). The link attribute allows the following values:

The link_type defaults to symlink but can also take the value of hardlink or copy.

TIP:

There are two shorthands for environments with a single view. If the environment at /path/to/env has a single view, with a root at /path/to/env/.spack-env/view, with default selection and exclusion and the default projection, we can put view: True in the environment manifest. Similarly, if the environment has a view with a different root, but default selection, exclusion, and projections, the manifest can say view: /path/to/view. These views are automatically named default, so that

spack:


  ...


  view: True

is equivalent to

spack:


  ...


  view:


    default:


      root: .spack-env/view

and

spack:


  ...


  view: /path/to/view

is equivalent to

spack:


  ...


  view:


    default:


      root: /path/to/view

By default, Spack environments are configured with view: True in the manifest. Environments can be configured without views using view: False. For backwards compatibility reasons, environments with no view key are treated the same as view: True.

From the command line, the spack env create command takes an argument --with-view [PATH] that sets the path for a single, default view. If no path is specified, the default path is used (view: True). The argument --without-view can be used to create an environment without any view configured.

The spack env view command can be used to change the manage views of an Environment. The subcommand spack env view enable will add a view named default to an environment. It takes an optional argument to specify the path for the new default view. The subcommand spack env view disable will remove the view named default from an environment if one exists. The subcommand spack env view regenerate will regenerate the views for the environment. This will apply any updates in the environment configuration that have not yet been applied.

View Projections¶

The default projection into a view is to link every package into the root of the view. The projections attribute is a mapping of partial specs to spec format strings, defined by the format() function, as shown in the example below:

projections:


  zlib: "{name}-{version}"


  ^mpi: "{name}-{version}/{^mpi.name}-{^mpi.version}-{compiler.name}-{compiler.version}"


  all: "{name}-{version}/{compiler.name}-{compiler.version}"

The entries in the projections configuration file must all be either specs or the keyword all. For each spec, the projection used will be the first non-all entry that the spec satisfies, or all if there is an entry for all and no other entry is satisfied by the spec. Where the keyword all appears in the file does not matter.

Given the example above, the spec zlib@1.2.8 will be linked into /my/view/zlib-1.2.8/, the spec hdf5@1.8.10+mpi %gcc@4.9.3 ^mvapich2@2.2 will be linked into /my/view/hdf5-1.8.10/mvapich2-2.2-gcc-4.9.3, and the spec hdf5@1.8.10~mpi %gcc@4.9.3 will be linked into /my/view/hdf5-1.8.10/gcc-4.9.3.

If the keyword all does not appear in the projections configuration file, any spec that does not satisfy any entry in the file will be linked into the root of the view as in a single-prefix view. Any entries that appear below the keyword all in the projections configuration file will not be used, as all specs will use the projection under all before reaching those entries.

Activating environment views¶

The spack env activate command will put the default view for the environment into the user's path, in addition to activating the environment for Spack commands. The arguments -v,--with-view and -V,--without-view can be used to tune this behavior. The default behavior is to activate with the environment view if there is one.

The environment variables affected by the spack env activate command and the paths that are used to update them are determined by the prefix inspections defined in your modules configuration; the defaults are summarized in the following table.

Each of these paths are appended to the view root, and added to the relevant variable if the path exists. For this reason, it is not recommended to use non-default projections with the default view of an environment.

The spack env deactivate command will remove the default view of the environment from the user's path.

Generating Depfiles from Environments¶

Spack can generate Makefiles to make it easier to build multiple packages in an environment in parallel. Generated Makefiles expose targets that can be included in existing Makefiles, to allow other targets to depend on the environment installation.

A typical workflow is as follows:

spack env create -d .
spack -e . add perl
spack -e . concretize
spack -e . env depfile -o Makefile
make -j64

This generates a Makefile from a concretized environment in the current working directory, and make -j64 installs the environment, exploiting parallelism across packages as much as possible. Spack respects the Make jobserver and forwards it to the build environment of packages, meaning that a single -j flag is enough to control the load, even when packages are built in parallel.

By default the following phony convenience targets are available:

TIP:

Specifying dependencies on generated make targets¶

An interesting question is how to include generated Makefiles in your own Makefiles. This comes up when you want to install an environment that provides executables required in a command for a make target of your own.

The example below shows how to accomplish this: the env target specifies the generated spack/env target as a prerequisite, meaning that the environment gets installed and is available for use in the env target.

SPACK ?= spack
.PHONY: all clean env
all: env
spack.lock: spack.yaml
	$(SPACK) -e . concretize -f
env.mk: spack.lock
	$(SPACK) -e . env depfile -o $@ --make-prefix spack
env: spack/env
	$(info Environment installed!)
clean:
	rm -rf spack.lock env.mk spack/
ifeq (,$(filter clean,$(MAKECMDGOALS)))
include env.mk
endif

This works as follows: when make is invoked, it first "remakes" the missing include env.mk as there is a target for it. This triggers concretization of the environment and makes spack output env.mk. At that point the generated target spack/env becomes available through include env.mk.

As it is typically undesirable to remake env.mk as part of make clean, the include is conditional.

NOTE:

Building a subset of the environment¶

The generated Makefiles contain install targets for each spec, identified by <name>-<version>-<hash>. This allows you to install only a subset of the packages in the environment. When packages are unique in the environment, it's enough to know the name and let tab-completion fill out the version and hash.

The following phony targets are available: install/<spec> to install the spec with its dependencies, and install-deps/<spec> to only install its dependencies. This can be useful when certain flags should only apply to dependencies. Below we show a use case where a spec is installed with verbose output (spack install --verbose) while its dependencies are installed silently:

$ spack env depfile -o Makefile
# Install dependencies in parallel, only show a log on error.
$ make -j16 install-deps/python-3.11.0-<hash> SPACK_INSTALL_FLAGS=--show-log-on-error
# Install the root spec with verbose output.
$ make -j16 install/python-3.11.0-<hash> SPACK_INSTALL_FLAGS=--verbose

Adding post-install hooks¶

Another advanced use-case of generated Makefiles is running a post-install command for each package. These "hooks" could be anything from printing a post-install message, running tests, or pushing just-built binaries to a buildcache.

This can be accomplished through the generated [<prefix>/]SPACK_PACKAGE_IDS variable. Assuming we have an active and concrete environment, we generate the associated Makefile with a prefix example:

$ spack env depfile -o env.mk --make-prefix example

And we now include it in a different Makefile, in which we create a target example/push/% with % referring to a package identifier. This target depends on the particular package installation. In this target we automatically have the target-specific HASH and SPEC variables at our disposal. They are respectively the spec hash (excluding leading /), and a human-readable spec. Finally, we have an entrypoint target push that will update the buildcache index once every package is pushed. Note how this target uses the generated example/SPACK_PACKAGE_IDS variable to define its prerequisites.

SPACK ?= spack
BUILDCACHE_DIR = $(CURDIR)/tarballs
.PHONY: all
all: push
include env.mk
example/push/%: example/install/%
	@mkdir -p $(dir $@)
	$(info About to push $(SPEC) to a buildcache)
	$(SPACK) -e . buildcache push --allow-root --only=package $(BUILDCACHE_DIR) /$(HASH)
	@touch $@
push: $(addprefix example/push/,$(example/SPACK_PACKAGE_IDS))
	$(info Updating the buildcache index)
	$(SPACK) -e . buildcache update-index $(BUILDCACHE_DIR)
	$(info Done!)
	@touch $@

A Quick Introduction¶

Consider having a Spack environment like the following:

spack:


  specs:


  - gromacs+mpi


  - mpich

Producing a Dockerfile from it is as simple as moving to the directory where the spack.yaml file is stored and giving the following command:

$ spack containerize > Dockerfile

The Dockerfile that gets created uses multi-stage builds and other techniques to minimize the size of the final image:

# Build stage with Spack pre-installed and ready to be used
FROM spack/ubuntu-bionic:latest as builder
# What we want to install and how we want to install it
# is specified in a manifest file (spack.yaml)
RUN mkdir /opt/spack-environment \
&&  (echo "spack:" \
&&   echo "  specs:" \
&&   echo "  - gromacs+mpi" \
&&   echo "  - mpich" \
&&   echo "  concretizer:" \
&&   echo "    unify: true" \
&&   echo "  config:" \
&&   echo "    install_tree: /opt/software" \
&&   echo "  view: /opt/view") > /opt/spack-environment/spack.yaml
# Install the software, remove unnecessary deps
RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast && spack gc -y
# Strip all the binaries
RUN find -L /opt/view/* -type f -exec readlink -f '{}' \; | \


    xargs file -i | \


    grep 'charset=binary' | \


    grep 'x-executable\|x-archive\|x-sharedlib' | \


    awk -F: '{print $1}' | xargs strip -s
# Modifications to the environment that are necessary to run
RUN cd /opt/spack-environment && \


    spack env activate --sh -d . >> /etc/profile.d/z10_spack_environment.sh
# Bare OS image to run the installed executables
FROM ubuntu:18.04
COPY --from=builder /opt/spack-environment /opt/spack-environment
COPY --from=builder /opt/software /opt/software
COPY --from=builder /opt/view /opt/view
COPY --from=builder /etc/profile.d/z10_spack_environment.sh /etc/profile.d/z10_spack_environment.sh
ENTRYPOINT ["/bin/bash", "--rcfile", "/etc/profile", "-l"]

The image itself can then be built and run in the usual way, with any of the tools suitable for the task. For instance, if we decided to use docker:

$ spack containerize > Dockerfile
$ docker build -t myimage .
[ ... ]
$ docker run -it myimage

The various components involved in the generation of the recipe and their configuration are discussed in details in the sections below.

Spack Images on Docker Hub¶

Docker images with Spack preinstalled and ready to be used are built when a release is tagged, or nightly on develop. The images are then pushed both to Docker Hub and to GitHub Container Registry. The OS that are currently supported are summarized in the table below:

Supported operating systems¶

All the images are tagged with the corresponding release of Spack: [image]

with the exception of the latest tag that points to the HEAD of the develop branch. These images are available for anyone to use and take care of all the repetitive tasks that are necessary to setup Spack within a container. The container recipes generated by Spack use them as default base images for their build stage, even though handles to use custom base images provided by users are available to accommodate complex use cases.

Creating Images From Environments¶

Any Spack Environment can be used for the automatic generation of container recipes. Sensible defaults are provided for things like the base image or the version of Spack used in the image. If a finer tuning is needed it can be obtained by adding the relevant metadata under the container attribute of environments:

spack:


  specs:


  - gromacs+mpi


  - mpich


  container:


    # Select the format of the recipe e.g. docker,


    # singularity or anything else that is currently supported


    format: docker


    # Sets the base images for the stages where Spack builds the


    # software or where the software gets installed after being built..


    images:


      os: "centos:7"


      spack: develop


    # Whether or not to strip binaries


    strip: true


    # Additional system packages that are needed at runtime


    os_packages:


      final:


      - libgomp


    # Labels for the image


    labels:


      app: "gromacs"


      mpi: "mpich"

A detailed description of the options available can be found in the Configuration Reference section.

Setting Base Images¶

The images subsection is used to select both the image where Spack builds the software and the image where the built software is installed. This attribute can be set in different ways and which one to use depends on the use case at hand.

Use Official Spack Images From Dockerhub¶

To generate a recipe that uses an official Docker image from the Spack organization to build the software and the corresponding official OS image to install the built software, all the user has to do is specify:

Any combination of these two values that can be mapped to one of the images discussed in Spack Images on Docker Hub is allowed. For instance, the following spack.yaml:

spack:


  specs:


  - gromacs+mpi


  - mpich


  container:


    images:


      os: centos:7


      spack: 0.15.4

uses spack/centos7:0.15.4 and centos:7 for the stages where the software is respectively built and installed:

# Build stage with Spack pre-installed and ready to be used
FROM spack/centos7:0.15.4 as builder
# What we want to install and how we want to install it
# is specified in a manifest file (spack.yaml)
RUN mkdir /opt/spack-environment \
&&  (echo "spack:" \
&&   echo "  specs:" \
&&   echo "  - gromacs+mpi" \
&&   echo "  - mpich" \
&&   echo "  concretizer:" \
&&   echo "    unify: true" \
&&   echo "  config:" \
&&   echo "    install_tree: /opt/software" \
&&   echo "  view: /opt/view") > /opt/spack-environment/spack.yaml
[ ... ]
# Bare OS image to run the installed executables
FROM centos:7
COPY --from=builder /opt/spack-environment /opt/spack-environment
COPY --from=builder /opt/software /opt/software
COPY --from=builder /opt/view /opt/view
COPY --from=builder /etc/profile.d/z10_spack_environment.sh /etc/profile.d/z10_spack_environment.sh
ENTRYPOINT ["/bin/bash", "--rcfile", "/etc/profile", "-l"]

This is the simplest available method of selecting base images, and we advise to use it whenever possible. There are cases though where using Spack official images is not enough to fit production needs. In these situations users can extend the recipe to start with the bootstrapping of Spack at a certain pinned version or manually select which base image to start from in the recipe, as we'll see next.

Use a Bootstrap Stage for Spack¶

In some cases users may want to pin the commit sha that is used for Spack, to ensure later reproducibility, or start from a fork of the official Spack repository to try a bugfix or a feature in the early stage of development. This is possible by being just a little more verbose when specifying information about Spack in the spack.yaml file:

images:


  os: amazonlinux:2


  spack:


    # URL of the Spack repository to be used in the container image


    url: <to-use-a-fork>


    # Either a commit sha, a branch name or a tag


    ref: <sha/tag/branch>


    # If true turn a branch name or a tag into the corresponding commit


    # sha at the time of recipe generation


    resolve_sha: <true/false>

url specifies the URL from which to clone Spack and defaults to https://github.com/spack/spack. The ref attribute can be either a commit sha, a branch name or a tag. The default value in this case is to use the develop branch, but it may change in the future to point to the latest stable release. Finally resolve_sha transform branch names or tags into the corresponding commit shas at the time of recipe generation, to allow for a greater reproducibility of the results at a later time.

The list of operating systems that can be used to bootstrap Spack can be obtained with:

$ spack containerize --list-os
==> The following operating systems can be used to bootstrap Spack:
alpine:3 amazonlinux:2 fedora:38 fedora:37 rockylinux:9 rockylinux:8 almalinux:9 almalinux:8 centos:stream centos:7 opensuse/leap:15 suse/sle:15 nvidia/cuda:11.2.1 ubuntu:22.04 ubuntu:20.04 ubuntu:18.04

NOTE:

Use Custom Images Provided by Users¶

Consider, as an example, building a production grade image for a CUDA application. The best strategy would probably be to build on top of images provided by the vendor and regard CUDA as an external package.

Spack doesn't currently provide an official image with CUDA configured this way, but users can build it on their own and then configure the environment to explicitly pull it. This requires users to:

A spack.yaml like the following:

spack:


  specs:


  - gromacs@2019.4+cuda build_type=Release


  - mpich


  - fftw precision=float


  packages:


    cuda:


      buildable: False


      externals:


      - spec: cuda%gcc


        prefix: /usr/local/cuda


  container:


    images:


      build: custom/cuda-10.1-ubuntu18.04:latest


      final: nvidia/cuda:10.1-base-ubuntu18.04

produces, for instance, the following Dockerfile:

# Build stage with Spack pre-installed and ready to be used
FROM custom/cuda-10.1-ubuntu18.04:latest as builder
# What we want to install and how we want to install it
# is specified in a manifest file (spack.yaml)
RUN mkdir /opt/spack-environment \
&&  (echo "spack:" \
&&   echo "  specs:" \
&&   echo "  - gromacs@2019.4+cuda build_type=Release" \
&&   echo "  - mpich" \
&&   echo "  - fftw precision=float" \
&&   echo "  packages:" \
&&   echo "    cuda:" \
&&   echo "      buildable: false" \
&&   echo "      externals:" \
&&   echo "      - spec: cuda%gcc" \
&&   echo "        prefix: /usr/local/cuda" \
&&   echo "  concretizer:" \
&&   echo "    unify: true" \
&&   echo "  config:" \
&&   echo "    install_tree: /opt/software" \
&&   echo "  view: /opt/view") > /opt/spack-environment/spack.yaml
# Install the software, remove unnecessary deps
RUN cd /opt/spack-environment && spack env activate . && spack install --fail-fast && spack gc -y
# Strip all the binaries
RUN find -L /opt/view/* -type f -exec readlink -f '{}' \; | \


    xargs file -i | \


    grep 'charset=binary' | \


    grep 'x-executable\|x-archive\|x-sharedlib' | \


    awk -F: '{print $1}' | xargs strip -s
# Modifications to the environment that are necessary to run
RUN cd /opt/spack-environment && \


    spack env activate --sh -d . >> /etc/profile.d/z10_spack_environment.sh
# Bare OS image to run the installed executables
FROM nvidia/cuda:10.1-base-ubuntu18.04
COPY --from=builder /opt/spack-environment /opt/spack-environment
COPY --from=builder /opt/software /opt/software
COPY --from=builder /opt/view /opt/view
COPY --from=builder /etc/profile.d/z10_spack_environment.sh /etc/profile.d/z10_spack_environment.sh
ENTRYPOINT ["/bin/bash", "--rcfile", "/etc/profile", "-l"]

where the base images for both stages are completely custom.

This second mode of selection for base images is more flexible than just choosing an operating system and a Spack version, but is also more demanding. Users may need to generate by themselves their base images and it's also their responsibility to ensure that:

Therefore we don't recommend its use in cases that can be otherwise covered by the simplified mode shown first.

Singularity Definition Files¶

In addition to producing recipes in Dockerfile format Spack can produce Singularity Definition Files by just changing the value of the format attribute:

$ cat spack.yaml
spack:


  specs:


  - hdf5~mpi


  container:


    format: singularity
$ spack containerize > hdf5.def
$ sudo singularity build hdf5.sif hdf5.def

The minimum version of Singularity required to build a SIF (Singularity Image Format) image from the recipes generated by Spack is 3.5.3.

Extending the Jinja2 Templates¶

The Dockerfile and the Singularity definition file that Spack can generate are based on a few Jinja2 templates that are rendered according to the environment being containerized. Even though Spack allows a great deal of customization by just setting appropriate values for the configuration options, sometimes that is not enough.

In those cases, a user can directly extend the template that Spack uses to render the image to e.g. set additional environment variables or perform specific operations either before or after a given stage of the build. Let's consider as an example the following structure:

$ tree /opt/environment
/opt/environment
├── data
│     └── data.csv
├── spack.yaml
├── data
└── templates


    └── container


        └── CustomDockerfile

containing both the custom template extension and the environment manifest file. To use a custom template, the environment must register the directory containing it, and declare its use under the container configuration:

spack:


  specs:


  - hdf5~mpi


  concretizer:


    unify: true


  config:


    template_dirs:


    - /opt/environment/templates


  container:


    format: docker


    depfile: true


    template: container/CustomDockerfile

The template extension can override two blocks, named build_stage and final_stage, similarly to the example below:

{% extends "container/Dockerfile" %}
{% block build_stage %}
RUN echo "Start building"
{{ super() }}
{% endblock %}
{% block final_stage %}
{{ super() }}
COPY data /share/myapp/data
{% endblock %}

The Dockerfile is generated by running:

$ spack -e /opt/environment containerize

Note that the environment must be active for spack to read the template. The recipe that gets generated contains the two extra instruction that we added in our template extension:

# Build stage with Spack pre-installed and ready to be used
FROM spack/ubuntu-jammy:latest as builder
RUN echo "Start building"
# What we want to install and how we want to install it
# is specified in a manifest file (spack.yaml)
RUN mkdir /opt/spack-environment \
&&  (echo "spack:" \
&&   echo "  specs:" \
&&   echo "  - hdf5~mpi" \
&&   echo "  concretizer:" \
&&   echo "    unify: true" \
&&   echo "  config:" \
&&   echo "    template_dirs:" \
&&   echo "    - /tmp/environment/templates" \
&&   echo "    install_tree: /opt/software" \
&&   echo "  view: /opt/view") > /opt/spack-environment/spack.yaml
# Install the software, remove unnecessary deps
RUN cd /opt/spack-environment && spack env activate . && spack concretize && spack env depfile -o Makefile && make -j $(nproc) && spack gc -y
# Strip all the binaries
RUN find -L /opt/view/* -type f -exec readlink -f '{}' \; | \


    xargs file -i | \


    grep 'charset=binary' | \


    grep 'x-executable\|x-archive\|x-sharedlib' | \


    awk -F: '{print $1}' | xargs strip -s
# Modifications to the environment that are necessary to run
RUN cd /opt/spack-environment && \


    spack env activate --sh -d . >> /etc/profile.d/z10_spack_environment.sh
# Bare OS image to run the installed executables
FROM ubuntu:22.04
COPY --from=builder /opt/spack-environment /opt/spack-environment
COPY --from=builder /opt/software /opt/software
COPY --from=builder /opt/._view /opt/._view
COPY --from=builder /opt/view /opt/view
COPY --from=builder /etc/profile.d/z10_spack_environment.sh /etc/profile.d/z10_spack_environment.sh
COPY data /share/myapp/data
ENTRYPOINT ["/bin/bash", "--rcfile", "/etc/profile", "-l", "-c", "$*", "--" ]
CMD [ "/bin/bash" ]

Configuration Reference¶

The tables below describe all the configuration options that are currently supported to customize the generation of container recipes:

General configuration options for the container section of spack.yaml¶

Configuration options specific to Singularity¶

Best Practices¶

MPI¶

Due to the dependency on Fortran for OpenMPI, which is the spack default implementation, consider adding gfortran to the apt-get install list.

Recent versions of OpenMPI will require you to pass --allow-run-as-root to your mpirun calls if started as root user inside Docker.

For execution on HPC clusters, it can be helpful to import the docker image into Singularity in order to start a program with an external MPI. Otherwise, also add openssh-server to the apt-get install list.

CUDA¶

Starting from CUDA 9.0, Nvidia provides minimal CUDA images based on Ubuntu. Please see their instructions. Avoid double-installing CUDA by adding, e.g.

packages:


  cuda:


    externals:


    - spec: "cuda@9.0.176%gcc@5.4.0 arch=linux-ubuntu16-x86_64"


      prefix: /usr/local/cuda


    buildable: False

to your spack.yaml.

Users will either need nvidia-docker or e.g. Singularity to execute device kernels.

Docker on Windows and OSX¶

On Mac OS and Windows, docker runs on a hypervisor that is not allocated much memory by default, and some spack packages may fail to build due to lack of memory. To work around this issue, consider configuring your docker installation to use more of your host memory. In some cases, you can also ease the memory pressure on parallel builds by limiting the parallelism in your config.yaml.

config:


  build_jobs: 2

spack mirror¶

Mirrors are managed with the spack mirror command. The help for spack mirror looks like this:

$ spack help mirror
usage: spack mirror [-hn] [--deprecated] SUBCOMMAND ...
manage mirrors (source and binary)
positional arguments:


  SUBCOMMAND


    create           create a directory to be used as a spack mirror, and fill it with package archives


    destroy          given a url, recursively delete everything under it


    add              add a mirror to Spack


    remove (rm)      remove a mirror by name


    set-url          change the URL of a mirror


    set              configure the connection details of a mirror


    list             print out available mirrors to the console
options:


  --deprecated       fetch deprecated versions without warning


  -h, --help         show this help message and exit


  -n, --no-checksum  do not use checksums to verify downloaded files (unsafe)

The create command actually builds a mirror by fetching all of its packages from the internet and checksumming them.

The other three commands are for managing mirror configuration. They control the URL(s) from which Spack downloads its packages.

spack mirror create¶

You can create a mirror using the spack mirror create command, assuming you're on a machine where you can access the internet.

The command will iterate through all of Spack's packages and download the safe ones into a directory structure like the one above. Here is what it looks like:

$ spack mirror create libelf libdwarf
==> Created new mirror in spack-mirror-2014-06-24
==> Trying to fetch from http://www.mr511.de/software/libelf-0.8.13.tar.gz
##########################################################                81.6%
==> Checksum passed for libelf@0.8.13
==> Added libelf@0.8.13
==> Trying to fetch from http://www.mr511.de/software/libelf-0.8.12.tar.gz
######################################################################    98.6%
==> Checksum passed for libelf@0.8.12
==> Added libelf@0.8.12
==> Trying to fetch from http://www.prevanders.net/libdwarf-20130207.tar.gz
######################################################################    97.3%
==> Checksum passed for libdwarf@20130207
==> Added libdwarf@20130207
==> Trying to fetch from http://www.prevanders.net/libdwarf-20130126.tar.gz
########################################################                  78.9%
==> Checksum passed for libdwarf@20130126
==> Added libdwarf@20130126
==> Trying to fetch from http://www.prevanders.net/libdwarf-20130729.tar.gz
#############################################################             84.7%
==> Added libdwarf@20130729
==> Added spack-mirror-2014-06-24/libdwarf/libdwarf-20130729.tar.gz to mirror
==> Added python@2.7.8.
==> Successfully updated mirror in spack-mirror-2015-02-24.


  Archive stats:


    0    already present


    5    added


    0    failed to fetch.

Once this is done, you can tar up the spack-mirror-2014-06-24 directory and copy it over to the machine you want it hosted on.

Custom package sets¶

Normally, spack mirror create downloads all the archives it has checksums for. If you want to only create a mirror for a subset of packages, you can do that by supplying a list of package specs on the command line after spack mirror create. For example, this command:

$ spack mirror create libelf@0.8.12: boost@1.44:

Will create a mirror for libelf versions greater than or equal to 0.8.12 and boost versions greater than or equal to 1.44.

Mirror files¶

If you have a very large number of packages you want to mirror, you can supply a file with specs in it, one per line:

$ cat specs.txt
libdwarf
libelf@0.8.12:
boost@1.44:
boost@1.39.0
...
$ spack mirror create --file specs.txt
...

This is useful if there is a specific suite of software managed by your site.

Mirror environment¶

To create a mirror of all packages required by a concrete environment, activate the environment and call spack mirror create -a. This is especially useful to create a mirror of an environment concretized on another machine.

[remote] $ spack env create myenv
[remote] $ spack env activate myenv
[remote] $ spack add ...
[remote] $ spack concretize
$ sftp remote:/spack/var/environment/myenv/spack.lock
$ spack env create myenv spack.lock
$ spack env activate myenv
$ spack mirror create -a

spack mirror add¶

Once you have a mirror, you need to let spack know about it. This is relatively simple. First, figure out the URL for the mirror. If it's a directory, you can use a file URL like this one:

file://$HOME/spack-mirror-2014-06-24

That points to the directory on the local filesystem. If it were on a web server, you could use a URL like this one:

https://example.com/some/web-hosted/directory/spack-mirror-2014-06-24

Spack will use the URL as the root for all of the packages it fetches. You can tell your Spack installation to use that mirror like this:

$ spack mirror add local_filesystem file://$HOME/spack-mirror-2014-06-24

Each mirror has a name so that you can refer to it again later.

spack mirror list¶

To see all the mirrors Spack knows about, run spack mirror list:

$ spack mirror list
local_filesystem    file:///home/username/spack-mirror-2014-06-24

spack mirror remove¶

To remove a mirror by name, run:

$ spack mirror remove local_filesystem
$ spack mirror list
==> No mirrors configured.

Mirror precedence¶

Adding a mirror really adds a line in ~/.spack/mirrors.yaml:

mirrors:


  local_filesystem: file:///home/username/spack-mirror-2014-06-24


  remote_server: https://example.com/some/web-hosted/directory/spack-mirror-2014-06-24

If you want to change the order in which mirrors are searched for packages, you can edit this file and reorder the sections. Spack will search the topmost mirror first and the bottom-most mirror last.

Local Default Cache¶

Spack caches resources that are downloaded as part of installs. The cache is a valid spack mirror: it uses the same directory structure and naming scheme as other Spack mirrors (so it can be copied anywhere and referenced with a URL like other mirrors). The mirror is maintained locally (within the Spack installation directory) at var/spack/cache/. It is always enabled (and is always searched first when attempting to retrieve files for an installation) but can be cleared with clean; the cache directory can also be deleted manually without issue.

Caching includes retrieved tarball archives and source control repositories, but only resources with an associated digest or commit ID (e.g. a revision number for SVN) will be cached.

Using module files via Spack¶

If you have installed a supported module system you should be able to run module avail to see what module files have been installed. Here is sample output of those programs, showing lots of installed packages:

$ module avail
--------------------------------------------------------------- ~/spack/share/spack/modules/linux-ubuntu14-x86_64 ---------------------------------------------------------------
autoconf/2.69-gcc-4.8-qextxkq       hwloc/1.11.6-gcc-6.3.0-akcisez             m4/1.4.18-gcc-4.8-ev2znoc                   openblas/0.2.19-gcc-6.3.0-dhkmed6        py-setuptools/34.2.0-gcc-6.3.0-fadur4s
automake/1.15-gcc-4.8-maqvukj       isl/0.18-gcc-4.8-afi6taq                   m4/1.4.18-gcc-6.3.0-uppywnz                 openmpi/2.1.0-gcc-6.3.0-go2s4z5          py-six/1.10.0-gcc-6.3.0-p4dhkaw
binutils/2.28-gcc-4.8-5s7c6rs       libiconv/1.15-gcc-4.8-at46wg3              mawk/1.3.4-gcc-4.8-acjez57                  openssl/1.0.2k-gcc-4.8-dkls5tk           python/2.7.13-gcc-6.3.0-tyehea7
bison/3.0.4-gcc-4.8-ek4luo5         libpciaccess/0.13.4-gcc-6.3.0-gmufnvh      mawk/1.3.4-gcc-6.3.0-ostdoms                openssl/1.0.2k-gcc-6.3.0-gxgr5or         readline/7.0-gcc-4.8-xhufqhn
bzip2/1.0.6-gcc-4.8-iffrxzn         libsigsegv/2.11-gcc-4.8-pp2cvte            mpc/1.0.3-gcc-4.8-g5mztc5                   pcre/8.40-gcc-4.8-r5pbrxb                readline/7.0-gcc-6.3.0-zzcyicg
bzip2/1.0.6-gcc-6.3.0-bequudr       libsigsegv/2.11-gcc-6.3.0-7enifnh          mpfr/3.1.5-gcc-4.8-o7xm7az                  perl/5.24.1-gcc-4.8-dg5j65u              sqlite/3.8.5-gcc-6.3.0-6zoruzj
cmake/3.7.2-gcc-6.3.0-fowuuby       libtool/2.4.6-gcc-4.8-7a523za              mpich/3.2-gcc-6.3.0-dmvd3aw                 perl/5.24.1-gcc-6.3.0-6uzkpt6            tar/1.29-gcc-4.8-wse2ass
curl/7.53.1-gcc-4.8-3fz46n6         libtool/2.4.6-gcc-6.3.0-n7zmbzt            ncurses/6.0-gcc-4.8-dcpe7ia                 pkg-config/0.29.2-gcc-4.8-ib33t75        tcl/8.6.6-gcc-4.8-tfxzqbr
expat/2.2.0-gcc-4.8-mrv6bd4         libxml2/2.9.4-gcc-4.8-ryzxnsu              ncurses/6.0-gcc-6.3.0-ucbhcdy               pkg-config/0.29.2-gcc-6.3.0-jpgubk3      util-macros/1.19.1-gcc-6.3.0-xorz2x2
flex/2.6.3-gcc-4.8-yf345oo          libxml2/2.9.4-gcc-6.3.0-rltzsdh            netlib-lapack/3.6.1-gcc-6.3.0-js33dog       py-appdirs/1.4.0-gcc-6.3.0-jxawmw7       xz/5.2.3-gcc-4.8-mew4log
gcc/6.3.0-gcc-4.8-24puqve           lmod/7.4.1-gcc-4.8-je4srhr                 netlib-scalapack/2.0.2-gcc-6.3.0-5aidk4l    py-numpy/1.12.0-gcc-6.3.0-oemmoeu        xz/5.2.3-gcc-6.3.0-3vqeuvb
gettext/0.19.8.1-gcc-4.8-yymghlh    lua/5.3.4-gcc-4.8-im75yaz                  netlib-scalapack/2.0.2-gcc-6.3.0-hjsemcn    py-packaging/16.8-gcc-6.3.0-i2n3dtl      zip/3.0-gcc-4.8-rwar22d
gmp/6.1.2-gcc-4.8-5ub2wu5           lua-luafilesystem/1_6_3-gcc-4.8-wkey3nl    netlib-scalapack/2.0.2-gcc-6.3.0-jva724b    py-pyparsing/2.1.10-gcc-6.3.0-tbo6gmw    zlib/1.2.11-gcc-4.8-pgxsxv7
help2man/1.47.4-gcc-4.8-kcnqmau     lua-luaposix/33.4.0-gcc-4.8-mdod2ry        netlib-scalapack/2.0.2-gcc-6.3.0-rgqfr6d    py-scipy/0.19.0-gcc-6.3.0-kr7nat4        zlib/1.2.11-gcc-6.3.0-7cqp6cj

The names should look familiar, as they resemble the output from spack find. For example, you could type the following command to load the cmake module:

$ module load cmake/3.7.2-gcc-6.3.0-fowuuby

Neither of these is particularly pretty, easy to remember, or easy to type. Luckily, Spack offers many facilities for customizing the module scheme used at your site.

Module file customization¶

Module files are generated by post-install hooks after the successful installation of a package.

NOTE:

The table below summarizes the essential information associated with the different file formats that can be generated by Spack:

Spack ships with sensible defaults for the generation of module files, but you can customize many aspects of it to accommodate package or site specific needs. In general you can override or extend the default behavior by:

The former method let you express changes in the run-time environment that are needed to use the installed software properly, e.g. injecting variables from language interpreters into their extensions. The latter two instead permit to fine tune the filesystem layout, content and creation of module files to meet site specific conventions.

Override API calls in package.py¶

There are two methods that you can override in any package.py to affect the content of the module files generated by Spack. The first one:

def setup_run_environment(self, env):


    pass

can alter the content of the module file associated with the same package where it is overridden. The second method:

def setup_dependent_run_environment(self, env, dependent_spec):


    pass

can instead inject run-time environment modifications in the module files of packages that depend on it. In both cases you need to fill env with the desired list of environment modifications.

    def setup_run_environment(self, env):


        env.prepend_path("LD_LIBRARY_PATH", join_path(self.prefix, "rlib", "R", "lib"))


        env.prepend_path("PKG_CONFIG_PATH", join_path(self.prefix, "rlib", "pkgconfig"))


        env.set("R_HOME", join_path(self.prefix, "rlib", "R"))


        if "+rmath" in self.spec:


            env.prepend_path("LD_LIBRARY_PATH", join_path(self.prefix, "rlib"))

    def setup_dependent_run_environment(self, env, dependent_spec):


        # For run time environment set only the path for dependent_spec and


        # prepend it to R_LIBS


        env.set("R_HOME", join_path(self.prefix, "rlib", "R"))


        if dependent_spec.package.extends(self.spec):


            env.prepend_path("R_LIBS", join_path(dependent_spec.prefix, self.r_lib_dir))

Write a configuration file¶

The configuration files that control module generation behavior are named modules.yaml. The default configuration:

# -------------------------------------------------------------------------
# This is the default configuration for Spack's module file generation.
#
# Settings here are versioned with Spack and are intended to provide
# sensible defaults out of the box. Spack maintainers should edit this
# file to keep it current.
#
# Users can override these settings by editing the following files.
#
# Per-spack-instance settings (overrides defaults):
#   $SPACK_ROOT/etc/spack/modules.yaml
#
# Per-user settings (overrides default and site settings):
#   ~/.spack/modules.yaml
# -------------------------------------------------------------------------
modules:


  # This maps paths in the package install prefix to environment variables


  # they should be added to. For example, <prefix>/bin should be in PATH.


  prefix_inspections:


    ./bin:


      - PATH


    ./man:


      - MANPATH


    ./share/man:


      - MANPATH


    ./share/aclocal:


      - ACLOCAL_PATH


    ./lib/pkgconfig:


      - PKG_CONFIG_PATH


    ./lib64/pkgconfig:


      - PKG_CONFIG_PATH


    ./share/pkgconfig:


      - PKG_CONFIG_PATH


    ./:


      - CMAKE_PREFIX_PATH


  # These are configurations for the module set named "default"


  default:


    # Where to install modules


    roots:


     tcl:    $spack/share/spack/modules


     lmod:   $spack/share/spack/lmod


    # What type of modules to use ("tcl" and/or "lmod")


    enable:


      - lmod


    tcl:


      all:


        autoload: direct


    # Default configurations if lmod is enabled


    lmod:


      all:


        autoload: direct


      hierarchy:


        - mpi

activates the hooks to generate tcl module files and inspects the installation folder of each package for the presence of a set of subdirectories (bin, man, share/man, etc.). If any is found its full path is prepended to the environment variables listed below the folder name.

Spack modules can be configured for multiple module sets. The default module set is named default. All Spack commands which operate on modules default to apply the default module set, but can be applied to any module set in the configuration.

Changing the modules root¶

As shown in the table above, the default module root for lmod is $spack/share/spack/lmod and the default root for tcl is $spack/share/spack/modules. This can be overridden for any module set by changing the roots key of the configuration.

modules:


  default:


    roots:


      tcl: /path/to/install/tcl/modules


  my_custom_lmod_modules:


    roots:


      lmod: /path/to/install/custom/lmod/modules


      ...

This configuration will create two module sets. The default module set will install its tcl modules to /path/to/install/tcl/modules (and still install its lmod modules, if any, to the default location). The set my_custom_lmod_modules will install its lmod modules to /path/to/install/custom/lmod/modules (and still install its tcl modules, if any, to the default location).

By default, an architecture-specific directory is added to the root directory. A module set may override that behavior by setting the arch_folder config value to False.

modules:


  default:


    roots:


      tcl: /path/to/install/tcl/modules


    arch_folder: false

Obviously, having multiple module sets install modules to the default location could be confusing to users of your modules. In the next section, we will discuss enabling and disabling module types (module file generators) for each module set.

Activate other hooks¶

Any other module file generator shipped with Spack can be activated adding it to the list under the enable key in the module file. Currently the only generator that is not active by default is lmod, which produces hierarchical lua module files.

Each module system can then be configured separately. In fact, you should list configuration options that affect a particular type of module files under a top level key corresponding to the generator being customized:

modules:


  default:


    enable:


      - tcl


      - lmod


    tcl:


      # contains environment modules specific customizations


    lmod:


      # contains lmod specific customizations

In general, the configuration options that you can use in modules.yaml will either change the layout of the module files on the filesystem, or they will affect their content. For the latter point it is possible to use anonymous specs to fine tune the set of packages on which the modifications should be applied.

Selection by anonymous specs¶

In the configuration file you can use anonymous specs (i.e. specs that are not required to have a root package and are thus used just to express constraints) to apply certain modifications on a selected set of the installed software. For instance, in the snippet below:

modules:


  default:


    tcl:


      # The keyword `all` selects every package


      all:


        environment:


          set:


            BAR: 'bar'


      # This anonymous spec selects any package that


      # depends on mpi. The double colon at the


      # end clears the set of rules that matched so far.


      ^mpi::


        environment:


          prepend_path:


            PATH: '{^mpi.prefix}/bin'


          set:


            BAR: 'baz'


      # Selects any zlib package


      zlib:


        environment:


          prepend_path:


            LD_LIBRARY_PATH: 'foo'


      # Selects zlib compiled with gcc@4.8


      zlib%gcc@4.8:


        environment:


          unset:


          - FOOBAR

you are instructing Spack to set the environment variable BAR=bar for every module, unless the associated spec satisfies the abstract dependency ^mpi in which case BAR=baz, and the directory containing the respective MPI executables is prepended to the PATH variable. In addition in any spec that satisfies zlib the value foo will be prepended to LD_LIBRARY_PATH and in any spec that satisfies zlib%gcc@4.8 the variable FOOBAR will be unset.

NOTE:

Exclude or include specific module files¶

You can use anonymous specs also to prevent module files from being written or to force them to be written. Consider the case where you want to hide from users all the boilerplate software that you had to build in order to bootstrap a new compiler. Suppose for instance that gcc@4.4.7 is the compiler provided by your system. If you write a configuration file like:

modules:


  default:


    tcl:


      include: ['gcc', 'llvm']  # include will have precedence over exclude


      exclude: ['%gcc@4.4.7']   # Assuming gcc@4.4.7 is the system compiler

you will prevent the generation of module files for any package that is compiled with gcc@4.4.7, with the only exception of any gcc or any llvm installation.

Customize the naming of modules¶

The names of environment modules generated by spack are not always easy to fully comprehend due to the long hash in the name. There are three module configuration options to help with that. The first is a global setting to adjust the hash length. It can be set anywhere from 0 to 32 and has a default length of 7. This is the representation of the hash in the module file name and does not affect the size of the package hash. Be aware that the smaller the hash length the more likely naming conflicts will occur. The following snippet shows how to set hash length in the module file names:

modules:


  default:


    tcl:


      hash_length: 7

To help make module names more readable, and to help alleviate name conflicts with a short hash, one can use the suffixes option in the modules configuration file. This option will add strings to modules that match a spec. For instance, the following config options,

modules:


  default:


    tcl:


      all:


        suffixes:


          ^python@2.7.12: 'python-2.7.12'


          ^openblas: 'openblas'

will add a python-2.7.12 version string to any packages compiled with python matching the spec, python@2.7.12. This is useful to know which version of python a set of python extensions is associated with. Likewise, the openblas string is attached to any program that has openblas in the spec, most likely via the +blas variant specification.

The most heavyweight solution to module naming is to change the entire naming convention for module files. This uses the projections format covered in View Projections.

modules:


  default:


    tcl:


      projections:


        all: '{name}/{version}-{compiler.name}-{compiler.version}-module'


        ^mpi: '{name}/{version}-{^mpi.name}-{^mpi.version}-{compiler.name}-{compiler.version}-module'

will create module files that are nested in directories by package name, contain the version and compiler name and version, and have the word module before the hash for all specs that do not depend on mpi, and will have the same information plus the MPI implementation name and version for all packages that depend on mpi.

When specifying module names by projection for Lmod modules, we recommend NOT including names of dependencies (e.g., MPI, compilers) that are already in the Lmod hierarchy.

NOTE:

modules:


  default:


    enable:


      - tcl


    tcl:


      projections:


        all: '{name}/{version}-{compiler.name}-{compiler.version}'


      all:


        conflict:


          - '{name}'


          - 'intel/14.0.1'

NOTE:

modules:


  default:


    enable:


      - lmod


    lmod:


      core_compilers:


        - 'gcc@4.8'


      core_specs:


        - 'python'


      hierarchy:


        - 'mpi'


        - 'lapack'

WARNING:

WARNING: