Python Environments and Package Installation
While using CHPC resources, you may need to install Python packages required by your software. You have several options for installing your own Python packages, and each method has benefits and drawbacks. We recommend using mamba or conda or creating a virtual environment, and we provide more information about each method, along with links to instructions, below.
mamba or conda with CHPC-installed Miniforge
The package managers mamba and conda are robust tools for setting up a Python environment and installing packages. You will need to learn some commands to load and unload your environment, however, as the environments are not a built-in feature of the Python language.
| Choose mamba or conda when a package installation guide suggests it; when a package has complex dependencies; and when you're working with packages written in multiple languages, like Python and R. This is a robust solution for installing your own packages. |
Virtual environments (venv)
A virtual environment (venv) allows you to install packages with pip. It's quick and easy to set up a venv, and you can create a module that will allow your environment to work seamlessly with other software installed on CHPC resources. However, for some packages, a package manager like mamba or conda is necessary or recommended by package developers. (If any of your packages requires mamba or conda, use mamba or conda for all packages.)
| Choose a virtual environment (venv) when you need to set up a Python environment with packages that are available through pip and don't have complex dependencies. This is a simple, lightweight solution for installing your own packages. |
mamba or conda with self-installed Miniforge
Installing Miniforge yourself allows to completely isolate your Python environments from each other, and load/unload them via environment modules. This provides a more natural plug-in into the CHPC module environments. However, it requires more steps to set up and installing environments on the top of base environment is not recommended due to potential dependency conflicts.
| Choose your own Miniforge installation if you want to have complete control your distribution, or prefer environment modules. |
Containerized Python installation with Micromamba
Installing Python packages in a container creates an unmodifiable Python distribution, which prevents accidental update or change of the environment. The container is portable and can be easily shared with the collaborators and the community. However, it adds complexity as you will need to use a few extra commands to build the container and run the Python from the container.
| Choose the container if you want to a fixed Python environment, or want to share it with the community. |