Mandatory Anaconda to Miniforge Transition

On March 25, 2025, the Anaconda software distribution environment was removed from the Computational Science Environment (CSE) on HPCMP systems and must no longer be used.

Anaconda was replaced with Miniforge, an open-source edition of Conda that provides access to freely available collections of recipes, build infrastructure, and distributions for the conda package manager via the conda-forge channel. Miniforge is a community led and supported environment, unlike Anaconda, which requires paid licensing for use on our HPC resources. For the vast majority of applications, Miniforge is essentially a drop-in replacement for Anaconda.

The program is working with the HPC vendors to evaluate other Anaconda installations that may be available outside of the purview of CSE. Users are advised that due to recent changes in the Anaconda Terms of Service, unlicensed Anaconda installations (self-installed or otherwise) must not be used on HPCMP resources.

If you still need to migrate an Anaconda environment but missed the transition deadline, you may still be able to export your environments by using the new Miniforge module instead.

The following guide explains how to convert your Anaconda environments to roughly equivalent Miniforge environments which should function exactly the same as existing Anaconda environments for most users. Note that the instructions for steps 1 and 2 below should work but may not as there is no guarantee that Miniforge can perfectly export an environment created with Anaconda. The remaining steps will continue to work as they do not rely on Anaconda.

Available Training

Migrating Anaconda Environments to Miniforge - https://training.hpc.mil/course/view.php?id=751

Conda and Miniforge Basics - https://training.hpc.mil/course/view.php?id=752

What's Wrong with Miniconda?

Miniconda is essentially Anaconda without preinstalled packages. It uses the licensed Anaconda package channels just like Anaconda, and those channels cannot be removed. However, miniForge is 100% free and the version being deployed by CSE contains many of the most common packages used by our HPCMP end user.

How to Migrate Your Conda Environments

Here is an overview of the steps needed to migrate an existing environment to a new Miniforge environment. A walkthrough of these steps is detailed below. You will need to repeat these steps for each conda environment that you have.

1. Activate your current environment to figure out what you already have.
2. Export a list of packages that need to be installed in the new environment.
3. Remove the licensed package channel from your .condarc.
4. Create a new Miniforge environment with your packages.
5. Troubleshoot errors (if any).

Step 1: Activate Your Current Environment to Discover What You Already Have

First, you'll need to know what kind of conda environments you currently have. You will need to "activate" (load) each environment in order to export their lists of installed packages..

Any environments built with CSE Anaconda or a user installation of Anaconda or Miniconda must be rebuilt in Miniforge to continue using them. If your environment was installed with a local instance of Miniforge then your environment is probably fine, but you should check anyway.

If any of your environments were built with Anaconda or Miniconda, then continue following this guide to migrate those environments to the new CSE instance of Miniforge to comply with the Anaconda license.

If your environment was built with CSE Anaconda, the CSE Anaconda module is no longer available. However, you can still attempt to activate an Anoconda environment with Miniforge purely for the purpose of exporting package lists. To load the new Miniforge module, run:

module load cseinit-noloads
module load cse/miniforge

If your environment was built with Miniconda or your own local install of Anaconda, run:

For bash, sh, zsh:

CONDA_ROOT=/path/to/conda
source "${CONDA_ROOT}/etc/profile.d/conda.sh"

For csh, tcsh:

set CONDA_ROOT /path/to/conda
source "${CONDA_ROOT}/etc/profile.d/conda.csh"

where /path/to/conda is the path to where your local Miniconda/Anaconda is installed

Verify the conda command is loaded with

Which conda

Next, run conda info --envs to see your conda environments, as follows:

% conda info --envs
# conda environments:
#
base                  *  /p/app/CSE/CSE.20231125/Release/anaconda3-2022.05
utils                    /p/home/alice/.conda/envs/utils

Note the environment names in the first column and environment paths in the second column. You will need these later.

To activate an environment from this list, copy the full environment path and paste it into the following command:

conda activate /path/to/environment

For example, to activate the "utils" environment from the output above, you would run:

conda activate /p/home/alice/.conda/envs/utils

Note: If you encounter a "CommandNotFoundError", do the following:

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run $ conda init <SHELL_NAME> Currently supported shells are: - bash - fish - tcsh - xonsh - zsh - powershell

While the error text suggests running conda init, this often does not resolve the problem. Instead, you simply need to load the conda start script manually (like how you would load a personal install of Miniconda) by copying and running the following:

For bash, sh, zsh:

source `which conda | sed 's|/bin/conda$|/etc/profile.d/conda.sh|'`

For csh, tcsh:

source `which conda | sed 's|/bin/conda$|/etc/profile.d/conda.csh|'`

Note the backtick characters (`) before and after the command. Then try running your conda activate command again. If this fails, double check the backtick and apostrophe characters.

Step 2: Export a List of Packages to be Installed in the New Environment

Packages in your conda environment may have been installed either with conda install or pip install. "pip" is a popular Python package manager that can be used both separately from and within conda environments to install packages from PyPI (The Python Package Index) and GitHub repos. Pip packages are sometimes also referred to as "wheels". It is not uncommon to add packages with pip install inside a conda environment alongside conda packages. You will need to export two package lists, one for conda install and one for pip install, so that you can recreate your environment.

Note that all Anaconda and Miniconda environments have licensed packages in them, even if all additional packages were installed with pip.

How to find out if your environment uses licensed Anaconda packages

For an active conda environment, to list all the packages and where they were installed from, run:

conda list

then look at the "Channel" column.

• For packages installed from the licensed Anaconda channel, the channel column will be blank. If you see any packages with a blank channel in your environment, you must rebuild your environment with Miniforge to continue using it.
• Packages installed from "conda-forge" will show "conda-forge"
• Packages installed with pip will show "pypi"

This example conda list output shows packages installed from all three locations:

# Name                    Version                   Build  Channel
scipy                     1.15.1          py311h08b1b3b_0
numpy                     2.2.1           py312h7e784f5_0    conda-forge
scikit-learn              1.6.1                    pypi_0    pypi

Note that the scipy package was installed from the licensed Anaconda channel.

To print only the lines where the channel is blank, run:

conda list | grep -v "^#" | awk '$4 == "" {print}'

How to export package lists from your environment

To recreate your environment with Miniforge, you need a list of all the conda packages and pip packages in your current environment. To save a manifest of your conda and pip packages, first copy the following bash script into a new file, export_conda_env_packages.sh:

#!/bin/bash

# Function to print error messages and exit
error() {
    echo "ERROR: $1" >&2
    exit 1
}

# Check if conda is available
if ! command -v conda &> /dev/null; then
    error "Conda is not loaded"
fi

# Get environment name (using CONDA_DEFAULT_ENV or fall back to conda info)
env_name="${CONDA_DEFAULT_ENV}"
if [ -z "$env_name" ]; then
    env_name=$(conda info --envs | grep '^*' | awk '{print $1}')
    if [ -z "$env_name" ]; then
        error "This script must be run from an active conda environment"
    fi
fi

# Get environment path
env_path="${CONDA_PREFIX}"
if [ -z "$env_path" ]; then
    # If CONDA_PREFIX is not set but we have a valid env_name,
    # try to get the path from conda info --envs
    if [ -n "$env_name" ]; then
        env_path=$(conda info --envs | grep "^${env_name}\s" | awk '{print $2}')
        if [ -z "$env_path" ]; then
            error_exit "Unable to determine environment path for '${env_name}' from conda info"
        fi
    else
        error_exit "CONDA_PREFIX is not set and no valid environment name available to determine path"
    fi
fi
# Check if environment path exists and is a directory
if [ ! -d "$env_path" ]; then
    error "Environment path '$env_path' does not exist or is not a directory"
fi
# Check if conda-meta directory exists in the environment
if [ ! -d "${env_path}/conda-meta" ]; then
    error "conda-meta directory not found in environment path '${env_path}'"
fi

# Define output files
conda_pkgs_file="${env_name}_conda.txt"
pip_pkgs_file="${env_name}_pip.txt"

# Export packages installed by conda
echo "Exporting conda packages to ${conda_pkgs_file}..."

# loop over lines
pkgs=()
while read -r line; do

    # loop over the comma delimited list
    IFS=, read -r -a items < <(echo "${line}" | \
        sed -E 's/# update specs: \[(.+)\]$/\1/')
    for item in "${items[@]}"; do

        # cleanup
        tmp="$(echo "${item}" | \
            sed -E "s/^[[:space:]]*['\"]|[[:space:]]*['\"]$//g")"

        # split into name and spec
        read -r name spec <<< $(echo "${tmp}" | \
            sed -E "s/^([^[]+)\[version='([^']+)'\]$/\1 \2/")

        # assemble final pkg
        [[ -n "${spec}" ]] && pkg="'${name}${spec}'" || pkg="${name}"
        pkgs+=("${pkg}")

    done
done < <(grep "update specs" "${env_path}/conda-meta/history")

# write to file
> ${conda_pkgs_file}
if [[ "${#pkgs[@]}" -gt 0 ]]; then
    for pkg in "${pkgs[@]}"; do
        echo "${pkg}" >> ${conda_pkgs_file}
    done
else
    error "Failed to export conda packages"
fi

# Export packages installed by pip
echo "Exporting pip packages to ${pip_pkgs_file}..."
if ! conda list | grep pypi | awk '{print $1 "==" $2}' > "${pip_pkgs_file}"; then
    error "Failed to export pip packages"
fi

echo "Successfully exported package lists:"
echo "- Conda packages: ${conda_pkgs_file}"
echo "- Pip packages: ${pip_pkgs_file}"

Then make the new script executable with:

chmod +x ./export_conda_env_packages.sh

First activate the environment you want to export from:

conda activate my_env_name

Then, run the export_conda_env_packages.sh script:

./export_conda_env_packages.sh

Using the "utils" environment as an example, the command sequence would look like the following:

conda activate utils
./export_conda_env_packages.sh

This script will generate 2 files in your current directory: one containing the list of conda packages and the other with a list of pip packages. You will need these two files for each environment you need to recreate.

You should look at the contents of the text files to ensure they contain the packages you expect. If you are unsure of what they should look like or want to understand more, then review the below sections "Listing Conda Packages" and "Listing Pip Packages" for details.

Further Reading: Listing Conda Packages

While it might seem easy to simply copy the entire list of packages (possibly hundreds) from the output of the conda list command, there is a much easier way that lets conda handle the many dependencies automatically. Simply run the following command:

grep "update specs" "${env_path}/conda-meta/history" | sed -e 's/# update specs://g'

where "${env_path}" is the absolute path to the root of your environment. This will return a list of packages specified in the initial conda create command followed by one line for each subsequent conda install command. Here is example output for a sample environment:

['python=3.11']
['scikit-learn' 'numba'] 
['plotly', 'dash'] 
['sympy', 'geopandas', 'pydantic']

In this example, the corresponding commands that installed these packages (and all their dependencies) would have been:

conda create -n my_env_name python=3.11
conda install scikit-learn numba
conda install plotly dash
conda install sympy geopandas pydantic

This indicates the environment was created with Python version 3.11, and then the other lines include one or more packages installed with each command. Notice there are only eight packages listed here (as opposed to hundreds). These are the only packages you would need to install by name in your new Miniforge environment.

Listing Pip Packages

Conda does not remember past pip install commands the same way it does conda packages. To list the pip packages, simply filter the conda list output for lines containing "pypi":

conda list | grep pypi

Step 3: Remove the Licensed Package Channel from Your .condarc File

Important! Before creating any new environments with Miniforge you must check your personal .condarc file to ensure it does not have the licensed Anaconda channel enabled. The channel will appear with the name "defaults" if it is enabled.

Open $HOME/.condarc in your preferred editor. Note the leading period character at the beginning of .condarc which indicates it's a hidden file.

In your .condarc file, if there is an entry "defaults" in the "channels" list, you must remove the entry. The channels list may appear on a single line, as follows:

channels: [conda-forge, defaults]

or on multiple consecutive lines:

channels:
- conda-forge
- defaults

In either case, you must delete the "defaults" entry.

If your .condarc file is empty or does not contain the "defaults" entry at all, then you can proceed to the next step.

Step 4: Create a New Miniforge Environment with Your Packages

Important! You must open a new shell before switching to CSE Miniforge. This is an essential step to prevent any previously loaded modules or commands from interfering with the new conda commands.

Once you have a clean shell, load CSE Miniforge by running:

module load cseinit-noloads
module load cse/miniforge

You can now use the conda command with the same syntax as with the old CSE Anaconda, but under the hood, the conda-forge channel will be used as the default for all new conda packages installed.

To begin recreating an environment, first copy the following bash script into a new file, print_conda_commands.sh:

#!/bin/bash
env_name="${1}"

# Function to print error messages and exit
error() {
    echo "ERROR: $1" >&2
    exit 1
}

# Check if conda is available
if ! command -v conda &> /dev/null; then
    error "Conda is not loaded"
fi

# Ensure env_name is defined
if [[ -z "${env_name}" ]]; then
    error "No environment name specified"
fi

# Define input files
conda_pkgs_file="${env_name}_conda.txt"
pip_pkgs_file="${env_name}_pip.txt"

# Check if input files exist and are readable
if [ ! -f "${conda_pkgs_file}" ]; then
    error "Conda packages file '${conda_pkgs_file}' does not exist"
fi
if [ ! -f "${pip_pkgs_file}" ]; then
    error "Pip packages file '${pip_pkgs_file}' does not exist"
fi

# Read conda packages into array
conda_pkgs=()
while IFS= read -r line; do
    [[ -n "$line" ]] && conda_pkgs+=("$line")
done < "${conda_pkgs_file}" || error "Failed to read conda packages from '${conda_pkgs_file}'"

# Read pip packages into array
pip_pkgs=()
while IFS= read -r line; do
    [[ -n "$line" ]] && pip_pkgs+=("$line")
done < "${pip_pkgs_file}" || error "Failed to read pip packages from '${pip_pkgs_file}'"

# blank line
echo ""

# If the directory already exists, print a command to rename it
envs_dir="$(conda config --show envs_dirs | grep "  - " | head -n 1 | sed 's/.* - //')"
if [[ -d "${envs_dir}/${env_name}" ]]; then
    echo "# Move the old environment to avoid overwriting it"
    echo "mv ${envs_dir}/${env_name} ${envs_dir}/old_${env_name}"
    echo ""
fi

# print conda create command
echo "# Create environment with conda packages"
echo "conda create --name ${env_name} ${conda_pkgs[@]}"
echo ""

# print conda activate command
echo "# Activate environment"
echo "conda activate ${env_name}"
echo ""

# print pip install command
echo "# Install pip packages (this line may be very long)"
echo "pip install ${pip_pkgs[@]}"
echo "" 

Then make the new script executable with:

chmod +x ./print_conda_commands.sh

Finally, from the same directory as your exported conda and pip package list files, run the script by passing the name of the environment you exported package lists from, as follows:

./print_conda_commands.sh env_name

Where env_name is the same environment name used to export your conda and pip package lists.

Running this script will generate sequential commands needed to create your new conda environment.

Note! If an environment already exists with the same name, the first command printed in the output will rename your old environment directory, so the new environment doesn't overwrite it. Later, if you need to temporarily restore your old environment, simply revert the name.

Verify the environment names, paths, and packages look correct, then copy and run these commands as printed to actually create your new environment.

Using an environment named "utils" as an example, you would print the commands to create a new environment with the same name using the following command:

./print_conda_commands.sh utils

The output of for this "utils" example might look like the following

mv /p/home/alice/.conda/envs/utils /p/home/alice/.conda/envs/old_utils
conda create --name utils scipy matplotlib plotly
conda activate utils
pip install multipledispatch==1.0.0 pandas==2.2.3 pytz==2025.1 tzdata==2025.1

Step 5: Troubleshooting Errors (if any)

In general, installing pip packages in your environment should go smoothly. However, several issues may arise which you may need to address:

1. Compiled pip packages which depend on other packages installed in the same command are likely to fail.
2. Packages installed from GitHub repositories may fail to automatically install or fail with "package not found", depending on the name.
3. Packages installed with the --index-url flag may fail to pull necessary dependencies if they are unavailable.

You should first try to simply install all packages with the line provided by the script. If there are issues, try the following:

1. Identify the package(s) with issues (usually referenced in an import command).
2. Manually install the problem package with pip install. If this fails, stop and contact the HPC Help Desk for assistance.
3. Backup your env_name_pip.txt file to env_name_pip.txt.bak for the environment with failed pip installations.
4. Edit the env_name_pip.txt, removing lines for packages that installed correctly so pip doesn't try to reinstall them.
5. Rerun print_conda_commands.sh with the same parameters as before.
6. Copy and run only the pip install command (ignore the first three commands).
7. Return to Step 1 if you encounter another error.

Example troubleshooting session

This is an example troubleshooting session with a common error many users of PyTorch will likely encounter.

The following is a pip install command generated by print_conda_commands.sh for an environment named "my_torch_env":

(Note this is a single command line)

pip install accelerate==1.3.0 annotated-types==0.7.0 bitsandbytes==0.45.2 bm25s==0.2.4 
cmake==3.31.4 dataclasses-json==0.6.7 datasets==3.3.0 deepspeed==0.16.3 einops==0.8.1 
faiss-cpu==1.10.0 filelock==3.13.1 fsspec==2024.6.1 greenlet==3.1.1 hjson==3.1.0 
httpx-sse==0.4.0 huggingface-hub==0.28.1 joblib==1.4.2 jsonpatch==1.33 jsonpointer==3.0.0 
langchain==0.3.7 langchain-community==0.3.5 langchain-core==0.3.15 langchain-text-splitters==0.3.2 
langsmith==0.1.147 lxml==5.3.1 marshmallow==3.26.1 mpi4py==4.0.3 mpmath==1.3.0 msgpack==1.1.0 
multiprocess==0.70.16 networkx==3.3 ninja==1.11.1.3 nvidia-cublas-cu11==11.11.3.6 
nvidia-cuda-cupti-cu11==11.8.87 nvidia-cuda-nvrtc-cu11==11.8.89 nvidia-cuda-runtime-cu11==11.8.89 
nvidia-cudnn-cu11==9.1.0.70 nvidia-cufft-cu11==10.9.0.58 nvidia-curand-cu11==10.3.0.86 
nvidia-cusolver-cu11==11.4.1.48 nvidia-cusparse-cu11==11.7.5.86 nvidia-nccl-cu11==2.21.5 
nvidia-nvtx-cu11==11.8.86 orjson==3.10.15 py-cpuinfo==9.0.0 pyarrow==19.0.0 pydantic==2.10.6 
pydantic-core==2.27.2 pydantic-settings==2.7.1 pymupdf==1.25.3 python-docx==1.1.2 python-dotenv==1.0.1 
regex==2024.11.6 requests-toolbelt==1.0.0 safetensors==0.5.2 scikit-learn==1.6.1 
sentence-transformers==3.4.1 sqlalchemy==2.0.35 sympy==1.13.1 tenacity==9.0.0 threadpoolctl==3.5.0 
tokenizers==0.21.0 torch==2.5.0+cu118 torchaudio==2.5.0+cu118 torchvision==0.20.0+cu118 tqdm==4.67.1 
transformers==4.48.3 triton==3.1.0 typing-inspect==0.9.0 xxhash==3.5.0

Running this command may return the following error:

ERROR: Could not find a version that satisfies the requirement torch==2.5.0+cu118 (from 
versions: 1.13.0, 1.13.1, 2.0.0, 2.0.1, 2.1.0, 2.1.1, 2.1.2, 2.2.0, 2.2.1, 2.2.2, 2.3.0, 
2.3.1, 2.4.0, 2.4.1, 2.5.0, 2.5.1, 2.6.0)
ERROR: No matching distribution found for torch==2.5.0+cu118

This error is caused by a missing parameter. PyTorch requires an --index-url argument specified on their website: https://pytorch.org/get-started/locally/.

To resolve this issue, you must manually install PyTorch using the exact pip install command found on that web page. It will look something like the following:

pip install torch==2.5.0 torchvision torchaudio --index-url
https://download.pytorch.org/whl/cu118

Note that this command also installs torchvision and torchaudio.

Next, backup the existing pip package list file for this environment:

cp my_torch_env_pip.txt my_torch_env_pip.txt.bak

Then edit the original pip package list file (in this example my_torch_env_pip.txt), removing the lines for torch, torchvision and torchaudio since you just manually installed them.

Now, re-run the print_conda_commands.sh to generate a new pip install command that will look similar to the following:

(Note that torch, torchvision, and torchaudio are no longer present)

pip install  accelerate==1.3.0 annotated-types==0.7.0 bitsandbytes==0.45.2 bm25s==0.2.4 
cmake==3.31.4 dataclasses-json==0.6.7 datasets==3.3.0 deepspeed==0.16.3 einops==0.8.1 
faiss-cpu==1.10.0 filelock==3.13.1 fsspec==2024.6.1 greenlet==3.1.1 hjson==3.1.0 
httpx-sse==0.4.0 huggingface-hub==0.28.1 joblib==1.4.2 jsonpatch==1.33 jsonpointer==3.0.0 
langchain==0.3.7 langchain-community==0.3.5 langchain-core==0.3.15 
langchain-text-splitters==0.3.2 langsmith==0.1.147 lxml==5.3.1 marshmallow==3.26.1 
mpi4py==4.0.3 mpmath==1.3.0 msgpack==1.1.0 multiprocess==0.70.16 networkx==3.3 
ninja==1.11.1.3 nvidia-cublas-cu11==11.11.3.6 nvidia-cuda-cupti-cu11==11.8.87 
nvidia-cuda-nvrtc-cu11==11.8.89 nvidia-cuda-runtime-cu11==11.8.89 nvidia-cudnn-cu11==9.1.0.70 
nvidia-cufft-cu11==10.9.0.58 nvidia-curand-cu11==10.3.0.86 nvidia-cusolver-cu11==11.4.1.48 
nvidia-cusparse-cu11==11.7.5.86 nvidia-nccl-cu11==2.21.5 nvidia-nvtx-cu11==11.8.86 
orjson==3.10.15 py-cpuinfo==9.0.0 pyarrow==19.0.0 pydantic==2.10.6 pydantic-core==2.27.2 
pydantic-settings==2.7.1 pymupdf==1.25.3 python-docx==1.1.2 python-dotenv==1.0.1 
regex==2024.11.6 requests-toolbelt==1.0.0 safetensors==0.5.2 scikit-learn==1.6.1 
sentence-transformers==3.4.1 sqlalchemy==2.0.35 sympy==1.13.1 tenacity==9.0.0 
threadpoolctl==3.5.0 tokenizers==0.21.0 tqdm==4.67.1 transformers==4.48.3 triton==3.1.0 
typing-inspect==0.9.0 xxhash==3.5.0

Copy and run the new pip install command, and the error should no longer occur.

Repeat this process for each new error.

If you've gone through this process and encounter an error you can't resolve, contact the HPC Help Desk for assistance.