Software: Using Miniconda to install software

From arccwiki
Jump to: navigation, search

To use an available Miniconda module

Miniconda2 and Miniconda3 are available as modules on Teton. To see their dependencies, run module spider miniconda. It will list the available versions of Miniconda and which modules are needed to load them. For example, to load Miniconda3, run:

module load swset/2018.05
module load miniconda3

Install packages into a Miniconda environment in your home directory

The following commands are useful for creating and using environments in your home directory:

conda info --envs-show environments
conda create -n NAME-create environment named NAME
conda create -n NAME python=3.4-create environment named NAME, with specified Python version
conda activate NAME-activate environment NAME
source activate NAME-Alternate activate environment NAME
conda install PKG-install package PKG into current environment
conda install -c CHANNEL PKG-install package PKG from channel CHANNEL into current environment
conda list-show packages in current environment
conda list -n NAME-show packages in environment NAME
conda deactivate-deactivate current environment
source deactivate-alternate deactivate current environment

Installing Python packages

After creating and activating an environment, packages available through Anaconda can be installed into that environment using the conda install command. For example, NumPy can be installed by:

conda install numpy

There are also packages available through different channels. For example, PyNIO and PyNGL from NCAR are available through the conda-forge channel, and can be installed by:

conda install -c conda-forge pynio
conda install -c conda-forge pyngl

Searching for packages in channels

To search if a package is available in the current channels, e.g. the package demultiplex:

conda search demultiplex

To search if a package is available from a particular channel (the conda-forge channel in this example):

conda search -c conda-forge demultiplex

If a package isn't available from a conda channel, but is available as a PyPi package (usually installed with pip), it can still be installed into a conda environment.

Examples

Example: Installing a conda package

Using the above commands, here is an example of creating an environment and installing the NumPy, PyNIO and PyNGL packages into it:

  1. Load Miniconda:
    module load miniconda3
    
  2. Create and activate environment:
    conda create -n hdf python=2.7
    conda activate hdf 
    
  3. Install the packages:
    conda install numpy
    conda install -c conda-forge pynio
    conda install -c conda-forge pyngl
    
  4. Deactivate environment:
    conda deactivate
    

It is also possible to interactively search/browse conda packages at https://anaconda.org/anaconda/repo.

Example: Installing a PyPi package

Using the above commands, here is an example of creating an environment and installing the PyPi demultiplex package into it:

  1. Load Miniconda:
    module load miniconda3
    
  2. Create and activate environment:
    conda create -n demux
    conda activate demux
    
  3. Install pip into this environment:
    conda install pip
    
  4. Update pip (in the conda environment):
    pip install --upgrade pip
    
  5. Use pip (from the conda environment) to install the demultiplex package:
    pip install demultiplex
    
  6. Deactivate environment:
    conda deactivate
    

Using installed Python packages

There are two ways to use Python packages installed in conda environments. For example, to use the demultiplex package installed above:

  • Activate the demux environment, then a Python script with 'import demultiplex' will find the package, and when done running it, deactivate the environment.
  • At the top of the Python script, use Python from the demux environment directly — this method does not require activating and deactivating the environment — the top line of the script would be:
#!/home/uname/.conda/envs/demux/bin/python

where uname is replaced with your username, and demux is replaced by the name of your environment.

Example: Installing a C/C++ package

Besides Python packages, conda can be used to install packages for various other languages such as R, C and C++. For example, the GNU Scientific Library (GSL) is a numerical library of various mathematical functions that can be used in C and C++ programs. It is available as a conda package and can be installed by:

  1. Load Miniconda:
    module load miniconda3
    
  2. Create and activate environment:
    conda create -n gsl
    conda activate gsl
    
  3. Install the packages:
    conda install gsl
    
  4. Deactivate environment:
    conda deactivate
    

After installing gsl into a conda environment as above, it is possible to use it in a C or C++ program by adding the necessary include and/or library paths to the paths that will be searched, e.g.:

  • add to INCLUDE path: ~/.conda/envs/gsl/include/gsl
  • add to LIBRARY path: ~/.conda/envs/gsl/lib

Note that the include and library paths will be different for different packages, and can be found by (from home directory) 'cd .conda/envs/NAME' where NAME is the name of the environment that the package was installed into, and then looking to see where the different types of files are.

Troubleshooting and other notes

  • Python packages: All Python packages that will be needed in programs using a package will need to be installed into the same environment, as the instance of Python from this environment will be used to run the programs. To install additional packages, activate the environment, install the packages (with conda if possible, otherwise with pip), then deactivate the environment.
  • Installing PyPi packages with pip: If an attempt to install a package gives an error about trying to write to a location to which there isn't permission, this is usually a result of a PATH issue.
    • It is important to use the instance of pip that was installed in the conda environment, and not one from another location that may be available in the path.
    • The current path can be seen by running 'echo $PATH'.
    • Issues with incorrect directories being included, or occurring in the wrong order, in a path can usually be resolved by logging out and back in, loading Miniconda, then activating the necessary environment.

Further information