Skip to content

Installation

  • Login to the cdds account. 
    xsudo -u cdds bash -l
  • Navigate to cdds home. 
    cd ~cdds
  • Activate the base conda environment. 
    conda activate
  • Create a folder to work from and enter it. 
    mkdir temporary_installation_folder &&  cd temporary_installation_folder
  • Obtain the conda environment file (replace "" with the version you're installing e.g. "v3.3.1"). 
    wget https://raw.githubusercontent.com/MetOffice/CDDS/refs/tags/<tagname>/environment.yml
  • Update locations pointed to within the environment file (replace "X.Y.Z" with the version you're installing e.g. "3.3.1"): 
    sed -i "s/<location>/X.Y.Z/" environment.yml

  • Create environment (replace "X.Y.Z" with the version you're installing e.g. "3.3.1"). 

    conda env create -f environment.yml -p $HOME/conda_environments/cdds-X.Y.Z

  • Activate environment and set CDDS_ENV_COMMAND variable (replace "X.Y.Z" with the version you're installing e.g. "3.3.1"): 

    conda activate $HOME/conda_environments/cdds-X.Y.Z
conda env config vars set CDDS_ENV_COMMAND="conda activate $HOME/conda_environments/cdds-X.Y.Z"

  • Set the CDDS_PLATFORM and CDDS_ETC variables. 

    conda env config vars set CDDS_PLATFORM=AZURE
conda env config vars set CDDS_ETC=$HOME/etc

  • Confirm environment variables: 

    echo $CYLC_VERSION
echo $LC_ALL
echo $TZ
echo $CDDS_PLATFORM
echo $CDDS_ETC
echo $CDDS_ENV_COMMAND

    Variable Value
    $CYLC_VERSION 8
    $LC_ALL en_GB.UTF-8
    $TZ UTC
    $CDDS_PLATFORM AZURE
    $CDDS_ETC $HOME/etc
    $CDDS_ENV_COMMAND $conda activate $HOME/conda_environments/cdds-X.Y.Z (X.Y.Z should be set to the environment you've created e.g.3.3.1)

  • Login to one of the JASMIN sci-servers as the cdds user.

  • Activate the base miniforge conda environment 

    source $HOME/software/miniforge3/bin/activate

  • Obtain the conda environment file for the release. 
    wget https://raw.githubusercontent.com/MetOffice/CDDS/refs/tags/<tagname>/environment.yml

  • Update locations pointed to within the environment file (omitting the leading v) 

    sed -i "s/<location>/X.Y.Z/" environment.yml

  • Create environment, where X.Y.Z is the new version number of CDDS (replace "X.Y.Z" with the version you're installing e.g. "3.3.1"). 

    conda env create -f environment.yml -p $HOME/conda_environments/cdds-X.Y.Z

  • Set the CDDS_PLATFORM and CDDS_ETC variables 

    conda activate cdds-X.Y.Z
conda env config vars set CDDS_PLATFORM=JASMIN
conda env config vars set CDDS_ETC=$HOME/etc

  • Set the CDDS_ENV_COMMAND variable making sure to substitute X.Y.Z with the appropriate version number. 

    conda env config vars set CDDS_ENV_COMMAND="$HOME/software/miniforge3/bin/activate $HOME/conda_environments/cdds-X.Y.Z"

  • Deactivate the environment 

    conda deactivate

  • Confirm that the environment variables were set correctly. 

    $HOME/software/miniforge3/bin/activate $HOME/conda_environments/cdds-X.Y.Z

    echo $CYLC_VERSION
echo $LC_ALL
echo $TZ
echo $CDDS_PLATFORM
echo $CDDS_ETC
echo $CDDS_ENV_COMMAND

    Variable Value
    $CYLC_VERSION 8
    $LC_ALL en_GB.UTF-8
    $TZ UTC
    $CDDS_PLATFORM JASMIN
    $CDDS_ETC $HOME/etc
    $CDDS_ENV_COMMAND $HOME/software/miniforge3/bin/activate $HOME/conda_environments/cdds-X.Y.Z

Manually add nccmp to the new environment

Info

The nccmp package currently causes the environment to fail solving due to a dependency conflict involving libnetcdf. Until this is fixed, we have to manually add the package to new environments after they're created with the following steps.

  • cd into the bin directory of the environment you've created within $HOME/conda_environments/

  • Run this command to manually add nccmp to the environment: 

    ln -s $HOME/conda_environments/cdds-3.3.3/bin/nccmp nccmp

Warning

This should be done by creation of a symlink (as demonstrated above). If you were to copy the nccmp folder instead of symlinking, there would be test failures.

Ensure all the tests pass in the 'real live environment'

Warning

Tests can only be run on Azure (Met Office).

  • The tests must be executed as the cdds user
  • Set the following environment variable, making sure to replace X.Y.Z with the relevant version. 
    export SRCDIR=$HOME/conda_environments/cdds-X.Y.Z/lib/python3.12/site-packages
  • Run the following tests. 
    echo "# Executing tests for cdds:"
pytest -s $SRCDIR/cdds --doctest-modules -m 'not slow and not integration and not rabbitMQ and not data_request'
pytest -s $SRCDIR/cdds -m slow
pytest -s $SRCDIR/cdds -m integration
pytest -s $SRCDIR/cdds -m data_request
echo "# Executing tests for mip_convert:"
pytest -s $SRCDIR/mip_convert --doctest-modules -m 'not slow and not mappings and not superslow'
pytest -s $SRCDIR/mip_convert -m mappings
pytest -s $SRCDIR/mip_convert -m slow

Info

Slow unit tests for transfer and cdds_configure will display error messages to standard output. This is intentional, and does not indicate the tests fail (see transfer.tests.test_command_line.TestMainStore.test_transfer_functional_failing_moo() for details).

  • The folder you created at the start to undertake the installation process can now be deleted. 
    rm -r temporary_installation_folder

Troubleshooting

Info

If the wheel installation fails then you can end up with #!python rather than the full paths – this is known to be caused by not having _DEV updated in the packages, possibly due to tagging without pulling the release branch from the repository first