Skip to content

Installation

  • Login to the cdds account.
    xsudo -u cdds bash -l
    
  • Activate the base conda environment
    conda activate
    
  • Obtain the conda environment file
    wget https://raw.githubusercontent.com/MetOffice/CDDS/refs/tags/<tagname>/environment.yml
    
  • Update locations pointed to within the environment file:
    sed -i "s/<location>/X.Y.Z/" environment.yml
    
  • Create environment

    conda env create -f environment.yml -p $HOME/conda_environments/cdds-X.Y.Z
    
    where X.Y.Z is the new version number of CDDS

  • Activate environment and set CDDS_ENV_COMMAND variable:

    conda activate cdds-X.Y.Z
    conda env config vars set CDDS_ENV_COMMAND="conda activate $HOME/conda_environments/cdds-X.Y.Z"
    
    where X.Y.Z is the new version number of CDDS

  • Set the CDDS_PLATFORM and CDDS_ETC variables

    conda env config vars set CDDS_PLATFORM=AZURE
    conda env config vars set CDDS_ETC=$HOME/etc
    

  • Confirm environment variables:

    echo $CYLC_VERSION
    echo $LC_ALL
    echo $TZ
    echo $CDDS_PLATFORM
    echo $CDDS_ETC
    echo $CDDS_ENV_COMMAND
    

    Variable Value
    $CYLC_VERSION 8
    $LC_ALL en_GB.UTF-8
    $TZ UTC
    $CDDS_PLATFORM AZURE
    $CDDS_ETC $HOME/etc
    $CDDS_ENV_COMMAND $conda activate $HOME/conda_environments/cdds-3.2.0
  • Login to one of the JASMIN sci-servers as the cdds user.

  • Activate the base miniforge conda environment

    source $HOME/software/miniforge3/bin/activate
    

  • Obtain the conda environment file for the release.
    wget https://raw.githubusercontent.com/MetOffice/CDDS/refs/tags/<tagname>/environment.yml
    
  • Update locations pointed to within the environment file (omitting the leading v)

    sed -i "s/<location>/X.Y.Z/" environment.yml
    

  • Create environment, where X.Y.Z is the new version number of CDDS

    conda env create -f environment.yml -p $HOME/conda_environments/cdds-X.Y.Z
    

  • Set the CDDS_PLATFORM and CDDS_ETC variables

    conda activate cdds-X.Y.Z
    conda env config vars set CDDS_PLATFORM=JASMIN
    conda env config vars set CDDS_ETC=$HOME/etc
    

  • Set the CDDS_ENV_COMMAND variable making sure to substitue X.Y.Z with the appropriate version number.

    conda env config vars set CDDS_ENV_COMMAND="$HOME/software/miniforge3/bin/activate $HOME/conda_environments/cdds-X.Y.Z"
    

  • Deactivate the environment

    conda deactivate
    

  • Confirm that the environment variables were set correctly.

    $HOME/software/miniforge3/bin/activate $HOME/conda_environments/cdds-X.Y.Z
    
    echo $CYLC_VERSION
    echo $LC_ALL
    echo $TZ
    echo $CDDS_PLATFORM
    echo $CDDS_ETC
    echo $CDDS_ENV_COMMAND
    

    Variable Value
    $CYLC_VERSION 8
    $LC_ALL en_GB.UTF-8
    $TZ UTC
    $CDDS_PLATFORM JASMIN
    $CDDS_ETC $HOME/etc
    $CDDS_ENV_COMMAND $HOME/software/miniforge3/bin/activate $HOME/conda_environments/cdds-X.Y.Z

Ensure all the tests pass in the 'real live environment'

Warning

Tests can only be run on Azure (Met Office).

  • The tests must be executed as the cdds user
  • Set the following environment variable, making sure to replace X.Y.Z with the relevant version.
    export SRCDIR=$HOME/conda_environments/cdds-X.Y.Z/lib/python3.10/site-packages
    
  • Run the following tests.
    echo "# Executing tests for cdds:"
    pytest -s $SRCDIR/cdds --doctest-modules -m 'not slow and not integration and not rabbitMQ and not data_request'
    pytest -s $SRCDIR/cdds -m slow
    pytest -s $SRCDIR/cdds -m integration
    pytest -s $SRCDIR/cdds -m data_request
    echo "# Executing tests for mip_convert:"
    pytest -s $SRCDIR/mip_convert --doctest-modules -m 'not slow and not mappings and not superslow'
    pytest -s $SRCDIR/mip_convert -m mappings
    pytest -s $SRCDIR/mip_convert -m slow
    

Info

Slow unit tests for transfer and cdds_configure will display error messages to standard output. This is intentional, and does not indicate the tests fail (see transfer.tests.test_command_line.TestMainStore.test_transfer_functional_failing_moo() for details).

Troubleshooting

Info

If the wheel installation fails then you can end up with #!python rather than the full paths – this is known to be caused by not having _DEV updated in the packages, possibly due to tagging without pulling the release branch from the repository first