Installation

On Azure (Met Office)On Jasmin

• Login to the cdds account.

  xsudo -u cdds bash -l
  

• Activate the base conda environment

  conda activate
  

• Obtain the conda environment file

  wget https://raw.githubusercontent.com/MetOffice/CDDS/refs/tags/<tagname>/environment.yml
  

• Update locations pointed to within the environment file:

  sed -i "s/<location>/X.Y.Z/" environment.yml
  

• Create environment

  conda env create -f environment.yml -p $HOME/conda_environments/cdds-X.Y.Z
  

  where X.Y.Z is the new version number of CDDS

Note

This has been updated following the roll out of Conda to MO systems. If the -p option is omitted then the installation will end up under $HOME/.conda and will not be visible to other users.

• Activate environment and set CDDS_ENV_COMMAND variable:

  conda activate cdds-X.Y.Z
  conda env config vars set CDDS_ENV_COMMAND="conda activate $HOME/conda_environments/cdds-X.Y.Z"
  

  where X.Y.Z is the new version number of CDDS ```
• Set platform in the CDDS_PLATFORM variable:

  conda activate cdds-X.Y.Z
  conda env config vars set CDDS_PLATFORM=AZURE
  

• Confirm environment variables:

  echo $CYLC_VERSION
  echo $LC_ALL
  echo $TZ
  echo $CDDS_PLATFORM
  echo $CDDS_ETC
  echo $CDDS_ENV_COMMAND
  

  Variable	Value
  $CYLC_VERSION	8
  $LC_ALL	en_GB.UTF-8
  $TZ	UTC
  $CDDS_PLATFORM	AZURE
  $CDDS_ETC	$HOME/etc
  $CDDS_ENV_COMMAND	$conda activate $HOME/conda_environments/cdds-3.2.0

• Login to one of the JASMIN sci-servers as the cdds user.
• Activate the base miniforge conda environment

  source $HOME/software/miniforge3/bin/activate
  

• Obtain the conda environment file for the release.

  wget https://raw.githubusercontent.com/MetOffice/CDDS/refs/tags/<tagname>/environment.yml
  

• Update locations pointed to within the environment file (omitting the leading v)

  sed -i "s/<location>/X.Y.Z/" environment.yml
  

• Uncomment the cdds and mip_convert python pip install lines.

    #- git+ssh://git@github.com-deploy/MetOffice/CDDS.git@v<location>#egg=cdds&subdirectory=cdds
    #- git+ssh://git@github.com-deploy/MetOffice/CDDS.git@v<location>#egg=mip_convert&subdirectory=mip_convert
  

• Create environment, where X.Y.Z is the new version number of CDDS

  conda env create -f environment.yml -p $HOME/conda_environments/cdds-X.Y.Z
  

• Set the CDDS variables making sure to replace X.Y.Z with the appropriate version number.

  conda env config vars set CDDS_PLATFORM=JASMIN
  conda env config vars set CDDS_ETC=$HOME/etc
  conda env config vars set CDDS_ENV_COMMAND="$HOME/software/miniforge3/bin/activate $HOME/conda_environments/cdds-X.Y.Z"
  

• Deactivate the environment

  conda deactivate
  

• Confirm that the environment variables were set correctly.

  $HOME/software/miniforge3/bin/activate $HOME/conda_environments/cdds-X.Y.Z
  

  echo $CYLC_VERSION
  echo $LC_ALL
  echo $TZ
  echo $CDDS_PLATFORM
  echo $CDDS_ETC
  echo $CDDS_ENV_COMMAND
  

  Variable	Value
  $CYLC_VERSION	8
  $LC_ALL	en_GB.UTF-8
  $TZ	UTC
  $CDDS_PLATFORM	JASMIN
  $CDDS_ETC	$HOME/etc
  $CDDS_ENV_COMMAND	$HOME/software/miniforge3/bin/activate $HOME/conda_environments/cdds-X.Y.Z

Ensure all the tests pass in the 'real live environment'

Warning

Tests can only be run on Azure (Met Office).

• The tests must be executed as the cdds user
• Set the following environment variable, making sure to replace X.Y.Z with the relevant version.

  export SRCDIR=$HOME/conda_environments/cdds-X.Y.Z/lib/python3.10/site-packages
  

• Run the following tests.

  echo "# Executing tests for cdds:"
  pytest -s $SRCDIR/cdds --doctest-modules -m 'not slow and not integration and not rabbitMQ and not data_request'
  pytest -s $SRCDIR/cdds -m slow
  pytest -s $SRCDIR/cdds -m integration
  pytest -s $SRCDIR/cdds -m data_request
  echo "# Executing tests for mip_convert:"
  pytest -s $SRCDIR/mip_convert --doctest-modules -m 'not slow and not mappings and not superslow'
  pytest -s $SRCDIR/mip_convert -m mappings
  pytest -s $SRCDIR/mip_convert -m slow
  

Info

Slow unit tests for transfer and cdds_configure will display error messages to standard output. This is intentional, and does not indicate the tests fail (see transfer.tests.test_command_line.TestMainStore.test_transfer_functional_failing_moo() for details).

Troubleshooting

Info

If the wheel installation fails then you can end up with #!python rather than the full paths – this is known to be caused by not having _DEV updated in the packages, possibly due to tagging without pulling the release branch from the repository first