Installation

On Azure (Met Office)

• Login to the cdds account.

  xsudo -u cdds bash -l
  

• Activate the base conda environment

  conda activate
  

• Obtain the conda environment file

  wget https://raw.githubusercontent.com/MetOffice/CDDS/refs/tags/<tagname>/environment.yml
  

• Update locations pointed to within the environment file:

  sed -i "s/<location>/X.Y.Z/" environment.yml
  

• Create environment

  conda env create -f environment.yml -p $HOME/conda_environments/cdds-X.Y.Z
  

  where X.Y.Z is the new version number of CDDS

<<<<<<< HEAD !!! note This has been updated following the roll out of Conda to MO systems. If the -p option is omitted then the installation will end up under $HOME/.conda and will not be visible to other users.

!!! info 
    If the `wheel` installation fails then you can end up with `#!python` rather than the full paths – this is known to be caused by not having 
    `_DEV` updated in the packages, possibly due to tagging without pulling the release branch from the repository first

=======

25ccf412 (Documentation improvements following v3.2.1 release) - [x] Activate environment and set CDDS_ENV_COMMAND variable:

conda activate cdds-X.Y.Z
conda env config vars set CDDS_ENV_COMMAND="conda activate $HOME/conda_environments/cdds-X.Y.Z"

where X.Y.Z is the new version number of CDDS

- [x] Set the `CDDS_PLATFORM` and `CDDS_ETC` variables
      ```bash
      conda env config vars set CDDS_PLATFORM=AZURE
      conda env config vars set CDDS_ETC=$HOME/etc
      ```

- [x] Confirm environment variables:
      ```bash
      echo $CYLC_VERSION
      echo $LC_ALL
      echo $TZ
      echo $CDDS_PLATFORM
      echo $CDDS_ETC
      echo $CDDS_ENV_COMMAND
      ```

      | Variable    | Value                                |
      | ----------- | ------------------------------------ |
      | `$CYLC_VERSION`     | `8` |
      | `$LC_ALL`           | `en_GB.UTF-8`  |
      | `$TZ`               | `UTC` |
      | `$CDDS_PLATFORM`    | `AZURE` |
      | `$CDDS_ETC`         | `$HOME/etc` |
      | `$CDDS_ENV_COMMAND` | `$conda activate $HOME/conda_environments/cdds-3.2.0` |

On Jasmin

<<<<<<< HEAD - [x] Login to one of the JASMIN science servers as the cdds user ======= - [x] Login to one of the JASMIN sci-servers as the cdds user.

25ccf412 (Documentation improvements following v3.2.1 release) - [x] Activate the base miniforge conda environment

source $HOME/software/miniforge3/bin/activate

- [x] Obtain the conda environment file for the release.

wget https://raw.githubusercontent.com/MetOffice/CDDS/refs/tags/<tagname>/environment.yml

- [x] Update locations pointed to within the environment file (omitting the leading v)

sed -i "s/<location>/X.Y.Z/" environment.yml

<<<<<<< HEAD =======

25ccf412 (Documentation improvements following v3.2.1 release) - [x] Create environment, where X.Y.Z is the new version number of CDDS

conda env create -f environment.yml -p $HOME/conda_environments/cdds-X.Y.Z

<<<<<<< HEAD - [x] Uncomment the cdds and mip_convert python pip install lines.

  #- git+ssh://git@github.com-deploy/MetOffice/CDDS.git@v<location>#egg=cdds&subdirectory=cdds
  #- git+ssh://git@github.com-deploy/MetOffice/CDDS.git@v<location>#egg=mip_convert&subdirectory=mip_convert

- [x] Create environment, where X.Y.Z is the new version number of CDDS

conda env create -f environment.yml -p $HOME/conda_environments/cdds-X.Y.Z

- [x] Set the CDDS variables making sure to replace X.Y.Z with the appropriate version number. =======

- [x] Set the `CDDS_PLATFORM` and `CDDS_ETC` variables

25ccf412 (Documentation improvements following v3.2.1 release) bash conda activate cdds-X.Y.Z conda env config vars set CDDS_PLATFORM=JASMIN conda env config vars set CDDS_ETC=$HOME/etc

- [x] Set the `CDDS_ENV_COMMAND` variable making sure to substitue `X.Y.Z` with the appropriate version number.
      ```bash
      conda env config vars set CDDS_ENV_COMMAND="$HOME/software/miniforge3/bin/activate $HOME/conda_environments/cdds-X.Y.Z"
      ```

- [x] Deactivate the environment
      ```bash
      conda deactivate
      ```
- [x] Confirm that the environment variables were set correctly.
      ```bash
      $HOME/software/miniforge3/bin/activate $HOME/conda_environments/cdds-X.Y.Z
      ```
      ```bash
      echo $CYLC_VERSION
      echo $LC_ALL
      echo $TZ
      echo $CDDS_PLATFORM
      echo $CDDS_ETC
      echo $CDDS_ENV_COMMAND
      ```

      | Variable    | Value                                |
      | ----------- | ------------------------------------ |
      | `$CYLC_VERSION`     | `8` |
      | `$LC_ALL`           | `en_GB.UTF-8`  |
      | `$TZ`               | `UTC` |
      | `$CDDS_PLATFORM`    | `JASMIN` |
      | `$CDDS_ETC`         | `$HOME/etc` |
      | `$CDDS_ENV_COMMAND` | `$HOME/software/miniforge3/bin/activate $HOME/conda_environments/cdds-X.Y.Z` |

Ensure all the tests pass in the 'real live environment'

Warning

Tests can only be run on Azure (Met Office).

• The tests must be executed as the cdds user
• Set the following environment variable, making sure to replace X.Y.Z with the relevant version.

  export SRCDIR=$HOME/conda_environments/cdds-X.Y.Z/lib/python3.10/site-packages
  

• Run the following tests.

  echo "# Executing tests for cdds:"
  pytest -s $SRCDIR/cdds --doctest-modules -m 'not slow and not integration and not rabbitMQ and not data_request'
  pytest -s $SRCDIR/cdds -m slow
  pytest -s $SRCDIR/cdds -m integration
  pytest -s $SRCDIR/cdds -m data_request
  echo "# Executing tests for mip_convert:"
  pytest -s $SRCDIR/mip_convert --doctest-modules -m 'not slow and not mappings and not superslow'
  pytest -s $SRCDIR/mip_convert -m mappings
  pytest -s $SRCDIR/mip_convert -m slow
  

Info

Slow unit tests for transfer and cdds_configure will display error messages to standard output. This is intentional, and does not indicate the tests fail (see transfer.tests.test_command_line.TestMainStore.test_transfer_functional_failing_moo() for details).

Troubleshooting

Info

If the wheel installation fails then you can end up with #!python rather than the full paths – this is known to be caused by not having _DEV updated in the packages, possibly due to tagging without pulling the release branch from the repository first