Adding Test Data#

Note

This page is a placeholder for information about test data. It is not yet complete and will be updated in due course.

These instructions apply to all Met Office managed systems and related external platforms (such as JULES data on JASMIN), though other sites may manage their test data differently.

Important

If you have questions about the process or concerns about the provenance of the data you want to include, please engage with the IAO as early as possible to prevent delays to your change later on.

For UM related datasets, please Email the MIAO team to discuss the best way to share the data.

Prerequisites#

Before adding test data, you should have a good understanding of the change you are making and the tests you will be adding. You should also have a good understanding of the codebase and the testing framework you will be using.

Licensing and Attribution#

All files require a licence and a record of where they have come from, both for legal and auditing purposes. In your request please describe where and how the data was generated, and the terms and conditions of its licence.

Before any files can be deployed, they must be approved by an IAO and this cannot be done without information about the licencing terms. It is your responsibility as a developer to obtain this approval.

Where the data is in netCDF format, it should use the global attributes described in the section below; data files in other formats should follow the existing UM ANCILDIR conventions. Attribution must reflect the original data source and be provided by the data creators before deployment, share, or distribution.

Metadata#

All file requirements should be recorded in or alongside the files being deployed.

If a source file has a licence that imposes requirements on derived works, then any ancillary file (or an intermediate file used to generate an ancillary) counts as a derived work for the purposes of recording metadata.

In cases where a file has been generated from multiple sources, the licences must be compatible with each other and it should be made clear where each licence/attribution/acknowledgement has come from.

NetCDF Files#

NetCDF files should have the relevant metadata included in the file itself. The metadata should include the following information:

  • The licence should be in a license global attribute as per ESIP Attribute Convention for Data Discovery.

  • If there is a paper attribution requirement, the relevant paper(s) should be cited in the references global attribute as per CF conventions.

  • If there is an organisation attribution requirement, it should be in the institution global attribute (again, as per CF).

  • If there is any other attribution requirement (e.g. for an individual), it should be in the acknowledgement global attribute (again, as per ACCD).

  • If there are restrictions on usage (e.g. “research only”), these should be in a restrictions global attribute.

Other Files#

  • Licence should be in an accompanying plain text file with the same name as the data file, but with a .license suffix.

  • Attribution should be in an accompanying plain text file with the same name as the data file, but with a .attribution suffix.

  • Restrictions on usage (e.g. “research only”) should be in an accompanying plain text file with the same name as the data file, but with a .restrictions suffix.