Loading and saving data with ANTS#
Every ANTS based application starts by loading source data and a target grid,
and ends by saving the resulting field(s) to disk. ANTS is built on top of
Iris, and while you can use iris.load() and iris.save()
directly, the ants.io.load and ants.io.save modules add
capability that ancillary generation specifically needs. You should prefer the
ANTS routines over the base Iris ones unless you have good reason not to.
Why use the ANTS load and save routines?#
ants.io.load uses Iris to read the fileformats Iris supports (and
some ANTS specific ones, see ants.fileformats), then additionally:
derives the global/regional status of the input data and sets the corresponding metadata on the cube,
guesses bounds for latitude and longitude coordinates where a source has none, and
removes any
forecast_reference_timeandforecast_periodcoordinates, which are not usually wanted in ancillary fields.preserves metadata by loading sidecar metdata files alongside the data files.
ants.io.save similarly extends iris.save() with support for
the F03 UM ancillary fileformat and a UKCA specific flavour of NetCDF, neither
of which Iris can write natively, and has the option to write metadata files
alongside the saved files in the case of saving an F03 UM ancillary fileformat.
Loading data#
Use ants.io.load.load() to load one or more cubes from a filepath (or
list of filepaths):
import ants
source_cubes = ants.io.load.load("/path/to/source_file.nc")
This returns a CubeList, even where the file contains a
single field. If you know your source contains exactly one field you may
prefer ants.io.load.load_cube(), which returns a single
Cube and raises an exception if more than one match is
found.
Loading a target grid#
Most ANTS applications need a target grid to process or regrid data onto.
ants.io.load.load_grid() loads a grid definition, without a data
payload, from one or more files, merging horizontal and vertical components
where necessary:
target_grid = ants.io.load.load_grid("/path/to/target_grid_namelist")
This supports both CAP compliant Fortran namelist grid definition files (use this where you do not need a land sea mask as part of your processing) and any fileformat that Iris can read (use this where you need the land sea mask of the target field).
Where a target’s land sea mask is needed directly, for example to make output
consistent with land or sea only data, use
ants.io.load.load_landsea_mask() instead. This function accepts either
a land binary mask field or a land fraction field:
# land_threshold is only used when the source is a land fraction field;
# fractions greater than this value are treated as land.
target_lsm = ants.io.load.load_landsea_mask(
"/path/to/target_lsm.nc", land_threshold=0.5
)
Loading metadata#
Metadata in this tutorial refers to information about the data in a file, that does not directly affect that data. For example, licensing information or attributions.
Some ancillary file formats cannot include metadata. Metadata can be very important to keep alongside the data as often data will have some form of requirement or restriction on it. Because of this, ANTS can load in external metadata files, provided they match the naming convention of filename.extension.accepted-metadata. The current accepted metadata attributes are “license”, “attribution”, “restrictions”, “institution”, “acknowledgement”, and “references”. These metadata files must be in the same directory as the data files and share the same name (including the extension).
Metadata can be loaded alongside all files, including NetCDF files, however ANTS will not allow you to load a file with a metadata attribute and a sidecar file with the same attribute. This is to prevent unintentional loss or overwriting of metadata information.
This functionality can be turned off through the use of the argument
--ignore-metadata-read on the command line or when calling the ANTS load:
cube = ants.io.load.load("/path/to/source_file", ignore_metadata_read=True)
Note
Any attributes or files referencing licensing should use the ‘license’ spelling for consistency.
Note
When using wildcards for loading with sidecar files, add the file extension to the end to prevent issues with ANTS attempting to load the sidecar files. E.g. data*.pp rather than data*.
Saving data#
Saving to NetCDF is done via ants.io.save.netcdf():
import ants.io.save as save
save.netcdf(source_cubes, "/path/to/output.nc")
To save to an F03 UM ancillary file, use ants.io.save.ancil() instead.
Every cube saved this way must have a STASH code attribute set, since this
is used to identify the field in the ancillary file:
import iris.fileformats.pp
for cube in source_cubes:
cube.attributes["STASH"] = iris.fileformats.pp.STASH.from_msi("m01s00i030")
save.ancil(source_cubes, "/path/to/output_ancil")
If your data is destined for UKCA, use ants.io.save.ukca_netcdf()
instead, which applies UKCA specific conventions (compression, bounds,
data type coercion and attribute renaming) on top of a standard NetCDF save:
save.ukca_netcdf(source_cubes, "/path/to/output_ukca.nc")
Saving metadata#
Metadata will only be saved to a sidecar file, if the ancil saver is used. This is because NetCDF files will include the attributes within the file.
This functionality can be turned off through the use of the argument
--ignore-metadata-write on the command line or when calling the ANTS save:
import ants.io.save as save
save.ancil(source_cubes, "/path/to/output_ancil", ignore_metadata_write=True)
The sidecar files produced by ANTS follow the same naming coventions at those loaded in filename.attribute.accepted-metadata. Only accepted metadata will be written out.
Key Points#
Prefer
ants.io.loadandants.io.saveover calling Iris directly - they add corrections and capability that ancillary generation depends on.
ants.io.save.ancil()requires every cube to have aSTASHattribute; NetCDF and UKCA NetCDF saves do not.Use
ants.io.load.load_grid()for a namelist based target grid, andants.io.load.load_landsea_mask()where the target’s land sea mask is needed directly.
See Also#
Regridding an ancillary field to a target grid for using a loaded target grid to regrid data.
ancil_2anc.py for a command line tool that wraps a load/save round trip to convert between fileformats.
Appendix A: F03 Ancillary file time metadata for the time metadata expectations placed on data being saved as an ancillary.