ants package#

class ants.AntsArgParser(target_lsm=False, target_grid=False, time_constraints=False)[source]#

Bases: ArgumentParser

Standardised ancillary commandline interface.

__init__(target_lsm=False, target_grid=False, time_constraints=False)[source]#

Parse command-line arguments and options.

The following arguments represent those common to all ancillary applications:

sources <SOURCE1> <SOURCE2> ... <SOURCEN>     Source data path(s).
--output <OUTPUT>, -o <OUTPUT>                Output filepath
--ants-config <ANTS_CONFIG>                   Configuration path.

Additionally, there are standardised optional arguments and these are activated by passing the relevant keyword argument. See ‘Parameters’ below for further detail on these standardised interfaces to ancillary generation applications. See ants.config for further details on ANTS run-time configuration of ANTS.

Parameters:

target_lsm (bool, optional) – Standardised optional argument. When True, the user is required to provide a land binary mask via a --target-lsm keyword argument followed by the path to the land sea mask.
target_grid (bool, optional) – Standardised optional argument. When True, the user is required to provide a target grid file via a --target-grid keyword argument followed by the path to the grid file.
time_constraints (bool, optional) – Standardised optional argument. When True, the user is able to provide two keyword arguments to constrain the search data by time: a “–begin” keyword argument which must be earlier or equal to the “–end” keyword argument. Both should be provided as years in the format YYYY.

Note

This class behaves in the same manner as the argparse.ArgumentParser class. Adding additional commandline arguments can be done in the standard way (see argparse). Two additional types are provided that can be used with the ‘type’ argument of argparse.ArgumentParser.add_argument(): ants.config.filepath_readable() and ants.config.dirpath_writeable().

This choice should be on the basis that the newly proposed interface is necessarily not common to the UI of other ancillary applications. Often this can be the case for pre-processing applications.

parse_args(args=None, namespace=None)[source]#

Return parsed commandline arguments according to the object specification.

Output filepaths are verified for write permissions. Input filepaths are verified for read permissions. Optional time constraints are verified if supplied that both exist and with the earlier date first. This ensures that exceptions are raised as soon as possible.

Note

The ants configuration is populated where an optional –ants-config has been provided during the parse.

class ants.ExtractConstraint(target, fix_period=False, pad_width=1)[source]#

Bases: Constraint

__init__(target, fix_period=False, pad_width=1)[source]#

Area overlap extraction constraint.

Parameters:

target (Cube) – Target cube used to define the area overlap with the source.
fix_period (bool, optional) – Ensure that the range of the returned cube in not altered. This may result in discontiguous bounds if the extract returns two disconnected regions of the source.
pad_width (int, optional) – Pad the slices by the specified number of cells. See utils.cube.get_slices().

extract(source)[source]#

Extract the source region which covers the target domain.

Parameters:: source (Cube) – Source cube.
Returns:: An Iris cube representing the sub-region of source which overlaps the area of the target.
Return type:: Cube

ants.load(*args, **kwargs)[source]#: Deprecated since version 2.1: Use ants.io.load.load() instead.

ants.load_cube(*args, **kwargs)[source]#: Deprecated since version 2.1: Use ants.io.load.load_cube() instead.

ants.load_cubes(*args, **kwargs)[source]#: Deprecated since version 2.1: Use ants.io.load.load_cubes() instead.

ants.load_grid(filenames, *args, **kwargs)[source]#: Deprecated since version 2.1: Use ants.io.load.load_grid() instead.

ants.load_raw(*args, **kwargs)[source]#: Deprecated since version 2.1: Use ants.io.load.load_raw() instead.

Subpackages#

Submodules#

ants.command_parse module#

class ants.command_parse.AntsArgParser(target_lsm=False, target_grid=False, time_constraints=False)[source]#

Bases: ArgumentParser

Standardised ancillary commandline interface.

__init__(target_lsm=False, target_grid=False, time_constraints=False)[source]#

Parse command-line arguments and options.

The following arguments represent those common to all ancillary applications:

sources <SOURCE1> <SOURCE2> ... <SOURCEN>     Source data path(s).
--output <OUTPUT>, -o <OUTPUT>                Output filepath
--ants-config <ANTS_CONFIG>                   Configuration path.

Parameters:

target_lsm (bool, optional) – Standardised optional argument. When True, the user is required to provide a land binary mask via a --target-lsm keyword argument followed by the path to the land sea mask.
target_grid (bool, optional) – Standardised optional argument. When True, the user is required to provide a target grid file via a --target-grid keyword argument followed by the path to the grid file.
time_constraints (bool, optional) – Standardised optional argument. When True, the user is able to provide two keyword arguments to constrain the search data by time: a “–begin” keyword argument which must be earlier or equal to the “–end” keyword argument. Both should be provided as years in the format YYYY.

Note

This choice should be on the basis that the newly proposed interface is necessarily not common to the UI of other ancillary applications. Often this can be the case for pre-processing applications.

parse_args(args=None, namespace=None)[source]#

Return parsed commandline arguments according to the object specification.

Note

The ants configuration is populated where an optional –ants-config has been provided during the parse.

ants.config module#

This module handles the run-time configuration of the ANTS library.

Certain hooks are present in the library for providing detailed control over ANTS run-time.

The configuration of ANTS follows a first-in first-out approach on parsing configuration options and is handled by ants.config.GlobalConfiguration. This means that any number of configuration files can be parsed. On import, ants.cfg is parsed when present in the lib/ants/ folder. Each successive configuration file parsing will override existing parameter values.

Additionally, the following environmental variable hooks are available:

ANTS_NPROCESSES: Number of processes to be used by ants decomposition.
- Takes an integer value corresponding to the number of processes desired. Additionally take the value ‘max’, which corresponds to utilising all available CPUs on the running hardware (defaults to 1).
- Redundant where a scheduler is used (SLURM, PBS, LSF). The relevant scheduler environment variable “SLURM_NTASKS”, “PBS_NP” or “LSB_DJOB_NUMPROC” is used instead.
ANTS_TEMPORARY_DIR: A user configured space used for temporary files. When decomposition is enabled large volumes of temporary data may be created, so it is useful to choose a location that can handle this (e.g. a personal SCRATCH space). ANTS cache and temporary file usage will utilise this location.

If this variable is set, it must be to a directory that already exists.

If this variable is not set, the default temporary directory is used. For further information about how this directory is chosen, please see: https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir
CARTOPY_DATA_DIR: Directory for Natural Earth data required by cartopy. To ensure reproducibility, this should be set to a curated location. Default is to use the cartopy default cache location (which means different users may see different results). This environment variable is read by cartopy directly.

class ants.config.GlobalConfiguration(*args, **kwargs)[source]#

Bases: object

The global configuration class handles any number of configuration files, where subsequent configuration entries act to override previous entries parsed.

All group names are prefixed with “ants” and all entries are then parsed strictly. Those groups not prefixed with “ants” are silently ignored.

The following represents a description of the runtime configuration options and their default values:

# Decomposition framework

[ants_decomposition]
# Decomposition breaks a computation down into smaller pieces.

# The total numbers of pieces is the product of the `x_split` and
# `y_split` parameters below.  Increasing the number of pieces results
# in more, but smaller, pieces so reduces total memory usage at the
# cost of needing more CPUs, more time or both.  The total number of
# pieces may be more than the available CPUs - remaining pieces will
# be queued and run when CPUs are available.

# The allowed combinations of x_split and y_split are as follows:
# * Both splits set to numbers greater than or equal to 1:
# decomposition occurs as described above. The two splits do not need
# to be the same number, but they must both be specified.
# * Both splits set to automatic: Ants will decompose to 800Mb chunks.
# * Both splits set to 0: decomposition will be disabled.
# * Neither split specified: decomposition will be disabled.

# Number of pieces to decompose over x. It is recommended that a
# user configure this. See above for the allowed combinations of
# x_split and y_split.
x_split

# Number of pieces to decompose over y. It is recommended that a
# user configure this. See above for the allowed combinations of
# x_split and y_split.
y_split

# Pad width to use when extracting source pieces which overlap with
# target pieces. Default is 1.
pad_width

# Regridding
[ants_regridding_horizontal]
# Specify the regridding scheme wanted (see ants.regrid).
# Either a value for ants_regridding_horizontal or
# a value for ants_regridding_vertical *must be provided*.
# It is also possible to specify both if required.
# Available horizontal regridding options are:
#  * Linear,
#  * TwoStage,
#  * ConservativeESMF,
#  * AreaWeighted,
#  * Nearest
# Default is None.
scheme

# Specify the extrapolation mode to be used for horizontal regridding.
# Note that not all schemes support extrapolation, e.g. AreaWeighted.
# In these cases, providing an extrapolation mode will result in an error.
# Permitted extrapolation modes will depend on the scheme, refer to the
# documentation for the selected scheme to find the extrapolation modes
# it supports.
# Default is None, i.e. use the selected scheme's default extrapolation mode.
extrapolation_mode

[ants_regridding_vertical]
# Specify the regridding scheme wanted (see ants.regrid).
# Either a value for ants_regridding_horizontal or
# a value for ants_regridding_vertical *must be provided*.
# It is also possible to specify both if required.
# Available vertical regridding options are:
#  * Linear,
#  * Conservative,
#  * Nearest
# Default is None.
scheme

# Specify the extrapolation mode to be used for vertical regridding.
# Note that not all schemes support extrapolation, e.g. Conservative.
# In these cases, providing an extrapolation mode will result in an error.
# Permitted extrapolation modes will depend on the scheme, refer to the
# documentation for the selected scheme to find the extrapolation modes
# it supports.
# Default is None, i.e. use the selected scheme's default extrapolation mode.
extrapolation_mode

# Additional metadata in the context of writing output (for those
# formats that support it - currently only NetCDF).
[ants_metadata]
history

[ants_tolerance]
# Specify the tolerance wanted for the checks conducted by the Raymond
# filter on a grid if isotropic filtering is requested. The default
# ANTS tolerance is used if this is not set.
raymond_filter_isotropy_tolerance

[ants_tuning]
# The behaviour of dask rechunking on save gives better performance in
# most cases.  However, if the source data is a netCDF with contiguous
# netCDF chunking, then it may be more performant to disable the dask
# rechunking by setting this option to True (which was the default
# behaviour up to and including ANTS 0.19).
disable_rechunking

Additionally there are some configurations which may be useful to debugging and or developers:

# Logging setup
[ants_logging]
# Simply set to True to enable application logging.
enabled

# Defaults to INFO.  See
# https://docs.python.org/3/library/logging.html#logging-levels for
# more options.
level

__init__()[source]#

parse_configuration(filename)[source]#

Parses a new configuration file.

Entries in ‘filename’ override existing entries in the configuration, while entries not set remain unchanged from the previous state.

Option to use a user defined temporary working directory. This is defined via the ANTS_TEMPORARY_DIR environment variable.

Parameters:: filename (str) – Name of the configuration file to read.

ants.config.dirpath_writeable(filepath)[source]#

Check read-write permissions of the provided path.

Parameters:: filepath (str) – File path to check read-write permissions. The path is expanded in the case of environmental variables and ‘~’. Symbolic links are also supported in the path. The directory the file is contained in is checked for read-write permissions.
Raises:: IOError – If the directory path does not have read-write permissions.
Returns:: filepath – Canonical expanded file path.
Return type:: str

ants.config.filepath_readable(filepath)[source]#

Check read permissions of the provided filepath.

Parameters:: filepath (str) – Filepath to check read permissions. The path is expanded in the case of environmental variables and ‘~’. Symbolic links are also supported in the path.
Raises:: argparse.ArgumentTypeError – If the filepath does not have read permissions or does not exist.
Returns:: filepath – Canonical expanded filepath.
Return type:: str

ants.config.formatted_time(time_string)[source]#

Converts provided string into a datetime object as required by iris.

Parameters:: time_string (str) – A string containing information about the time constraints for the data to enable it to be split into year long chunks.
Raises:: TimeConstraintFormatException – If the string is not of the required (YYYY) format.
Returns:: datetime – datetime as required by iris.
Return type:: int

ants.config.set_temporary_directory()[source]#

ants.coord_systems module#

ANTS defines its own coordinate systems derived by CFCRS. This acts to encapsulate an iris coordinate system along with CF metadata.

Additionally, ANTS extends the iris.coord_systems.CoordSystem baseclass such that every coordinate system has a new method (see ants.coord_systems.as_ants_crs()). This allows one coordinate system to be treated as another, where required. Examples include performing a regrid, where a WGS84 can be approximated as a UMSPHERE. Currently, the following ANTS routines/modules utilise this coordinate system equivalence:

ants.ExtractConstraint()

ants.regrid.rectilinear()

class ants.coord_systems.CFCRS(crs)[source]#

Bases: object

Ants encapsulation of coordinate system and metadata corresponding to grid metadata.

__init__(crs)[source]#

Return an object describing a coordinate system along with metadata required for a grid defined on this coordinate system.

Parameters:

crs (iris.coord_systems.CoordSystem) –
x_metadata (dict) – Metadata describing the x axis of a grid with corresponding coordinate system. Of the form {‘standard_name’: …, ‘units’:, …}.
y_metatada (dict) – Metadata describing the x axis of a grid with corresponding coordinate system. Of the form {‘standard_name’: …, ‘units’:, …}.

property crs#: Return coordinate system.

property x#: Return metadata corresponding to the x axis.

property y#: Return metadata corresponding to the y axis.

class ants.coord_systems.DefinitionsABC[source]#

Bases: object

Abstract base class for creating coordinate reference systems mappers.

abstract definitions()[source]#

classmethod get_crs(value)[source]#

Get ANTS crs.

Parameters:: value (str) – String object to parse.
Returns:: Ants coordinate system.
Return type:: ants.coord_systems.CFCRS

class ants.coord_systems.Name2CRS[source]#

Bases: DefinitionsABC, object

Coordinate name to ANTS crs mapper.

definitions = [['OSGB 1936 British National Grid', TransverseMercator(latitude_of_projection_origin=49.0, longitude_of_central_meridian=-2.0, false_easting=400000.0, false_northing=-100000.0, scale_factor_at_central_meridian=0.9996012717, ellipsoid=GeogCS(semi_major_axis=6377563.396, semi_minor_axis=6356256.909))]]#

class ants.coord_systems.Proj2CRS[source]#

Bases: DefinitionsABC, object

Proj4 to ANTS crs mapper.

definitions = [['+proj=longlat +a=6371229 +b=6371229 +no_defs', GeogCS(6371229.0)], ['+proj=longlat +R=6371229 +no_defs', GeogCS(6371229.0)]]#

ants.coord_systems.as_ants_crs(self)[source]#

Return an ANTS coordinate system.

Under most circumstances, return itself, unless the coordinate system can be treated as another, in which case, return its counterpart. Here are the mappings between coordinate systems and their counterparts:

if self == WGS84: return UM_SPHERE
if self == OSGB: return TransverseMercator
if self == RotatedGeogCS and grid_north_pole_latitude == 90:
    return Unrotated version of our crs
else: return self

Return type:: ants.coord_systems.CFCRS

Note

This function is defined in ANTS but monkey patches iris.coord_systems.CoordSystem

ants.decomposition module#

Decomposition in ants is achieved via decompose().

ants decomposition can utilise the hardware available. To that end, ANTS respects scheduler configuration (SLURM, SPICE, LSF), utilising only the hardware configured by the scheduler for the job. Where no scheduler is used, the user may desire to configure ants themselves (see ants.config for configuring ANTS_NPROCESSES). Having both scheduler and user configuration will cause an exception to be raised.

Additionally, decomposition is sensitive to the dataset(s) being decomposed. To that end, configuration is necessary to tell ants how you wish source datasets to be decomposed (see ants.config.GlobalConfiguration).

Decomposition relies on being able to write temporary files to disk. The temporary directory used can be configured by setting the ANTS_TEMPORARY_DIR to a local directory capable of handling the volumes of temporary output data created by the ANTS process you are running. (see: ants.config).

See ants.utils.cube.defer_cube().

class ants.decomposition.CallableMosaic[source]#

Bases: object

Abstract mosaic generator factory.

abstract __call__()[source]#

Mosaic generator.

Called each time we require a slice through our sliceable object. Particularly beneficial where we are restricted by memory.

Returns:: Mosaic generator of sliceable pieces.

property sliceable#

class ants.decomposition.DomainDecompose(pad_width=1)[source]#

Bases: object

Domain decompose an operation for a given cube for both unary and binary operations.

__call__(operation, mosaics, sources=None)[source]#

Perform the provided operation over each decomposed piece.

Parameters:

operation (Callable) – Binary or unary operation on which to apply over each decomposed piece.
mosaic (CallableMosaic object) – Callable which returns a generator of cubes.
sources (One or more iris.cube.Cube, optional) – Source cube(s), where an extracted overlap is performed with each target piece in order to perform our binary operation.

__init__(pad_width=1)[source]#

Create a decomposable cube wrapper to which we can apply unary and binary operations.

Parameters:: pad_width (int) – Pad the slices of the extracted source domain by the specified number of cells. Default is 1.

property mosaic_generator#

An iterator over all the mosaic pieces.

Type:: Iterator

source_piece_generator(source, mosaic)[source]#

Create a generator for source pieces that overlap with target pieces.

This method uses ants.ExtractConstraint to extract source pieces that overlap with each target piece returned by the mosaic generator. The region extraction uses the pad_width provided when initialising the class instance.

Parameters:

source (iris.cube.Cube) – The source cube to be split into pieces
mosaic (Iterable[iris.cube.Cube]) – Iterable of target pieces for source to overlap

Returns:

Pieces of the source cube which overlap with each target cube in the mosaic

Return type:

Generator[iris.cube.Cube]

property src_generator#

The source which overlaps each decomposed target piece.

Type:: list of Cube

class ants.decomposition.MosaicBySplit(sliceable, split)[source]#

Bases: CallableMosaic

Mosaic generator factory where mosaic piece size is determined by the number of pieces requested for the mosaic.

For example:

>>> import numpy as np
>>> arr = np.array([[1, 2], [3, 4]])
>>> splitter = MosaicBySplit(arr, (2, 1))
>>> # Return our generator
>>> slices = splitter()
>>> print(list(slices))
[array([[1, 2]]), array([[3, 4]])]

__call__()[source]#

Mosaic generator.

Called each time we require a slice through our sliceable object. Particularly beneficial where we are restricted by memory.

Returns:: Mosaic generator of sliceable pieces.

__init__(sliceable, split)[source]#

Mosaic generator factory for the given sliceable with specified target shape.

Parameters:

sliceable (array_like) – Object with shape property and numpy style indexing. Mosaic pieces correspond to a pieces of this given sliceable object.
split (tuple) – Specified how each dimension should be split.

property split#

Number of split corresponding to each dimension.

Type:: tuple

class ants.decomposition.MultiprocessingDomainDecompose(pad_width=1)[source]#

Bases: DomainDecompose

Domain decompose an operation in parallel for a given cube for both unary and binary operations.

ants.decomposition.decompose(operation, sources, targets=None)[source]#

Decompose source(s) [and optional targets] and apply operation on each segment.

Where only sources are provided, these sources are turned into mosaics. Each mosaic piece then has the provided operation applied to it. If targets are also provided, then it is these targets that are turned into mosaics. The sources which overlap each target mosaic piece then has the provided operation applied to it. See this module documentation on how decomposition is configured.

Parameters:

operation (Callable) – Operation to be computed on each decomposed piece within the decomposition framework, whether unary or binary.
sources (One or more iris.cube.Cube) – Cube(s) upon which the operation will be performed.
targets (One of more iris.cube.Cube, optional) – Target grid cube(s), utilised in binary operations. See the note below on providing suitable targets.

Returns:

Result from applying source and optional targets to the specified operation via the decomposition framework.

Return type:

One or more iris.cube.Cube

Notes

One of the following rules must apply for sources and targets provided:

len(sources) > 0 AND len(targets) in [0, 1]
len(sources) == 1 AND len(targets) > 1 AND (targets on identical horizontal grids and with equal shape)
len(sources) == len(targets)

Where these conditions are not met, the relationship between source and target is ambiguous and an exception is thrown. Under such circumstances, the user is referred to the userguide where the ‘partial’ library will allow arbitrary arguments/relationships to be utilised with the decomposition framework.

ants.exceptions module#

exception ants.exceptions.DateRangeNotFullyAvailableException(time_string, available_range)[source]#

Bases: ValueError

Raised when the years requested in the cli are only partially found in the data.

__init__(time_string, available_range)[source]#

Creates a ValueError that tells the user the years they requested are not all available in the source data and tells the user which years are available.

Args:

time_string: the years requested by the user.
available_range: the years available in the source data.

exception ants.exceptions.FloodfillError(message)[source]#

Bases: ValueError

Raised where the seed point already has the value being floodfilled.

__init__(message)[source]#

Raised where the seed point already has the value being floodfilled.

Args:

Message to explain the seed point already has the value being floodfilled.

exception ants.exceptions.InvalidOutputPathException[source]#

Bases: RuntimeError

Raised when the user may overwrite the source files with the output.

__init__()[source]#: Provides the user with a message to explain the output path must be different from the source file.

exception ants.exceptions.NoCoverageError[source]#

Bases: IndexError

Raised when the source has no coverage over the target.

__init__()[source]#

Raised when the source has no coverage over the target.

Args:

Message to explain the source has no coverage over the target.

exception ants.exceptions.TimeConstraintFormatException(time_string)[source]#

Bases: RuntimeError

Raised when the time format is not correctly formatted

__init__(time_string)[source]#

Provides the user with a RuntimeError to explain how the request should be formatted.

Args:

time_string: the years in the format requested by the user.
message: information on how the request should be formatted.

exception ants.exceptions.TimeConstraintMissingException[source]#

Bases: ValueError

Raised when the user only supplies one of the two required time constraints in the cli.

__init__()[source]#: Provides users with information that both an end and start time must be provided. To request a single year, these can be the same.

exception ants.exceptions.TimeConstraintOutOfBoundsException(time_string)[source]#

Bases: ValueError

Raised when the years requested in the cli are not found in the data.

__init__(time_string)[source]#

Raises a ValueError that tells the user the years they requested are not available in the data.

Args:

time_string: the years requested by the user.

exception ants.exceptions.TimeConstraintUnorderedException[source]#

Bases: ValueError

Raised when the cli arguments for a time constraint have been given out of order.

__init__()[source]#: Raises a ValueError and lets the user know the begin time must be before the end time.

ants.stats module#

class ants.stats.TimeIt(t1=None)[source]#

Bases: object

__init__(t1=None)[source]#

Timing a code snippet.

Context manager returns the time between entering and exiting the context manager.

Parameters:: t1 (float, optional) – Reference start time in seconds since epoch, in UTC. If not provided, this is set to the time of initialisation of this class.

property time_taken#

ants.stats.proc_stat()[source]#

Process specific memory statistics

Return process specific statistics (running process as determined by os.getpid) by parsing /proc/<pid>/status

Returns:: Dictionary of process specific statistics ‘Memory Statistics PID’, ‘VmSize’, ‘VmRSS’, ‘VmPeak’ and ‘VmHWM’.
Return type:: dict

ants.stats.sys_stat()[source]#

System memory statistics

Return system memory statistics by parsing the host /proc/meminfo.

Returns:: Dictionary containing system memory statistics: ‘Committed_AS’; ‘MemFree’; ‘Buffers’; ‘Cached’ and ‘MemTotal’.
Return type:: dict