Internal Functions¶

This page details various internal functions within CSET. Most people should never have to worry about these, but they are documented here for the convenience of those developing CSET.

CSET.operators¶

Subpackage contains all of CSET’s operators.

CSET.operators.execute_recipe_collate(recipe_yaml: Path | str, output_directory: Path, recipe_variables: dict = None, style_file: Path = None) → None¶

Parse and execute the collation steps from a recipe file.

Parameters:

recipe_yaml (Path or str) – Path to a file containing, or string of, a recipe’s YAML describing the operators that need running. If a Path is provided it is opened and read.
output_directory (Path) – Pathlike indicating desired location of output. Must already exist.
recipe_variables (dict) – Dictionary of variables for the recipe.

Raises:

ValueError – The recipe is not well formed.
TypeError – The provided recipe is not a stream or Path.

CSET.operators.execute_recipe_parallel(recipe_yaml: Path | str, input_directory: Path, output_directory: Path, recipe_variables: dict = None, style_file: Path = None) → None¶

Parse and executes the parallel steps from a recipe file.

Parameters:

recipe_yaml (Path or str) – Path to a file containing, or string of, a recipe’s YAML describing the operators that need running. If a Path is provided it is opened and read.
input_file (Path) – Pathlike to netCDF (or something else that iris read) file to be used as input.
output_directory (Path) – Pathlike indicating desired location of output.
recipe_variables (dict) – Dictionary of variables for the recipe.

Raises:

FileNotFoundError – The recipe or input file cannot be found.
FileExistsError – The output directory as actually a file.
ValueError – The recipe is not well formed.
TypeError – The provided recipe is not a stream or Path.

CSET.operators.get_operator(name: str)¶

Get an operator by its name.

Parameters:: name (str) – The name of the desired operator.
Returns:: The named operator.
Return type:: function
Raises:: ValueError – If name is not an operator.

Examples

>>> CSET.operators.get_operator("read.read_cubes")
<function read_cubes at 0x7fcf9353c8b0>

CSET.recipes¶

Operations on recipes.

exception CSET.recipes.FileExistsWarning¶: Warning a file already exists, and some unusual action shall be taken.

CSET.recipes.detail_recipe(recipe_name: str) → None¶

Detail the recipe to stdout.

If multiple recipes match the given name they will all be displayed.

Parameters:: recipe_name (str) – Partial match for the recipe name.

CSET.recipes.list_available_recipes() → None¶: List available recipes to stdout.

CSET.recipes.unpack_recipe(recipe_dir: Path, recipe_name: str) → None¶

Unpacks recipes files into a directory, creating it if it doesn’t exist.

Parameters:

recipe_dir (Path) – Path to a directory into which to unpack the recipe files.
recipe_name (str) – Name of recipe to unpack.

Raises:

FileExistsError – If recipe_dir already exists, and is not a directory.
OSError – If recipe_dir cannot be created, such as insufficient permissions, or lack of space.

CSET.graph¶

Visualise recipe into a graph.

CSET.graph.save_graph(recipe_file: Path | str, save_path: Path = None, auto_open: bool = False, detailed: bool = False)¶

Draws out the graph of a recipe, and saves it to a file.

Parameters:

recipe_file (Path | str) – The recipe to be graphed.
save_path (Path) – Path where to save the generated image. Defaults to a temporary file.
auto_open (bool) – Whether to automatically open the graph with the default image viewer.
detailed (bool) – Whether to include operator arguments on the graph.

Raises:

ValueError – Recipe is invalid.

CSET._common¶

Note

While no functions documented on this page are part of any API promise, the functions inside _common are especially liable to change without any kind of notice.

Common functionality used across CSET.

exception CSET._common.ArgumentError¶: Provided arguments are not understood.

exception CSET._common.TemplateError¶: Rendering a template failed due a placeholder without a value.

CSET._common.check_recipe_has_steps(recipe: dict)¶

Check a recipe has the minimum required steps.

Checking that the recipe actually has some steps, and providing helpful error messages otherwise. We must have at least a parallel step, as that reads the raw data.

Parameters:

recipe (dict) – The recipe as a python dictionary.

Raises:

ValueError – If the recipe is invalid. E.g. invalid YAML, missing any steps, etc.
TypeError – If recipe isn’t a dict.
KeyError – If needed recipe variables are not supplied.

CSET._common.get_recipe_metadata() → dict¶: Get the metadata of the running recipe.

CSET._common.iter_maybe(thing) → Iterable¶: Ensure thing is Iterable. Strings count as atoms.

CSET._common.parse_recipe(recipe_yaml: Path | str, variables: dict = None)¶

Parse a recipe into a python dictionary.

Parameters:

recipe_yaml (Path | str) – Path to recipe file, or the recipe YAML directly.
variables (dict) – Dictionary of recipe variables. If None templating is not attempted.

Returns:

recipe – The recipe as a python dictionary.

Return type:

dict

Raises:

ValueError – If the recipe is invalid. E.g. invalid YAML, missing any steps, etc.
TypeError – If recipe_yaml isn’t a Path or string.
KeyError – If needed recipe variables are not supplied.

Examples

>>> CSET._common.parse_recipe(Path("myrecipe.yaml"))
{'parallel': [{'operator': 'misc.noop'}]}

CSET._common.parse_variable_options(arguments: list[str]) → dict¶

Parse a list of arguments into a dictionary of variables.

The variable name arguments start with two hyphen-minus (–), consisting of only capital letters (A-Z) and underscores (_). While the variable name is restricted, the value of the variable can be any string.

Parameters:: arguments (list[str]) – List of arguments, e.g: [”–LEVEL”, “2”, “–STASH=m01s01i001”]
Returns:: recipe_variables – Dictionary keyed with the variable names.
Return type:: dict
Raises:: ValueError – If any arguments cannot be parsed.

CSET._common.render(template: str, /, **variables) → str¶

Render the template with the provided variables.

The template should contain placeholders that will be replaced. These placeholders consist of the placeholder name within double curly braces. The name of the placeholder should be a valid python identifier. Whitespace between the braces and the name is ignored. E.g.: {{ placeholder_name }}

An exception will be raised if there are placeholders without corresponding values. It is acceptable to provide unused values; they will be ignored.

Parameters:

template (str) – Template to fill with variables.
**variables (Any) – Keyword arguments for the placeholder values. The argument name should be the same as the placeholder’s name. You can unpack a dictionary of value with render(template, **my_dict).

Returns:

rendered_template – Filled template.

Return type:

str

Raises:

TemplateError – Value not given for a placeholder in the template.
TypeError – If the template is not a string, or a variable cannot be casted to a string.

Examples

>>> template = "<p>Hello {{myplaceholder}}!</p>"
>>> simple_template.render(template, myplaceholder="World")
"<p>Hello World!</p>"

CSET._common.render_file(template_path: str, /, **variables) → str¶

Render a template directly from a file.

Otherwise the same as simple_template.render().

Examples

>>> simple_template.render_file("/path/to/template.html", myplaceholder="World")
"<p>Hello World!</p>"

CSET._common.replace_template_variable(s: str, variables)¶: Fill all variable placeholders in the string.

CSET._common.slugify(s: str) → str¶

Turn a string into a version that can be used everywhere.

The resultant string will only consist of a-z, 0-9, dots, dashes, and underscores.

CSET._common.template_variables(recipe: dict | list, variables: dict) → dict¶

Insert variables into recipe.

Parameters:

recipe (dict | list) – The recipe as a python dictionary. It is updated in-place.
variables (dict) – Dictionary of variables for the recipe.

Returns:

recipe – Filled recipe as a python dictionary.

Return type:

dict

Raises:

KeyError – If needed recipe variables are not supplied.