Examples: Modifying data in individual fields¶

Example 1: Set all real data fields to zero¶

import mule
from mule.operators import ScaleFactorOperator

zero_operator = ScaleFactorOperator(0.0)

ff = mule.FieldsFile.from_file("InputFile")

ff_out = ff.copy()

for field in ff.fields:
    if field.lbrel in (2,3) and field.lbuser4 != 30:
        ff_out.fields.append(zero_operator(field))
    else:
        ff_out.fields.append(field)

ff_out.to_file("OutputFile")

Note

A copy of the original file object is not strictly necessary, but it avoids overwriting the original field objects.
The built-in operators in mule.operators handle mdi points in a sensible way - in this example points set to mdi will not be changed.
Notice the check on lbrel - this is to make sure only fields with valid header release numbers are processed.
The loop also filters out fields with the STASH code 30 - this is the land-sea mask, and probably should not be zeroed; note that since only fields with valid header releases define the lbuser4 property it is important that the two if conditions are placed in the order above.
Since the packing code (lbpack) of the modified fields is not changed Mule will try to write the fields out with the same packing method and accuracy - in some cases this might not be suitable so you may need to consider whether to update the value of lbpack on each field before appending it to the file.
It is also common practice to update the lbproc value of the field’s lookup header when the data has been modified (some applications use this to determine certain types of field) - you should update this value before appending the modified fields to the file if required (or do it inside the operator’s new_field method).

Example 2: Set level-1 u fields (STASH code 2) to their absolute values¶

import mule
import numpy as np

class AbsoluteOperator(mule.DataOperator):
    """Operator which sets all non-missing points to their absolute value"""
    def __init__(self):
        pass
    def new_field(self, source_field):
        """Creates the new field object"""
        field = source_field.copy()
        return field
    def transform(self, source_field, new_field):
        """Performs the data manipulation"""
        data = source_field.get_data()
        data_out = np.abs(data)
        # If the field defines MDI, reset missing points from original
        # field back to MDI in the output
        if hasattr(source_field, "bmdi"):
            mdi = source_field.bmdi
            mask = (data == mdi)
            data_out[mask] = mdi
        return data_out

abs_operator = AbsoluteOperator()

ff = mule.FieldsFile.from_file("InputFile")

for ifield, field in enumerate(ff.fields):
    if field.lbrel in (2,3) and field.lbuser4 == 2 and field.lblev == 1:
        ff.fields[ifield] = abs_operator(field)

ff.to_file("OutputFile")

Note

There aren’t any built-in operators in mule.operators which provide this functionality, so this example creates a custom operator.
Unlike example 1 - this example modifies the fields in the file object in-place; this is neater but overwrites the point of access to the original fields so might not always be ideal.
Notice the check on lbrel - this is to make sure only fields with valid header release numbers are processed.
The loop also filters for the required STASH code and level; note that since only fields with valid header release numbers define lbuser4 and lblev, so it is important that the if conditions for these appear after the check on lbrel.
Since the packing code (lbpack) of the modified fields is not changed Mule will try to write the fields out with the same packing method and accuracy - in some cases this might not be suitable so you may need to consider whether to update the value of lbpack on each field before appending it to the file (or inside the operator’s new_field method).
It is also common practice to update the lbproc value of the field’s lookup header when the data has been modified (some applications use this to determine certain types of field) - you should update this value before appending the modified fields to the file if required (or do it inside the operator’s new_field method).

Example 3: Raise all fields valid at 12Z to the power 3¶

import mule

class ExponentOperator(mule.DataOperator):
    """Operator which raises all non-missing points to an exponent"""
    def __init__(self, exponent):
        """Initialise the operator, setting the exponent value"""
        self.exponent = exponent
    def new_field(self, source_field):
        """Creates the new field object"""
        field = source_field.copy()
        return field
    def transform(self, source_field, new_field):
        """Performs the data manipulation"""
        data = source_field.get_data()
        data_out = data**self.exponent
        # If the field defines MDI, reset missing points from original
        # field back to MDI in the output
        if hasattr(source_field, "bmdi"):
            mdi = source_field.bmdi
            mask = (data == mdi)
            data_out[mask] = mdi
        return data_out

exp_operator = ExponentOperator(3)

ff = mule.FieldsFile.from_file("InputFile")

for ifield, field in enumerate(ff.fields):
    if field.lbrel in (2,3) and field.lbhr == 12:
        ff.fields[ifield] = exp_operator(field)

ff.to_file("OutputFile")

Note

There aren’t any built-in operators in mule.operators which provide this functionality, so this example creates a custom operator.
Unlike example 1 - this example modifies the fields in the file object in-place; this is neater but overwrites the point of access to the original fields so might not always be ideal.
Notice the check on lbrel - this is to make sure only fields with valid header release numbers are processed.
The loop also filters for the required forecast time; note that since only fields with valid header release numbers define lbhr it is important that the if conditions for these appear after the check on lbrel.
Since the packing code (lbpack) of the modified fields is not changed Mule will try to write the fields out with the same packing method and accuracy - in some cases this might not be suitable so you may need to consider whether to update the value of lbpack on each field before appending it to the file (or inside the operator’s new_field method).
It is also common practice to update the lbproc value of the field’s lookup header when the data has been modified (some applications use this to determine certain types of field) - you should update this value before appending the modified fields to the file if required (or do it inside the operator’s new_field method).

Example 4: Remove negative specific humidities (STASH code 10)¶

import mule
from mule.operators import HardLimitOperator

neg_to_zero_operator = HardLimitOperator(lower_limit=0.0)

ff = mule.FieldsFile.from_file("InputFile")

for ifield, field in enumerate(ff.fields):
    if field.lbrel in (2,3) and field.lbuser4 == 10:
        ff.fields[ifield] = neg_to_zero_operator(field)

ff.to_file("OutputFile")

Note

Unlike example 1 - this example modifies the fields in the file object in-place; this is neater but overwrites the point of access to the original fields so might not always be ideal.
Notice the check on lbrel - this is to make sure only fields with valid header release numbers are processed.
The loop also filters for the required STASH code; note that since only fields with valid header release numbers define lbuser4 it is important that the if conditions for these appear after the check on lbrel.

Example 5: Set all level-38 fields to -1¶

import mule
from mule.operators import ScaleFactorOperator, AddScalarOperator

zero_operator = ScaleFactorOperator(0.0)
subtract_one_operator = AddScalarOperator(-1.0)

ff = mule.FieldsFile.from_file("InputFile")

for ifield, field in enumerate(ff.fields):
    if field.lbrel in (2,3) and field.lblev == 38:
        ff.fields[ifield] = (
            subtract_one_operator(zero_operator(field)))

ff.to_file("OutputFile")

Note

There isn’t a single built-in operator in mule.operators which provides this functionality, but it can be achieved by combining the effects of 2 operators - note the way in which they can be “chained” together to apply multiple operations.
Unlike example 1 - this example modifies the fields in the file object in-place; this is neater but overwrites the point of access to the original fields so might not always be ideal.
Notice the check on lbrel - this is to make sure only fields with valid header release numbers are processed.
The loop also filters for the required level; note that since only fields with valid header release numbers define lblev it is important that the if conditions for these appear after the check on lbrel.
Since the packing code (lbpack) of the modified fields is not changed Mule will try to write the fields out with the same packing method and accuracy - in some cases this might not be suitable so you may need to consider whether to update the value of lbpack on each field before appending it to the file (or inside the operator’s new_field method).
It is also common practice to update the lbproc value of the field’s lookup header when the data has been modified (some applications use this to determine certain types of field) - you should update this value before appending the modified fields to the file if required (or do it inside the operator’s new_field method).

Example 6: Set all level-38 32 bit packed (LBPACK 21 = 2) fields to -1¶

import mule
from mule.operators import ScaleFactorOperator, AddScalarOperator

zero_operator = ScaleFactorOperator(0.0)
subtract_one_operator = AddScalarOperator(-1.0)

ff = mule.FieldsFile.from_file("InputFile")

for ifield, field in enumerate(ff.fields):
    if field.lbrel in (2,3) and field.lblev == 38 and field.lbpack == 2:
        ff.fields[ifield] = (
            subtract_one_operator(zero_operator(field)))

ff.to_file("OutputFile")

Note

There isn’t a single built-in operator in mule.operators which provides this functionality, but it can be achieved by combining the effects of 2 operators - note the way in which they can be “chained” together to apply multiple operations.
Unlike example 1 - this example modifies the fields in the file object in-place; this is neater but overwrites the point of access to the original fields so might not always be ideal.
Notice the check on lbrel - this is to make sure only fields with valid header release numbers are processed.
The loop also filters for the required level; note that since only fields with valid header release numbers define lblev and lbpack it is important that the if conditions for these appear after the check on lbrel.
It is common practice to update the lbproc value of the field’s lookup header when the data has been modified (some applications use this to determine certain types of field) - you should update this value before appending the modified fields to the file if required (or do it inside the operator’s new_field method).

Example 7: Add random perturbations at the level of the least significant bit to all theta fields (STASH code 4)¶

import sys
import mule
import numpy as np

EPSILON = sys.float_info.epsilon

class LSBPerturbOperator(mule.DataOperator):
    """Operator which adds LSB perturbations to a field"""
    def __init__(self):
        pass
    def new_field(self, source_field):
        """Creates the new field object"""
        field = source_field.copy()
        return field
    def transform(self, source_field, new_field):
        """Performs the data manipulation"""
        data = source_field.get_data()
        # Create an array of perturbations and apply them to the data
        random_numbers = (2.0*np.random.random(data.shape) - 1.0)*EPSILON
        data_out = data + data*random_numbers
        # If the field defines MDI, reset missing points from original
        # field back to MDI in the output
        if hasattr(source_field, "bmdi"):
            mdi = source_field.bmdi
            mask = (data == mdi)
            data_out[mask] = mdi
        return data_out

lsb_perturb_operator = LSBPerturbOperator()

ff = mule.FieldsFile.from_file("InputFile")

for ifield, field in enumerate(ff.fields):
    if field.lbrel in (2,3) and field.lbuser4 == 4:
        ff.fields[ifield] = lsb_perturb_operator(field)

ff.to_file("OutputFile")

Note

There aren’t any built-in operators in mule.operators which provide this functionality, so this example creates a custom operator.
Unlike example 1 - this example modifies the fields in the file object in-place; this is neater but overwrites the point of access to the original fields so might not always be ideal.
Notice the check on lbrel - this is to make sure only fields with valid header release numbers are processed.
The loop also filters for the required STASH code; note that since only fields with valid header release numbers define lbuser4 it is important that the if conditions for these appear after the check on lbrel.
Since the packing code (lbpack) of the modified fields is not changed Mule will try to write the fields out with the same packing method and accuracy - in some cases this might not be suitable so you may need to consider whether to update the value of lbpack on each field before appending it to the file (or inside the operator’s new_field method).
It is also common practice to update the lbproc value of the field’s lookup header when the data has been modified (some applications use this to determine certain types of field) - you should update this value before appending the modified fields to the file if required (or do it inside the operator’s new_field method).

Example 8: Add 1.0 to fields with Item Numbers 2 or 3, only outputting fields operated on¶

import mule
from mule.operators import AddScalarOperator

add_1_operator = AddScalarOperator(1.0)

ff = mule.FieldsFile.from_file("InputFile")

ff_out = ff.copy()

for field in ff.fields:
    if field.lbrel in (2,3) and field.lbuser4 in (2,3):
        ff_out.fields.append(add_1_operator(field))

ff_out.to_file("OutputFile")

Note

Unlike many of the other examples, this example makes a copy of the original object - this allows only the desired fields to be added and hence written out.
Notice the check on lbrel - this is to make sure only fields with valid header release numbers are processed.
The loop also filters for the required STASH codes; note that since only fields with valid header release numbers define lbuser4 it is important that the if conditions for these appear after the check on lbrel.
It is common practice to update the lbproc value of the field’s lookup header when the data has been modified (some applications use this to determine certain types of field) - you should update this value before appending the modified fields to the file if required (or do it inside the operator’s new_field method).

Examples: Modifying data in individual fields¶

Example 1: Set all real data fields to zero¶

Example 2: Set level-1 u fields (STASH code 2) to their absolute values¶

Example 3: Raise all fields valid at 12Z to the power 3¶

Example 4: Remove negative specific humidities (STASH code 10)¶

Example 5: Set all level-38 fields to -1¶

Example 6: Set all level-38 32 bit packed (LBPACK 21 = 2) fields to -1¶

Example 7: Add random perturbations at the level of the least significant bit to all theta fields (STASH code 4)¶

Example 8: Add 1.0 to fields with Item Numbers 2 or 3, only outputting fields operated on¶

Table of Contents

This Page