fab.tools#

A simple init file to make it shorter to import tools.

class fab.tools.Ar#

This is the base class for ar.

create(output_fpath, members)#

Create the archive with the specified name, containing the listed members.

Parameters:

output_fpath (Path) – the output path.
members (List[Union[Path, str]]) – the list of objects to be added to the archive.

class fab.tools.Category(value)#

This class defines the allowed tool categories.

property is_compiler#: Returns if the category is either a C or a Fortran compiler.

class fab.tools.CCompiler(name, exec_name, suite, mpi=False, compile_flag=None, output_flag=None, openmp_flag=None)#

This is the base class for a C compiler. It just sets the category of the compiler as convenience.

Parameters:

name (str) – name of the compiler.
exec_name (str) – name of the executable to start.
suite (str) – name of the compiler suite.
mpi (bool) – whether the compiler or linker support MPI. (default: False)
compile_flag (Optional[str]) – the compilation flag to use when only requesting compilation (not linking). (default: None)
output_flag (Optional[str]) – the compilation flag to use to indicate the name of the output file (default: None)
openmp_flag (Optional[str]) – the flag to use to enable OpenMP (default: None)

class fab.tools.Compiler(name, exec_name, suite, category, mpi=False, compile_flag=None, output_flag=None, openmp_flag=None)#

This is the base class for any compiler. It provides flags for

compilation only (-c),
naming the output file (-o),
OpenMP

Parameters:

name (str) – name of the compiler.
exec_name (Union[str, Path]) – name of the executable to start.
suite (str) – name of the compiler suite this tool belongs to.
category (Category) – the Category (C_COMPILER or FORTRAN_COMPILER).
mpi (bool) – whether the compiler or linker support MPI. (default: False)
compile_flag (Optional[str]) – the compilation flag to use when only requesting compilation (not linking). (default: None)
output_flag (Optional[str]) – the compilation flag to use to indicate the name of the output file (default: None)
openmp_flag (Optional[str]) – the flag to use to enable OpenMP. If no flag is specified, it is assumed that the compiler does not support OpenMP. (default: None)

get_hash()#

Return type:: int
Returns:: a hash based on the compiler name and version.

property openmp: bool#

Returns:: if the compiler supports openmp or not

property openmp_flag: str#

Returns:: The flag to enable OpenMP for this compiler.

compile_file(input_file, output_file, openmp, add_flags=None)#

Compiles a file. It will add the flag for compilation-only automatically, as well as the output directives. The current working directory for the command is set to the folder where the source file lives when compile_file is called. This is done to stop the compiler inserting folder information into the mod files, which would cause them to have different checksums depending on where they live.

Parameters:

input_file (Path) – the path of the input file.
output_file (Path) – the path of the output file.
opemmp – whether OpenMP should be used or not.
add_flags (Optional[List[str]]) – additional compiler flags. (default: None)

check_available()#

Checks if the compiler is available. While the method in the Tools base class would be sufficient (when using –version), in case of a compiler we also want to store the compiler version. So, re-implement check_available in a way that will automatically store the compiler version for later usage.

Return type:: bool
Returns:: whether the compiler is available or not. We do this by requesting the compiler version.

get_version()#

Try to get the version of the given compiler.

Expects a version in a certain part of the –version output, which must adhere to the n.n.n format, with at least 2 parts.

Return type:: Tuple[int, ...]
Returns:: a tuple of at least 2 integers, representing the version e.g. (6, 10, 1) for version ‘6.10.1’.
Raises:: RuntimeError – if the compiler was not found, or if it returned an unrecognised output from the version command.

run_version_command(version_command='--version')#

Run the compiler’s command to get its version.

Parameters:: version_command (Optional[str]) – The compiler argument used to get version info. (default: '--version')
Return type:: str
Returns:: The output from the version command.
Raises:: RuntimeError – if the compiler was not found, or raised an error.

parse_version_output(category, version_output)#

Extract the numerical part from the version output. Implemented in specific compilers.

Return type:: str

get_version_string()#

Get a string representing the version of the given compiler.

Return type:: str
Returns:: a string of at least 2 numeric version components, i.e. major.minor[.patch, …]
Raises:: RuntimeError – if the compiler was not found, or if it returned an unrecognised output from the version command.

class fab.tools.CompilerSuiteTool(name, exec_name, suite, category, mpi=False)#

A tool that is part of a compiler suite (typically compiler and linker).

Parameters:

name (str) – name of the tool.
exec_name (Union[str, Path]) – name of the executable to start.
suite (str) – name of the compiler suite.
category (Category) – the Category to which this tool belongs.
mpi (bool) – whether the compiler or linker support MPI. (default: False)

property suite: str#

Returns:: the compiler suite of this tool.

property mpi: bool#

Returns:: whether this tool supports MPI or not.

class fab.tools.Cpp#: Class for cpp.

class fab.tools.CppFortran#: Class for cpp when used as a Fortran preprocessor

class fab.tools.Fcm#

This is the base class for FCM. All commands will be mapped back to the corresponding subversion commands.

Constructor.

This is class is extended by the FCM interface which is why name and executable are mutable.

Parameters:

name – Tool name, defaults to “subversion.”
exec_name – Tool executable, defaults to “svn.”
category – Tool category, defaults to SUBVERSION.

class fab.tools.Flags(list_of_flags=None)#

This class represents a list of parameters for a tool. It is a list with some additional functionality.

TODO #22: This class and build_config.FlagsConfig should be combined.

Parameters:: list_of_flags (Optional[List[str]]) – List of parameters to initialise this object with. (default: None)

checksum()#

Return type:: str
Returns:: a checksum of the flags.

remove_flag(remove_flag, has_parameter=False)#

Removes all occurrences of remove_flag in flags`. If has_parameter is defined, the next entry in flags will also be removed, and if this object contains this flag+parameter without space (e.g. -J/tmp), it will be correctly removed. Note that only the flag itself must be specified, you cannot remove a flag only if a specific parameter is given (i.e. remove_flag=”-J/tmp” will not work if this object contains […,”-J”, “/tmp”]).

Parameters:

remove_flag (str) – the flag to remove
has_parameter (bool) – if the flag to remove takes a parameter (default: False)

class fab.tools.FortranCompiler(name, exec_name, suite, mpi=False, compile_flag=None, output_flag=None, openmp_flag=None, module_folder_flag=None, syntax_only_flag=None)#

This is the base class for a Fortran compiler. It is a compiler that needs to support a module output path and support for syntax-only compilation (which will only generate the .mod files).

Parameters:

name (str) – name of the compiler.
exec_name (str) – name of the executable to start.
suite (str) – name of the compiler suite.
mpi (bool) – whether the compiler or linker support MPI. (default: False)
compile_flag (Optional[str]) – the compilation flag to use when only requesting compilation (not linking). (default: None)
output_flag (Optional[str]) – the compilation flag to use to indicate the name of the output file (default: None)
module_folder_flag (Optional[str]) – the compiler flag to indicate where to store created module files. (default: None)
openmp_flag (Optional[str]) – the flag to use to enable OpenMP (default: None)
syntax_only_flag (Optional[str]) – flag to indicate to only do a syntax check. The side effect is that the module files are created. (default: None)

property has_syntax_only: bool#

Returns:: whether this compiler supports a syntax-only feature.

set_module_output_path(path)#

Sets the output path for modules.

Params path:: the path to the output directory.

compile_file(input_file, output_file, openmp, add_flags=None, syntax_only=False)#

Compiles a file.

Parameters:

input_file (Path) – the name of the input file.
output_file (Path) – the name of the output file.
add_flags (Optional[List[str]]) – additional flags for the compiler. (default: None)
syntax_only (bool) – if set, the compiler will only do a syntax check (default: False)

class fab.tools.Fpp#: Class for Intel’s Fortran-specific preprocessor.

class fab.tools.Gcc(name='gcc', exec_name='gcc', mpi=False)#

Class for GNU’s gcc compiler.

Parameters:

name (str) – name of this compiler. (default: 'gcc')
exec_name (str) – name of the executable. (default: 'gcc')
mpi (bool) – whether the compiler supports MPI. (default: False)

class fab.tools.Gfortran(name='gfortran', exec_name='gfortran', mpi=False)#

Class for GNU’s gfortran compiler.

Parameters:

name (str) – name of this compiler. (default: 'gfortran')
exec_name (str) – name of the executable. (default: 'gfortran')
mpi (bool) – whether the compiler supports MPI. (default: False)

class fab.tools.Git#

Interface to Git version control system.

Constructor.

Parameters:

name – Display name of this tool.
exec_name – Executable for this tool.
category – Tool belongs to this category.

current_commit(folder=None)#

Return type:: str
Returns:: the hash of the current commit.
Parameters:: folder (Union[Path, str, None]) – the folder for which to determine the current commitf (defaults to .). (default: None)

init(folder)#

Initialises a directory.

Parameters:: folder (Union[Path, str]) – the directory to initialise.

clean(folder)#

Removes all non versioned files in a directory.

Parameters:: folder (Union[Path, str]) – the directory to clean.

fetch(src, dst, revision)#

Runs git fetch in the specified directory

Parameters:

src (Union[str, Path]) – the source directory from which to fetch
revision (Optional[str]) – the revision to fetch (can be “” for latest revision)
dst (Union[str, Path]) – the directory in which to run fetch.

checkout(src, dst='', revision=None)#

Checkout or update a Git repo.

Parameters:

src (str) – the source directory from which to checkout.
dst (str) – the directory in which to run checkout. (default: '')
revision (Optional[str]) – the revision to check out (can be “” for latest revision). (default: None)

merge(dst, revision=None)#

Merge a git repo into a local working copy. If the merge fails, it will run git merge –abort to clean the directory.

Parameters:

dst (Union[str, Path]) – the directory to merge in.
revision (Optional[str]) – the revision number (only used for error message, it relies on git fetch running previously). (default: None)

class fab.tools.GnuVersionHandling#

Mixin to handle version information from GNU compilers

parse_version_output(category, version_output)#

Extract the numerical part from a GNU compiler’s version output

Parameters:

name – the compiler’s name
category (Category) – the compiler’s Category
version_output (str) – the full version output from the compiler

Return type:

str

Returns:

the actual version as a string

Raises:

RuntimeError – if the output is not in an expected format.

class fab.tools.Icc(name='icc', exec_name='icc', mpi=False)#

Class for the Intel’s icc compiler.

Parameters:

name (str) – name of this compiler. (default: 'icc')
exec_name (str) – name of the executable. (default: 'icc')
mpi (bool) – whether the compiler supports MPI. (default: False)

class fab.tools.Ifort(name='ifort', exec_name='ifort', mpi=False)#

Class for Intel’s ifort compiler.

Parameters:

name (str) – name of this compiler. (default: 'ifort')
exec_name (str) – name of the executable. (default: 'ifort')
mpi (bool) – whether the compiler supports MPI. (default: False)

class fab.tools.IntelVersionHandling#

Mixin to handle version information from Intel compilers

parse_version_output(category, version_output)#

Extract the numerical part from an Intel compiler’s version output

Parameters:

name – the compiler’s name
version_output (str) – the full version output from the compiler

Return type:

str

Returns:

the actual version as a string

Raises:

RuntimeError – if the output is not in an expected format.

class fab.tools.Linker(name=None, exec_name=None, suite=None, compiler=None, output_flag='-o')#

This is the base class for any Linker. If a compiler is specified, its name, executable, and compile suite will be used for the linker (if not explicitly set in the constructor).

Parameters:

name (Optional[str]) – the name of the linker. (default: None)
exec_name (Optional[str]) – the name of the executable. (default: None)
suite (Optional[str]) – optional, the name of the suite. (default: None)
compiler (Optional[Compiler]) – optional, a compiler instance (default: None)
output_flag (str) – flag to use to specify the output name. (default: '-o')

check_available()#

Return type:: bool
Returns:: whether the linker is available or not. We do this by requesting the linker version.

link(input_files, output_file, openmp, add_libs=None)#

Executes the linker with the specified input files, creating output_file.

Parameters:

input_files (List[Path]) – list of input files to link.
output_file (Path) – output file.
openm – whether OpenMP is requested or not.
add_libs (Optional[List[str]]) – additional linker flags. (default: None)

Return type:

str

Returns:

the stdout of the link command

class fab.tools.MpiGcc#: Class for a simple wrapper around gcc that supports MPI. It calls mpicc.

class fab.tools.MpiGfortran#: Class for a simple wrapper around gfortran that supports MPI. It calls mpif90.

class fab.tools.MpiIcc#: Class for a simple wrapper around icc that supports MPI. It calls mpicc.

class fab.tools.MpiIfort#: Class for a simple wrapper around ifort that supports MPI. It calls mpif90.

class fab.tools.Preprocessor(name, exec_name, category, availablility_option=None)#

This is the base class for any preprocessor.

Parameters:

name (str) – the name of the preprocessor.
exec_name (Union[str, Path]) – the name of the executable.
category (Category) – the category (C_PREPROCESSOR or FORTRAN_PREPROCESSOR)

preprocess(input_file, output_file, add_flags=None)#

Calls the preprocessor to process the specified input file, creating the requested output file.

Parameters:

input_file (Path) – input file.
output_file (Path) – the output filename.
add_flags (Optional[List[Union[Path, str]]]) – List with additional flags to be used. (default: None)

class fab.tools.Psyclone(api=None)#

This is the base class for PSyclone.

process(config, x90_file, psy_file, alg_file, transformation_script=None, additional_parameters=None, kernel_roots=None, api=None)#

Run PSyclone with the specified parameters.

Parameters:

api (Optional[str]) – the PSyclone API. (default: None)
x90_file (Path) – the input file for PSyclone
psy_file (Path) – the output PSy-layer file.
alg_file (Union[Path, str]) – the output modified algorithm file.
transformation_script (Optional[Callable[[Path, BuildConfig], Path]]) – an optional transformation script (default: None)
additional_parameters (Optional[List[str]]) – optional additional parameters for PSyclone (default: None)
kernel_roots (Optional[List[Union[Path, str]]]) – optional directories with kernels. (default: None)

class fab.tools.Rsync#

This is the base class for rsync.

execute(src, dst)#

Execute an rsync command from src to dst. It supports ~ expansion for src, and makes sure that src end with a / so that rsync does not create a sub-directory.

Parameters:

src (Path) – the input path.
dst (Path) – destination path.

class fab.tools.Subversion(name=None, exec_name=None, category=Category.SUBVERSION)#

Interface to the Subversion version control system.

Constructor.

This is class is extended by the FCM interface which is why name and executable are mutable.

Parameters:

name (Optional[str]) – Tool name, defaults to “subversion.” (default: None)
exec_name (Union[Path, str, None]) – Tool executable, defaults to “svn.” (default: None)
category (Category) – Tool category, defaults to SUBVERSION. (default: <Category.SUBVERSION: 9>)

execute(pre_commands=None, revision=None, post_commands=None, env=None, cwd=None, capture_output=True)#

Executes a svn command.

Parameters:

pre_commands (Optional[List[str]]) – List of strings to be sent to subprocess.run() as the command. (default: None)
revision (Union[int, str, None]) – optional revision number as argument (default: None)
post_commands (Optional[List[str]]) – List of additional strings to be sent to subprocess.run() after the optional revision number. (default: None)
env (Optional[Dict[str, str]]) – Optional env for the command. By default it will use the current session’s environment. (default: None)
capture_output – If True, capture and return stdout. If False, the command will print its output directly to the console. (default: True)

Return type:

str

export(src, dst, revision=None)#

Runs svn export.

Parameters:

src (Union[str, Path]) – from where to export.
dst (Union[str, Path]) – destination path.
revision (Optional[str]) – revision to export. (default: None)

checkout(src, dst, revision=None)#

Runs svn checkout.

Parameters:

src (Union[str, Path]) – from where to check out.
dst (Union[str, Path]) – destination path.
revision (Optional[str]) – revision to check out. (default: None)

update(dst, revision=None)#

Runs svn checkout.

Parameters:

dst (Union[str, Path]) – destination path.
revision (Optional[str]) – revision to check out. (default: None)

merge(src, dst, revision=None)#

Runs svn merge.

Parameters:

src (Union[str, Path]) – the src URI.
dst (Union[str, Path]) – destination path.
revision (Optional[str]) – revision to check out. (default: None)

class fab.tools.Tool(name, exec_name, category=Category.MISC, availablility_option=None)#

This is the base class for all tools. It stores the name of the tool, the name of the executable, and provides a run method.

Parameters:

name (str) – name of the tool.
exec_name (Union[str, Path]) – name or full path of the executable to start.
category (Category) – the Category to which this tool belongs. (default: <Category.MISC: 12>)
availability_option – a command line option for the tool to test if the tool is available on the current system. Defaults to –version.

check_available()#: Run a ‘test’ command to check if this tool is available in the system. :rtype: bool :returns: whether the tool is working (True) or not.

property is_available: bool#

Checks if the tool is available or not. It will call a tool-specific function check_available to determine this, but will cache the results to avoid testing a tool more than once.

Returns:: whether the tool is available (i.e. installed and working).

property is_compiler: bool#: Returns whether this tool is a (Fortran or C) compiler or not.

property exec_name: str#

Returns:: the name of the executable.

property name: str#

Returns:: the name of the tool.

property category: Category#

Returns:: the category of this tool.

property flags: Flags#

Returns:: the flags to be used with this tool.

property logger: Logger#

Returns:: a logger object for convenience.

run(additional_parameters=None, env=None, cwd=None, capture_output=True)#

Run the binary as a subprocess.

Parameters:

additional_parameters (Union[str, List[Union[Path, str]], None]) – List of strings or paths to be sent to subprocess.run() as additional parameters for the command. Any path will be converted to a normal string. (default: None)
env (Optional[Dict[str, str]]) – Optional env for the command. By default it will use the current session’s environment. (default: None)
capture_output – If True, capture and return stdout. If False, the command will print its output directly to the console. (default: True)

Raises:

RuntimeError – if the code is not available.
RuntimeError – if the return code of the executable is not 0.

Return type:

str

class fab.tools.ToolBox#

This class implements the tool box. It stores one tool for each category to be used in a FAB build.

add_tool(tool, silent_replace=False)#

Adds a tool for a given category.

Parameters:

tool (Tool) – the tool to add.
silent_replace (bool) – if set, no warning will be printed if an existing tool is replaced. (default: False)

Raises:

RuntimeError – if the tool to be added is not available.

Return type:

None

get_tool(category, mpi=None, openmp=None)#

Returns the tool for the specified category.

Parameters:

category (Category) – the name of the category in which to look for the tool.
mpi (Optional[bool]) – if no compiler or linker is explicitly specified in this tool box, use the MPI and OpenMP setting to find an appropriate default from the tool repository. (default: None)
mpi – if no compiler or linker is explicitly specified in this tool box, use the MPI and OpenMP setting to find an appropriate default from the tool repository.

Raises:

KeyError – if the category is not known.

Return type:

Tool

class fab.tools.ToolRepository#

This class implements the tool repository. It stores a list of tools for various categories. For each compiler, it will automatically create a tool called “linker-{compiler-name}” which can be used for linking with the specified compiler.

Singleton access. Changes the value of _singleton so that the constructor can verify that it is indeed called from here.

add_tool(cls)#

Creates an instance of the specified class and adds it to the tool repository.

Parameters:: cls (Type[Any]) – the tool to instantiate.

get_tool(category, name)#

Return type:

Tool

Returns:

the tool with a given name in the specified category.

Parameters:

category (Category) – the name of the category in which to look for the tool.
name (str) – the name of the tool to find.

Raises:

KeyError – if there is no tool in this category.
KeyError – if no tool in the given category has the requested name.

set_default_compiler_suite(suite)#

Sets the default for linker and compilers to be of the given compiler suite.

Parameters:: suite (str) – the name of the compiler suite to make the default.

get_default(category, mpi=None, openmp=None)#

Returns the default tool for a given category. For most tools that will be the first entry in the list of tools. The exception are compilers and linker: in this case it must be specified if MPI support is required or not. And the default return will be the first tool that either supports MPI or not.

Parameters:

category (Category) – the category for which to return the default tool.
mpi (Optional[bool]) – if a compiler or linker is required that supports MPI. (default: None)
open – if a compiler or linker is required that supports OpenMP.

Raises:

KeyError – if the category does not exist.
RuntimeError – if no compiler/linker is found with the requested level of MPI support (yes or no).

class fab.tools.Versioning(name, exec_name, category)#

Base class for versioning tools like Git and Subversion.

Constructor.

Parameters:

name (str) – Display name of this tool.
exec_name (Union[str, Path]) – Executable for this tool.
category (Category) – Tool belongs to this category.

Modules

`ar`	This file contains the Ar class for archiving files.
`category`	This simple module defines an Enum for all allowed categories.
`compiler`	This file contains the base class for any compiler, and derived classes for gcc, gfortran, icc, ifort
`flags`	This file contains a simple Flag class to manage tool flags.
`linker`	This file contains the base class for any Linker.
`preprocessor`	This file contains the base class for any preprocessor, and two derived classes for cpp and fpp.
`psyclone`	This file contains the tool class for PSyclone.
`rsync`	This file contains the Rsync class for synchronising file trees.
`tool`	This file contains the base class for all tools, i.e. compiler, preprocessor, linker, archiver, Psyclone, rsync, versioning tools.
`tool_box`	This file contains the ToolBox class.
`tool_repository`	This file contains the ToolRepository class.
`versioning`	Versioning tools such as Subversion and Git.