FUTILE Python modules

This modules are provided with the library to ease the interoperability between I/O of the Fortran library with python scripting

Utils module

This file contains some low-level useful functions

futile.Utils.dict_merge(dest, src)[source]

Recursive dict merge. Inspired by dict.update(), instead of updating only top-level keys, dict_merge recurses down into dicts nested to an arbitrary depth, updating keys. The src is merged into dest. From angstwad/dict-merge.py

Parameters:
  • dest (dict) – dict onto which the merge is executed
  • src (dict) – dict merged into dest
futile.Utils.dict_set(inp, *subfields)[source]

Ensure the provided fields and set the value

Provide a entry point to the dictionary. Useful to define a key in a dictionary that may not have the previous keys already defined.

Parameters:
  • inp (dict) – the top-level dictionary
  • subfields (str,object) – keys, ordered by level, that have to be retrieved from topmost level of inp. The last item correspond to the value to be set .

Example

>>> inp={}
>>> dict_set(inp,'dft','nspin','mpol',2)
>>> print (inp)
{'dft': {'nspin': {'mpol': 2}}}
futile.Utils.ensure_copy(src, dest)[source]

Copy src into dest.

Guarantees that the file indicated by dest is a copy of the file src

Parameters:
  • src (str) – path of the source file. Should be valid.
  • dest (src) – path of the destination file
Returns:

True if the file needed to be copied, False if src and dest are identical

Return type:

bool

futile.Utils.ensure_dir(file_path)[source]

Guarantees the existance on the directory given by the (relative) file_path

Parameters:file_path (str) – path of thh directory to be created
Returns:True if the directory needed to be created, False if it existed already
Return type:bool
futile.Utils.file_time(filename)[source]

Gives the date of the creation of the file, if exists.

Parameters:filename (str) – name of the file
Returns:if the file exists, the date of the filename as per os.path.getmtime. Otherwise it returns 0
futile.Utils.find_files(regexp, archive=None)[source]

Returns a list of the paths to the files that follow the regular expression regexp. They are searched from the current working directory or from an archive given as optional argument.

Parameters:
  • regexp (string) – A regular expression
  • archive – an opened tarfile archive (optional)
Returns:

a list of all the paths that agree with the regexp

Return type:

list of strings

Raises:

ValueError if the regexp does not find a single path.

Example:

#Find all python files in the current working directory
find_files('*py')

#An exmple outside of the current working directory
find_files('*/log-*.yaml')

#Example using a tarfile
import tarfile
my_archive = tarfile.open('archive.tar.gz')
find_files('*/*/log-*.yaml', archive=my_archive)
futile.Utils.function_signature_regenerator(target_kwargs_function, fun_name='', fun_docstring='', **kwargs)[source]

Generate the function of the name provided by fun_name, with signature provided by the kwargs dictionary.

Parameters:
  • target_kwargs_function (func) – keyword arguments function that will be used for the generated function.
  • fun_name (str) – name of the regenerated function. If empty it will be the target_kwargs_functon.__name__ prefixed by regenerated, which will be copied in the docstring of the regenerated function.
  • fun_docstring (str) – docstring of the generated function, if empty it will take the docstring from target_kwargs_function.
  • **kwargs – keyword arguments which will represent the signature of the generated function.

Example

>>> def write_kwargs(**kwargs):
>>>     """
>>>     Convert keyword arguments into a string
>>>     """
>>>     return str(kwargs)
>>> write_opts=function_signature_regenerator(write_kwargs,fun_name='write_opts',opt1='default1',opt2='default2')
>>> help(write_opts)
>>> print (write_opts())
Help on function write_opts:
write_opts(opt1=’default1’, opt2=’default2’)
Convert keyword arguments into a string

{‘opt1’: ‘default1’, ‘opt2’: ‘default2’}

futile.Utils.kw_pop(*args, **kwargs)[source]

Treatment of kwargs. Eliminate from kwargs the tuple in args.

futile.Utils.make_dict(inp)[source]

Transform the instance inp into a python dictionary. If inp is already a dictionary, it perfroms a copy.

Parameters:inp (dict) – a instance of a Class which inherits from dict
Returns:the copy of the class, converted as a dictionary
Return type:dict
futile.Utils.option_line_generator(separator='--', **kwargs)[source]

Associate to each of the keyword arguments a command line argument.

Parameters:
  • separator (str) – The string needed to separate the options.
  • be '--' for command-line arguments, but also ',' for function signatures. (Might) –

Warning

The separator comes before the first argument therefore pay attention to lstrip it in case you want to use it as a function signature string.

Example

>>> option_line_generator(arg1='val1',arg2='val2')
'--arg1=val1 --arg2=val2'
futile.Utils.push_path(inp, *keys)[source]

Follow in the dictionary inp the path indicated by the keys. If this path does not exists creates it.

Parameters:
  • inp (dict) – dictionary
  • keys (str) – keys of the path to follow
Returns:

(branch,``key``) tuple, where

  • branch (dict): the dictionary of the second-last item of the path
  • key (str): the last item of the path

futile.Utils.write(*args, **kwargs)[source]

Wrapper for print function or print to ensure compatibility with python 2 The arguments are used similarly as the print_function They can also be generalized to python 2 cases

YamlIO module

futile.YamlIO.load(file=None, stream=None, doc_lists=True, safe_mode=False)[source]

Encapsulate the loading of yaml documents.

Provides a dictionary, or a list of dictionaries, which represents the structure of the stream to be loaded. It also wraps the yaml loader to perform a optimized parsing when the minloader of PyYaml 3.13 is available. This wrapper ensures to extract from the stream the maximum possible information by choosing the best loader available.

Parameters:
  • file (str) – path of the yaml-compliant file containing the stream to be loaded
  • stream (str) – the stream to load, overrides the file argument if present
  • doc_lists (bool) – if True, ensures that the results is always in a form of lists of documents, even in the case of a single doc When False, the return type is either a dictionary or a generator according to the specifications of yaml.load and yaml.load_all respectively.
  • safe_mode (bool) – When true, in the case of multiple documents in the stream, it loads the document one after another. This is useful to avoid losing of all the document list in the case when one of the document is not yaml compliant, like in the case of a broken logfile. It may works only when the separation of the documents is indicated by the usual syntax "---\n" (i.e. no yaml tags between documents)
Returns:

  • a list of dictionaries, if doc_lists is set to True;
  • a dictionary, if the stream or the file contains a single yaml document;
  • a generator if the parsed stream is made of multiple documents and safe_mode = False;
  • a list of dictionaries if the stream is made of multiple documents and safe_mode is True.

futile.YamlIO.dump(data, filename=None, raw=False, tar=False)[source]

Encapsulate the dumping of dictionaries.

This function is useful to dump a dictionary in yaml or json-compliant form. This may be used as an alternative to the usual yaml.dump method, especially when the dictionary to be dump’ed is heavy. No particular attention is paid in human readability of the output. The dumped information can then be parsed either from json or yaml interpreter.

Parameters:
  • data (dict,list) – the information to be dumped
  • filename (str) – path of the file in which the information will be stored. If absent, the information is written on sys.stdout().
  • raw (bool) – if True the output is in json-style, otherwise it is pre-processed by :py:meth:yaml.dump, but None is passed to default_flow_style.
  • tar (bool) – if True the filename is assumed to be a compressed tarfile. The tarfile module is used to create and append information.
futile.YamlIO.clean_logfile(logfile_lines, to_remove)[source]

Remove yaml fields from a list of lines.

Removes from a set of lines the yaml_fields contained in the to_remove list.

Parameters:
  • logfile_lines (list) – list of the lines of the logfile. Generated from a file by e.g. readlines().
  • to_remove (list) – list of keys to remove from logfile_lines
Returns:

list of lines where the removed keys have as values the “<folded>” string

Inputvars module

Handle the input variable specifications.

This module is the python complement to the specification of the input variables of a file as provided by the f_input_file module. It uses the same syntax as defined there and make possible the interplay between a python-based serialization of the input dictionary and the construction of

class futile.Inputvars.InputVariable(name, spec)[source]

Define a input variable of a library. Such object can be initialized and inspected from the dictionary used by futile and then initialized according to the provided specification.

is_valid()[source]
set(val)[source]

Set the value of the input variable

set_dependent_variable(var)[source]

Set the dependent variable from which impose the profile

set_master_variable(var)[source]

Set the variable which is activated when the present has suitable values or profiles

ArgParse module

Handle and automatize the parsing of input arguments

This module uses the same convention of the yaml_parse fortran module to define the command line arguments. Such module can be used to define command line arguments of python scripts that follows the same conventions or to generate python functions that have the same signature than the provided command arguments.

futile.YamlArgparse.get_python_function(target_kwargs_function, func_name, func_spec)[source]

Convert a argparse spec into a python function

This function provides a python function with a signature indicated by the fun_spec dictionary With the conventions of the yaml_argparse modules. The futile.Utils.function_signature_regenerator() function is used for the conversion

Parameters:
  • target_kwargs_function (func) – the keyword arguments function we want to give the signature to.
  • func_name (str) – Name of the function, usually the key of the dictionary whose func_spec is the value
  • func_spec (dict) – dictionary of the function specifications to be provided to the futile.Utils.function_signature_regenerator() function.
Returns:

the genreated function with signature given by the arguments of func_spec

defaulting to their default value.

Return type:

func

Todo

Create the docstring of the generated function by also including the docstring of the arguments

Figures module

class futile.Figures.AxisSet(fig, rect, facecolor=None, frameon=True, sharex=None, sharey=None, label=u'', xscale=None, yscale=None, **kwargs)[source]

Bases: matplotlib.axes._axes.Axes

cla()[source]
create_twin()[source]
classmethod twinify(ax)[source]

Include the axis provided as a radix of the set

class futile.Figures.FigureSet(**kwargs)[source]

Container for multiple figures.

Define a container for a plot with the possiblity to switch between simple and gnuplot plotting

Arguments: title: The title of the master figure **kwargs: arguments for the axis instance

add(**kwargs)[source]
exists(figname)[source]

True if the Figure exists in the Set

invoke(idx)[source]
show(figname=None)[source]
class futile.Figures.VertSlider(ax, label, valmin, valmax, valinit=0.5, valfmt='%1.2f', closedmin=True, closedmax=True, slidermin=None, slidermax=None, dragging=True, **kwargs)[source]

Bases: matplotlib.widgets.AxesWidget

A slider representing a floating point range.

For the slider to remain responsive you must maintain a reference to it.

*ax*

the slider matplotlib.axes.Axes instance

*val*

the current slider value

*hline*

a matplotlib.lines.Line2D instance representing the initial value of the slider

*poly*

A matplotlib.patches.Polygon instance which is the slider knob

*valfmt*

the format string for formatting the slider text

*label*

a matplotlib.text.Text instance for the slider label

*closedmin*

whether the slider is closed on the minimum

*closedmax*

whether the slider is closed on the maximum

*slidermin*

another slider - if not None, this slider must be greater than slidermin

*slidermax*

another slider - if not None, this slider must be less than slidermax

*dragging*

allow for mouse dragging on slider

Call on_changed() to connect to the slider event

disconnect(cid)[source]

remove the observer with connection id cid

on_changed(func)[source]

When the slider value is changed, call func with the new slider position

A connection id is returned which can be used to disconnect

reset()[source]

reset the slider to the initial value if needed

set_val(val)[source]
futile.Figures.axis_from_data(fig, ax, data)[source]

Transform a data tuple into axis coordinates

futile.Figures.data_from_data(fig, dst, src, data)[source]

Transform a data tuple of anothe axis in the figure into data of another axis

futile.Figures.show_image(imgfile, title=None)[source]

Show image file using matplotlib imgread. Useful to bypass the Jupyter bug for converting a notebook into a pdf file

Time module

class futile.Time.TimeData(*filenames, **kwargs)[source]
barwidth = 0.9
collect_categories(dict_list, vals)[source]

Collect all the categories which belong to all the dictionaries

counters()[source]

Inspect the available counters

draw_barfigure(fig, axis, data, title)[source]
draw_lineplot(ax, data, label)[source]
find_items(category, dict_list)[source]

For a given category find the items which have them

find_unbalanced(data)[source]

Determine lookup array of unbalanced categories

gnuplot_figure(lookup=None, epsfile=None, aggregate=None, select_category=None)[source]
ignored_counters = ['CPU parallelism', 'Routines timing and number of calls', 'SUMMARY', 'Report timestamp', 'Hostnames']
inspect_category(cat)[source]
inspect_counter(counter, unit=None)[source]
load_unbalancing(ax, dict, hosts)[source]

Extract the data for plotting the hostname balancings between different categories in bar chart

replot(label)[source]
routines_plot(index, event=None)[source]

Draw the plot of the routines level for the run identified by index

show()[source]
unbalanced(val)[source]

Criterion for unbalancing

workload_plot(index, event=None)[source]

Draw the plot of the workload of different classes for the run identified by index

class futile.Time.polar_axis(fig, ax, data)[source]
draw_polarplot()[source]
dump_timing_level(level, starting_point=None, ilev=0, theta=0, data=None)[source]

Inspect the first level of the given dictionary and dump the profile subroutines at this level

step = 5