Plugin¶

BuildStream supports third party plugins to define additional kinds of elements and sources.

Plugin Structure¶

A plugin should consist of a setuptools package that advertises contained plugins using entry points.

A plugin entry point must be a module that extends a class in the Core Framework to be discovered by BuildStream. A YAML file defining plugin default settings with the same name as the module can also be defined in the same directory as the plugin module.

Note

BuildStream does not support function/class entry points.

A sample plugin could be structured as such:

.
├── elements
│   ├── autotools.py
│   ├── autotools.yaml
│   └── __init__.py
├── MANIFEST.in
└── setup.py

The setuptools configuration should then contain at least:

setup.py:

from setuptools import setup, find_packages

setup(name='BuildStream Autotools',
      version="0.1",
      description="A better autotools element for BuildStream",
      packages=find_packages(),
      install_requires=[
          'setuptools'
      ],
      include_package_data=True,
      entry_points={
          'buildstream.plugins': [
              'autotools = elements.autotools'
          ]
      })

MANIFEST.in:

global-include *.yaml

Class Reference¶

class Plugin¶

Bases: object

Base Plugin class.

Some common features to both Sources and Elements are found in this class.

Note

Derivation of plugins is not supported. Plugins may only derive from the base Source and Element types, and any convenience subclasses (like BuildElement) which are included in the buildstream namespace.

BST_REQUIRED_VERSION_MAJOR = 0¶: Minimum required major version

BST_REQUIRED_VERSION_MINOR = 0¶: Minimum required minor version

BST_FORMAT_VERSION = 0¶

The plugin’s YAML format version

This should be set to 1 the first time any new configuration is understood by your configure() implementation and subsequently bumped every time your configuration is enhanced.

Note

Plugins are expected to maintain backward compatibility in the format and configurations they expose. The versioning is intended to track availability of new features only.

name = None¶

The plugin name

For elements, this is the project relative bst filename, for sources this is the owning element’s name with a suffix indicating it’s index on the owning element.

For sources this is for display purposes only.

get_kind()¶

Fetches the kind of this plugin

Returns:	The kind of this plugin
Return type:	(str)

node_items(node)¶

Iterate over a dictionary loaded from YAML

Parameters:	node (dict) – The YAML loaded dictionary object
Returns:	List of key/value tuples to iterate over
Return type:	list

BuildStream holds some private data in dictionaries loaded from the YAML in order to preserve information to report in errors.

This convenience function should be used instead of the dict.items() builtin function provided by python.

node_provenance(node, member_name=None)¶

Gets the provenance for node and member_name

This reports a string with file, line and column information suitable for reporting an error or warning.

Parameters:	node (dict) – The YAML loaded dictionary object member_name (str) – The name of the member to check, or None for the node itself
Returns:	A string describing the provenance of the node and member
Return type:	(str)

node_get_member(node, expected_type, member_name, default=None)¶

Fetch the value of a node member, raising an error if the value is missing or incorrectly typed.

Parameters:	node (dict) – A dictionary loaded from YAML expected_type (type) – The expected type of the node member member_name (str) – The name of the member to fetch default (expected_type) – A value to return when member_name is not specified in node
Returns:	The value of member_name in node, otherwise default
Raises:	`LoadError` – When member_name is not found and no default was provided

Note

Returned strings are stripped of leading and trailing whitespace

Example:

# Expect a string 'name' in 'node'
name = self.node_get_member(node, str, 'name')

# Fetch an optional integer
level = self.node_get_member(node, int, 'level', -1)

node_validate(node, valid_keys)¶

This should be used in configure() implementations to assert that users have only entered valid configuration keys.

Parameters:	node (dict) – A dictionary loaded from YAML valid_keys (iterable) – A list of valid keys for the node
Raises:	`LoadError` – When an invalid key is found

Example:

# Ensure our node only contains valid autotools config keys
self.node_validate(node, [
    'configure-commands', 'build-commands',
    'install-commands', 'strip-commands'
])

node_get_list_element(node, expected_type, member_name, indices)¶

Fetch the value of a list element from a node member, raising an error if the value is incorrectly typed.

Parameters:	node (dict) – A dictionary loaded from YAML expected_type (type) – The expected type of the node member member_name (str) – The name of the member to fetch indices (list of int) – List of indices to search, in case of nested lists
Returns:	The value of the list element in member_name at the specified indices
Raises:	`LoadError`

Note

Returned strings are stripped of leading and trailing whitespace

Example:

# Fetch the list itself
things = self.node_get_member(node, list, 'things')

# Iterate over the list indices
for i in range(len(things)):

    # Fetch dict things
    thing = self.node_get_list_element(
        node, dict, 'things', [ i ])

configure(node)¶

Configure the Plugin from loaded configuration data

Parameters:	node (dict) – The loaded configuration dictionary
Raises:	`SourceError` – If its a `Source` implementation `ElementError` – If its an `Element` implementation `LoadError` – If one of the node handling methods fail

Plugin implementors should implement this method to read configuration data and store it. Use of the node_get_member() convenience method will ensure that a nice LoadError is triggered whenever the YAML input configuration is faulty.

Implementations may raise SourceError or ElementError for other errors.

Note

During configure, logging is suppressed unless buildstream is run with debugging output enabled.

preflight()¶

Preflight Check

Raises:	`SourceError` – If its a `Source` implementation `ElementError` – If its an `Element` implementation `ProgramNotFoundError` – If a required host tool is not found

This method is run after configure() and after the pipeline is fully constructed. Element plugins are free to use the dependencies() method and inspect public data at this time.

Implementors should simply raise SourceError or ElementError with an informative message in the case that the host environment is unsuitable for operation.

Plugins which require host tools (only sources usually) should obtain them with utils.get_host_tool() which will raise ProgramNotFoundError automatically.

get_unique_key()¶

Return something which uniquely identifies the plugin input

Returns:	A string, list or dictionary which uniquely identifies the sources to use

This is used to construct unique cache keys for elements and sources, sources should return something which uniquely identifies the payload, such as an sha256 sum of a tarball content. Elements should implement this by collecting any configurations which could possibly effect the output and return a dictionary of these settings.

debug(brief, *, detail=None)¶

Print a debugging message

Parameters:	brief (str) – The brief message detail (str) – An optional detailed message, can be multiline output

status(brief, *, detail=None)¶

Print a status message

Parameters:	brief (str) – The brief message detail (str) – An optional detailed message, can be multiline output

Note: Status messages tell about what a plugin is currently doing

info(brief, *, detail=None)¶

Print an informative message

Parameters:	brief (str) – The brief message detail (str) – An optional detailed message, can be multiline output

Note: Informative messages tell the user something they might want: to know, like if refreshing an element caused it to change.

warn(brief, *, detail=None)¶

Print a warning message

Parameters:	brief (str) – The brief message detail (str) – An optional detailed message, can be multiline output

log(brief, *, detail=None)¶

Log a message into the plugin’s log file

The message will not be shown in the master log at all (so it will not be displayed to the user on the console).

Parameters:	brief (str) – The brief message detail (str) – An optional detailed message, can be multiline output

timed_activity(activity_name, *, detail=None, silent_nested=False)¶

Context manager for performing timed activities in plugins

Parameters:	activity_name (str) – The name of the activity detail (str) – An optional detailed message, can be multiline output silent_nested (bool) – If specified, nested messages will be silenced

This function lets you perform timed tasks in your plugin, the core will take care of timing the duration of your task and printing start / fail / success messages.

Example

# Activity will be logged and timed
with self.timed_activity("Mirroring {}".format(self.url)):

    # This will raise SourceError on its own
    self.call(... command which takes time ...)

call(*popenargs, fail=None, **kwargs)¶

A wrapper for subprocess.call()

Parameters:	popenargs (list) – Popen() arguments fail (str) – A message to display if the process returns a non zero exit code rest_of_args (kwargs) – Remaining arguments to subprocess.call()
Returns:	The process exit code.
Return type:	(int)
Raises:	(`PluginError`) – If a non-zero return code is received and fail is specified

Note: If fail is not specified, then the return value of subprocess.call(): is returned even on error, and no exception is automatically raised.

Example

# Call some host tool
self.tool = utils.get_host_tool('toolname')
self.call(
    [self.tool, '--download-ponies', self.mirror_directory],
    "Failed to download ponies from {}".format(
        self.mirror_directory))

check_output(*popenargs, fail=None, **kwargs)¶

A wrapper for subprocess.check_output()

Parameters:	popenargs (list) – Popen() arguments fail (str) – A message to display if the process returns a non zero exit code rest_of_args (kwargs) – Remaining arguments to subprocess.call()
Returns:	The process exit code (str): The process standard output
Return type:	(int)
Raises:	(`PluginError`) – If a non-zero return code is received and fail is specified

Note: If fail is not specified, then the return value of subprocess.check_output(): is returned even on error, and no exception is automatically raised.

Example

# Get the tool at preflight time
self.tool = utils.get_host_tool('toolname')

# Call the tool, automatically raise an error
_, output = self.check_output(
    [self.tool, '--print-ponies'],
    "Failed to print the ponies in {}".format(
        self.mirror_directory),
    cwd=self.mirror_directory)

# Call the tool, inspect exit code
exit_code, output = self.check_output(
    [self.tool, 'get-ref', tracking],
    cwd=self.mirror_directory)

if exit_code == 128:
    return
elif exit_code != 0:
    fmt = "{plugin}: Failed to get ref for tracking: {track}"
    raise SourceError(
        fmt.format(plugin=self, track=tracking)) from e

Plugin¶

Plugin Structure¶

Class Reference¶

Table Of Contents

Related Topics