Node - Parsed YAML configuration

This module contains the building blocks for handling YAML configuration.

Everything that is loaded from YAML is encapsulated in such nodes, which provide helper methods to validate configuration on access.

Using node methods when reading configuration will ensure that errors are always coherently notified to the user.

Note

Plugins are not expected to handle exceptions thrown by node methods for the above reason; They are private. There should always be a way to acquire information without resorting to exception handling.

Node types

The most important classes defined here are:

  • MappingNode: represents a YAML Mapping (dictionary)

  • ScalarNode: represents a YAML Scalar (string, boolean, integer)

  • SequenceNode: represents a YAML Sequence (list)

Class Reference

class MappingNode

Bases: Node

This class represents a Mapping (dict) in a YAML document.

It behaves mostly like a dict, but doesn’t allow untyped value access (Nothing of the form my_dict[my_value].

It also doesn’t allow anything else than str as keys, to align with YAML.

You can however use common dict operations in it:

# Assign a new value to a key
my_mapping[key] = my_value

# Delete an entry
del my_mapping[key]

When assigning a key/value pair, the key must be a string, and the value can be any of:

  • a Node, in which case the node is just assigned like normally

  • a list, dict, int, str, bool or None. In which case, the value will be converted to a Node for you.

Therefore, all values in a MappingNode will be Node.

Note

You should never create an instance directly. Use Node.from_dict() instead, which will ensure your node is correctly formatted.

clone()

Clone the node and return the copy.

Returns

a clone of the current node

Return type

Node

get_bool(key, default=sentinel)

Get the value of the node for key as a boolean.

This is equivalent to: mapping.get_scalar(my_key, my_default).as_bool().

Parameters
  • key (str) – key for which to get the value

  • default (bool) – default value to return if key is not in the mapping

Raises

buildstream._exceptions.LoadError – if the value at key is not a ScalarNode or isn’t a valid boolean

Returns

the value at key or the default

Return type

bool

get_enum()

Get the value of the node as an enum member from constraint

Parameters
  • key (str) – key for which to get the value

  • constraint (buildstream.types.FastEnum or Enum) – an enum from which to extract the value for the current node.

  • default (object) – default value to return if key is not in the mapping

Raises

buildstream._exceptions.LoadError – if the value is not is not found or not part of the provided enum.

Returns

the value contained in the node, as a member of

constraint

Return type

buildstream.types.Enum or Enum

get_int(key, default=sentinel)

Get the value of the node for key as an integer.

This is equivalent to: mapping.get_scalar(my_key, my_default).as_int().

Parameters
  • key (str) – key for which to get the value

  • default (int, None) – default value to return if key is not in the mapping

Raises

buildstream._exceptions.LoadError – if the value at key is not a ScalarNode or isn’t a valid integer

Returns

the value at key or the default

Return type

int or None

get_mapping(key, default=sentinel)

Get the value of the node for key as a MappingNode.

Parameters
  • key (str) – key for which to get the value

  • default (dict) – default value to return if key is not in the mapping. It will be converted to a MappingNode before being returned

Raises

buildstream._exceptions.LoadError – if the value at key is not a MappingNode

Returns

the value at key or the default

Return type

MappingNode

get_node(key, allowed_types=None, allow_none=False)

Get the value of the node for key as a Node.

This is useful if you have configuration that can be either a ScalarNode or a MappingNode for example.

This method will validate that the value is indeed exactly one of those types (not a subclass) and raise an exception accordingly.

Parameters
  • key (str) – key for which to get the value

  • allowed_types (list) – list of valid subtypes of Node that are valid return values. If this is None, no checks are done on the return value.

  • allow_none (bool) – whether to allow the return value to be None or not

Raises

buildstream._exceptions.LoadError – if the value at key is not one of the expected types or if it doesn’t exist.

Returns

the value at key or None

Return type

Node

get_scalar(key, default=sentinel)

Get the value of the node for key as a ScalarNode.

Parameters
  • key (str) – key for which to get the value

  • default (str, int, bool, None) – default value to return if key is not in the mapping. It will be converted to a ScalarNode before being returned.

Raises

buildstream._exceptions.LoadError – if the value at key is not a MappingNode

Returns

the value at key or the default

Return type

ScalarNode

get_sequence(key, default=sentinel)

Get the value of the node for key as a SequenceNode.

Parameters
  • key (str) – key for which to get the value

  • default (list) – default value to return if key is not in the mapping. It will be converted to a SequenceNode before being returned

  • allowed_types (list) – list of valid subtypes of Node that are valid for nodes in the sequence.

Raises

buildstream._exceptions.LoadError – if the value at key is not a SequenceNode

Returns

the value at key or the default

Return type

SequenceNode

get_str(key, default=sentinel)

Get the value of the node for key as an string.

This is equivalent to: mapping.get_scalar(my_key, my_default).as_str().

Parameters
  • key (str) – key for which to get the value

  • default (str) – default value to return if key is not in the mapping

Raises

buildstream._exceptions.LoadError – if the value at key is not a ScalarNode or isn’t a valid str

Returns

the value at key or the default

Return type

str

get_str_list(key, default=sentinel)

Get the value of the node for key as a list of strings.

This is equivalent to: mapping.get_sequence(my_key, my_default).as_str_list().

Parameters
  • key (str) – key for which to get the value

  • default (str) – default value to return if key is not in the mapping

Raises

buildstream._exceptions.LoadError – if the value at key is not a SequenceNode or if any of its internal values is not a ScalarNode.

Returns

the value at key or the default

Return type

list

items()

Get a new view of the mapping items ((key, value) pairs).

This is equivalent to running my_dict.item() on a dict.

Returns

a view on the underlying dictionary

Return type

dict_items

keys()

Get the list of all keys in the mapping.

This is equivalent to running my_dict.keys() on a dict.

Returns

a list of all keys in the mapping

Return type

list

safe_del(key)

Remove the entry at key in the dictionary if it exists.

This method is a safe equivalent to del mapping[key], that doesn’t throw anything if the key doesn’t exist.

Parameters

key (str) – key to remove from the mapping

strip_node_info()

Remove all the node information (provenance) and return the underlying data as plain python objects

Returns

the underlying data that was held in the node structure.

Return type

(list, dict, str, None)

validate_keys(valid_keys)

Validate that the node doesn’t contain extra keys

This validates the node so as to ensure the user has not specified any keys which are unrecognized by BuildStream (usually this means a typo which would otherwise not trigger an error).

Parameters

valid_keys (list) – A list of valid keys for the specified node

Raises

buildstream._exceptions.LoadError – In the case that the specified node contained one or more invalid keys

values()

Get the values in the mapping.

This is equivalent to running my_dict.values() on a dict.

Returns

a list of all values in the mapping

Return type

dict_values

class Node

Bases: object

This is the base class for YAML document nodes.

YAML Nodes contain information to describe the provenance of the YAML which resulted in the Node, allowing mapping back from a Node to the place in the file it came from.

Note

You should never need to create a Node manually. If you do, you can create Node from dictionaries with Node.from_dict(). If something else is needed, please open an issue.

clone()

Clone the node and return the copy.

Returns

a clone of the current node

Return type

Node

from_dict(value)

Create a new node from the given dictionary.

This is a recursive operation, and will transform every value in the dictionary to a Node instance

Valid values for keys are str Valid values for values are list, dict, str, int, bool or None. list and dict can also only contain such types.

Parameters

value (dict) – dictionary from which to create a node.

Raises

TypeError – when the value cannot be converted to a Node

Returns

a new mapping containing the value

Return type

MappingNode

get_provenance()

A convenience accessor to obtain the node’s ProvenanceInformation

The provenance information allows you to inform the user of where a node came. Transforming the information to a string will show the file, line and column in the file where the node is.

An example usage would be:

# With `config` being your node
max_jobs_node = config.get_node('max-jobs')
max_jobs = max_jobs_node.as_int()

if max_jobs < 1:  # We can't get a negative number of jobs
    raise LoadError("Error at {}: Max jobs needs to be >= 1".format(
        max_jobs_node.get_provenance()
    )

# Will print something like:
# element.bst [line 4, col 7]: Max jobs needs to be >= 1
Returns

the provenance information for the node.

Return type

ProvenanceInformation

strip_node_info()

Remove all the node information (provenance) and return the underlying data as plain python objects

Returns

the underlying data that was held in the node structure.

Return type

(list, dict, str, None)

class ProvenanceInformation

Bases: object

Represents the location of a YAML node in a file.

This can effectively be used as a pretty print to display those information in errors consistently.

You can retrieve this information for a Node with Node.get_provenance()

class ScalarNode

Bases: Node

This class represents a Scalar (int, str, bool, None) in a YAML document.

Note

If you need to store another type of scalars, please open an issue on the project.

Note

You should never have to create a ScalarNode directly

as_bool()

Get the value of the node as a boolean.

Note

BuildStream treats the values ‘True’ and ‘true’ as True, and the values ‘False’ and ‘false’ as False. Any other string values (such as the valid YAML ‘TRUE’ or ‘FALSE’ will be considered as an error)

Raises

buildstream._exceptions.LoadError – if the value cannot be coerced to a bool correctly.

Returns

the value contained in the node, as a boolean

Return type

bool

as_enum()

Get the value of the node as an enum member from constraint

The constraint must be a buildstream.types.FastEnum or a plain python Enum.

For example you could do:

from buildstream.types import FastEnum

class SupportedCompressions(FastEnum):
  NONE = "none"
  GZIP = "gzip"
  XZ = "xz"


x = config.get_scalar('compress').as_enum(SupportedCompressions)

if x == SupportedCompressions.GZIP:
    print("Using GZIP")
Parameters

constraint (buildstream.types.FastEnum or Enum) – an enum from which to extract the value for the current node.

Returns

the value contained in the node, as a member of constraint

Return type

FastEnum or Enum

as_int()

Get the value of the node as an integer.

Raises

buildstream._exceptions.LoadError – if the value cannot be coerced to an integer correctly.

Returns

the value contained in the node, as a integer

Return type

int

as_str()

Get the value of the node as a string.

Returns

the value contained in the node, as a string, or None if the content

is None.

Return type

str

clone()

Clone the node and return the copy.

Returns

a clone of the current node

Return type

Node

is_none()

Determine whether the current scalar is None.

Returns

True if the value of the scalar is None, else False

Return type

bool

strip_node_info()

Remove all the node information (provenance) and return the underlying data as plain python objects

Returns

the underlying data that was held in the node structure.

Return type

(list, dict, str, None)

class SequenceNode

Bases: Node

This class represents a Sequence (list) in a YAML document.

It behaves mostly like a list, but doesn’t allow untyped value access (Nothing of the form my_list[my_value]).

You can however perform common list operations on it:

# Assign a value
my_sequence[key] = value

# Get the length
len(my_sequence)

# Reverse it
reversed(my_sequence)

# And iter over it
for value in my_sequence:
    print(value)

All values in a SequenceNode will be Node.

append(value)

Append the given object to the sequence.

Parameters

value (object) –

the value to append to the list. This can either be:

  • a Node

  • a int, bool, str, None, dict or list. In which case, this will be converted into a Node beforehand

Raises

TypeError – when the value cannot be converted to a Node

as_str_list()

Get the values of the sequence as a list of strings.

Raises

buildstream._exceptions.LoadError – if the sequence contains more than ScalarNode

Returns

the content of the sequence as a list of strings

Return type

list

clone()

Clone the node and return the copy.

Returns

a clone of the current node

Return type

Node

mapping_at(index)

Retrieve the entry at index as a MappingNode.

Parameters

index (int) – index for which to get the value

Raises
  • buildstream._exceptions.LoadError – if the value at key is not a MappingNode

  • IndexError – if no value exists at this index

Returns

the value at index

Return type

MappingNode

node_at(index, allowed_types=None)

Retrieve the entry at index as a Node.

This is useful if you have configuration that can be either a ScalarNode or a MappingNode for example.

This method will validate that the value is indeed exactly one of those types (not a subclass) and raise an exception accordingly.

Parameters
  • index (int) – index for which to get the value

  • allowed_types (list) – list of valid subtypes of Node that are valid return values. If this is None, no checks are done on the return value.

Raises
  • buildstream._exceptions.LoadError – if the value at index is not of one of the expected types

  • IndexError – if no value exists at this index

Returns

the value at index

Return type

Node

scalar_at(index)

Retrieve the entry at index as a ScalarNode.

Parameters

index (int) – index for which to get the value

Raises
  • buildstream._exceptions.LoadError – if the value at key is not a ScalarNode

  • IndexError – if no value exists at this index

Returns

the value at index

Return type

ScalarNode

sequence_at(index)

Retrieve the entry at index as a SequenceNode.

Parameters

index (int) – index for which to get the value

Raises
  • buildstream._exceptions.LoadError – if the value at key is not a SequenceNode

  • IndexError – if no value exists at this index

Returns

the value at index

Return type

SequenceNode

strip_node_info()

Remove all the node information (provenance) and return the underlying data as plain python objects

Returns

the underlying data that was held in the node structure.

Return type

(list, dict, str, None)