Node - Parsed YAML configuration

This module contains the building blocks for handling YAML configuration.

Everything that is loaded from YAML is encapsulated in such nodes, which provide helper methods to validate configuration on access.

Using node methods when reading configuration will ensure that errors are always coherently notified to the user.

Note

Plugins are not expected to handle exceptions thrown by node methods for the above reason; They are private. There should always be a way to acquire information without resorting to exception handling.

Node types

The most important classes defined here are:

MappingNode: represents a YAML Mapping (dictionary)
ScalarNode: represents a YAML Scalar (string, boolean, integer)
SequenceNode: represents a YAML Sequence (list)

Class Reference

class MappingNode

Bases: Node

This class represents a Mapping (dict) in a YAML document.

It behaves mostly like a dict, but doesn’t allow untyped value access (Nothing of the form my_dict[my_value].

It also doesn’t allow anything else than str as keys, to align with YAML.

You can however use common dict operations in it:

# Assign a new value to a key
my_mapping[key] = my_value

# Delete an entry
del my_mapping[key]

When assigning a key/value pair, the key must be a string, and the value can be any of:

a Node, in which case the node is just assigned like normally
a list, dict, int, str, bool or None. In which case, the value will be converted to a Node for you.

Therefore, all values in a MappingNode will be Node.

Note

You should never create an instance directly. Use Node.from_dict() instead, which will ensure your node is correctly formatted.

clone()

Clone the node and return the copy.

Returns:: a clone of the current node
Return type:: Node

get_bool(key, default=sentinel)

Get the value of the node for key as a boolean.

This is equivalent to: mapping.get_scalar(my_key, my_default).as_bool().

Parameters:

key (str) – key for which to get the value
default (bool) – default value to return if key is not in the mapping

Raises:

buildstream._exceptions.LoadError – if the value at key is not a ScalarNode or isn’t a valid boolean

Returns:

the value at key or the default

Return type:

bool

get_enum()

Get the value of the node as an enum member from constraint

Parameters:

key (str) – key for which to get the value
constraint (buildstream.types.FastEnum or Enum) – an enum from which to extract the value for the current node.
default (object) – default value to return if key is not in the mapping

Raises:

buildstream._exceptions.LoadError – if the value is not is not found or not part of the provided enum.

Returns:

the value contained in the node, as a member of: constraint

Return type:

buildstream.types.Enum or Enum

get_int(key, default=sentinel)

Get the value of the node for key as an integer.

This is equivalent to: mapping.get_scalar(my_key, my_default).as_int().

Parameters:

key (str) – key for which to get the value
default (int, None) – default value to return if key is not in the mapping

Raises:

buildstream._exceptions.LoadError – if the value at key is not a ScalarNode or isn’t a valid integer

Returns:

the value at key or the default

Return type:

int or None

get_mapping(key, default=sentinel)

Get the value of the node for key as a MappingNode.

Parameters:

key (str) – key for which to get the value
default (dict) – default value to return if key is not in the mapping. It will be converted to a MappingNode before being returned

Raises:

buildstream._exceptions.LoadError – if the value at key is not a MappingNode

Returns:

the value at key or the default

Return type:

MappingNode

get_node(key, allowed_types=None, allow_none=False)

Get the value of the node for key as a Node.

This is useful if you have configuration that can be either a ScalarNode or a MappingNode for example.

This method will validate that the value is indeed exactly one of those types (not a subclass) and raise an exception accordingly.

Parameters:

key (str) – key for which to get the value
allowed_types (list) – list of valid subtypes of Node that are valid return values. If this is None, no checks are done on the return value.
allow_none (bool) – whether to allow the return value to be None or not

Raises:

buildstream._exceptions.LoadError – if the value at key is not one of the expected types or if it doesn’t exist.

Returns:

the value at key or None

Return type:

Node

get_scalar(key, default=sentinel)

Get the value of the node for key as a ScalarNode.

Parameters:

key (str) – key for which to get the value
default (str, int, bool, None) – default value to return if key is not in the mapping. It will be converted to a ScalarNode before being returned.

Raises:

buildstream._exceptions.LoadError – if the value at key is not a MappingNode

Returns:

the value at key or the default

Return type:

ScalarNode

get_sequence(key, default=sentinel)

Get the value of the node for key as a SequenceNode.

Parameters:

key (str) – key for which to get the value
default (list) – default value to return if key is not in the mapping. It will be converted to a SequenceNode before being returned
allowed_types (list) – list of valid subtypes of Node that are valid for nodes in the sequence.

Raises:

buildstream._exceptions.LoadError – if the value at key is not a SequenceNode

Returns:

the value at key or the default

Return type:

SequenceNode

get_str(key, default=sentinel)

Get the value of the node for key as an string.

This is equivalent to: mapping.get_scalar(my_key, my_default).as_str().

Parameters:

key (str) – key for which to get the value
default (str) – default value to return if key is not in the mapping

Raises:

buildstream._exceptions.LoadError – if the value at key is not a ScalarNode or isn’t a valid str

Returns:

the value at key or the default

Return type:

str

get_str_list(key, default=sentinel)

Get the value of the node for key as a list of strings.

This is equivalent to: mapping.get_sequence(my_key, my_default).as_str_list().

Parameters:

key (str) – key for which to get the value
default (str) – default value to return if key is not in the mapping

Raises:

buildstream._exceptions.LoadError – if the value at key is not a SequenceNode or if any of its internal values is not a ScalarNode.

Returns:

the value at key or the default

Return type:

list

items()

Get a new view of the mapping items ((key, value) pairs).

This is equivalent to running my_dict.item() on a dict.

Returns:: a view on the underlying dictionary
Return type:: dict_items

keys()

Get the list of all keys in the mapping.

This is equivalent to running my_dict.keys() on a dict.

Returns:: a list of all keys in the mapping
Return type:: list

safe_del(key)

Remove the entry at key in the dictionary if it exists.

This method is a safe equivalent to del mapping[key], that doesn’t throw anything if the key doesn’t exist.

Parameters:: key (str) – key to remove from the mapping

strip_node_info()

Remove all the node information (provenance) and return the underlying data as plain python objects

Returns:: the underlying data that was held in the node structure.
Return type:: (list, dict, str, None)

validate_keys(valid_keys)

Validate that the node doesn’t contain extra keys

This validates the node so as to ensure the user has not specified any keys which are unrecognized by BuildStream (usually this means a typo which would otherwise not trigger an error).

Parameters:: valid_keys (list) – A list of valid keys for the specified node
Raises:: buildstream._exceptions.LoadError – In the case that the specified node contained one or more invalid keys

values()

Get the values in the mapping.

This is equivalent to running my_dict.values() on a dict.

Returns:: a list of all values in the mapping
Return type:: dict_values

class Node

Bases: object

This is the base class for YAML document nodes.

YAML Nodes contain information to describe the provenance of the YAML which resulted in the Node, allowing mapping back from a Node to the place in the file it came from.

Note

You should never need to create a Node manually. If you do, you can create Node from dictionaries with Node.from_dict(). If something else is needed, please open an issue.

clone()

Clone the node and return the copy.

Returns:: a clone of the current node
Return type:: Node

from_dict(value)

Create a new node from the given dictionary.

This is a recursive operation, and will transform every value in the dictionary to a Node instance

Valid values for keys are str Valid values for values are list, dict, str, int, bool or None. list and dict can also only contain such types.

Parameters:: value (dict) – dictionary from which to create a node.
Raises:: TypeError – when the value cannot be converted to a Node
Returns:: a new mapping containing the value
Return type:: MappingNode

get_provenance()

A convenience accessor to obtain the node’s ProvenanceInformation

The provenance information allows you to inform the user of where a node came. Transforming the information to a string will show the file, line and column in the file where the node is.

An example usage would be:

# With `config` being your node
max_jobs_node = config.get_node('max-jobs')
max_jobs = max_jobs_node.as_int()

if max_jobs < 1:  # We can't get a negative number of jobs
    raise LoadError("Error at {}: Max jobs needs to be >= 1".format(
        max_jobs_node.get_provenance()
    )

# Will print something like:
# element.bst [line 4, col 7]: Max jobs needs to be >= 1

Returns:: the provenance information for the node.
Return type:: ProvenanceInformation

strip_node_info()

Remove all the node information (provenance) and return the underlying data as plain python objects

Returns:: the underlying data that was held in the node structure.
Return type:: (list, dict, str, None)

class ProvenanceInformation

Bases: object

Represents the location of a YAML node in a file.

This can effectively be used as a pretty print to display those information in errors consistently.

You can retrieve this information for a Node with Node.get_provenance()

class ScalarNode

Bases: Node

This class represents a Scalar (int, str, bool, None) in a YAML document.

Note

If you need to store another type of scalars, please open an issue on the project.

Note

You should never have to create a ScalarNode directly

as_bool()

Get the value of the node as a boolean.

Note

BuildStream treats the values ‘True’ and ‘true’ as True, and the values ‘False’ and ‘false’ as False. Any other string values (such as the valid YAML ‘TRUE’ or ‘FALSE’ will be considered as an error)

Raises:: buildstream._exceptions.LoadError – if the value cannot be coerced to a bool correctly.
Returns:: the value contained in the node, as a boolean
Return type:: bool

as_enum()

Get the value of the node as an enum member from constraint

The constraint must be a buildstream.types.FastEnum or a plain python Enum.

For example you could do:

from buildstream.types import FastEnum

class SupportedCompressions(FastEnum):
  NONE = "none"
  GZIP = "gzip"
  XZ = "xz"


x = config.get_scalar('compress').as_enum(SupportedCompressions)

if x == SupportedCompressions.GZIP:
    print("Using GZIP")

Parameters:: constraint (buildstream.types.FastEnum or Enum) – an enum from which to extract the value for the current node.
Returns:: the value contained in the node, as a member of constraint
Return type:: FastEnum or Enum

as_int()

Get the value of the node as an integer.

Raises:: buildstream._exceptions.LoadError – if the value cannot be coerced to an integer correctly.
Returns:: the value contained in the node, as a integer
Return type:: int

as_str()

Get the value of the node as a string.

Returns:

the value contained in the node, as a string, or None if the content: is None.

Return type:

str

clone()

Clone the node and return the copy.

Returns:: a clone of the current node
Return type:: Node

is_none()

Determine whether the current scalar is None.

Returns:: True if the value of the scalar is None, else False
Return type:: bool

strip_node_info()

Remove all the node information (provenance) and return the underlying data as plain python objects

Returns:: the underlying data that was held in the node structure.
Return type:: (list, dict, str, None)

class SequenceNode

Bases: Node

This class represents a Sequence (list) in a YAML document.

It behaves mostly like a list, but doesn’t allow untyped value access (Nothing of the form my_list[my_value]).

You can however perform common list operations on it:

# Assign a value
my_sequence[key] = value

# Get the length
len(my_sequence)

# Reverse it
reversed(my_sequence)

# And iter over it
for value in my_sequence:
    print(value)

All values in a SequenceNode will be Node.

append(value)

Append the given object to the sequence.

Parameters:

value (object) –

the value to append to the list. This can either be:

a Node
a int, bool, str, None, dict or list. In which case, this will be converted into a Node beforehand

Raises:

TypeError – when the value cannot be converted to a Node

as_str_list()

Get the values of the sequence as a list of strings.

Raises:: buildstream._exceptions.LoadError – if the sequence contains more than ScalarNode
Returns:: the content of the sequence as a list of strings
Return type:: list

clone()

Clone the node and return the copy.

Returns:: a clone of the current node
Return type:: Node

mapping_at(index)

Retrieve the entry at index as a MappingNode.

Parameters:

index (int) – index for which to get the value

Raises:

buildstream._exceptions.LoadError – if the value at key is not a MappingNode
IndexError – if no value exists at this index

Returns:

the value at index

Return type:

MappingNode

node_at(index, allowed_types=None)

Retrieve the entry at index as a Node.

This is useful if you have configuration that can be either a ScalarNode or a MappingNode for example.

This method will validate that the value is indeed exactly one of those types (not a subclass) and raise an exception accordingly.

Parameters:

index (int) – index for which to get the value
allowed_types (list) – list of valid subtypes of Node that are valid return values. If this is None, no checks are done on the return value.

Raises:

buildstream._exceptions.LoadError – if the value at index is not of one of the expected types
IndexError – if no value exists at this index

Returns:

the value at index

Return type:

Node

scalar_at(index)

Retrieve the entry at index as a ScalarNode.

Parameters:

index (int) – index for which to get the value

Raises:

buildstream._exceptions.LoadError – if the value at key is not a ScalarNode
IndexError – if no value exists at this index

Returns:

the value at index

Return type:

ScalarNode

sequence_at(index)

Retrieve the entry at index as a SequenceNode.

Parameters:

index (int) – index for which to get the value

Raises:

buildstream._exceptions.LoadError – if the value at key is not a SequenceNode
IndexError – if no value exists at this index

Returns:

the value at index

Return type:

SequenceNode

strip_node_info()

Remove all the node information (provenance) and return the underlying data as plain python objects

Returns:: the underlying data that was held in the node structure.
Return type:: (list, dict, str, None)