Declaring elements
Element basics
Here is a rather complete example using the autotools element kind and git source kind:
# Specify the kind of element this is
kind: autotools
# Specify some dependencies
depends:
- element1.bst
- element2.bst
# Specify the source which should be built
sources:
- kind: git
url: upstream:modulename.git
track: master
ref: d0b38561afb8122a3fc6bafc5a733ec502fcaed6
# Override some variables
variables:
sysconfdir: "%{prefix}/etc"
# Tweak the sandbox shell environment
environment:
LD_LIBRARY_PATH: /some/custom/path
# Specify the configuration of the element
config:
# Override autotools element default configure-commands
configure-commands:
- "%{configure} --enable-fancy-feature"
# Specify public domain data, visible to other elements.
public:
bst:
integration-commands:
- /usr/bin/update-fancy-feature-cache
# Specify a user id and group id to use in the build sandbox.
sandbox:
build-uid: 0
build-gid: 0
For most use cases you would not need to specify this much detail, we’ve provided details here in order to have a more complete initial example.
Let’s break down the above and give a brief explanation of what these attributes mean.
Kind
# Specify the kind of element this is
kind: autotools
The kind
attribute specifies which plugin will be operating on the element’s input to
produce its output. Plugins define element types and each of them can be referred to by
name with the kind
attribute.
To refer to a third party plugin, prefix the plugin with its package, for example:
kind: buildstream-plugins:dpkg_build
Depends
# Specify some dependencies
depends:
- element1.bst
- element2.bst
Relationships between elements are specified with the depends
attribute. Elements
may depend on other elements by specifying the element path
relative filename to the elements they depend on here.
See Dependencies for more information on the dependency model.
Build-Depends
# Specify some build-dependencies
build-depends:
- element1.bst
- element2.bst
Build dependencies between elements can be specified with the build-depends
attribute.
The above code snippet is equivalent to:
# Specify some build-dependencies
depends:
- filename: element1.bst
type: build
- filename: element2.bst
type: build
See Dependencies for more information on the dependency model.
Note
The build-depends
configuration is available since format version 14
Runtime-Depends
# Specify some runtime-dependencies
runtime-depends:
- element1.bst
- element2.bst
Runtime dependencies between elements can be specified with the runtime-depends
attribute.
The above code snippet is equivalent to:
# Specify some runtime-dependencies
depends:
- filename: element1.bst
type: runtime
- filename: element2.bst
type: runtime
See Dependencies for more information on the dependency model.
Note
The runtime-depends
configuration is available since format version 14
Sources
# Specify the source which should be built
sources:
- kind: git
url: upstream:modulename.git
track: master
ref: d0b38561afb8122a3fc6bafc5a733ec502fcaed6
Here we specify some input for the element, any number of sources may be specified.
By default the sources will be staged in the root of the element’s build directory
in the build sandbox, but sources may specify a directory
attribute to control
where the sources will be staged. The directory
attribute may specify a build
sandbox relative subdirectory.
For example, one might encounter a component which requires a separate data package in order to build itself, in this case the sources might be listed as:
sources:
# Specify the source which should be built
- kind: git
url: upstream:modulename.git
track: master
ref: d0b38561afb8122a3fc6bafc5a733ec502fcaed6
# Specify the data package we need for build frobnication,
# we need it to be unpacked in a src/frobdir
- kind: tarball
directory: src/frobdir
url: data:frobs.tgz
ref: 9d4b1147f8cf244b0002ba74bfb0b8dfb3...
Like Elements, Source types are plugins which are indicated by the kind
attribute.
Asides from the common kind
and directory
attributes which may be applied to all
Sources, refer to the Source specific documentation for meaningful attributes for the
particular Source.
Variables
# Override some variables
variables:
sysconfdir: "%{prefix}/etc"
Variables can be declared or overridden from an element. Variables can also be declared and overridden in the Project configuration
See Using variables below for a more in depth discussion on variables in BuildStream.
Environment
# Tweak the sandbox shell environment
environment:
LD_LIBRARY_PATH: /some/custom/path
Environment variables can be set to literal values here, these environment
variables will be effective in the Sandbox
where
build instructions are run for this element.
Environment variables can also be declared and overridden in the Project configuration
Config
# Specify the configuration of the element
config:
# Override autotools element default configure-commands
configure-commands:
- "%{configure} --enable-fancy-feature"
Here we configure the element itself. The autotools element provides sane defaults for
building sources which use autotools. Element default configurations can be overridden
in the project.conf
file and additionally overridden in the declaration of an element.
For meaningful documentation on what can be specified in the config
section for a given
element kind
, refer to the element specific documentation.
Public
# Specify public domain data, visible to other elements.
public:
bst:
integration-commands:
- /usr/bin/update-fancy-feature-cache
Metadata declared in the public
section of an element is visible to
any other element which depends on the declaring element in a given pipeline.
BuildStream itself consumes public data from the bst
domain. The integration-commands
demonstrated above for example, describe commands which should be run in an
environment where the given element is installed but before anything should be run.
An element is allowed to read domain data from any element it depends on, and users may specify additional domains to be understood and processed by their own element plugins.
The public data keys which are recognized under the bst
domain
can be viewed in detail in the builtin public data section.
Sandbox
Configuration for the build sandbox (other than environment variables)
can be placed in the sandbox
configuration. At present, only the
UID and GID used by the user in the group can be specified.
# Specify a user id and group id to use in the build sandbox.
sandbox:
build-uid: 1003
build-gid: 1001
BuildStream normally uses uid 0 and gid 0 (root) to perform all builds. However, the behaviour of certain tools depends on user id, behaving differently when run as non-root. To support those builds, you can supply a different uid or gid for the sandbox. Only bwrap-style sandboxes support custom user IDs at the moment, and hence this will only work on Linux host platforms.
Note
The sandbox
configuration is available since format version 6
Dependencies
The dependency model in BuildStream is simplified by treating software distribution and software building as separate problem spaces. This is to say that one element can only ever depend on another element but never on a subset of the product which another element produces.
In this section we’ll quickly go over the few features BuildStream offers in its dependency model.
Expressing dependencies
Dependencies in BuildStream are parameterizable objects, however as demonstrated in the above example, they can also be expressed as simple strings as a convenience shorthand in most cases, whenever the default dependency attributes are suitable.
Note
Note the order in which element dependencies are declared in the depends
,
build-depends
and runtime-depends
lists are not meaningful.
Dependency dictionary:
# Fully specified dependency
depends:
- filename: foo.bst
type: build
junction: baseproject.bst
strict: false
Attributes:
filename
The element path relative filename of the element to depend on in the project.
type
This attribute is used to express the dependency type. This field is not permitted in Build-Depends or Runtime-Depends.
junction
This attribute can be used to depend on elements in other projects.
If a junction is specified, then it must be an element path relative filename of the junction element in the project.
In the case that a junction is specified, the
filename
attribute indicates an element in the junctioned project.See
junction
.Note
The
junction
attribute is available since format version 1strict
This attribute can be used to specify that this element should be rebuilt when the dependency changes, even when strict mode has been turned off.
This is appropriate whenever a dependency’s output is consumed verbatim in the output of the depending element, for instance when static linking is in use.
Cross-junction dependencies
As mentioned above, cross-junction dependencies can be specified using the
junction
attribute. They can also be expressed as simple strings as a
convenience shorthand. You can refer to cross-junction elements using the
syntax {junction-name}:{element-name}
.
For example, the following is logically same as the example above:
build-depends:
- baseproject.bst:foo.bst
Similarly, you can also refer to cross-junction elements via the filename
attribute, like so:
depends:
- filename: baseproject.bst:foo.bst
type: build
Note
BuildStream does not allow recursice lookups for junction elements. If a
filename contains more than one :
(colon) character, an error will be
raised. See nested junctions for more details
on nested junctions.
Note
This shorthand is available since format version 15
Dependency types
The dependency type
attribute defines what the dependency is required for
and is essential to how BuildStream plots a build plan.
There are two types which one can specify for a dependency:
build
A
build
dependency type states that the given element’s product must be staged in order to build the depending element. Depending on an element which hasbuild
dependencies will not implicitly depend on that element’sbuild
dependencies.runtime
A
runtime
dependency type states that the given element’s product must be present for the depending element to function. An element’sruntime
dependencies need not be staged in order to build the element.
If type
is not specified, then it is assumed that the dependency is
required both at build time and runtime.
Note
It is assumed that a dependency which is required for building an
element must run while building the depending element. This means that
build
depending on a given element implies that that element’s
runtime
dependencies will also be staged for the purpose of building.
Using variables
Variables in BuildStream are a way to make your build instructions and element configurations more dynamic.
Referring to variables
Variables are expressed as %{...}
, where ...
must contain only
alphanumeric characters and the separators _
and -
. Further, the
first letter of ...
must be an alphabetic character.
This is release version %{version}
Declaring and overriding variables
To declare or override a variable, one need only specify a value in the relevant variables section:
variables:
hello: Hello World
You can refer to another variable while declaring a variable:
variables:
release-text: This is release version %{version}
The order in which you declare variables is arbitrary, so long as there is no cyclic dependency and that all referenced variables are declared, the following is fine:
variables:
release-text: This is release version %{version}
version: 5.5
Note
It should be noted that variable resolution only happens after all Element Composition has already taken place.
This is to say that overriding %{version}
at a higher priority will effect
the final result of %{release-text}
.
Example:
kind: autotools
# Declare variable, expect %{version} was already declared
variables:
release-text: This is release version %{version}
config:
# Customize the installation
install-commands:
- |
%{make-install} RELEASE_TEXT="%{release-text}"
Variables declared by BuildStream
BuildStream declares a set of builtin variables that may be overridden. In addition, the following read-only variables are also dynamically declared by BuildStream:
element-name
The name of the element being processed (e.g base/alpine.bst).
project-name
The name of project where BuildStream is being used.
max-jobs
Maximum number of parallel build processes within a given build, support for this is conditional on the element type and the build system used (any element using ‘make’ can implement this).