Commands

This page contains documentation for each BuildStream command, along with their possible options and arguments. Each command can be invoked on the command line, where, in most cases, this will be from the project’s main directory.

Commonly used parameters

Remotes

Remote cache servers can be specified on the command line for commands which may result in communicating with such servers.

Any command which has arguments to specify a REMOTE, such as --artifact-remote or --source-remote, will override whatever was set in the user configuration, and will have an accompanying switch which allows the command to decide whether to ignore any remote artifact or source caches suggested by project configuration.

Remotes can be specified on the command line either as a simple URI, or as a comma separated list of key value pairs.

Specifying a remote using a URI

bst artifact push --remote https://artifacts.com/artifacts:8088 element.bst

Specifying a remote using key value pairs

bst build --artifact-remote \
    url=https://artifacts.com/artifacts:8088,type=index,server-cert=~/artifacts.cert \
    element.bst

Attributes

Here is the list attributes which can be spefied when providing a REMOTE on the command line:

url

The URL of the remote, possibly including a port number.
instance-name

The instance name of this remote, used for sharding by some implementations.
type

Whether this remote is to be used for indexing, storage or both, as explained in the corresponding user configuration documentation
push

Normally one need not specify this, as it is often inferred by the command being used. In some cases, like bst build, it can be useful to specify multiple remotes, and only allow pushing to some of the remotes.

If unspecified, this is assumed to be True and BuildStream will attempt to push to the remote, but fallback to only pulling if insufficient credentials were provided.
server-cert, client-cert, client-key:

These keys specify the attributes of the authentication configuration.

When specifying these on the command line, they are interpreted as paths relative to the current working directory.

Top-level commands

bst

Build and manipulate BuildStream projects

Most of the main options override options in the user preferences configuration file.

bst [OPTIONS] COMMAND [ARGS]...

Options

--version

-c, --config <config>: Configuration file to use

-C, --directory <directory>: Project directory (default: current directory)

--on-error <on_error>

What to do when an error is encountered

Options:: continue | quit | terminate

--fetchers <fetchers>: Maximum simultaneous download tasks

--builders <builders>: Maximum simultaneous build tasks

--pushers <pushers>: Maximum simultaneous upload tasks

--max-jobs <max_jobs>: Number of parallel jobs allowed for a given build task

--network-retries <network_retries>: Maximum retries for network tasks

--no-interactive: Force non interactive mode, otherwise this is automatically decided

--verbose, --no-verbose: Be extra verbose

--debug, --no-debug: Print debugging output

--error-lines <error_lines>: Maximum number of lines to show from a task log

--message-lines <message_lines>: Maximum number of lines to show in a detailed message

--log-file <log_file>: A file to store the main log (allows storing the main log while in interactive mode)

--colors, --no-colors: Force enable/disable ANSI color codes in output

--strict, --no-strict: Elements must be rebuilt when their dependencies have changed

-o, --option <OPTION VALUE>: Specify a project option

--default-mirror <default_mirror>: The mirror to fetch from first, before attempting other mirrors

--pull-buildtrees: Include an element’s build tree when pulling remote element artifacts

--cache-buildtrees <cache_buildtrees>

Cache artifact build tree content on creation

Options:: always | auto | never

Commands

artifact: Manipulate cached artifacts.

build: Build elements in a pipeline

help: Print usage information

init: Initialize a new BuildStream project

shell: Shell into an element’s sandbox environment

show: Show elements in the pipeline

source: Manipulate sources for an element

workspace: Manipulate developer workspaces

bst artifact

Manipulate cached artifacts

Some subcommands take artifact references as arguments. Artifacts can be specified in two ways:

- artifact refs: triples of the form <project name>/<element name>/<cache key>

- element names

When elements are given, the artifact is looked up by observing the element and it’s current cache key.

The commands also support shell-style wildcard expansion: ? matches a single character, * matches zero or more characters but does not match the / path separator, and ** matches zero or more characters including / path separators.

If the wildcard expression ends with .bst, then it will be used to search element names found in the project, otherwise, it will be used to search artifacts that are present in the local artifact cache.

Some example arguments are:

- myproject/hello/8276376b077eda104c812e6ec2f488c7c9eea211ce572c83d734c10bf241209f
- myproject/he*/827637*
- core/*.bst (all elements in the core directory)
- **.bst (all elements)
- myproject/** (all artifacts from myproject)
- myproject/myelement/* (all cached artifacts for a specific element)

bst artifact [OPTIONS] COMMAND [ARGS]...

Commands

checkout: Checkout contents of an artifact

delete: Remove artifacts from the local cache

list-contents: List the contents of an artifact

log: Show logs of artifacts

pull: Pull a built artifact

push: Push a built artifact

show: Show the cached state of artifacts

bst init

Initialize a new BuildStream project

Creates a new BuildStream project.conf in the project directory.

Unless –project-name is specified, this will be an interactive session.

bst init [OPTIONS] [TARGET_DIRECTORY]

Options

--project-name <project_name>: The project name to use

--min-version <min_version>

The required format version

Default:: '2.3'

--element-path <element_path>

The subdirectory to store elements in

Default:: 'elements'

-f, --force: Allow overwriting an existing project.conf

Arguments

TARGET_DIRECTORY: Optional argument

bst build

Build elements in a pipeline

Specifying no elements will result in building the default targets of the project. If no default targets are configured, all project elements will be built.

When this command is executed from a workspace directory, the default is to build the workspace element.

Specify –deps to control which dependencies must be built:

none:  No dependencies, just the element itself
build: Build time dependencies, excluding the element itself
all:   All dependencies

Dependencies that are consequently required to build the requested elements will be built on demand.

bst build [OPTIONS] [ELEMENTS]...

Options

-d, --deps <deps>

The dependencies to build

Options:: none | build | all

--artifact-remote <artifact_remotes>: A remote for uploading and downloading artifacts

--source-remote <source_remotes>: A remote for uploading and downloading cached sources

--ignore-project-artifact-remotes: Ignore remote artifact cache servers recommended by projects

--ignore-project-source-remotes: Ignore remote source cache servers recommended by projects

-r, --retry-failed: Try to build elements for which a failed build artifact is found

Arguments

ELEMENTS: Optional argument(s)

bst show

Show elements in the pipeline

Specifying no elements will result in showing the default targets of the project. If no default targets are configured, all project elements will be shown.

When this command is executed from a workspace directory, the default is to show the workspace element.

By default this will show all of the dependencies of the specified target element.

Specify --deps to control which elements to show:

none:  No dependencies, just the element itself
run:   Runtime dependencies, including the element itself
build: Build time dependencies, excluding the element itself
all:   All dependencies

FORMAT

The --format option controls what should be printed for each element, the following symbols can be used in the format string:

%{name}           The element name
%{description}    The element description, on a single line (Since: 2.3)
%{key}            The abbreviated cache key (if all sources are consistent)
%{full-key}       The full cache key (if all sources are consistent)
%{state}          cached, buildable, waiting, inconsistent or junction
%{config}         The element configuration
%{vars}           Variable configuration
%{env}            Environment settings
%{public}         Public domain data
%{workspaced}     If the element is workspaced
%{workspace-dirs} A list of workspace directories
%{deps}           A list of all dependencies
%{build-deps}     A list of build dependencies
%{runtime-deps}   A list of runtime dependencies

The value of the %{symbol} without the leading ‘%’ character is understood as a pythonic formatting string, so python formatting features apply, example:

bst show target.bst –format

‘Name: %{name: ^20} Key: %{key: ^8} State: %{state}’

If you want to use a newline in a format string in bash, use the ‘$’ modifier:

bst show target.bst –format

$’———- %{name} ———-n%{vars}’

bst show [OPTIONS] [ELEMENTS]...

Options

--except <except_>: Except certain dependencies

-d, --deps <deps>

The dependencies to show

Default:: <buildstream.types._PipelineSelection object at 0x7f871086b050>
Options:: none | run | build | all

--order <order>

Staging or alphabetic ordering of dependencies

Default:: 'stage'
Options:: stage | alpha

-f, --format <FORMAT>: Format string for each element

Arguments

ELEMENTS: Optional argument(s)

bst shell

Run a command in the target element’s sandbox environment

When this command is executed from a workspace directory, the default is to shell into the workspace element.

This will stage a temporary sysroot for running the target element, assuming it has already been built and all required artifacts are in the local cache.

Use ‘–’ to separate a command from the options to bst, otherwise bst may respond to them instead. e.g.

bst shell example.bst – df -h

Use the –build option to create a temporary sysroot for building the element instead.

If no COMMAND is specified, the default is to attempt to run an interactive shell.

bst shell [OPTIONS] [TARGET] [COMMAND]...

Options

-b, --build: Stage dependencies and sources to build

--mount <HOSTPATH PATH>: Mount a file or directory into the sandbox

--isolate: Create an isolated build sandbox

-t, --use-buildtree: Stage a buildtree. Will fail if a buildtree is not available. pull-buildtrees configuration is needed if the buildtree is not available locally.

--artifact-remote <artifact_remotes>: A remote for uploading and downloading artifacts

--source-remote <source_remotes>: A remote for uploading and downloading cached sources

--ignore-project-artifact-remotes: Ignore remote artifact cache servers recommended by projects

--ignore-project-source-remotes: Ignore remote source cache servers recommended by projects

Arguments

TARGET: Optional argument

COMMAND: Optional argument(s)

bst source

Manipulate sources for an element

bst source [OPTIONS] COMMAND [ARGS]...

Commands

checkout: Checkout sources of an element

fetch: Fetch sources in a pipeline

push: Push sources in a pipeline

track: Track new source references

bst workspace

Manipulate developer workspaces

bst workspace [OPTIONS] COMMAND [ARGS]...

Commands

close: Close workspaces

list: List open workspaces

open: Open a new workspace

reset: Reset a workspace to its original state

Artifact subcommands

Artifact names

Various artifact subcommands accept either element names, which will operate on artifacts by deriving the artifact from local project state, or artifact names interchangeably as targets. Artifact names allow the user to operate directly on cached artifacts, without requiring local project data.

An artifact name is composed of the following identifiers:

The project name
The element name, without any trailing .bst extension
The cache key of the element at the time it was built.

To compose an artifact name, simply join these using a forward slash (/) character, like so: <project-name>/<element-name>/<cache-key>.

An artifact name might look like: project/target/788da21e7c1b5818b7e7b60f7eb75841057ff7e45d362cc223336c606fe47f27

bst artifact checkout

Checkout contents of an artifact

When this command is executed from a workspace directory, the default is to checkout the artifact of the workspace element.

bst artifact checkout [OPTIONS] [TARGET]

Options

-f, --force: Allow files to be overwritten

-d, --deps <deps>

The dependencies to checkout

Default:: <buildstream.types._PipelineSelection object at 0x7f871086b0b0>
Options:: run | build | none | all

--integrate, --no-integrate: Whether to run integration commands

--hardlinks: Checkout hardlinks instead of copying if possible

--tar <LOCATION>: Create a tarball from the artifact contents instead of a file tree. If LOCATION is ‘-’, the tarball will be dumped to the standard output.

--compression <compression>

The compression option of the tarball created.

Options:: gz | xz | bz2

--directory <directory>: The directory to checkout the artifact to

--artifact-remote <artifact_remotes>: A remote for downloading artifacts

--ignore-project-artifact-remotes: Ignore remote artifact cache servers recommended by projects

Arguments

TARGET: Optional argument

bst artifact log

Show build logs of artifacts

bst artifact log [OPTIONS] [ARTIFACTS]...

Options

--out <out>: Output logs to individual files in the specified path. If absent, logs are written to stdout.

Arguments

ARTIFACTS: Optional argument(s)

bst artifact pull

Pull a built artifact from the configured remote artifact cache.

Specifying no elements will result in pulling the default targets of the project. If no default targets are configured, all project elements will be pulled.

When this command is executed from a workspace directory, the default is to pull the workspace element.

By default the artifact will be pulled one of the configured caches if possible, following the usual priority order. If the –artifact-remote flag is given, only the specified cache will be queried.

Specify –deps to control which artifacts to pull:

none:  No dependencies, just the element itself
run:   Runtime dependencies, including the element itself
build: Build time dependencies, excluding the element itself
all:   All dependencies

bst artifact pull [OPTIONS] [ARTIFACTS]...

Options

-d, --deps <deps>

The dependency artifacts to pull

Default:: <buildstream.types._PipelineSelection object at 0x7f871086aff0>
Options:: build | none | run | all

--artifact-remote <artifact_remotes>: A remote for downloading artifacts

--ignore-project-artifact-remotes: Ignore remote artifact cache servers recommended by projects

Arguments

ARTIFACTS: Optional argument(s)

bst artifact push

Push built artifacts to a remote artifact cache, possibly pulling them first.

Specifying no elements will result in pushing the default targets of the project. If no default targets are configured, all project elements will be pushed.

When this command is executed from a workspace directory, the default is to push the workspace element.

If bst has been configured to include build trees on artifact pulls, an attempt will be made to pull any required build trees to avoid the skipping of partial artifacts being pushed.

Specify –deps to control which artifacts to push:

none:  No dependencies, just the element itself
run:   Runtime dependencies, including the element itself
build: Build time dependencies, excluding the element itself
all:   All dependencies

bst artifact push [OPTIONS] [ARTIFACTS]...

Options

-d, --deps <deps>

The dependencies to push

Default:: <buildstream.types._PipelineSelection object at 0x7f871086aff0>
Options:: build | none | run | all

--artifact-remote <artifact_remotes>: A remote for uploading artifacts

--ignore-project-artifact-remotes: Ignore remote artifact cache servers recommended by projects

Arguments

ARTIFACTS: Optional argument(s)

bst artifact delete

Remove artifacts from the local cache

bst artifact delete [OPTIONS] [ARTIFACTS]...

Options

-d, --deps <deps>

The dependencies to delete

Default:: <buildstream.types._PipelineSelection object at 0x7f871086aff0>
Options:: none | run | build | all

Arguments

ARTIFACTS: Optional argument(s)

bst artifact show

show the cached state of artifacts

bst artifact show [OPTIONS] [ARTIFACTS]...

Options

-d, --deps <deps>

The dependencies we also want to show

Default:: <buildstream.types._PipelineSelection object at 0x7f871086aff0>
Options:: build | run | all | none

Arguments

ARTIFACTS: Optional argument(s)

bst artifact list-contents

List the contents of an artifact.

Note that ‘artifacts’ can be element names, which must end in ‘.bst’, or artifact references, which must be in the format <project_name>/<element>/<key>.

bst artifact list-contents [OPTIONS] [ARTIFACTS]...

Options

-l, --long: Provide more information about the contents of the artifact.

Arguments

ARTIFACTS: Optional argument(s)

Source subcommands

bst source fetch

Fetch sources required to build the pipeline

Specifying no elements will result in fetching the default targets of the project. If no default targets are configured, all project elements will be fetched.

When this command is executed from a workspace directory, the default is to fetch the workspace element.

By default this will only try to fetch sources for the specified elements.

Specify –deps to control which sources to fetch:

none:  No dependencies, just the element itself
run:   Runtime dependencies, including the element itself
build: Build time dependencies, excluding the element itself
all:   All dependencies

bst source fetch [OPTIONS] [ELEMENTS]...

Options

--except <except_>: Except certain dependencies from fetching

-d, --deps <deps>

The dependencies to fetch

Default:: <buildstream.types._PipelineSelection object at 0x7f871086aff0>
Options:: none | build | run | all

--source-remote <source_remotes>: A remote for downloading sources

--ignore-project-source-remotes: Ignore remote source cache servers recommended by projects

Arguments

ELEMENTS: Optional argument(s)

bst source track

Consults the specified tracking branches for new versions available to build and updates the project with any newly available references.

Specifying no elements will result in tracking the default targets of the project. If no default targets are configured, all project elements will be tracked.

When this command is executed from a workspace directory, the default is to track the workspace element.

If no default is declared, all elements in the project will be tracked

By default this will track just the specified element, but you can also update a whole tree of dependencies in one go.

Specify –deps to control which sources to track:

none: No dependencies, just the specified elements

all: All dependencies of all specified elements

bst source track [OPTIONS] [ELEMENTS]...

Options

--except <except_>: Except certain dependencies from tracking

-d, --deps <deps>

The dependencies to track

Default:: <buildstream.types._PipelineSelection object at 0x7f871086aff0>
Options:: build | run | all | none

-J, --cross-junctions: Allow crossing junction boundaries

Arguments

ELEMENTS: Optional argument(s)

bst source push

Push sources required to build the pipeline

Specifying no elements will result in pushing the sources of the default targets of the project. If no default targets are configured, sources of all project elements will be pushed.

When this command is executed from a workspace directory, the default is to push the sources of the workspace element.

By default this will only try to push sources for the specified elements.

Specify –deps to control which sources to fetch:

none:  No dependencies, just the element itself
run:   Runtime dependencies, including the element itself
build: Build time dependencies, excluding the element itself
all:   All dependencies

bst source push [OPTIONS] [ELEMENTS]...

Options

--except <except_>: Except certain dependencies from pushing

-d, --deps <deps>

The dependencies to push

Default:: <buildstream.types._PipelineSelection object at 0x7f871086aff0>
Options:: none | build | run | all

--source-remote <source_remotes>: A remote for uploading sources

--ignore-project-source-remotes: Ignore remote source cache servers recommended by projects

Arguments

ELEMENTS: Optional argument(s)

bst source checkout

Checkout sources of an element to the specified location

When this command is executed from a workspace directory, the default is to checkout the sources of the workspace element.

bst source checkout [OPTIONS] [ELEMENT]

Options

-f, --force: Allow files to be overwritten

--except <except_>: Except certain dependencies

-d, --deps <deps>

The dependencies whose sources to checkout

Default:: <buildstream.types._PipelineSelection object at 0x7f871086aff0>
Options:: build | none | run | all

--tar <LOCATION>: Create a tarball containing the sources instead of a file tree.

--compression <compression>

The compression option of the tarball created.

Options:: gz | xz | bz2

--include-build-scripts

--directory <directory>: The directory to checkout the sources to

--source-remote <source_remotes>: A remote for downloading cached sources

--ignore-project-source-remotes: Ignore remote source cache servers recommended by projects

Arguments

ELEMENT: Optional argument

Workspace subcommands

bst workspace open

Open a workspace for manual source modification

bst workspace open [OPTIONS] ELEMENTS...

Options

--no-checkout: Do not checkout the source, only link to the given directory

-f, --force: The workspace will be created even if the directory in which it will be created is not empty or if a workspace for that element already exists

--directory <directory>: Only for use when a single Element is given: Set the directory to use to create the workspace

--source-remote <source_remotes>: A remote for downloading cached sources

--ignore-project-source-remotes: Ignore remote source cache servers recommended by projects

Arguments

ELEMENTS: Required argument(s)

bst workspace close

Close a workspace

bst workspace close [OPTIONS] [ELEMENTS]...

Options

--remove-dir: Remove the path that contains the closed workspace

-a, --all: Close all open workspaces

Arguments

ELEMENTS: Optional argument(s)

bst workspace reset

Reset a workspace to its original state

bst workspace reset [OPTIONS] [ELEMENTS]...

Options

--soft: Mark workspace to re-execute configuration steps (if any) on next build. Does not alter workspace contents.

-a, --all: Reset all open workspaces

Arguments

ELEMENTS: Optional argument(s)

bst workspace list

List open workspaces

bst workspace list [OPTIONS]