Commands

This page contains documentation for each BuildStream command, along with their possible options and arguments. Each command can be invoked on the command line, where, in most cases, this will be from the project’s main directory.

Commonly used parameters

Remotes

Remote cache servers can be specified on the command line for commands which may result in communicating with such servers.

Any command which has arguments to specify a REMOTE, such as --artifact-remote or --source-remote, will override whatever was set in the user configuration, and will have an accompanying switch which allows the command to decide whether to ignore any remote artifact or source caches suggested by project configuration.

Remotes can be specified on the command line either as a simple URI, or as a comma separated list of key value pairs.

Specifying a remote using a URI

bst artifact push --remote https://artifacts.com/artifacts:8088 element.bst

Specifying a remote using key value pairs

bst build --artifact-remote \
    url=https://artifacts.com/artifacts:8088,type=index,server-cert=~/artifacts.cert \
    element.bst

Attributes

Here is the list attributes which can be spefied when providing a REMOTE on the command line:

  • url

    The URL of the remote, possibly including a port number.

  • instance-name

    The instance name of this remote, used for sharding by some implementations.

  • type

    Whether this remote is to be used for indexing, storage or both, as explained in the corresponding user configuration documentation

  • push

    Normally one need not specify this, as it is often inferred by the command being used. In some cases, like bst build, it can be useful to specify multiple remotes, and only allow pushing to some of the remotes.

    If unspecified, this is assumed to be True and BuildStream will attempt to push to the remote, but fallback to only pulling if insufficient credentials were provided.

  • server-cert, client-cert, client-key:

    These keys specify the attributes of the authentication configuration.

    When specifying these on the command line, they are interpreted as paths relative to the current working directory.

Top-level commands

bst

Build and manipulate BuildStream projects

Most of the main options override options in the user preferences configuration file.

bst [OPTIONS] COMMAND [ARGS]...

Options

--version
-c, --config <config>

Configuration file to use

-C, --directory <directory>

Project directory (default: current directory)

--on-error <on_error>

What to do when an error is encountered

Options

continue | quit | terminate

--fetchers <fetchers>

Maximum simultaneous download tasks

--builders <builders>

Maximum simultaneous build tasks

--pushers <pushers>

Maximum simultaneous upload tasks

--max-jobs <max_jobs>

Number of parallel jobs allowed for a given build task

--network-retries <network_retries>

Maximum retries for network tasks

--no-interactive

Force non interactive mode, otherwise this is automatically decided

--verbose, --no-verbose

Be extra verbose

--debug, --no-debug

Print debugging output

--error-lines <error_lines>

Maximum number of lines to show from a task log

--message-lines <message_lines>

Maximum number of lines to show in a detailed message

--log-file <log_file>

A file to store the main log (allows storing the main log while in interactive mode)

--colors, --no-colors

Force enable/disable ANSI color codes in output

--strict, --no-strict

Elements must be rebuilt when their dependencies have changed

-o, --option <OPTION VALUE>

Specify a project option

--default-mirror <default_mirror>

The mirror to fetch from first, before attempting other mirrors

--pull-buildtrees

Include an element’s build tree when pulling remote element artifacts

--cache-buildtrees <cache_buildtrees>

Cache artifact build tree content on creation

Options

always | auto | never

Commands

artifact

Manipulate cached artifacts.

build

Build elements in a pipeline

help

Print usage information

init

Initialize a new BuildStream project

shell

Shell into an element’s sandbox environment

show

Show elements in the pipeline

source

Manipulate sources for an element

workspace

Manipulate developer workspaces


bst artifact

Manipulate cached artifacts

Some subcommands take artifact references as arguments. Artifacts can be specified in two ways:

- artifact refs: triples of the form <project name>/<element name>/<cache key>
- element names

When elements are given, the artifact is looked up by observing the element and it’s current cache key.

The commands also support shell-style wildcard expansion: ? matches a single character, * matches zero or more characters but does not match the / path separator, and ** matches zero or more characters including / path separators.

If the wildcard expression ends with .bst, then it will be used to search element names found in the project, otherwise, it will be used to search artifacts that are present in the local artifact cache.

Some example arguments are:

- myproject/hello/8276376b077eda104c812e6ec2f488c7c9eea211ce572c83d734c10bf241209f
- myproject/he*/827637*
- core/*.bst (all elements in the core directory)
- **.bst (all elements)
- myproject/** (all artifacts from myproject)
- myproject/myelement/* (all cached artifacts for a specific element)
bst artifact [OPTIONS] COMMAND [ARGS]...

Commands

checkout

Checkout contents of an artifact

delete

Remove artifacts from the local cache

list-contents

List the contents of an artifact

log

Show logs of artifacts

pull

Pull a built artifact

push

Push a built artifact

show

Show the cached state of artifacts


bst init

Initialize a new BuildStream project

Creates a new BuildStream project.conf in the project directory.

Unless –project-name is specified, this will be an interactive session.

bst init [OPTIONS] [TARGET_DIRECTORY]

Options

--project-name <project_name>

The project name to use

--min-version <min_version>

The required format version

Default

2.0

--element-path <element_path>

The subdirectory to store elements in

Default

elements

-f, --force

Allow overwriting an existing project.conf

Arguments

TARGET_DIRECTORY

Optional argument


bst build

Build elements in a pipeline

Specifying no elements will result in building the default targets of the project. If no default targets are configured, all project elements will be built.

When this command is executed from a workspace directory, the default is to build the workspace element.

Specify –deps to control which dependencies must be built:

none: No dependencies, just the element itself
build: Build time dependencies, excluding the element itself
all: All dependencies

Dependencies that are consequently required to build the requested elements will be built on demand.

bst build [OPTIONS] [ELEMENTS]...

Options

-d, --deps <deps>

The dependencies to build

Options

none | build | all

--artifact-remote <artifact_remotes>

A remote for uploading and downloading artifacts

--source-remote <source_remotes>

A remote for uploading and downloading cached sources

--ignore-project-artifact-remotes

Ignore remote artifact cache servers recommended by projects

--ignore-project-source-remotes

Ignore remote source cache servers recommended by projects

Arguments

ELEMENTS

Optional argument(s)


bst show

Show elements in the pipeline

Specifying no elements will result in showing the default targets of the project. If no default targets are configured, all project elements will be shown.

When this command is executed from a workspace directory, the default is to show the workspace element.

By default this will show all of the dependencies of the specified target element.

Specify --deps to control which elements to show:

none: No dependencies, just the element itself
run: Runtime dependencies, including the element itself
build: Build time dependencies, excluding the element itself
all: All dependencies

FORMAT

The --format option controls what should be printed for each element, the following symbols can be used in the format string:

%{name} The element name
%{key} The abbreviated cache key (if all sources are consistent)
%{full-key} The full cache key (if all sources are consistent)
%{state} cached, buildable, waiting, inconsistent or junction
%{config} The element configuration
%{vars} Variable configuration
%{env} Environment settings
%{public} Public domain data
%{workspaced} If the element is workspaced
%{workspace-dirs} A list of workspace directories
%{deps} A list of all dependencies
%{build-deps} A list of build dependencies
%{runtime-deps} A list of runtime dependencies

The value of the %{symbol} without the leading ‘%’ character is understood as a pythonic formatting string, so python formatting features apply, example:

bst show target.bst –format
‘Name: %{name: ^20} Key: %{key: ^8} State: %{state}’

If you want to use a newline in a format string in bash, use the ‘$’ modifier:

bst show target.bst –format
$’———- %{name} ———-n%{vars}’
bst show [OPTIONS] [ELEMENTS]...

Options

--except <except_>

Except certain dependencies

-d, --deps <deps>

The dependencies to show

Default

all

Options

none | run | build | all

--order <order>

Staging or alphabetic ordering of dependencies

Default

stage

Options

stage | alpha

-f, --format <FORMAT>

Format string for each element

Arguments

ELEMENTS

Optional argument(s)


bst shell

Run a command in the target element’s sandbox environment

When this command is executed from a workspace directory, the default is to shell into the workspace element.

This will stage a temporary sysroot for running the target element, assuming it has already been built and all required artifacts are in the local cache.

Use ‘–’ to separate a command from the options to bst, otherwise bst may respond to them instead. e.g.

bst shell example.bst – df -h

Use the –build option to create a temporary sysroot for building the element instead.

If no COMMAND is specified, the default is to attempt to run an interactive shell.

bst shell [OPTIONS] [TARGET] [COMMAND]...

Options

-b, --build

Stage dependencies and sources to build

--mount <HOSTPATH PATH>

Mount a file or directory into the sandbox

--isolate

Create an isolated build sandbox

-t, --use-buildtree

Stage a buildtree. Will fail if a buildtree is not available. pull-buildtrees configuration is needed if the buildtree is not available locally.

--artifact-remote <artifact_remotes>

A remote for uploading and downloading artifacts

--source-remote <source_remotes>

A remote for uploading and downloading cached sources

--ignore-project-artifact-remotes

Ignore remote artifact cache servers recommended by projects

--ignore-project-source-remotes

Ignore remote source cache servers recommended by projects

Arguments

TARGET

Optional argument

COMMAND

Optional argument(s)


bst source

Manipulate sources for an element

bst source [OPTIONS] COMMAND [ARGS]...

Commands

checkout

Checkout sources of an element

fetch

Fetch sources in a pipeline

push

Push sources in a pipeline

track

Track new source references


bst workspace

Manipulate developer workspaces

bst workspace [OPTIONS] COMMAND [ARGS]...

Commands

close

Close workspaces

list

List open workspaces

open

Open a new workspace

reset

Reset a workspace to its original state

Artifact subcommands

Artifact names

Various artifact subcommands accept either element names, which will operate on artifacts by deriving the artifact from local project state, or artifact names interchangeably as targets. Artifact names allow the user to operate directly on cached artifacts, without requiring local project data.

An artifact name is composed of the following identifiers:

  • The project name

  • The element name, without any trailing .bst extension

  • The cache key of the element at the time it was built.

To compose an artifact name, simply join these using a forward slash (/) character, like so: <project-name>/<element-name>/<cache-key>.

An artifact name might look like: project/target/788da21e7c1b5818b7e7b60f7eb75841057ff7e45d362cc223336c606fe47f27

bst artifact checkout

Checkout contents of an artifact

When this command is executed from a workspace directory, the default is to checkout the artifact of the workspace element.

bst artifact checkout [OPTIONS] [TARGET]

Options

-f, --force

Allow files to be overwritten

-d, --deps <deps>

The dependencies to checkout

Default

run

Options

run | build | none | all

--integrate, --no-integrate

Whether to run integration commands

Checkout hardlinks instead of copying if possible

--tar <LOCATION>

Create a tarball from the artifact contents instead of a file tree. If LOCATION is ‘-’, the tarball will be dumped to the standard output.

--compression <compression>

The compression option of the tarball created.

Options

gz | xz | bz2

--directory <directory>

The directory to checkout the artifact to

--artifact-remote <artifact_remotes>

A remote for downloading artifacts

--ignore-project-artifact-remotes

Ignore remote artifact cache servers recommended by projects

Arguments

TARGET

Optional argument


bst artifact log

Show build logs of artifacts

bst artifact log [OPTIONS] [ARTIFACTS]...

Options

--out <out>

Output logs to individual files in the specified path. If absent, logs are written to stdout.

Arguments

ARTIFACTS

Optional argument(s)


bst artifact pull

Pull a built artifact from the configured remote artifact cache.

Specifying no elements will result in pulling the default targets of the project. If no default targets are configured, all project elements will be pulled.

When this command is executed from a workspace directory, the default is to pull the workspace element.

By default the artifact will be pulled one of the configured caches if possible, following the usual priority order. If the –artifact-remote flag is given, only the specified cache will be queried.

Specify –deps to control which artifacts to pull:

none: No dependencies, just the element itself
run: Runtime dependencies, including the element itself
build: Build time dependencies, excluding the element itself
all: All dependencies
bst artifact pull [OPTIONS] [ARTIFACTS]...

Options

-d, --deps <deps>

The dependency artifacts to pull

Default

none

Options

build | none | run | all

--artifact-remote <artifact_remotes>

A remote for downloading artifacts

--ignore-project-artifact-remotes

Ignore remote artifact cache servers recommended by projects

Arguments

ARTIFACTS

Optional argument(s)


bst artifact push

Push built artifacts to a remote artifact cache, possibly pulling them first.

Specifying no elements will result in pushing the default targets of the project. If no default targets are configured, all project elements will be pushed.

When this command is executed from a workspace directory, the default is to push the workspace element.

If bst has been configured to include build trees on artifact pulls, an attempt will be made to pull any required build trees to avoid the skipping of partial artifacts being pushed.

Specify –deps to control which artifacts to push:

none: No dependencies, just the element itself
run: Runtime dependencies, including the element itself
build: Build time dependencies, excluding the element itself
all: All dependencies
bst artifact push [OPTIONS] [ARTIFACTS]...

Options

-d, --deps <deps>

The dependencies to push

Default

none

Options

build | none | run | all

--artifact-remote <artifact_remotes>

A remote for uploading artifacts

--ignore-project-artifact-remotes

Ignore remote artifact cache servers recommended by projects

Arguments

ARTIFACTS

Optional argument(s)


bst artifact delete

Remove artifacts from the local cache

bst artifact delete [OPTIONS] [ARTIFACTS]...

Options

-d, --deps <deps>

The dependencies to delete

Default

none

Options

none | run | build | all

Arguments

ARTIFACTS

Optional argument(s)


bst artifact show

show the cached state of artifacts

bst artifact show [OPTIONS] [ARTIFACTS]...

Options

-d, --deps <deps>

The dependencies we also want to show

Default

none

Options

build | run | all | none

Arguments

ARTIFACTS

Optional argument(s)


bst artifact list-contents

List the contents of an artifact.

Note that ‘artifacts’ can be element names, which must end in ‘.bst’, or artifact references, which must be in the format <project_name>/<element>/<key>.

bst artifact list-contents [OPTIONS] [ARTIFACTS]...

Options

-l, --long

Provide more information about the contents of the artifact.

Arguments

ARTIFACTS

Optional argument(s)

Source subcommands

bst source fetch

Fetch sources required to build the pipeline

Specifying no elements will result in fetching the default targets of the project. If no default targets are configured, all project elements will be fetched.

When this command is executed from a workspace directory, the default is to fetch the workspace element.

By default this will only try to fetch sources for the specified elements.

Specify –deps to control which sources to fetch:

none: No dependencies, just the element itself
run: Runtime dependencies, including the element itself
build: Build time dependencies, excluding the element itself
all: All dependencies
bst source fetch [OPTIONS] [ELEMENTS]...

Options

--except <except_>

Except certain dependencies from fetching

-d, --deps <deps>

The dependencies to fetch

Default

none

Options

none | build | run | all

--source-remote <source_remotes>

A remote for downloading sources

--ignore-project-source-remotes

Ignore remote source cache servers recommended by projects

Arguments

ELEMENTS

Optional argument(s)


bst source track

Consults the specified tracking branches for new versions available to build and updates the project with any newly available references.

Specifying no elements will result in tracking the default targets of the project. If no default targets are configured, all project elements will be tracked.

When this command is executed from a workspace directory, the default is to track the workspace element.

If no default is declared, all elements in the project will be tracked

By default this will track just the specified element, but you can also update a whole tree of dependencies in one go.

Specify –deps to control which sources to track:

none: No dependencies, just the specified elements
all: All dependencies of all specified elements
bst source track [OPTIONS] [ELEMENTS]...

Options

--except <except_>

Except certain dependencies from tracking

-d, --deps <deps>

The dependencies to track

Default

none

Options

build | run | all | none

-J, --cross-junctions

Allow crossing junction boundaries

Arguments

ELEMENTS

Optional argument(s)


bst source push

Push sources required to build the pipeline

Specifying no elements will result in pushing the sources of the default targets of the project. If no default targets are configured, sources of all project elements will be pushed.

When this command is executed from a workspace directory, the default is to push the sources of the workspace element.

By default this will only try to push sources for the specified elements.

Specify –deps to control which sources to fetch:

none: No dependencies, just the element itself
run: Runtime dependencies, including the element itself
build: Build time dependencies, excluding the element itself
all: All dependencies
bst source push [OPTIONS] [ELEMENTS]...

Options

--except <except_>

Except certain dependencies from pushing

-d, --deps <deps>

The dependencies to push

Default

none

Options

none | build | run | all

--source-remote <source_remotes>

A remote for uploading sources

--ignore-project-source-remotes

Ignore remote source cache servers recommended by projects

Arguments

ELEMENTS

Optional argument(s)


bst source checkout

Checkout sources of an element to the specified location

When this command is executed from a workspace directory, the default is to checkout the sources of the workspace element.

bst source checkout [OPTIONS] [ELEMENT]

Options

-f, --force

Allow files to be overwritten

--except <except_>

Except certain dependencies

-d, --deps <deps>

The dependencies whose sources to checkout

Default

none

Options

build | none | run | all

--tar <LOCATION>

Create a tarball containing the sources instead of a file tree.

--compression <compression>

The compression option of the tarball created.

Options

gz | xz | bz2

--include-build-scripts
--directory <directory>

The directory to checkout the sources to

--source-remote <source_remotes>

A remote for downloading cached sources

--ignore-project-source-remotes

Ignore remote source cache servers recommended by projects

Arguments

ELEMENT

Optional argument

Workspace subcommands

bst workspace open

Open a workspace for manual source modification

bst workspace open [OPTIONS] ELEMENTS...

Options

--no-checkout

Do not checkout the source, only link to the given directory

-f, --force

The workspace will be created even if the directory in which it will be created is not empty or if a workspace for that element already exists

--directory <directory>

Only for use when a single Element is given: Set the directory to use to create the workspace

--source-remote <source_remotes>

A remote for downloading cached sources

--ignore-project-source-remotes

Ignore remote source cache servers recommended by projects

Arguments

ELEMENTS

Required argument(s)


bst workspace close

Close a workspace

bst workspace close [OPTIONS] [ELEMENTS]...

Options

--remove-dir

Remove the path that contains the closed workspace

-a, --all

Close all open workspaces

Arguments

ELEMENTS

Optional argument(s)


bst workspace reset

Reset a workspace to its original state

bst workspace reset [OPTIONS] [ELEMENTS]...

Options

--soft

Mark workspace to re-execute configuration steps (if any) on next build. Does not alter workspace contents.

-a, --all

Reset all open workspaces

Arguments

ELEMENTS

Optional argument(s)


bst workspace list

List open workspaces

bst workspace list [OPTIONS]