User configuration

User configuration and preferences can be specified in a user provided configuration file, and usually also on the command line.

Values specified in a user provided configuration file override the defaults, while command line options take precedence over any other specified configurations.

Configuration file

Users can provide a configuration file to override parameters in the default configuration.

Unless a configuration file is explicitly specified on the command line when invoking bst, an attempt is made to load user specific configuration from $XDG_CONFIG_HOME/buildstream.conf. On most Linux based systems, the location will be ~/.config/buildstream.conf

Project specific value

The projects key can be used to specify project specific configurations, the supported configurations on a project wide basis are listed here.

Artifact server

Although project’s often specify a remote artifact cache in their project.conf, you may also want to specify extra caches.

Assuming that your host/server is reachable on the internet as artifacts.com (for example), there are two ways to declare remote caches in your user configuration:

  1. Adding global caches:
#
# Artifacts
#
artifacts:
  # Add a cache to pull from
  - url: https://artifacts.com/artifacts:11001
    server-cert: server.crt
  # Add a cache to push/pull to/from
  - url: https://artifacts.com/artifacts:11002
    server-cert: server.crt
    client-cert: client.crt
    client-key: client.key
    push: true
  # Add another cache to pull from
  - url: https://anothercache.com/artifacts:8080
    server-cert: another_server.crt

Note

Caches declared here will be used by all BuildStream project’s on the user’s machine and are considered a lower priority than those specified in the project configuration.

  1. Specifying caches for a specific project within the user configuration:
projects:
  project-name:
    artifacts:
      # Add a cache to pull from
      - url: https://artifacts.com/artifacts:11001
        server-cert: server.crt
      # Add a cache to push/pull to/from
      - url: https://artifacts.com/artifacts:11002
        server-cert: server.crt
        client-cert: client.crt
        client-key: client.key
        push: true
      # Add another cache to pull from
      - url: https://ourprojectcache.com/artifacts:8080
        server-cert: project_server.crt

Note

Caches listed here will be considered a higher priority than those specified by the project. Furthermore, for a given list of URLs, earlier entries will have higher priority.

Notice that the use of different ports for the same server distinguishes between pull only access and push/pull access. For information regarding this and the server/client certificates and keys, please see: Key pair for the server.

Remote execution

The same configuration for remote execution in project.conf can be provided in the user configuation.

There is only one remote execution configuration used per project.

The project overrides will be taken in priority. The global configuration will be used as fallback.

  1. Global remote execution fallback:
remote-execution:
  execution-service:
    url: http://execution.fallback.example.com:50051
    instance-name: main
  storage-service:
    url: https://storage.fallback.example.com:11002/
    server-cert: /keys/server.crt
    client-cert: /keys/client.crt
    client-key: /keys/client.key
    instance-name: main
  action-cache-service:
    url: http://action.flalback.example.com:50052
  1. Project override:
projects:
  some_project:
    remote-execution:
      execution-service:
        url: http://execution.some_project.example.com:50051
        instance-name: main
      storage-service:
        url: https://storage.some_project.example.com:11002/
        server-cert: /some_project_keys/server.crt
        client-cert: /some_project_keys/client.crt
        client-key: /some_project_keys/client.key
        instance-name: main
      action-cache-service:
        url: http://action.some_project.example.com:50052

Strict build plan

The strict build plan option decides whether you want elements to rebuild when their dependencies have changed. This is enabled by default, but recommended to turn off in developer scenarios where you might want to build a large system and test it quickly after modifying some low level component.

Example

projects:
  project-name:
    strict: False

Note

It is always possible to override this at invocation time using the --strict and --no-strict command line options.

Default Mirror

When using mirrors, a default mirror can be defined to be fetched first. The default mirror is defined by its name, e.g.

projects:
  project-name:
    default-mirror: oz

Note

It is possible to override this at invocation time using the --default-mirror command-line option.

Local cache expiry

BuildStream locally caches artifacts, build trees, log files and sources within a cache located at ~/.cache/buildstream (unless a $XDG_CACHE_HOME environment variable exists). When building large projects, this cache can get very large, thus BuildStream will attempt to clean up the cache automatically by expiring the least recently used artifacts.

By default, cache expiry will begin once the file system which contains the cache approaches maximum usage. However, it is also possible to impose a quota on the local cache in the user configuration. This can be done in two ways:

  1. By restricting the maximum size of the cache directory itself.

For example, to ensure that BuildStream’s cache does not grow beyond 100 GB, simply declare the following in your user configuration (~/.config/buildstream.conf):

cache:
  quota: 100G

This quota defines the maximum size of the artifact cache in bytes. Other accepted values are: K, M, G or T (or you can simply declare the value in bytes, without the suffix). This uses the same format as systemd’s resource-control.

  1. By expiring artifacts once the file system which contains the cache exceeds a specified usage.

To ensure that we start cleaning the cache once we’ve used 80% of local disk space (on the file system which mounts the cache):

cache:
  quota: 80%

Default configuration

The default BuildStream configuration is specified here for reference:

# Default BuildStream user configuration.

#
#    Work Directories
#
#
# Note that BuildStream forces the XDG Base Directory names
# into the environment if they are not already set, and allows
# expansion of '~' and environment variables when specifying
# paths.
#

# Location to store sources
sourcedir: ${XDG_CACHE_HOME}/buildstream/sources

# Location to perform builds
builddir: ${XDG_CACHE_HOME}/buildstream/build

# Location to store local binary artifacts
artifactdir: ${XDG_CACHE_HOME}/buildstream/artifacts

# Location to store build logs
logdir: ${XDG_CACHE_HOME}/buildstream/logs

# Default root location for workspaces, blank for no default set.
workspacedir: .

#
#    Cache
#
cache:
  # Size of the artifact cache in bytes - BuildStream will attempt to keep the
  # artifact cache within this size.
  # If the value is suffixed with K, M, G or T, the specified memory size is
  # parsed as Kilobytes, Megabytes, Gigabytes, or Terabytes (with the base
  # 1024), respectively.
  # Alternatively, a percentage value may be specified, which is taken relative
  # to the isize of the file system containing the cache.
  quota: infinity

  # Whether to pull build trees when downloading element artifacts
  pull-buildtrees: False

#
#    Scheduler
#
scheduler:

  # Maximum number of simultaneous downloading tasks.
  fetchers: 10

  # Maximum number of simultaneous build tasks.
  builders: 4

  # Maximum number of simultaneous uploading tasks.
  pushers: 4

  # Maximum number of retries for network tasks.
  network-retries: 2

  # What to do when an element fails, if not running in
  # interactive mode:
  #
  #  continue  - Continue queueing jobs as much as possible
  #  quit      - Exit after all ongoing jobs complete
  #  terminate - Terminate any ongoing jobs and exit
  #
  on-error: quit


#
#    Logging
#
logging:

  # The abbreviated cache key length to display in the UI
  key-length: 8

  # Whether to show extra detailed messages
  verbose: True

  # Maximum number of lines to print from the
  # end of a failing build log
  error-lines: 20

  # Maximum number of lines to print in a detailed
  # message on the console or in the master log (the full
  # messages are always recorded in the individual build
  # logs)
  message-lines: 20

  # Whether to enable debugging messages
  debug: False

  # Format string for printing the pipeline at startup, this
  # also determines the default display format for `bst show`
  element-format: |

    %{state: >12} %{full-key} %{name} %{workspace-dirs}

  # Format string for all log messages.
  message-format: |

    [%{elapsed}][%{key}][%{element}] %{action} %{message}

#
#    Prompt overrides
#
# Here you can suppress 'are you sure?' and other kinds of prompts by supplying
# override values. Note that e.g. 'yes' and 'no' have the same meaning here as
# they do in the actual cli prompt.
#
prompt:

  # Whether to really proceed with 'bst workspace close --remove-dir' removing
  # a workspace directory, potentially losing changes.
  #
  #  ask - Ask the user if they are sure.
  #  yes - Always remove, without asking.
  #
  really-workspace-close-remove-dir: ask

  # Whether to really proceed with 'bst workspace close' when doing so would
  # stop them from running bst commands in this workspace.
  #
  #  ask - Ask the user if they are sure.
  #  yes - Always close, without asking.
  #
  really-workspace-close-project-inaccessible: ask

  # Whether to really proceed with 'bst workspace reset' doing a hard reset of
  # a workspace, potentially losing changes.
  #
  #  ask - Ask the user if they are sure.
  #  yes - Always hard reset, without asking.
  #
  really-workspace-reset-hard: ask