1. Junction elements

BuildStream’s junction elements are the mechanism which allow projects to interact and depend on eachother.

Junction elements represent the BuildStream project you are depending, and behave much like other elements in the sense that they can be fetched and tracked like other elements, except that regular elements cannot depend on junctions directly, nor can junctions be built. Instead, junctions act like a window into another project you depend on, and allow elements of your project to depend on elements exposed by the project referenced by the junction.

Projects which are junctioned by your project are referred to as subprojects.

1.1. A simple example

Note

This example is distributed with BuildStream in the doc/examples/junctions subdirectory.

Below is a simple example of bst file for the junction element, which we have called hello-junction.bst in this project:

kind: junction
sources:
- kind: local
  path: autotools

This element imports the autotools example subproject distributed with BuildStream in the doc/examples/junctions/autotools subdirectory.

Note

For the sake of this example we are using a local source in a subdirectory of the example project.

Since junctions allow interoperability of projects, it would be more common to use a junction to a remote project under separate revision control, possibly using a kind: git source.

The below bst file describes the element callHello.bst, which depends on the hello.bst element from the autotools example:

kind: import

sources:
- kind: local
  path: files/callHello.sh

depends:
  - filename: hello.bst
    junction: hello-junction.bst

Note how this element refers to the previously declared hello-junction.bst junction in its dependency dictionary. This dependency expresses that we are depending on the hello.bst element in the project which hello-junction.bst refers to.

The callHello.bst element simply imports a callHello.sh shell script which calls the hello command provided by hello.bst:

#!/bin/sh
echo "Calling hello:"
hello

1.2. Building and running

Building the callHello.bst element which requires an external project is just a matter of invoking bst build in the regular way:

user@host:~/junctions$ bst build callHello.bst

[--:--:--][        ][    main:core activity                 ] START   Build
[--:--:--][        ][    main:core activity                 ] START   Loading elements
[--:--:--][57dd4ab5][   fetch:hello-junction.bst            ] START   junctions/hello-junction/57dd4ab5-fetch.3361.log
[00:00:00][57dd4ab5][   fetch:hello-junction.bst            ] SUCCESS junctions/hello-junction/57dd4ab5-fetch.3361.log
[00:00:00][        ][    main:core activity                 ] SUCCESS Loading elements
[--:--:--][        ][    main:core activity                 ] START   Resolving elements
[--:--:--][        ][    main:callHello.bst                 ] START   Staging local files into CAS
[00:00:00][        ][    main:callHello.bst                 ] SUCCESS Staging local files into CAS
[00:00:00][        ][    main:core activity                 ] SUCCESS Resolving elements
[--:--:--][        ][    main:core activity                 ] START   Initializing remote caches
[00:00:00][        ][    main:core activity                 ] SUCCESS Initializing remote caches
[--:--:--][        ][    main:core activity                 ] START   Checking sources
[00:00:00][        ][    main:core activity                 ] SUCCESS Checking sources
[--:--:--][        ][    main:core activity                 ] START   Query cache
[00:00:00][        ][    main:core activity                 ] SUCCESS Query cache
[--:--:--][        ][    main:core activity                 ] START   Preparing work plan
[00:00:00][        ][    main:core activity                 ] SUCCESS Preparing work plan

BuildStream Version 1.93.6+36.ga935429a
    Session Start: Thursday, 25-03-2021 at 15:48:24
    Project:       junctions (/home/user/buildstream/doc/examples/junctions)
    Targets:       callHello.bst

User Configuration
    Configuration File:      /home/user/buildstream/doc/run-bst-tzwm794q/buildstream.conf
    Cache Directory:         /home/user/buildstream/doc/run-bst-tzwm794q
    Log Files:               /home/user/buildstream/doc/run-bst-tzwm794q/logs
    Source Mirrors:          /home/user/buildstream/doc/run-bst-tzwm794q/sources
    Build Area:              /home/user/buildstream/doc/run-bst-tzwm794q/build
    Strict Build Plan:       Yes
    Maximum Fetch Tasks:     10
    Maximum Build Tasks:     4
    Maximum Push Tasks:      4
    Maximum Network Retries: 2

Project: junctions

    Element Plugins
        junction: core plugin
        import:   core plugin

    Source Plugins
        local: core plugin

Project: autotools
    Junction path: hello-junction.bst
    Loaded by:     callHello.bst [line 8 column 4]

    Element Plugins
        autotools: core plugin
        stack:     core plugin
        import:    core plugin

    Source Plugins
        tar: core plugin

Pipeline
fetch needed 5c7d4b22fc6bf3cb925fe125ed5107982569e8ef6b7a3bc1c195675342d70ed9 hello-junction.bst:base/alpine.bst 
     waiting 419075c865ea9d85849e124301e0c892a04094aab40edbdade394e94150367bb hello-junction.bst:base.bst 
fetch needed 6cecb6f80ef126464c5218e26ac60646fc643469eaa96765ac508a39dc0162ab hello-junction.bst:hello.bst 
     waiting 6826f1d0c7939ddf4fd4c7170b2246de58b6a32ca00fce7b6e647171185df1c1 callHello.bst 
===============================================================================
[--:--:--][5c7d4b22][   fetch:hello-junction.bst:base/alpine.bst] START   autotools/base-alpine/5c7d4b22-fetch.3361.log
[--:--:--][5c7d4b22][   fetch:hello-junction.bst:base/alpine.bst] START   Fetching https://bst-integration-test-images.ams3.cdn.digitaloceanspaces.com/integration-tests-base.v1.x86_64.tar.xz
[--:--:--][419075c8][   fetch:hello-junction.bst:base.bst   ] START   autotools/base/419075c8-fetch.3361.log
[--:--:--][6826f1d0][   fetch:callHello.bst                 ] START   junctions/callHello/6826f1d0-fetch.3361.log
[--:--:--][6cecb6f8][   fetch:hello-junction.bst:hello.bst  ] START   autotools/hello/6cecb6f8-fetch.3361.log
[--:--:--][6cecb6f8][   fetch:hello-junction.bst:hello.bst  ] START   Fetching https://ftpmirror.gnu.org/gnu/automake/automake-1.16.tar.gz
[00:00:00][6826f1d0][   fetch:callHello.bst                 ] SUCCESS junctions/callHello/6826f1d0-fetch.3361.log
[00:00:00][419075c8][   fetch:hello-junction.bst:base.bst   ] SUCCESS autotools/base/419075c8-fetch.3361.log
[00:00:00][5c7d4b22][   fetch:hello-junction.bst:base/alpine.bst] SUCCESS Fetching https://bst-integration-test-images.ams3.cdn.digitaloceanspaces.com/integration-tests-base.v1.x86_64.tar.xz
[00:00:03][6cecb6f8][   fetch:hello-junction.bst:hello.bst  ] SUCCESS Fetching https://ftpmirror.gnu.org/gnu/automake/automake-1.16.tar.gz
[00:00:04][6cecb6f8][   fetch:hello-junction.bst:hello.bst  ] SUCCESS autotools/hello/6cecb6f8-fetch.3361.log
[00:00:08][5c7d4b22][   fetch:hello-junction.bst:base/alpine.bst] SUCCESS autotools/base-alpine/5c7d4b22-fetch.3361.log
[--:--:--][5c7d4b22][   build:hello-junction.bst:base/alpine.bst] START   autotools/base-alpine/5c7d4b22-build.3361.log
[--:--:--][5c7d4b22][   build:hello-junction.bst:base/alpine.bst] START   Staging sources
[00:00:00][5c7d4b22][   build:hello-junction.bst:base/alpine.bst] SUCCESS Staging sources
[--:--:--][5c7d4b22][   build:hello-junction.bst:base/alpine.bst] START   Caching artifact
[00:00:00][5c7d4b22][   build:hello-junction.bst:base/alpine.bst] SUCCESS Caching artifact
[00:00:00][5c7d4b22][   build:hello-junction.bst:base/alpine.bst] SUCCESS autotools/base-alpine/5c7d4b22-build.3361.log
[--:--:--][419075c8][   build:hello-junction.bst:base.bst   ] START   autotools/base/419075c8-build.3361.log
[--:--:--][419075c8][   build:hello-junction.bst:base.bst   ] START   Caching artifact
[00:00:00][419075c8][   build:hello-junction.bst:base.bst   ] SUCCESS Caching artifact
[00:00:00][419075c8][   build:hello-junction.bst:base.bst   ] SUCCESS autotools/base/419075c8-build.3361.log
[--:--:--][6cecb6f8][   build:hello-junction.bst:hello.bst  ] START   autotools/hello/6cecb6f8-build.3361.log
[--:--:--][6cecb6f8][   build:hello-junction.bst:hello.bst  ] START   Staging dependencies at: /
[00:00:00][6cecb6f8][   build:hello-junction.bst:hello.bst  ] SUCCESS Staging dependencies at: /
[--:--:--][6cecb6f8][   build:hello-junction.bst:hello.bst  ] START   Integrating sandbox
[00:00:00][6cecb6f8][   build:hello-junction.bst:hello.bst  ] SUCCESS Integrating sandbox
[--:--:--][6cecb6f8][   build:hello-junction.bst:hello.bst  ] START   Staging sources
[00:00:00][6cecb6f8][   build:hello-junction.bst:hello.bst  ] SUCCESS Staging sources
[--:--:--][6cecb6f8][   build:hello-junction.bst:hello.bst  ] START   Running commands

    export NOCONFIGURE=1;
    
    if [ -x ./configure ]; then true;
    elif [ -x ./autogen ]; then ./autogen;
    elif [ -x ./autogen.sh ]; then ./autogen.sh;
    elif [ -x ./bootstrap ]; then ./bootstrap;
    elif [ -x ./bootstrap.sh ]; then ./bootstrap.sh;
    else autoreconf -ivf .;
    fi
    ./configure --prefix=/usr \
    --exec-prefix=/usr \
    --bindir=/usr/bin \
    --sbindir=/usr/sbin \
    --sysconfdir=/etc \
    --datadir=/usr/share \
    --includedir=/usr/include \
    --libdir=/usr/lib \
    --libexecdir=/usr/libexec \
    --localstatedir=/var \
    --sharedstatedir=/usr/com \
    Message contains 23 additional lines

[00:00:06][6cecb6f8][   build:hello-junction.bst:hello.bst  ] SUCCESS Running commands
[--:--:--][6cecb6f8][   build:hello-junction.bst:hello.bst  ] START   Caching artifact
[00:00:00][6cecb6f8][   build:hello-junction.bst:hello.bst  ] SUCCESS Caching artifact
[00:00:07][6cecb6f8][   build:hello-junction.bst:hello.bst  ] SUCCESS autotools/hello/6cecb6f8-build.3361.log
[--:--:--][6826f1d0][   build:callHello.bst                 ] START   junctions/callHello/6826f1d0-build.3361.log
[--:--:--][6826f1d0][   build:callHello.bst                 ] START   Staging sources
[00:00:00][6826f1d0][   build:callHello.bst                 ] SUCCESS Staging sources
[--:--:--][6826f1d0][   build:callHello.bst                 ] START   Caching artifact
[00:00:00][6826f1d0][   build:callHello.bst                 ] SUCCESS Caching artifact
[00:00:00][6826f1d0][   build:callHello.bst                 ] SUCCESS junctions/callHello/6826f1d0-build.3361.log
[00:00:16][        ][    main:core activity                 ] SUCCESS Build

Pipeline Summary
    Total:       4
    Session:     4
    Fetch Queue: processed 4, skipped 0, failed 0 
    Build Queue: processed 4, skipped 0, failed 0

You can see that the hello.bst element and its dependencies from the autotools project have been built as a part of the pipeline for callHello.bst.

We can now invoke bst shell and run our callHello.sh script, which in turn also calls the hello program installed by the subproject’s hello.bst element.

user@host:~/junctions$ bst shell callHello.bst -- /bin/sh callHello.sh

[--:--:--][        ][    main:core activity                 ] START   Loading elements
[00:00:00][        ][    main:core activity                 ] SUCCESS Loading elements
[--:--:--][        ][    main:core activity                 ] START   Resolving elements
[--:--:--][        ][    main:callHello.bst                 ] START   Staging local files into CAS
[00:00:00][        ][    main:callHello.bst                 ] SUCCESS Staging local files into CAS
[00:00:00][        ][    main:core activity                 ] SUCCESS Resolving elements
[--:--:--][        ][    main:core activity                 ] START   Initializing remote caches
[00:00:00][        ][    main:core activity                 ] SUCCESS Initializing remote caches
[--:--:--][        ][    main:core activity                 ] START   Query cache
[00:00:00][        ][    main:core activity                 ] SUCCESS Query cache
[--:--:--][6826f1d0][    main:callHello.bst                 ] START   Staging dependencies
[00:00:00][6826f1d0][    main:callHello.bst                 ] SUCCESS Staging dependencies
[--:--:--][6826f1d0][    main:callHello.bst                 ] START   Integrating sandbox
[00:00:00][6826f1d0][    main:callHello.bst                 ] SUCCESS Integrating sandbox
[--:--:--][6826f1d0][    main:callHello.bst                 ] STATUS  Running command

    /bin/sh callHello.sh

WARNING: The --use-localcas option will be deprecated. LocalCAS support is now enabled by default.
2021-03-25T15:48:41.029+0000 [4460:140011828600640] [buildboxcommon_client.cpp:86] [INFO] Setting d_maxBatchTotalSizeBytes = 4128768 bytes by default
Calling hello:
Hello World!
This is amhello 1.0.

1.3. Further reading

For an example of junction elements being used in a real project, take a look at the freedesktop-sdk junction in the gnome-build-meta project.