1. Junction elements
BuildStream’s junction elements are the mechanism which
allow projects to interact and depend on eachother.
Junction elements represent the BuildStream project you are depending, and behave much like other elements in the sense that they can be fetched and tracked like other elements, except that regular elements cannot depend on junctions directly, nor can junctions be built. Instead, junctions act like a window into another project you depend on, and allow elements of your project to depend on elements exposed by the project referenced by the junction.
Projects which are junctioned by your project are referred to as subprojects.
1.1. A simple example
Note
This example is distributed with BuildStream in the doc/examples/junctions subdirectory.
Below is a simple example of bst file for the junction element, which
we have called hello-junction.bst in this project:
kind: junction
sources:
- kind: local
path: autotools
This element imports the autotools example subproject distributed with BuildStream in the doc/examples/junctions/autotools subdirectory.
Note
For the sake of this example we are using a local source in a subdirectory of the example project.
Since junctions allow interoperability of projects, it would be more common
to use a junction to a remote project under separate revision control, possibly
using a kind: git source.
The below bst file describes the element callHello.bst, which depends on the
hello.bst element from the autotools example:
kind: import
sources:
- kind: local
path: files/callHello.sh
depends:
- filename: hello.bst
junction: hello-junction.bst
Note how this element refers to the previously declared hello-junction.bst
junction in its dependency dictionary. This dependency
expresses that we are depending on the hello.bst element in the project
which hello-junction.bst refers to.
The callHello.bst element simply imports a callHello.sh shell script which
calls the hello command provided by hello.bst:
#!/bin/sh
echo "Calling hello:"
hello
1.2. Building and running
Building the callHello.bst element which requires an external project
is just a matter of invoking bst build in the
regular way:
user@host:~/junctions$ bst build callHello.bst
[--:--:--][ ][ main:core activity ] START Build
[--:--:--][ ][ main:core activity ] START Loading elements
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1730888235.804244 5363 config.cc:230] gRPC experiments enabled: call_status_override_on_cancellation, event_engine_dns, event_engine_listener, http2_stats_fix, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
[--:--:--][6d2e5da0][ fetch:hello-junction.bst ] START junctions/hello-junction/6d2e5da0-fetch.20241106-101715.log
[00:00:00][6d2e5da0][ fetch:hello-junction.bst ] SUCCESS junctions/hello-junction/6d2e5da0-fetch.20241106-101715.log
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Initializing remote caches
[00:00:00][ ][ main:core activity ] SUCCESS Initializing remote caches
[--:--:--][ ][ main:core activity ] START Query cache
[00:00:00][ ][ main:core activity ] SUCCESS Query cache
BuildStream Version 2.3.0+26.g65e2074ad
Session Start: Wednesday, 06-11-2024 at 10:17:15
Project: junctions (/home/user/buildstream/doc/examples/junctions)
Targets: callHello.bst
User Configuration
Configuration File: /home/user/buildstream/doc/run-bst-phxqlxaq/buildstream.conf
Cache Directory: /home/user/buildstream/doc/run-bst-phxqlxaq
Log Files: /home/user/buildstream/doc/run-bst-phxqlxaq/logs
Source Mirrors: /home/user/buildstream/doc/run-bst-phxqlxaq/sources
Build Area: /home/user/buildstream/doc/run-bst-phxqlxaq/build
Strict Build Plan: Yes
Maximum Fetch Tasks: 10
Maximum Build Tasks: 4
Maximum Push Tasks: 4
Maximum Network Retries: 2
Project: junctions
Element Plugins
junction: core plugin
import: core plugin
Source Plugins
local: core plugin
Project: autotools
Junction path: hello-junction.bst
Loaded by: callHello.bst [line 8 column 4]
Element Plugins
autotools: python package 'buildstream-plugins 2.2.0' at: /home/user/buildstream/.tox/docs/lib/python3.12/site-packages
stack: core plugin
import: core plugin
Source Plugins
tar: core plugin
Pipeline
fetch needed 6a78c03097648e558c803c20d8d20f61d3e87e3e616f3a45923704cfe95019af hello-junction.bst:base/alpine.bst
waiting 644e964abe673c8a8723cf5a3fceeeda44a7bea364f55604fc8132af44a4fb59 hello-junction.bst:base.bst
fetch needed ffeab61203b5940714c33b1cc29822b9391b9fea5876faef23c024bf6acd99cf hello-junction.bst:hello.bst
waiting b6da6abe68420499ec616c50433619027a915e37217b85450a5590a28bd28820 callHello.bst
===============================================================================
[--:--:--][644e964a][ fetch:hello-junction.bst:base.bst ] START autotools/base/644e964a-fetch.20241106-101715.log
[--:--:--][6a78c030][ fetch:hello-junction.bst:base/alpine.bst] START autotools/base-alpine/6a78c030-fetch.20241106-101715.log
[--:--:--][ffeab612][ fetch:hello-junction.bst:hello.bst ] START autotools/hello/ffeab612-fetch.20241106-101715.log
[--:--:--][b6da6abe][ fetch:callHello.bst ] START junctions/callHello/b6da6abe-fetch.20241106-101715.log
[--:--:--][ffeab612][ fetch:hello-junction.bst:hello.bst ] START Fetching https://ftpmirror.gnu.org/gnu/automake/automake-1.16.tar.gz
[--:--:--][6a78c030][ fetch:hello-junction.bst:base/alpine.bst] START Fetching https://bst-integration-test-images.ams3.cdn.digitaloceanspaces.com/integration-tests-base.v1.x86_64.tar.xz
[00:00:00][644e964a][ fetch:hello-junction.bst:base.bst ] SUCCESS autotools/base/644e964a-fetch.20241106-101715.log
[00:00:00][b6da6abe][ fetch:callHello.bst ] SUCCESS junctions/callHello/b6da6abe-fetch.20241106-101715.log
[00:00:00][6a78c030][ fetch:hello-junction.bst:base/alpine.bst] SUCCESS Fetching https://bst-integration-test-images.ams3.cdn.digitaloceanspaces.com/integration-tests-base.v1.x86_64.tar.xz
[00:00:00][ffeab612][ fetch:hello-junction.bst:hello.bst ] SUCCESS Fetching https://ftpmirror.gnu.org/gnu/automake/automake-1.16.tar.gz
[00:00:01][ffeab612][ fetch:hello-junction.bst:hello.bst ] SUCCESS autotools/hello/ffeab612-fetch.20241106-101715.log
[00:00:06][6a78c030][ fetch:hello-junction.bst:base/alpine.bst] SUCCESS autotools/base-alpine/6a78c030-fetch.20241106-101715.log
[--:--:--][6a78c030][ build:hello-junction.bst:base/alpine.bst] START autotools/base-alpine/6a78c030-build.20241106-101722.log
[--:--:--][6a78c030][ build:hello-junction.bst:base/alpine.bst] START Staging sources
[00:00:00][6a78c030][ build:hello-junction.bst:base/alpine.bst] SUCCESS Staging sources
[--:--:--][6a78c030][ build:hello-junction.bst:base/alpine.bst] START Caching artifact
[00:00:00][6a78c030][ build:hello-junction.bst:base/alpine.bst] SUCCESS Caching artifact
[00:00:00][6a78c030][ build:hello-junction.bst:base/alpine.bst] SUCCESS autotools/base-alpine/6a78c030-build.20241106-101722.log
[--:--:--][644e964a][ build:hello-junction.bst:base.bst ] START autotools/base/644e964a-build.20241106-101722.log
[--:--:--][644e964a][ build:hello-junction.bst:base.bst ] START Caching artifact
[00:00:00][644e964a][ build:hello-junction.bst:base.bst ] SUCCESS Caching artifact
[00:00:00][644e964a][ build:hello-junction.bst:base.bst ] SUCCESS autotools/base/644e964a-build.20241106-101722.log
[--:--:--][ffeab612][ build:hello-junction.bst:hello.bst ] START autotools/hello/ffeab612-build.20241106-101722.log
[--:--:--][ffeab612][ build:hello-junction.bst:hello.bst ] START Staging dependencies at: /
[00:00:00][ffeab612][ build:hello-junction.bst:hello.bst ] SUCCESS Staging dependencies at: /
[--:--:--][ffeab612][ build:hello-junction.bst:hello.bst ] START Integrating sandbox
[00:00:00][ffeab612][ build:hello-junction.bst:hello.bst ] SUCCESS Integrating sandbox
[--:--:--][ffeab612][ build:hello-junction.bst:hello.bst ] START Staging sources
[00:00:00][ffeab612][ build:hello-junction.bst:hello.bst ] SUCCESS Staging sources
[--:--:--][ffeab612][ build:hello-junction.bst:hello.bst ] START Running commands
export NOCONFIGURE=1;
if [ -x ./configure ]; then true;
elif [ -x ./autogen ]; then ./autogen;
elif [ -x ./autogen.sh ]; then ./autogen.sh;
elif [ -x ./bootstrap ]; then ./bootstrap;
elif [ -x ./bootstrap.sh ]; then ./bootstrap.sh;
else autoreconf -ivf .;
fi
./configure --prefix=/usr \
--exec-prefix=/usr \
--bindir=/usr/bin \
--sbindir=/usr/sbin \
--sysconfdir=/etc \
--datadir=/usr/share \
--includedir=/usr/include \
--libdir=/usr/lib \
--libexecdir=/usr/libexec \
--localstatedir=/var \
--sharedstatedir=/usr/com \
Message contains 23 additional lines
[00:00:03][ffeab612][ build:hello-junction.bst:hello.bst ] SUCCESS Running commands
[--:--:--][ffeab612][ build:hello-junction.bst:hello.bst ] START Caching artifact
[00:00:00][ffeab612][ build:hello-junction.bst:hello.bst ] SUCCESS Caching artifact
[00:00:04][ffeab612][ build:hello-junction.bst:hello.bst ] SUCCESS autotools/hello/ffeab612-build.20241106-101722.log
[--:--:--][b6da6abe][ build:callHello.bst ] START junctions/callHello/b6da6abe-build.20241106-101726.log
[--:--:--][b6da6abe][ build:callHello.bst ] START Staging sources
[00:00:00][b6da6abe][ build:callHello.bst ] SUCCESS Staging sources
[--:--:--][b6da6abe][ build:callHello.bst ] START Caching artifact
[00:00:00][b6da6abe][ build:callHello.bst ] SUCCESS Caching artifact
[00:00:00][b6da6abe][ build:callHello.bst ] SUCCESS junctions/callHello/b6da6abe-build.20241106-101726.log
[00:00:10][ ][ main:core activity ] SUCCESS Build
Pipeline Summary
Total: 4
Session: 4
Fetch Queue: processed 4, skipped 0, failed 0
Build Queue: processed 4, skipped 0, failed 0
You can see that the hello.bst element and its dependencies from the autotools project have been built as a part of the pipeline for callHello.bst.
We can now invoke bst shell and run our callHello.sh
script, which in turn also calls the hello program installed by the
subproject’s hello.bst element.
user@host:~/junctions$ bst shell callHello.bst -- /bin/sh callHello.sh
[--:--:--][ ][ main:core activity ] START Loading elements
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1730888247.185803 6452 config.cc:230] gRPC experiments enabled: call_status_override_on_cancellation, event_engine_dns, event_engine_listener, http2_stats_fix, monitoring_experiment, pick_first_new, trace_record_callops, work_serializer_clears_time_cache
[00:00:00][ ][ main:core activity ] SUCCESS Loading elements
[--:--:--][ ][ main:core activity ] START Resolving elements
[00:00:00][ ][ main:core activity ] SUCCESS Resolving elements
[--:--:--][ ][ main:core activity ] START Initializing remote caches
[00:00:00][ ][ main:core activity ] SUCCESS Initializing remote caches
[--:--:--][ ][ main:core activity ] START Query cache
[00:00:00][ ][ main:core activity ] SUCCESS Query cache
[--:--:--][b6da6abe][ main:callHello.bst ] START Staging dependencies
[00:00:00][b6da6abe][ main:callHello.bst ] SUCCESS Staging dependencies
[--:--:--][b6da6abe][ main:callHello.bst ] START Integrating sandbox
[00:00:00][b6da6abe][ main:callHello.bst ] SUCCESS Integrating sandbox
[--:--:--][b6da6abe][ main:callHello.bst ] STATUS Running command
/bin/sh callHello.sh
Calling hello:
Hello World!
This is amhello 1.0.
1.3. Further reading
For an example of junction elements being used in a real project, take a look at the freedesktop-sdk junction in the gnome-build-meta project.