1. Workspaces

In this section we will cover the use of BuildStream’s workspaces feature when devloping on a BuildStream project.

Note

This example is distributed with BuildStream in the doc/examples/developing subdirectory.

We will start with the project used in the running commands tutorial. Recall the element hello.bst, which builds the bellow C file:

/*
 * hello.c - Simple hello world program
 */
#include <stdio.h>

int main(int argc, char *argv[])
{
  printf("Hello World\n");
  return 0;
}

Suppose we now want to alter the functionality of the hello command. We can make changes to the source code of Buildstream elements by making use of BuildStream’s workspace command.

1.1. Opening a workspace

First we need to open a workspace, we can do this by running

user@host:~/developing$ bst workspace open --directory workspace_hello hello.bst

[--:--:--][        ][ main:core activity                 ] START   Loading elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Loading elements
[--:--:--][        ][ main:core activity                 ] START   Resolving elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving elements
[--:--:--][        ][ main:core activity                 ] START   Resolving cached state
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving cached state
[--:--:--][        ][ main:core activity                 ] START   Checking sources
[00:00:00][        ][ main:core activity                 ] SUCCESS Checking sources
[--:--:--][        ][ main:core activity                 ] INFO    Creating workspace for element hello.bst
[--:--:--][41b51a92][ main:hello.bst                     ] START   Staging sources to /home/user/developing/workspace_hello
[--:--:--][        ][ main:hello.bst-0                   ] START   Staging local files at files/src
[00:00:00][        ][ main:hello.bst-0                   ] SUCCESS Staging local files at files/src
[00:00:00][41b51a92][ main:hello.bst                     ] SUCCESS Staging sources to /home/user/developing/workspace_hello
[--:--:--][        ][ main:core activity                 ] INFO    Created a workspace for element: hello.bst

This command has created the workspace_hello directory in which you can see the source for the hello.bst element, i.e. hello.c and the corresponding makefile.

You can view existing workspaces using

user@host:~/developing$ bst workspace list

workspaces:
- element: hello.bst
  directory: /home/user/developing/workspace_hello

1.2. Making code changes

Let’s say we want to alter the message printed when the hello command is run. We can open workspace_hello/hello.c and make the following change:

-- hello.c	2018-06-25 14:48:32.077568920 +0100
+++ hello.c	2018-06-25 14:49:23.025553785 +0100
@@ -5,6 +5,6 @@
 
 int main(int argc, char *argv[])
 {
-  printf("Hello World\n");
+  printf("Hello World\nWe can use workspaces!\n");
   return 0;
 }

Now, rebuild the hello.bst element.

user@host:~/developing$ bst build hello.bst

[--:--:--][        ][ main:core activity                 ] START   Build
[--:--:--][        ][ main:core activity                 ] START   Loading elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Loading elements
[--:--:--][        ][ main:core activity                 ] START   Resolving elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving elements
[--:--:--][        ][ main:core activity                 ] START   Resolving cached state
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving cached state
[--:--:--][        ][ main:core activity                 ] START   Checking sources
[00:00:00][        ][ main:core activity                 ] SUCCESS Checking sources

BuildStream Version 1.3.0+1587.g36a60abe
  Session Start: Tuesday, 19-02-2019 at 18:54:56
  Project:       developing (/home/user/developing)
  Targets:       hello.bst
  Cache Usage:   12K / None (0%)

User Configuration
  Configuration File:      /home/user/.config/buildstream.conf
  Cache Directory:         /home/user/.cache/buildstream
  Log Files:               /home/user/.cache/buildstream/logs
  Source Mirrors:          /home/user/.cache/buildstream/sources
  Build Area:              /home/user/.cache/buildstream/build
  Strict Build Plan:       Yes
  Maximum Fetch Tasks:     10
  Maximum Build Tasks:     4
  Maximum Push Tasks:      4
  Maximum Network Retries: 2

Pipeline
   buildable 4a31d9cf5fcd8bed64c35acd0451ad5c3c06813ee2febab29149d1b849e32f16 base/alpine.bst 
     waiting 20182cc27932cf6633aa2cbaee86fe04f56ac112feef21d9a3759f0991d48f35 base.bst 
     waiting ???????????????????????????????????????????????????????????????? hello.bst Workspace: /home/user/developing/workspace_hello
===============================================================================
[--:--:--][4a31d9cf][build:base/alpine.bst               ] START   developing/base-alpine/4a31d9cf-build.690.log
[--:--:--][4a31d9cf][build:base/alpine.bst               ] START   Staging sources
[00:00:11][4a31d9cf][build:base/alpine.bst               ] SUCCESS Staging sources
[--:--:--][4a31d9cf][build:base/alpine.bst               ] START   Caching artifact
[00:00:06][4a31d9cf][build:base/alpine.bst               ] SUCCESS Caching artifact
[00:00:20][4a31d9cf][build:base/alpine.bst               ] SUCCESS developing/base-alpine/4a31d9cf-build.690.log
[--:--:--][20182cc2][build:base.bst                      ] START   developing/base/20182cc2-build.695.log
[--:--:--][20182cc2][build:base.bst                      ] START   Caching artifact
[00:00:00][20182cc2][build:base.bst                      ] SUCCESS Caching artifact
[00:00:00][20182cc2][build:base.bst                      ] SUCCESS developing/base/20182cc2-build.695.log
[--:--:--][????????][build:hello.bst                     ] START   developing/hello/????????-build.700.log
[--:--:--][????????][build:hello.bst                     ] START   Staging dependencies
[00:00:00][????????][build:hello.bst                     ] SUCCESS Staging dependencies
[--:--:--][????????][build:hello.bst                     ] START   Integrating sandbox
[00:00:00][????????][build:hello.bst                     ] SUCCESS Integrating sandbox
[--:--:--][????????][build:hello.bst                     ] START   Staging sources
[00:00:00][????????][build:hello.bst                     ] SUCCESS Staging sources
[--:--:--][????????][build:hello.bst                     ] START   Running build-commands
[--:--:--][????????][build:hello.bst                     ] STATUS  Running command

    make PREFIX="/usr"

[00:00:00][????????][build:hello.bst                     ] SUCCESS Running build-commands
[--:--:--][????????][build:hello.bst                     ] START   Running install-commands
[--:--:--][????????][build:hello.bst                     ] STATUS  Running command

    make -j1 PREFIX="/usr" DESTDIR="/buildstream-install" install

[00:00:00][????????][build:hello.bst                     ] SUCCESS Running install-commands
[--:--:--][????????][build:hello.bst                     ] START   Running strip-commands
[00:00:00][????????][build:hello.bst                     ] SUCCESS Running strip-commands
[--:--:--][????????][build:hello.bst                     ] START   Caching artifact
[00:00:00][????????][build:hello.bst                     ] SUCCESS Caching artifact
[00:00:01][????????][build:hello.bst                     ] SUCCESS developing/hello/????????-build.700.log
[00:00:22][        ][ main:core activity                 ] SUCCESS Build

Resolved key Summary
      cached e430437a3875406ae99ca798f98e703067330996443dee0850cf521155eaecc0 hello.bst Workspace: /home/user/developing/workspace_hello

Pipeline Summary
  Total:       3
  Session:     3
  Fetch Queue: processed 0, skipped 3, failed 0 
  Build Queue: processed 3, skipped 0, failed 0

Note that if you run the command from inside the workspace, the element name is optional.

user@host:~/workspace_hello$ bst build

[--:--:--][        ][ main:core activity                 ] START   Build
[--:--:--][        ][ main:core activity                 ] START   Loading elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Loading elements
[--:--:--][        ][ main:core activity                 ] START   Resolving elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving elements
[--:--:--][        ][ main:core activity                 ] START   Resolving cached state
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving cached state
[--:--:--][        ][ main:core activity                 ] START   Checking sources
[00:00:00][        ][ main:core activity                 ] SUCCESS Checking sources

BuildStream Version 1.3.0+1587.g36a60abe
  Session Start: Tuesday, 19-02-2019 at 18:55:19
  Project:       developing (/builds/BuildStream/buildstream/doc/examples/developing)
  Targets:       hello.bst
  Cache Usage:   135.3M / None (0%)

User Configuration
  Configuration File:      /home/user/.config/buildstream.conf
  Cache Directory:         /home/user/.cache/buildstream
  Log Files:               /home/user/.cache/buildstream/logs
  Source Mirrors:          /home/user/.cache/buildstream/sources
  Build Area:              /home/user/.cache/buildstream/build
  Strict Build Plan:       Yes
  Maximum Fetch Tasks:     10
  Maximum Build Tasks:     4
  Maximum Push Tasks:      4
  Maximum Network Retries: 2

Pipeline
      cached 4a31d9cf5fcd8bed64c35acd0451ad5c3c06813ee2febab29149d1b849e32f16 base/alpine.bst 
      cached 20182cc27932cf6633aa2cbaee86fe04f56ac112feef21d9a3759f0991d48f35 base.bst 
      cached e430437a3875406ae99ca798f98e703067330996443dee0850cf521155eaecc0 hello.bst Workspace: /home/user/workspace_hello
===============================================================================
[00:00:00][        ][ main:core activity                 ] SUCCESS Build

Pipeline Summary
  Total:       3
  Session:     0
  Fetch Queue: processed 0, skipped 0, failed 0 
  Build Queue: processed 0, skipped 0, failed 0

Now running the hello command using bst shell:

user@host:~/developing$ bst shell hello.bst -- hello

[--:--:--][        ][ main:core activity                 ] START   Loading elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Loading elements
[--:--:--][        ][ main:core activity                 ] START   Resolving elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving elements
[--:--:--][        ][ main:core activity                 ] START   Resolving cached state
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving cached state
[--:--:--][e430437a][ main:hello.bst                     ] START   Staging dependencies
[00:00:00][e430437a][ main:hello.bst                     ] SUCCESS Staging dependencies
[--:--:--][e430437a][ main:hello.bst                     ] START   Integrating sandbox
[00:00:00][e430437a][ main:hello.bst                     ] SUCCESS Integrating sandbox
[--:--:--][e430437a][ main:hello.bst                     ] STATUS  Running command

    hello

Hello World
We can use workspaces!

This gives us the new message we changed in hello.c.

From this point we have several options. If the source is under version control we can commit our changes and push them to the remote repository.

1.3. Closing your workspace

If we want to close the workspace and come back to our changes later, we can

user@host:~/developing$ bst workspace close hello.bst

[--:--:--][        ][ main:core activity                 ] START   Loading elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Loading elements
[--:--:--][        ][ main:core activity                 ] START   Resolving elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving elements
[--:--:--][        ][ main:core activity                 ] START   Resolving cached state
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving cached state
[--:--:--][        ][ main:core activity                 ] INFO    Closed workspace for hello.bst

We can then reopen the workspace later using:

user@host:~/developing$ bst workspace open --no-checkout --directory workspace_hello hello.bst

[--:--:--][        ][ main:core activity                 ] START   Loading elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Loading elements
[--:--:--][        ][ main:core activity                 ] START   Resolving elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving elements
[--:--:--][        ][ main:core activity                 ] START   Resolving cached state
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving cached state
[--:--:--][        ][ main:core activity                 ] INFO    Creating workspace for element hello.bst
[--:--:--][        ][ main:core activity                 ] INFO    Created a workspace for element: hello.bst

The –no-checkout option tells BuildStream not to check the source out but to instead hard-link to the workspace_hello directory.

Alternatively, if we wish to discard the changes we can use

user@host:~/developing$ bst workspace reset hello.bst

[--:--:--][        ][ main:core activity                 ] START   Loading elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Loading elements
[--:--:--][        ][ main:core activity                 ] START   Resolving elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving elements
[--:--:--][        ][ main:core activity                 ] START   Resolving cached state
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving cached state
[--:--:--][e430437a][ main:hello.bst                     ] START   Removing workspace directory /home/user/developing/workspace_hello
[00:00:00][e430437a][ main:hello.bst                     ] SUCCESS Removing workspace directory /home/user/developing/workspace_hello
[--:--:--][e430437a][ main:hello.bst                     ] START   Staging sources to /home/user/developing/workspace_hello
[--:--:--][        ][ main:hello.bst-0                   ] START   Staging local files at files/src
[00:00:00][        ][ main:hello.bst-0                   ] SUCCESS Staging local files at files/src
[00:00:00][e430437a][ main:hello.bst                     ] SUCCESS Staging sources to /home/user/developing/workspace_hello
[--:--:--][        ][ main:core activity                 ] INFO    Reset workspace for hello.bst at: /home/user/developing/workspace_hello

This resets the workspace to its original state.

To discard the workspace completely we can do:

user@host:~/developing$ bst workspace close --remove-dir hello.bst

[--:--:--][        ][ main:core activity                 ] START   Loading elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Loading elements
[--:--:--][        ][ main:core activity                 ] START   Resolving elements
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving elements
[--:--:--][        ][ main:core activity                 ] START   Resolving cached state
[00:00:00][        ][ main:core activity                 ] SUCCESS Resolving cached state
[--:--:--][        ][ main:core activity                 ] START   Removing workspace directory /home/user/developing/workspace_hello
[00:00:00][        ][ main:core activity                 ] SUCCESS Removing workspace directory /home/user/developing/workspace_hello
[--:--:--][        ][ main:core activity                 ] INFO    Closed workspace for hello.bst

This will close the workspace and completely remove the workspace_hello directory.