.. _tutorial_running_commands: Running commands ================ In :ref:`the first chapter ` we only imported a file to create an artifact, this time lets run some commands inside the :ref:`isolated build sandbox `. .. note:: This example is distributed with BuildStream in the `doc/examples/running-commands `_ subdirectory. Overview -------- In this chapter, we'll be running commands inside the sandboxed execution environment and producing build output. We'll be compiling the following simple C file: ``files/src/hello.c`` ~~~~~~~~~~~~~~~~~~~~~ .. literalinclude:: ../../examples/running-commands/files/src/hello.c :language: c And we're going to build it using ``make``, using the following Makefile: ``files/src/Makefile`` ~~~~~~~~~~~~~~~~~~~~~~ .. literalinclude:: ../../examples/running-commands/files/src/Makefile :language: Makefile We'll be using the most fundamental :ref:`build element `, the :mod:`manual ` build element. The :mod:`manual ` element is the backbone on which all the other build elements are built, so understanding how it works at this level is helpful. Project structure ----------------- In this project we have a ``project.conf``, a directory with some source code, and 3 element declarations. Let's first take a peek at what we need to build using :ref:`bst show `: .. raw:: html :file: ../sessions/running-commands-show-before.html This time we have loaded a pipeline with 3 elements, let's go over what they do in detail. .. _tutorial_running_commands_project_conf: ``project.conf`` ~~~~~~~~~~~~~~~~ .. literalinclude:: ../../examples/running-commands/project.conf :language: yaml Our ``project.conf`` is very much like the last one, except that we have defined a :ref:`source alias ` for ``alpine``. .. tip:: Using :ref:`source aliases ` for groups of sources which are generally hosted together is encouraged. This allows one to globally change the access scheme or URL for a group of repositories which belong together. ``elements/base/alpine.bst`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. literalinclude:: ../../examples/running-commands/elements/base/alpine.bst :language: yaml This :mod:`import ` element uses a :mod:`tar ` source to download our Alpine Linux tarball to create our base runtime. This tarball is a sysroot which provides the C runtime libraries and some programs - this is what will be providing the programs we're going to run in this example. ``elements/base.bst`` ~~~~~~~~~~~~~~~~~~~~~ .. literalinclude:: ../../examples/running-commands/elements/base.bst :language: yaml This is just a symbolic :mod:`stack ` element which declares that anything which depends on it, will implicitly depend on ``base/alpine.bst``. It is typical to use stack elements in places where the implementing logical software stack could change, but you rather not have your higher level components carry knowledge about those changing components. Any element which :ref:`runtime depends ` on the ``base.bst`` will now be able to execute programs provided by the imported ``base/alpine.bst`` runtime. ``elements/hello.bst`` ~~~~~~~~~~~~~~~~~~~~~~ .. literalinclude:: ../../examples/running-commands/elements/hello.bst :language: yaml Finally we have the element which executes commands. Looking at the :mod:`manual ` element's documentation, we can see that the element configuration exposes four command lists: * ``configure-commands`` Commands which are run in preparation of a build. This is where you would normally call any configure stage build tools to configure the build how you like and generate some files needed for the build. * ``build-commands`` Commands to run the build, usually a build system will invoke the compiler for you here. * ``install-commands`` Commands to install the build results. Commands to install the build results into the target system, these should install files somewhere under ``%{install-root}``. * ``strip-commands`` Commands to doctor the build results after the install. Typically this involves stripping binaries of debugging symbols or stripping timestamps from build results to ensure reproducibility. .. tip:: All other :ref:`build elements ` implement exactly the same command lists too, except that they provide default commands specific to invoke the build systems they support. The :mod:`manual ` element however is the most basic and does not provide any default commands, so we have instructed it to use ``make`` to build and install our program. Using the project ----------------- Build the hello.bst element ~~~~~~~~~~~~~~~~~~~~~~~~~~~ To build the project, run :ref:`bst build ` in the following way: .. raw:: html :file: ../sessions/running-commands-build.html Now we've built our hello world program, using ``make`` and the C compiler provided by the Alpine Linux image. In the :ref:`first chapter ` we observed that the inputs and output of an element are *directory trees*. In this example, the directory tree generated by ``base/alpine.bst`` is consumed by ``hello.bst`` due to the :ref:`implicit runtime dependency ` introduced by ``base.bst``. .. tip:: All of the :ref:`dependencies ` which are required to run for the sake of a build, are staged at the root of the build sandbox. These comprise the runtime environment in which the depending element will run commands. The result is that the ``make`` program and C compiler provided by ``base/alpine.bst`` were already in ``$PATH`` and ready to run when the commands were needed by ``hello.bst``. Now observe that all of the elements in the loaded pipeline are ``cached``, the element is *built*: .. raw:: html :file: ../sessions/running-commands-show-after.html Run the hello world program ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Now that we've built everything, we can indulge ourselves in running the hello world program using :ref:`bst shell `: .. raw:: html :file: ../sessions/running-commands-shell.html Here, :ref:`bst shell ` created a runtime environment for running the ``hello.bst`` element. This was done by staging all of the dependencies of ``hello.bst`` including the ``hello.bst`` output itself into a directory. Once a directory with all of the dependencies was staged and ready, we ran the ``hello`` command from within the build sandbox environment. .. tip:: When specifying a command for :ref:`bst shell ` to run, we always specify ``--`` first. This is a commonly understood shell syntax to indicate that the remaining arguments are to be treated literally. Specifying ``--`` is optional and disambiguates BuildStream's arguments and options from those of the program being run by :ref:`bst shell `. Summary ------- In this chapter we've explored how to use the :mod:`manual ` element, which forms the basis of all build elements. We've also observed how the directory tree from the output *artifact* of one element is later *staged* at the root of the sandbox, as input for use by any build elements which :ref:`depend ` on that element. .. tip:: The way that elements consume their dependency input can vary across the different *kinds* of elements. This chapter describes how it works for :mod:`build elements ` implementations, which are the most commonly used element type.