Utilities

exception UtilError(message, reason=None)

Bases: BstError

Raised by utility functions when system calls fail.

This will be handled internally by the BuildStream core, if you need to handle this error, then it should be reraised, or either of the ElementError or SourceError exceptions should be raised from this error.

exception ProgramNotFoundError(message, reason=None)

Bases: BstError

Raised if a required program is not found.

It is normally unneeded to handle this exception from plugin code.

exception DirectoryExistsError

Bases: OSError

Raised when a os.rename is attempted but the destination is an existing directory.

class FileListResult

Bases: object

An object which stores the result of one of the operations which run on a list of files.

overwritten

List of files which were overwritten in the target directory

ignored

List of files which were ignored, because they would have replaced a non empty directory

failed_attributes

List of files for which attributes could not be copied over

files_written

List of files that were written.

list_relative_paths(directory: str) → Iterator[str]

A generator for walking directory relative paths

This generator is useful for checking the full manifest of a directory.

Symbolic links will not be followed, but will be included in the manifest.

Parameters:

directory – The directory to list files in

Yields:

Relative filenames in directory

glob(paths: Iterable[str], pattern: str) → Iterator[str]

A generator to yield paths which match the glob pattern

Parameters:

• paths (iterable) – The paths to check

• pattern (str) – A glob pattern

This generator will iterate over the passed paths and yield only the filenames which matched the provided pattern.

Meta	Description
*	Zero or more of any character, excepting path separators
**	Zero or more of any character, including path separators
?	One of any character, except for path separators
[abc]	One of any of the specified characters
[a-z]	One of the characters in the specified range
[!abc]	Any single character, except the specified characters
[!a-z]	Any single character, except those in the specified range

Note

Escaping of the metacharacters is not possible

sha256sum(filename: str) → str

Calculate the sha256sum of a file

Parameters:

filename – A path to a file on disk

Returns:

An sha256 checksum string

Raises:

UtilError – In the case there was an issue opening or reading filename

safe_copy(src: str, dest: str, *, copystat: bool = True, result: FileListResult | None = None) → None

Copy a file while optionally preserving attributes

Parameters:

• src – The source filename

• dest – The destination filename

• copystat – Whether to preserve attributes

• result – An optional collective result

Raises:

UtilError – In the case of unexpected system call failures

This is almost the same as shutil.copy2() when copystat is True, except that we unlink dest before overwriting it if it exists, just incase dest is a hardlink to a different file.

safe_link(src: str, dest: str, *, result: FileListResult | None = None, _unlink=False) → None

Try to create a hardlink, but resort to copying in the case of cross device links.

Parameters:

• src – The source filename

• dest – The destination filename

• result – An optional collective result

Raises:

UtilError – In the case of unexpected system call failures

safe_remove(path: str) → bool

Removes a file or directory

This will remove a file if it exists, and will remove a directory if the directory is empty.

Parameters:

path – The path to remove

Returns:

True if path was removed or did not exist, False if path was a non empty directory.

Raises:

UtilError – In the case of unexpected system call failures

copy_files(src: str, dest: str, *, filter_callback: Callable[[str], bool] | None = None, ignore_missing: bool = False, report_written: bool = False) → FileListResult

Copy files from source to destination.

Parameters:

• src – The source directory

• dest – The destination directory

• filter_callback – Optional filter callback. Called with the relative path as argument for every file in the source directory. The file is copied only if the callable returns True. If no filter callback is specified, all files will be copied.

• ignore_missing – Dont raise any error if a source file is missing

• report_written – Add to the result object the full list of files written

Returns:

The result describing what happened during this file operation

Raises:

UtilError – In the case of unexpected system call failures

Note

Directories in dest are replaced with files from src, unless the existing directory in dest is not empty in which case the path will be reported in the return value.

UNIX domain socket files from src are ignored.

link_files(src: str, dest: str, *, filter_callback: Callable[[str], bool] | None = None, ignore_missing: bool = False, report_written: bool = False) → FileListResult

Hardlink files from source to destination.

Parameters:

• src – The source directory

• dest – The destination directory

• filter_callback – Optional filter callback. Called with the relative path as argument for every file in the source directory. The file is hardlinked only if the callable returns True. If no filter callback is specified, all files will be hardlinked.

• ignore_missing – Dont raise any error if a source file is missing

• report_written – Add to the result object the full list of files written

Returns:

The result describing what happened during this file operation

Raises:

UtilError – In the case of unexpected system call failures

Note

Directories in dest are replaced with files from src, unless the existing directory in dest is not empty in which case the path will be reported in the return value.

Note

If a hardlink cannot be created due to crossing filesystems, then the file will be copied instead.

UNIX domain socket files from src are ignored.

get_host_tool(name: str) → str

Get the full path of a host tool

Parameters:

name (str) – The name of the program to search for

Returns:

The full path to the program, if found

Raises:

.ProgramNotFoundError –

get_bst_version() → Tuple[int, int]

Gets the major, minor release portion of the BuildStream version.

Returns:

A 2-tuple of form (major version, minor version)

move_atomic(source: Path | str, destination: Path | str, *, ensure_parents: bool = True) → None

Move the source to the destination using atomic primitives.

This uses os.rename to move a file or directory to a new destination. It wraps some OSError thrown errors to ensure their handling is correct.

The main reason for this to exist is that rename can throw different errors for the same symptom (https://www.unix.com/man-page/POSIX/3posix/rename/) when we are moving a directory.

We are especially interested here in the case when the destination already exists, is a directory and is not empty. In this case, either EEXIST or ENOTEMPTY can be thrown.

In order to ensure consistent handling of these exceptions, this function should be used instead of os.rename

Parameters:

• source – source to rename

• destination – destination to which to move the source

• ensure_parents – Whether or not to create the parent’s directories of the destination (default: True)

Raises:

• DirectoryExistsError – if the destination directory already exists and is not empty

• OSError – if another filesystem level error occured

save_file_atomic(filename: str, mode: str = 'w', *, buffering: int = -1, encoding: str | None = None, errors: str | None = None, newline: str | None = None, closefd: bool = True, opener: Callable[[str, int], int] | None = None, tempdir: str | None = None) → Iterator[IO]

Save a file with a temporary name and rename it into place when ready.

This is a context manager which is meant for saving data to files. The data is written to a temporary file, which gets renamed to the target name when the context is closed. This avoids readers of the file from getting an incomplete file.

Example:

with save_file_atomic('/path/to/foo', 'w') as f:
    f.write(stuff)

The file will be called something like tmpCAFEBEEF until the context block ends, at which point it gets renamed to foo. The temporary file will be created in the same directory as the output file. The filename parameter must be an absolute path.

If an exception occurs or the process is terminated, the temporary file will be deleted.

get_umask()