Utilities

exception UtilError(message, reason=None)

Bases: buildstream._exceptions.BstError

Raised by utility functions when system calls fail.

This will be handled internally by the BuildStream core, if you need to handle this error, then it should be reraised, or either of the ElementError or SourceError exceptions should be raised from this error.

exception ProgramNotFoundError(message, reason=None)

Bases: buildstream._exceptions.BstError

Raised if a required program is not found.

It is normally unneeded to handle this exception from plugin code.

class FileListResult

Bases: object

An object which stores the result of one of the operations which run on a list of files.

overwritten = None

List of files which were overwritten in the target directory

ignored = None

List of files which were ignored, because they would have replaced a non empty directory

failed_attributes = None

List of files for which attributes could not be copied over

files_written = None

List of files that were written.

combine(other)

Create a new FileListResult that contains the results of both.

list_relative_paths(directory, *, list_dirs=True)

A generator for walking directory relative paths

This generator is useful for checking the full manifest of a directory.

Note that directories will be yielded only if they are empty.

Symbolic links will not be followed, but will be included in the manifest.

Parameters:
  • directory (str) – The directory to list files in
  • list_dirs (bool) – Whether to list directories
Yields:

Relative filenames in directory
glob(paths, pattern)

A generator to yield paths which match the glob pattern

Parameters:
  • paths (iterable) – The paths to check
  • pattern (str) – A glob pattern

This generator will iterate over the passed paths and yield only the filenames which matched the provided pattern.

Meta Description
* Zero or more of any character, excepting path separators
** Zero or more of any character, including path separators
? One of any character, except for path separators
[abc] One of any of the specified characters
[a-z] One of the characters in the specified range
[!abc] Any single character, except the specified characters
[!a-z] Any single character, except those in the specified range

Note

Escaping of the metacharacters is not possible

sha256sum(filename)

Calculate the sha256sum of a file

Parameters:filename (str) – A path to a file on disk
Returns:An sha256 checksum string
Return type:(str)
Raises:UtilError – In the case there was an issue opening or reading filename
safe_copy(src, dest, *, result=None)

Copy a file while preserving attributes

Parameters:
  • src (str) – The source filename
  • dest (str) – The destination filename
  • result (FileListResult) – An optional collective result
Raises:

UtilError – In the case of unexpected system call failures

This is almost the same as shutil.copy2(), except that we unlink dest before overwriting it if it exists, just incase dest is a hardlink to a different file.

Try to create a hardlink, but resort to copying in the case of cross device links.

Parameters:
  • src (str) – The source filename
  • dest (str) – The destination filename
  • result (FileListResult) – An optional collective result
Raises:

UtilError – In the case of unexpected system call failures
safe_remove(path)

Removes a file or directory

This will remove a file if it exists, and will remove a directory if the directory is empty.

Parameters:path (str) – The path to remove
Returns:True if path was removed or did not exist, False if path was a non empty directory.
Raises:UtilError – In the case of unexpected system call failures
copy_files(src, dest, *, files=None, ignore_missing=False, report_written=False)

Copy files from source to destination.

Parameters:
  • src (str) – The source file or directory
  • dest (str) – The destination directory
  • files (list) – Optional list of files in src to copy
  • ignore_missing (bool) – Dont raise any error if a source file is missing
  • report_written (bool) – Add to the result object the full list of files written
Returns:

The result describing what happened during this file operation
Return type:

(FileListResult)
Raises:

UtilError – In the case of unexpected system call failures

Note

Directories in dest are replaced with files from src, unless the existing directory in dest is not empty in which case the path will be reported in the return value.

Hardlink files from source to destination.

Parameters:
  • src (str) – The source file or directory
  • dest (str) – The destination directory
  • files (list) – Optional list of files in src to link
  • ignore_missing (bool) – Dont raise any error if a source file is missing
  • report_written (bool) – Add to the result object the full list of files written
Returns:

The result describing what happened during this file operation
Return type:

(FileListResult)
Raises:

UtilError – In the case of unexpected system call failures

Note

Directories in dest are replaced with files from src, unless the existing directory in dest is not empty in which case the path will be reported in the return value.

Note

If a hardlink cannot be created due to crossing filesystems, then the file will be copied instead.

get_host_tool(name)

Get the full path of a host tool

Parameters:name (str) – The name of the program to search for
Returns:The full path to the program, if found
Raises:ProgramNotFoundError
url_directory_name(url)

Normalizes a url into a directory name

Parameters:url (str) – A url string
Returns:A string which can be used as a directory name
get_bst_version()

Gets the major, minor release portion of the BuildStream version.

Returns:The major version (int): The minor version
Return type:(int)
save_file_atomic(filename, mode='w', *, buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None, tempdir=None)

Save a file with a temporary name and rename it into place when ready.

This is a context manager which is meant for saving data to files. The data is written to a temporary file, which gets renamed to the target name when the context is closed. This avoids readers of the file from getting an incomplete file.

Example:

with save_file_atomic('/path/to/foo', 'w') as f:
    f.write(stuff)

The file will be called something like tmpCAFEBEEF until the context block ends, at which point it gets renamed to foo. The temporary file will be created in the same directory as the output file. The filename parameter must be an absolute path.

If an exception occurs or the process is terminated, the temporary file will be deleted.