backup_files#

qx_utilities.general.snapshots.backup_files(source, target, filelist, store='original', overwrite=False, _log=None)#

backup_files source=<path to source folder> target=<path to target folder> filelist=<list of files> [store="original"] [overwrite=False]

Creates a backup of specified files from a source folder to a target location. Files are stored with sequential backup prefixes (b001_, b002_, etc.) and a file_list.txt manifest is created to track the original locations.

Parameters

--source (str):

The path to the source folder containing the files to back up. All file paths in filelist are resolved relative to this folder if they are not absolute paths.

--target (str):

The path to the target folder where backups will be stored. If the folder does not exist, it will be created. For store=zip mode, this should be the path without .zip extension (the .zip extension will be added automatically).

--filelist (list of str):

A list of file paths to back up. Paths can be absolute or relative to the source folder. The list can be provided as a Python list or as a comma-separated string.

--store (str, default 'original'):

Storage mode for the backup files:

  • 'original': Files are copied as-is to the target folder with backup prefixes. Directory structure is flattened - all files are stored in the root of the target folder.

  • 'gzip': Files are gzipped during backup. Files without .gz extension are compressed and get .gz added. Files already ending in .gz are copied as-is without further compression.

  • 'zip': All files are packaged into a single ZIP archive named <target>.zip. The target folder itself is not created. Parent directories in the path are created if needed. The file_list.txt manifest is included inside the ZIP archive.

--overwrite (bool, default False):

Controls behavior when target already exists and contains files:

  • False: If target exists and contains files, raise an error and exit without creating backup.

  • True: If target exists and contains files, remove existing files before creating the backup.

File Naming:

Each backed up file receives a sequential prefix: - b001_<filename> for the first file - b002_<filename> for the second file - b003_<filename> for the third file - And so on...

For gzip mode, non-compressed files also get .gz extension: - b001_config.json → b001_config.json.gz - b002_data.nii.gz → b002_data.nii.gz (already compressed, no change)

Manifest File (file_list.txt):

A manifest file named file_list.txt is created with the backup: - First line: Source folder path (source folder: <path>) - Second line: Storage mode (store: <mode>) - Following lines: <prefix>: <relative path within source folder>

Example:

source folder: /home/user/project/data
store: original
b001: configs/settings.json
b002: results/output.txt
b003: logs/process.log

For store=zip mode, file_list.txt is included inside the ZIP archive. For other modes, it's placed in the target folder.

Directory Structure:

Backups use a flat structure - all files are stored in the root of the target folder or ZIP archive, regardless of their original directory structure in the source. The original paths are preserved only in the file_list.txt manifest.

Notes

  • The function creates all necessary parent directories automatically

  • By default (overwrite=False), the function will fail if target exists and contains files, preventing accidental data loss

  • With overwrite=True, existing files in target are removed before backup

  • For large files, gzip mode can significantly reduce storage space

  • For many small files, zip mode provides better organization

  • The manifest file allows easy restoration of original directory structure

Examples

Back up configuration files as-is:

backup_files(
    source="/path/to/project",
    target="/path/to/backups/config_backup",
    filelist=["configs/settings.json", "configs/database.ini", "README.md"],
    store="original",
)

Back up data files with gzip compression:

backup_files(
    source="/path/to/study/sessions/subject01",
    target="/path/to/backups/subject01_data",
    filelist=["bold/run1.nii", "bold/run2.nii", "anat/T1w.nii"],
    store="gzip",
)

Create a ZIP archive of analysis results:

backup_files(
    source="/path/to/analysis",
    target="/path/to/archives/results_2024",
    filelist=["output.csv", "plots/figure1.png", "plots/figure2.png"],
    store="zip",
)