qx_utilities.hcp.process_hcp.hcp_diffusion(sinfo, options, overwrite=False, thread=0)#

hcp_diffusion [... processing options]

Runs the Diffusion step of HCP Pipeline (DiffPreprocPipeline.sh). This command uses GPUs by default so CUDA Libraries are required for this to work. Use the hcp_dwi_nogpu flag to run without a GPU if needed, note that this results in much slower processing speed.


The code expects the first HCP preprocessing step (hcp_pre_freesurfer) to have been run and finished successfully. It expects the DWI data to have been acquired in phase encoding reversed pairs, which should be present in the Diffusion folder in the sessions's root hcp folder.


--batchfile (str, default ''):

The batch.txt file with all the sessions information.

--sessionsfolder (str, default '.'):

The path to the study/sessions folder, where the imaging data is supposed to go.

--parsessions (int, default 1):

How many sessions to run in parallel.

--overwrite (str, default 'no'):

Whether to overwrite existing data (yes) or not (no).

--hcp_suffix (str, default ''):

Specifies a suffix to the session id if multiple variants are run, empty otherwise.

--logfolder (str, default ''):

The path to the folder where runlogs and comlogs are to be stored, if other than default.

--log (str, default 'keep'):

Whether to keep ("keep") or remove ("remove") the temporary logs once jobs are completed. When a comma or pipe ("|") separated list is given, the log will be created at the first provided location and then linked or copied to other locations. The valid locations are:

  • "study" (for the default: "<study>/processing/logs/comlogs" location)

  • "session" (for "<sessionid>/logs/comlogs")

  • "hcp" (for "<hcp_folder>/logs/comlogs")

  • "<path>" (for an arbitrary directory).

--hcp_dwi_echospacing (str, default detailed below):

Echo Spacing or Dwelltime of DWI images in msec. Default is image specific.

--use_sequence_info (str, default 'all'):

A pipe, comma or space separated list of inline sequence information to use in preprocessing of specific image modalities.

Example specifications:

  • all

    Use all present inline information for all modalities.

  • DwellTime

    Use DwellTime information for all modalities.

  • T1w:all

    Use all present inline information for T1w modality.

  • SE:EchoSpacing

    Use EchoSpacing information for Spin-Echo fieldmap images.

  • none

    Do not use inline information.

Modalities: 'T1w', 'T2w', 'SE', 'BOLD', 'dMRi'. Inline information: 'TR', 'PEDirection', 'EchoSpacing' 'DwellTime', 'ReadoutDirection'.

If information is not specified it will not be used. More general specification (e.g. all) implies all more specific cases (e.g. T1w:all).

--hcp_dwi_phasepos (str, default 'PA'):

The direction of unwarping for positive phase. Can be AP, PA, LR, or RL. Negative phase isset automatically based on this setting.

--hcp_dwi_gdcoeffs (str, default 'NONE'):

A path to a file containing gradient distortion coefficients, alternatively a string describing multiple options (see below), or "NONE", if not used.

--hcp_dwi_topupconfig (str, default $HCPPIPEDIR/global/config/b02b0.cnf):

A full path to the topup configuration file to use.

--hcp_dwi_dof (int, default 6):

Degrees of Freedom for post eddy registration to structural images.

--hcp_dwi_b0maxbval (int, default 50):

Volumes with a bvalue smaller than this value will be considered as b0s.

--hcp_dwi_combinedata (int, default 1):

Specified value is passed as the CombineDataFlag value for the eddy_postproc.sh script. If JAC resampling has been used in eddy, this value determines what to do with the output file.

  • 2 ... include in the output all volumes uncombined (i.e. output file of eddy)

  • 1 ... include in the output and combine only volumes where both LR/RL (or AP/PA) pairs have been acquired

  • 0 ... As 1, but also include uncombined single volumes.

--hcp_dwi_selectbestb0 (flag, optional):

If set selects the best b0 for each phase encoding direction to pass on to topup rather than the default behaviour of using equally spaced b0's throughout the scan. The best b0 is identified as the least distorted (i.e., most similar to the average b0 after registration). The flag is not set by default.

--hcp_dwi_extraeddyarg (str, default ''):

A string specifying additional arguments to pass to the DiffPreprocPipeline_Eddy.sh script and subsequently to the run_eddy.sh script and finally to the command that actually invokes the eddy binary. The string is to be written as a contiguous set of arguments to be added. Each argument needs to be provided together with dashes if it needs them. To provide multiple arguments divide them with the pipe (|) character, e.g. --hcp_dwi_extraeddyarg="--niter=8|--nvoxhp=2000".

--hcp_dwi_name (str, default 'Diffusion'):

Name to give DWI output directories.

--hcp_dwi_nogpu (flag, optional):

If specified, use the non-GPU-enabled version of eddy. The flag is not set by default.

--hcp_dwi_even_slices (flag, optional):

If set will ensure the input images to FSL's topup and eddy have an even number of slices by removing one slice if necessary. This behaviour used to be the default, but is now optional, because discarding a slice is incompatible with using slice-to-volume correction in FSL's eddy. The flag is not set by default.

Output files

The results of this command will be present in the Diffusion folder in the sessions's root hcp folder.


Gradient Coefficient File Specification:

--hcp_dwi_gdcoeffs parameter can be set to either "NONE", a path to a specific file to use, or a string that describes, which file to use in which case. Each option of the string has to be divided by a pipe "|" character and it has to specify, which information to look up, a possible value, and a file to use in that case, separated by a colon ":" character. The information too look up needs to be present in the description of that session. Standard options are e.g.:

institution: Yale
device: Siemens|Prisma|123456

Where device is formatted as "<manufacturer>|<model>|<serial number>".

If specifying a string it also has to include a "default" option, which will be used in the information was not found. An example could be:


With the information present above, the file "/data/gc/Prisma.conf" would be used.

Apptainer (Singularity) and GPU support:

If nogpu is not provided, this command will facilitate GPUs to speed up processing. Since the command uses CUDA binaries, an NVIDIA GPU is required. To give access to CUDA drivers to the system inside the Apptainer (Singularity) container, you need to use the --nv flag of the qunex_container script.

Mapping of QuNex parameters onto HCP Pipelines parameters:

Below is a detailed specification about how QuNex parameters are mapped onto the HCP Pipelines parameters.

QuNex parameter

HCPpipelines parameter


posData, negData and PEdir


























Runs the Diffusion step of HCP Pipeline. It preprocesses diffusion weighted images (DWI). Specifically, after b0 intensity normalization, the b0 images of both phase encoding directions are used to calculate the susceptibility-induced B0 field deviations. The full timeseries from both phase encoding directions is used in the “eddy” tool for modeling of eddy current distortions and subject motion. Gradient distortion is corrected and the b0 image is registered to the T1w image using BBR. The diffusion data output from eddy are then resampled into 1.25mm native structural space and masked. Diffusion directions and the gradient deviation estimates are also appropriately rotated and registered into structural space. The function enables the use of a number of parameters to customize the specific preprocessing steps.


Example run from the base study folder with test flag:

qunex hcp_diffusion \
    --sessionsfolder="<path_to_study_folder>/sessions" \
    --batchfile="<path_to_study_folder>/processing/batch.txt" \
    --overwrite="no" \

Run with scheduler, the compute node also loads the required CUDA module:

qunex hcp_diffusion \
    --sessionsfolder="<path_to_study_folder>/sessions" \
    --batchfile="<path_to_study_folder>/processing/batch.txt" \
    --overwrite="yes" \
    --bash="module load CUDA/11.3.1" \

Run without a scheduler and without GPU support:

qunex hcp_diffusion \
    --sessionsfolder="<path_to_study_folder>/sessions" \
    --batchfile="<path_to_study_folder>/processing/batch.txt" \
    --overwrite="yes" \