# HCP Pipeline Mapping in QuNex To be able to process neuroimaging data, the HCP minimal processing pipeline needs to be provided with the correct files. When importing data to QuNex—e.g. from raw DICOM files—the information about the session is recorded in the [session acquisition information files](../Overview/FileSession), or `session.txt`. The information stored, however, often does not enable automatic identification of the image types, e.g., which sequence to use as a high-resolution T1w structural image for atlas registration and brain tissue segmentation. To enable easy and robust identification of sequences, these `session.txt` files are converted to HCP session pipeline information files (`session_hcp.txt`) by adding descriptors that identify sequence type and (in the case of BOLD and DWI images) provide additional relevant information. This step comes after the `import_` QuNex command. For an overview of the order in which to run QuNex, see [this schematic](../Overview/VisualSchematic) ## The QuNex session file (`session.txt`) Session acqusition information files, or QuNex session files, are text files. They are expected to be present in each `/sessions/` folder, and are usually automatically generated on data import (eg. [`import_dicom`](../../api/gmri/import_dicom.rst)). For an overview of the format, please see information on [session acquisition information files](../Overview/FileSession). Below is an example of what one might look like: ```bash session: subject: dicom: /dicom raw_data: /nii hcp: /hcp 11: SequenceName_for_Localizer 21: SequenceName_for_T1_Scan 31: SequenceName_for_T2_Scan 41: SequenceName_for_SpinEchoDirection_AnteriorPosterior 51: SequenceName_for_SpinEchoDirection_PosteriorAnterior 61: SequenceName_for_BOLD_Task_SingleBandReference 71: SequenceName_for_BOLD_Task 81: SequenceName_for_BOLD_Rest_SingleBandReference 91: SequenceName_for_BOLD_Rest 101: SequenceName_for_DiffusionSequenceDirection_AnteriorPosterior_90directions 111: SequenceName_for_DiffusionSequenceDirection_PosteriorAnterior_90directions 121: SequenceName_for_DiffusionSequenceDirection_AnteriorPosterior_91directions 131: SequenceName_for_DiffusionSequenceDirection_PosteriorAnterior_91directions ``` ## HCP session pipeline information file (`session_hcp.txt`) Similar to QuNex session files, HCP session pipeline information files, or HCP session files, are expected to be present in each `/sessions/` folder. These files are created by adapting the QuNex session file, and can be done so manually or in batches by using a mapping file in conjunction with `create_session_info`. Each of the images collected that are to be used and processed via the HCP pipeline need to be described using standard names, which are used for proper HCP folder mapping. For a table of accepted HCP tags, see [HCP descriptor tags](#hcp-descriptor-tags) below. For each individual image you want to process, you must add the above standard names where appropriate. So, in the case of the previous example, the new `session_hcp.txt` file would look like: ```bash session: subject: dicom: /dicom raw_data: /nii hcp: /hcp hcpready: true 11: :SequenceName_for_Localizer 21: T1w :SequenceName_for_T1_Scan 31: T2w :SequenceName_for_T2_Scan 41: SE-FM-AP :SequenceName_for_SpinEchoDirection_AnteriorPosterior 51: SE-FM-PA :SequenceName_for_SpinEchoDirection_PosteriorAnterior 61: boldref1:task :SequenceName_for_BOLD_Task_SingleBandReference 71: bold1:task :SequenceName_for_BOLD_Task 81: boldref2:rest :SequenceName_for_BOLD_Rest_SingleBandReference 91: bold2:rest :SequenceName_for_BOLD_Rest 101: DWI:dir90_AP :SequenceName_for_DiffusionSequenceDirection_AnteriorPosterior_90directions 111: DWI:dir90_PA :SequenceName_for_DiffusionSequenceDirection_PosteriorAnterior_90directions 121: DWI:dir91_AP :SequenceName_for_DiffusionSequenceDirection_AnteriorPosterior_91directions 131: DWI:dir91_PA :SequenceName_for_DiffusionSequenceDirection_PosteriorAnterior_91directions ``` \*Note that BOLD files have a sequential number appended to `bold` specification. This is the number that will be used in many of the QuNex commands and functions. Each BOLD needs to have a unique number, preferentially starting with 1 and increasing in a sequential order. In addition each BOLD file also has an additional descriptor specifying the kind of BOLD that was recorded (`task`, `control` and `rest` above). These descriptors can be used in the QuNex commands to specify which BOLD files to run specific steps on. Finally, note that for DWI data there is a specification that denotes number of directions and the phase-encoding direction (e.g. dir90_PA). Once generated for each session, these files are then used by the [`setup_hcp`](../../api/gmri/setup_hcp.rst) command to copy and map imported session images (located in `/sessions//nii`) into an HCP friendly folder structure inside the session folder (`/sessions//hcp/`) ## HCP session file creation using a mapping file Though the above file can be created manually, the process can be simplified using the [`create_session_info`](../../api/gmri/create_session_info.rst) command. This command produces the HCP session file, by default `session_hcp.txt`, in each session directory. To run, the `create_session_info` command needs 2 plain text input files. The first, `sourcefile`, is the aformentioned QuNex session file containing sequence/image names and numbers. This file should automatically be generated upon using a QuNex data import command. The second, `mapping`, must be created by the user. This will be used to assign each sequence to the corresponding standard name specified in your mapping file. This is particularly useful if each sequence name uniquely pairs with an HCP descriptor. * The default location of this mapping file is `//sessions/specs/`. When the initial QuNex study hierarchy is generated using `create_study`, an example mapping text file will be generated here to allow users to proceed. * If not specified otherwise, the command will look for the `//sessions/specs/hcp_mapping.txt`. ### Example mapping file To create the above HCP session file based off the provided example QuNex session file, one would create the following mapping file: ```bash SequenceName_for_T1_Scan => T1w SequenceName_for_T2_Scan => T2w SequenceName_for_SpinEchoDirection_AnteriorPosterior => SE-FM-AP SequenceName_for_SpinEchoDirection_PosteriorAnterior => SE-FM-PA SequenceName_for_BOLD_Task_SingleBandReference => boldref:task SequenceName_for_BOLD_Task => bold:task SequenceName_for_BOLD_Rest_SingleBandReference => boldref:rest SequenceName_for_BOLD_Rest => bold:rest SequenceName_for_DiffusionSequenceDirection_AnteriorPosterior_90directions => DWI:dir90_AP SequenceName_for_DiffusionSequenceDirection_PosteriorAnterior_90directions => DWI:dir90_PA SequenceName_for_DiffusionSequenceDirection_AnteriorPosterior_91directions => DWI:dir91_AP SequenceName_for_DiffusionSequenceDirection_PosteriorAnterior_91directions => DWI:dir91_PA ``` ### Creating the mapping file #### General mapping guidelines * The file should be a plain text file in which individual mappings are specified one per line: `SequenceName_for_T1_Scan => T1w` * Lines that include `=>` will be interpreted as mapping descriptions with the convention ` => ` * The `` before the `=>` is mapped to ``. * Any spaces before or after each string are ignored. * Any lines not including `=>` or starting with `#`are ignored. * Multiple sources for the same target can be specified. Sequences can be mapped by specifying a unique sequence name that can be obtained from the `session.txt` file to be assigned a unique user defined sequence descriptor tag: ```bash => ``` * `SequenceName_for_T1_Scan => T1w` -\> the following line specifies that the sequence named `SequenceName_for_T1_Scan` will be mapped to the descriptor tag `T1w`. * Note that if using the \ specification in the mapping file then this sequence has to always be associated with the same name throughout the study across all sessions and subjects. If a unique paring between \ and an HCP descriptor tag is not possible, the sequence can be identified using the unique number (often the order of acquisition) that can be obtained from the `session.txt` file to be assigned a unique user defined sequence descriptor tag: ```bash => ``` * `20 => T1w` -\> the following line specifies that the acquisition sequence in the session that is associated with the number `20` will always be mapped to the sequence descriptor tag `T1w` across all sessions. Sometimes, it is not possible to uniquely map all sequences for a study using just one mapping (For example, some sessions may have an additional `T1w` scan due to scan interruption or poor quality). In these situations, users are encouraged to create multiple mapping files and selectively run `create_session_info` on the appropriate sessions. #### HCP descriptor tags This table contains all QuNex compatible HCP sequence descriptor tags: | HCP Tag | Definition | |---------|------------| | `T1w` | T1 weighted high resolution structural image | | `T2w` | T2 weighted high resolution structural image | | `FM-GE` | Gradient echo field map image used for distortion correction | | `FM-Magnitude` | Field mapping magnitude image used for distortion correction | | `FM-Phase` | Field mapping phase image used for distortion correction | | `boldref` | Reference image for the following (multiband) BOLD image | | `bold` | BOLD image | | `SE-FM-AP` | Spin-echo fieldmap image recorded using the A-to-P frequency readout direction | | `SE-FM-PA` | Spin-echo fieldmap image recorded using the P-to-A frequency readout direction | | `SE-FM-LR` | Spin-echo fieldmap image recorded using the L-to-R frequency readout direction | | `SE-FM-RL` | Spin-echo fieldmap image recorded using the R-to-L frequency readout direction | | `DWI` | Diffusion weighted images, usually in pairs of phase-encoding reversed acquisition, followed by `dir_` specification (e.g. `dir90_PA`) | | `TB1AFI` | Actual Flip Angle Imaging (AFI) | | `TB1TFL-Magnitude` | B1+ data acquired using tfl_b1_map product sequence by Siemens - anatomical image | | `TB1TFL-Phase` | B1+ data acquired using tfl_b1_map product sequence by Siemens - flip angle map | | `RB1COR-Head` | Low resolution images for B1^- field correction acquired by the head coil | | `RB1COR-Body` | Low resolution images for B1^- field correction acquired by the body coil | #### Functional data mapping In the case of BOLD sequences mapping can become more complicated, as users will also want to specifiy additional sequence info: ```bash => :[:] ``` * `SequenceName_for_BOLD_Rest => bold:rest`-\> the following line specifies that the acquisition sequence called `SequenceName_for_BOLD_Rest` will always be mapped to the sequence descriptor tag `bold:rest` across all sessions where `bold` represents the `` and `rest` represents the optional user defined ``. * You will also want to do the same with the corresponding reference if present: `SequenceName_for_BOLD_Task_SingleBandReference => boldref:task`. * The number of the BOLD image should not be provided as it is added automatically. The number can be explicitly specified under certain conditions (see the [section below](#explicitly-assigning-bold-numbers)). * For BOLD images the mapping file can handle `SpinEcho` field maps. When running `create_session_info` the `se()` will be added automatically using the following logic: * the first spin-echo pair will be assigned number 1, * the next spin-echo pair will be assigned number 2, etc. * structural and functional images will be assigned the number of the immediately preceding spin-echo pair * if no spin-echo pair was present before the structural or functional image, it will be assigned the number of the first following spin-echo pair. * Similarly, if EPI fieldmaps are present (for Siemens this would be a pair of images tagged as `FM-Magnitude` and `FM-Phase`, or a single `FM-GE` image), then these images will be assigned a `fm()` descriptor, and structural and functional images will receive the same descriptor using the logic described above. **Please note** that the logic for associating spin-echo and EPI sequence-based field map images depends on the images being listed in the order they were acquired. This might not be the case if the `session.txt` file was not generated by QuNex when importing DICOM or PAR/REC files. For instance, if the `session.txt` file was generated when importing images from BIDS datasets, the order of the files is determined by image modalities, image types, and BIDS labels since sequence numbers might not be available or can't be reliably extracted. In this case, the automatic assignment feature should not be used when there are multiple field map images. Currently, the automatic assignment of `fm([N])` and `se([N])` can be augmented if that information is provided in the `session.txt` file or the mapping file, and the exact semantics is described in [this section](#explicitly-associating-field-maps). #### Explicitly assigning bold numbers By default, QuNex sequentially assigns a bold number to each bold image, starting from 1. Internally, QuNex uses the bold tag (e.g., in `bold1:rest` rest is the bold tag) to select and filter bold scans for processing. However, this behavior can be overridden by defining the `bold_num()` tag for certain BOLD scans in the mapping file. For BOLD scans mapped without the `bold_num` tag, their bold number will still be assigned sequentially starting from 1, and any number used in the `bold_num()` tag will be skipped. \*Note this will only work if the bold number can be **uniquely** assigned to a single BOLD scan. For example, consider the following three sessions: ```bash session: sess01 ... 11: TASK1 21: TASK2 31: REST 41: REST ``` ```bash session: sess02 ... 11: TASK2 21: REST ``` ```bash session: sess03 ... 11: TASK2 21: REST 31: REST 41: REST ``` By defining the `bold_num` tag in the following mapping file, it is possible to achieve more consistent bold numbering across sessions. ```bash TASK1 => bold:task1 : bold_num(3) TASK2 => bold:task2 : bold_num(4) REST => bold:rest ``` With this mapping file, `TASK1` scans will always get the bold number 3 and `TASK2` scans will always get the bold number 4. ```bash session: sess01 ... 11: bold3:task1 : TASK1 21: bold4:task2 : TASK2 31: bold1:rest : REST 41: bold2:rest : REST ``` ```bash session: sess02 ... 11: bold4:task2 : TASK2 21: bold1:rest : REST ``` In this case, image `04` gets `bold5` because `bold3` is reserved for `TASK1` in the mapping file. ```bash session: sess03 ... 11: bold4:task2 : TASK2 21: bold1:rest : REST 31: bold2:rest : REST 41: bold5:rest : REST ``` #### Explicitly assigning boldref numbers By default, boldref images are assumed to be associated with the preceding bold image in the session file, and will be mapped accordingly. However, in the case that the ordering of images in the session file disallows this, like above you can explicitly map the bold number using `bold_num(n)`. **Note: doing this will disable automatic bold-boldref pairing, and the user will have to assign the number for _every_ bold and boldref.** Therefore, we recommend doing this only when it would be impossible otherwise. #### Explicitly associating field maps The algorithm automatically associating field maps with structural and functional images has two phases. The first phase identifies valid consecutive field map pairs with associativity that favors field maps appearing later. Specifically, in the following example, both 11/21 and 21/31 can be considered a valid pair. Here we favor the latter case since it is more common. The field map pairs are numbered sequentially, starting from one(1), and spin-echo (`se`) and EPI field maps (`fm`) are numbered separately. ```bash # mapping SpinEcho_AP => SE-FM-AP SpinEcho_PA => SE-FM-PA # session_hcp session: sess01 ... 11: : SpinEcho_PA 21: SE-FM-AP : SpinEcho_AP : se(1) 31: SE-FM-PA : SpinEcho_PA : se(1) ``` The second phase of the algorithm associates field map pairs identified in the previous step with other images with the logic described in previous sections. The `se`/`fm` descriptors can be added to session and mapping files. User-defined se/fm descriptors are treated differently when defined on field map images versus structural and functional images. If any se/fm descriptors are defined on spin-echo or EPI field map images, the first phase of the algorithm will not execute, so only user-defined `se`/`fm` descriptors will be used. The `se`/`fm` descriptors defined on structural/functional images do not affect the first stage of the algorithm in any way. During the second phase, if `se`/`fm` descriptors are defined on structural and functional images, we will verify that the referred field map pair exists. Otherwise, we will follow the same rule as described previously. By allowing custom field map pairing, the images in a pair are no longer guaranteed to be consecutive. We define relative order between field maps and other images by only looking at the position of the **first** image in a field map pair. If multiple mappings are specified for fieldmap magnitude images only the last fieldmap magnitude image will be used. To pair two fieldmap magnitude images with the same fieldmap phase image, `fm` tags must be explicitly specified in the mapping file, e.g: ```bash fieldmap_phase => FM-Phase: fm(1) fieldmap_magnitude1 => FM-Magnitude: fm(1) fieldmap_magnitude2 => FM-Magnitude: fm(1) ```