XNAT Management for advanced users & administrators#

Data stored within XNAT is organized according to projects. To manage data within XNAT, one creates a project to which data can be assigned. In some situations, these projects correspond directly to grant approved studies. In other situations, projects can correspond to individual collection sites for a multi-site study. User permissions for access to data are governed by the user's access to particular projects.

All data stored in XNAT must be associated with a project. This association is the basis of the XNAT security model. Users are given access to data which belongs to a particular project. In order to upload any data to XNAT, you must first create a project.

STEP 1: Creating a new project#

The following sections describe how to create a new project for the first time.

STEP 1A: Login to an XNAT instance using your browser.

STEP 1B: Click on → New.

New

STEP 1C: Select Project.

Project

STEP 1D: Provide Project Information.

Project information

Required Fields

Variable name

Description

Project Title

The full title of the project, which will appear in the site-wide project listing.

Running Title

This is an abbreviated project name, which will appear in search listings and data tables.

Project ID

This ID is used within the XNAT database and in all API calls related to your project and its data. Once set, it cannot be changed.

Optional Fields

Variable name

Description

Description

Add as much detail as you like describing your project. This will show up in the project page, and in the site-wide project listing.

Keywords

Keywords can be searched for in the primary search field, and can help you find project data in an XNAT system with many projects.

Aliases

These are optional project identifiers. A common reason to use them would be for billing codes.

Investigators

Project Investigators can be added to projects as reference values only. These do not need to correspond to user accounts in XNAT. A common reason to use this would be for identification of primary investigators or researchers associated with a study, or with multiple studies.

Setting Project Accessibility

This setting allows you to manage who can view project data, or even have awareness of the existence of your project in XNAT. There are three possible accessibility settings.

Variable name

Description

Public

All XNAT users will be able to view or download your project and its data. Owners of other XNAT projects can share your data into their studies. If your XNAT security settings allow guest access, then any visitor to the web application will be able to view and download your data.

Protected

All XNAT users will be able to see your project title and description in site-wide listings, but only users assigned by project owners will be able to access its data.

Private

Only XNAT users assigned by project owners will be able to see the project in XNAT, or access its data.

Note: The default value of the project accessibility is Private.

STEP 1E: → Click Create Project. At this stage, a project is created and users can now be given access to data within a project (if the project accessibility is other than Public).

Create project

Note: The Manage tab allows project owners access to components that can be configured for the project. More details here.

STEP 2: Managing user access#

The following sections describe how to define and manage specific access privileges for users.

Step 2A: Understanding accessibility settings.

By default, XNAT defines three common project roles:  Owners,  Members, and Collaborators. A project Owner has the ability to add specific users to the project within these roles. These users will then have the permissions associated with their defined role (which overrules the default accessibility permissions).

Role/Activity

Project Owners

Project Members

Project Collaborators

Create Data

C

C

Read/Download Data

R

R

R

Update Data

D

U

Delete Data

U

The "CRUD" model of permissions

  • Project Owners: If you define additional project owners, they will have all of the permissions on your project that you do. They can read, insert, modify, and delete anything (and everything) associated with your project. They can also add additional users to your project and modify the data types associated with your project.

  • Project Members: Members have the ability to manage the data in your project. They can read, insert, and modify sessions (subjects) and experiments in your project. They cannot modify the project users and data types.

  • Project Collaborators: Collaborators have read-only access on all of the data in your project. They cannot insert or modify data owned by your project. They can download your data and use it within their projects.

Note: Unless a project is set to "Public", in the Project Accessibility setting above (Step 1D), access to project data will be controlled by users specified in the "Access" tab on the Project page.

Accessibility

STEP 2B: → Add users to the Project.

Users are added to a XNAT site by the Site Administrator. As a project owner, invite existing users by clicking the "Show List" button, or invite new users by their email address by clicking the "Add User" button. In either case, you have to choose what access level to give them: Project Owners, Project Members or Project Collaborators.

Collaborators

STEP 3: Uploading imaging session data to XNAT and routing data to a project#

There are various channels by which data can be sent to a XNAT instance. Data can be sent from the Scanner directly, from a PACS system, manual data upload or shell scripts which use the XNAT REST API to import data.

XNAT, as it receives data, looks into the DICOM tags to figure out the project to assign data to. If XNAT is unable to figure out the project, data is held in an area, called the prearchive, and its project is Unassigned. To view the data in the prearchive, navigate from the main XNAT landing page to _Upload _****→ Go To Prearchive.

Upload

More information on how XNAT extracts project information from the DICOM tags is provided here.

Using the QuNex command line tool#

One route to upload the data into an XNAT instance is to use the QuNex Suite script xnat_upload_download.sh for direct upload via the command line. This script is run locally on the user's computer. This script is provided as part of the QuNex Suite repository and is located in:

> ~/qunex/bash/qx_utilities/#

To get details on usage of this tool refer to the help by invoking the command directly from the terminal:

> xnat_upload_download.sh --help#

Advanced users and administrators information#

This section is for Site Administrators/Data Managers and contains info regarding the QuNex Suite installation, XNAT build and pipeline directories and strategies for debugging errors.

Location of the QuNex suite installation on the AWS AMI#

/opt/qunex

Location of the build directory for processing#

/opt/data/build

How to find out jobs which are running#

Login as XNAT Site Admin, navigate to **Administer **→ Plugin Settings → Command History.

How to debug a container failure#

From the log and err files (as Container executes, it captures the stdout in .log file and the stderr in .err file). These files would contain some clues as to why the pipeline failed. Typically, the cause of failure would be:

  • Expectations by the QuNex Suite not met correctly

  • Data not uploaded completely into XNAT

  • Incorrect value passed for some argument in the QuNex Suite

  • Pipeline not configured properly: QUNEX_PROC resource not created, batch and mapping file not uploaded to the QUNEX_PROC resource at the project level, name of batch and the mapping file not supplied correctly at the pipeline configuration step.

To investigate more, QuNex+XNAT super users, would login to the XNAT Host Server and look into the build directory for cause of failure. The build directory path would be in the log file. Navigate to the build directory and look into the subfolder LOGS.

The subfolders, comlogs and runlogs, contain log files generated by QuNex Suite that contain the commands executed by the QuNex Suite and the outputs generated by the workflow.

XNAT container service#

XNAT Container Service is a Java-based framework that facilitates launching of docker containers.

APPENDIX I: Command JSON#

{
  "name": "qunex-turnkey-initial-hcp_minimal-QC-BOLDfc-uploadstudy-cleanup",
  "label": "QUNEX TURNKEY INITIAL, HCP1-5, QC, BOLD FC & DATA CLEANUP WITH FULL STUDY UPLOAD",
  "description": "Runs QuNex Turnkey initial, HCP1-5 & QC, BOLD fc w/ full study upload",
  "version": "1.10",
  "schema-version": "1.0",
  "info-url": "https://gitlab.qunex.yale.edu/qunex/qunex",
  "image": "qunex/qunex_suite:0_45_07",
  "type": "docker",
  "working-directory": "/output",
  "command-line": "/opt/qunex/qx_library/etc/cs-wrapper.sh #BATCH_PARAMETERS_FILENAME_ARG# #TURNKEYSTEPS_ARG# #OVERWRITESTEP_ARG# #OVERWRITEPROJECTXNAT_ARG# #SCAN_MAPPING_FILENAME_ARG# #ID# #LABEL# #XNAT_PROJECT# #SUBJECTID#",
  "mounts": [
    {
      "name": "in",
      "writable": false,
      "path": "/input"
    },
    {
      "name": "out",
      "writable": true,
      "path": "/output"
    }
  ],
  "environment-variables": {},
  "ports": {},
  "inputs": [
    {
      "name": "batch_parameters_filename",
      "label": null,
      "description": "Name of the batch parameters file. This file is expected to be present at the QUNEX_PROC resource at the project level",
      "type": "string",
      "matcher": null,
      "default-value": "batch_parameters.txt",
      "required": true,
      "replacement-key": "#BATCH_PARAMETERS_FILENAME_ARG#",
      "sensitive": null,
      "command-line-flag": "--batchfile",
      "command-line-separator": "=",
      "true-value": null,
      "false-value": null,
      "select-values": [],
      "multiple-delimiter": null
    },
    {
      "name": "scan_mapping_filename",
      "label": null,
      "description": "Name of the scan mapping file. This file is expected to be present at the QUNEX_PROC resource at the project level",
      "type": "string",
      "matcher": null,
      "default-value": "hcp_mapping.txt",
      "required": true,
      "replacement-key": "#SCAN_MAPPING_FILENAME_ARG#",
      "sensitive": null,
      "command-line-flag": "--mappingfile",
      "command-line-separator": "=",
      "true-value": null,
      "false-value": null,
      "select-values": [],
      "multiple-delimiter": null
    },
    {
      "name": "session_id",
      "label": null,
      "description": "Session Accession ID",
      "type": "string",
      "matcher": null,
      "default-value": null,
      "required": true,
      "replacement-key": "#ID#",
      "sensitive": null,
      "command-line-flag": "--xnataccsessionid",
      "command-line-separator": "=",
      "true-value": null,
      "false-value": null,
      "select-values": [],
      "multiple-delimiter": null
    },
    {
      "name": "session_label",
      "label": null,
      "description": "Session Label",
      "type": "string",
      "matcher": null,
      "default-value": null,
      "required": true,
      "replacement-key": "#LABEL#",
      "sensitive": null,
      "command-line-flag": "--xnatsessionlabels",
      "command-line-separator": "=",
      "true-value": null,
      "false-value": null,
      "select-values": [],
      "multiple-delimiter": null
    },
    {
      "name": "session_project",
      "label": null,
      "description": "Project ID of the session",
      "type": "string",
      "matcher": null,
      "default-value": null,
      "required": true,
      "replacement-key": "#XNAT_PROJECT#",
      "sensitive": null,
      "command-line-flag": "--xnatprojectid",
      "command-line-separator": "=",
      "true-value": null,
      "false-value": null,
      "select-values": [],
      "multiple-delimiter": null
    },
    {
      "name": "session_subject",
      "label": null,
      "description": "Subject ID of the Session",
      "type": "string",
      "matcher": null,
      "default-value": null,
      "required": true,
      "replacement-key": "#SUBJECTID#",
      "sensitive": null,
      "command-line-flag": "--xnatsubjectid",
      "command-line-separator": "=",
      "true-value": null,
      "false-value": null,
      "select-values": [],
      "multiple-delimiter": null
    },
    {
      "name": "overwritestep",
      "label": null,
      "description": "Flag to decide if a specific step is to be overwritten by QuNex scripts",
      "type": "string",
      "matcher": null,
      "default-value": "yes",
      "required": true,
      "replacement-key": "#OVERWRITESTEP_ARG#",
      "sensitive": null,
      "command-line-flag": "--overwritestep",
      "command-line-separator": "=",
      "true-value": null,
      "false-value": null,
      "select-values": [],
      "multiple-delimiter": null
    },
    {
      "name": "cleanup",
      "label": null,
      "description": "Flag to decide if the data is to be cleaned up after all turnkey steps",
      "type": "string",
      "matcher": null,
      "default-value": "yes",
      "required": true,
      "replacement-key": "#CLEANUP_ARG#",
      "sensitive": null,
      "command-line-flag": "--cleanup",
      "command-line-separator": "=",
      "true-value": null,
      "false-value": null,
      "select-values": [],
      "multiple-delimiter": null
    },
    {
      "name": "turnkeysteps",
      "label": null,
      "description": "Steps to execute",
      "type": "string",
      "matcher": null,
      "default-value": "create_study,map_raw_data,import_dicom,create_session_info,setup_hcp,create_batch,hcp_pre_freesurfer,hcp_freesurfer,hcp_post_freesurfer,run_qc_t1w,run_qc_t2w,run_qc_myelin,hcp_fmri_volume,hcp_fmri_surface,run_qc_bold,map_hcp_data,create_bold_brain_masks,compute_bold_stats,create_stats_report,extract_nuisance_signal,preprocess_bold,parcellate_bold,compute_bold_fc_seed,compute_bold_fc_gbc",
      "required": true,
      "replacement-key": "#TURNKEYSTEPS_ARG#",
      "sensitive": null,
      "command-line-flag": "--turnkeysteps",
      "command-line-separator": "=",
      "true-value": null,
      "false-value": null,
      "select-values": [],
      "multiple-delimiter": null
    },
    {
      "name": "overwriteprojectxnat",
      "label": null,
      "description": "Flag to decide if the entire XNAT project is to be deleted before start",
      "type": "string",
      "matcher": null,
      "default-value": "yes",
      "required": true,
      "replacement-key": "#OVERWRITEPROJECTXNAT_ARG#",
      "sensitive": null,
      "command-line-flag": "--overwriteprojectxnat",
      "command-line-separator": "=",
      "true-value": null,
      "false-value": null,
      "select-values": [],
      "multiple-delimiter": null
    }
  ],
  "outputs": [
    {
      "name": "qunex_study",
      "description": null,
      "required": false,
      "mount": "out",
      "path": "#XNAT_PROJECT#",
      "glob": null
    }
  ],
  "xnat": [
    {
      "name": "qunex-init-to-hcp_minimal-QC-BOLDfc",
      "label": null,
      "description": "Execute QuNex Turnkey initial, hcp_minimal with QC, BOLD fc with full study upload and data cleanup",
      "contexts": [
        "xnat:imageSessionData"
      ],
      "external-inputs": [
        {
          "name": "session",
          "label": null,
          "description": "Input session",
          "type": "Session",
          "matcher": null,
          "default-value": null,
          "required": true,
          "replacement-key": null,
          "sensitive": null,
          "provides-value-for-command-input": null,
          "provides-files-for-command-mount": "in",
          "via-setup-command": null,
          "user-settable": null,
          "load-children": true
        }
      ],
      "derived-inputs": [
        {
          "name": "id",
          "label": null,
          "description": "Session Accession ID. Used as QuNex id variable.",
          "type": "string",
          "matcher": null,
          "default-value": null,
          "required": true,
          "replacement-key": null,
          "sensitive": null,
          "provides-value-for-command-input": "session_id",
          "provides-files-for-command-mount": null,
          "user-settable": null,
          "load-children": true,
          "derived-from-wrapper-input": "session",
          "derived-from-xnat-object-property": "id",
          "via-setup-command": null,
          "multiple": false
        },
        {
          "name": "label",
          "label": null,
          "description": "Session label. Used as QuNex subject identifier.",
          "type": "string",
          "matcher": null,
          "default-value": null,
          "required": true,
          "replacement-key": null,
          "sensitive": null,
          "provides-value-for-command-input": "session_label",
          "provides-files-for-command-mount": null,
          "user-settable": null,
          "load-children": true,
          "derived-from-wrapper-input": "session",
          "derived-from-xnat-object-property": "label",
          "via-setup-command": null,
          "multiple": false
        },
        {
          "name": "xnat_project",
          "label": null,
          "description": "Project in which session is stored",
          "type": "string",
          "matcher": null,
          "default-value": null,
          "required": true,
          "replacement-key": null,
          "sensitive": null,
          "provides-value-for-command-input": "session_project",
          "provides-files-for-command-mount": null,
          "user-settable": null,
          "load-children": true,
          "derived-from-wrapper-input": "session",
          "derived-from-xnat-object-property": "project-id",
          "via-setup-command": null,
          "multiple": false
        },
        {
          "name": "subjectid",
          "label": null,
          "description": "Subject Accession ID",
          "type": "string",
          "matcher": null,
          "default-value": null,
          "required": true,
          "replacement-key": null,
          "sensitive": null,
          "provides-value-for-command-input": "session_subject",
          "provides-files-for-command-mount": null,
          "user-settable": null,
          "load-children": true,
          "derived-from-wrapper-input": "session",
          "derived-from-xnat-object-property": "subject-id",
          "via-setup-command": null,
          "multiple": false
        }
      ],
      "output-handlers": [
        {
          "name": "output-resource-study",
          "accepts-command-output": "qunex_study",
          "via-wrapup-command": null,
          "as-a-child-of": "session",
          "type": "Resource",
          "label": "qunex_study",
          "format": null
        }
      ]
    }
  ],
  "reserve-memory": 8000
}