Quality Assurance in QuNex#

Quality Assurance is an important but highly tedious step in running QuNex preprocessing. The run_qa command helps ease this process, and supports the following:

  • Raw Data QA (--datatype=raw_data)

  • Config File QA (--datatype=config)

Quality Assurance, QA, is not to be confused with Quality Control, QC, and it's command run_qc. In short, QA is responsible for processing efficiency and completion, whereas QC is responsible for the results of the processing.

Using the run_qa command#

qunex run_qa \
    --datatype=<Type of QA>
    --sessionsfolder=<QuNex sessions folder>
    --sessions=<Sessions to QA>
    --configfile=<QA config file>
    --tag=<Output identifier>
    --overwrite=<Overwrite, yes or no>

This command will run QA on all specified sessions according to a highly-customizable user-created configuration YAML file. Usually, this entails checking specified files exist and that parameters have expected values.

Once complete, run_qa will output lists of sessions that have passed and failed the declared QA, as well as reports, both human and machine readable, that detail why and how these sessions failed.

The QA performed is highly dependent on two flags: configfile and datatype. The configfile flag should point towards a configuration file. The datatype flag must be a String referring to the type of data on which you want to run QA. This page will focus primarily on these.

For more precise info on other flags and actually running the command, see the command's page.

The Configuration file#

Because the QA needed varies greatly between datasets, run_qa is designed to be highly user-customizable, controlled through a user-created configuration YAML file. If you're unfamiliar with YAML format, see the documentation here.

In this file, users can define nested parameter-value pairs and sequences pertaining to your data. Basically, it allows you to tell run_qa what things you want to check in your data and what you expect them to be.

The contents will be quite different depending on the QA type you're running and your data, but it should follow this basic format:

datatypes:
    <Specified Data-type 1>:
        <param>:<value>
        <param>:
            <sub-param>:<value>

    <Specified Data-type 2>:
        - <sequence param>:
            <sub-param>:<value>
            <sub-param>:
                <sub-sub-param>:<value>

config:
    <Additional config options>

Parameters and sub-parameters must be within the scope of their corresponding datatype or parameter. They can either be specified directly as key-value pairs, or as yaml sequences starting with - depending on the data type. See below for data type specific parameters and config creation.

Data Types#

Only the below data types are currently supported.

Raw Data QA (--datatype=raw_data)#