Logging and log files#

Whenever you run a QuNex command, QuNex writes down what it did. This is worth knowing about in some detail, because sooner or later you will want to answer one of two questions: "did that run actually work?" and, when it did not, "what exactly went wrong, and where?" QuNex keeps a separate kind of log for each of those questions, and once you know which is which, finding the answer takes seconds.

This page describes the kinds of log QuNex writes, where they are placed, how they are named, and the parameters and settings you can use to change any of that.

The two kinds of log#

Runlogs answer "what did QuNex do?"

A runlog is written once per qunex call. It records the command you ran and every parameter it ran with, then the report for each session as that session finishes, and finally a summary listing which sessions succeeded and which did not. If you run one command over forty sessions, you get one runlog covering all forty. It is the place to look first, and often the only place you need to look.

Comlogs answer "what did the tool say?"

Underneath, QuNex calls other software — FSL, FreeSurfer, MATLAB, HCP Pipelines scripts. A comlog captures the raw output of one such call, together with the QuNex report lines recorded while it ran. This is where you go when the runlog tells you that a particular session failed and you want to see the actual error the underlying tool printed.

There is also a third, more specialised kind. Batchlogs hold the output of jobs submitted to a cluster scheduler, and they matter only when you run with --scheduler.

Where the logs are placed#

Each qunex call collects its logs in a folder of its own, inside the study's logs folder. The folder is named after the moment the run started and the command that was run, so runs never overwrite or mix with each other:

<study>/logs/2026-08-11_10.02.10.104512_hcp_fmri_volume/
├── Log-hcp_fmri_volume-2026-08-11_10.02.10.104512.log      <- the runlog
├── comlogs                                                 <- one file per call
│   ├── done_hcp_fmri_volume_OP101_2026-08-11_10.02.30.118324.log
│   ├── done_hcp_fmri_volume_OP102_2026-08-11_10.19.14.902551.log
│   └── error_hcp_fmri_volume_OP103_2026-08-11_10.44.51.663018.log
└── batchlogs                                               <- only with --scheduler
    └── SLURM_hcp_fmri_volume_job1240.2026-08-11_10.02.11.207446.log

When you run a command outside a study — with no study folder for QuNex to find — the same folder is created under ~/qunex_logs/ instead. You can send it elsewhere with the outside_study setting described further below.

Two parameters change the placement for a single run:

  • --logfolder="<path>" places this run's logs at a path you choose;

  • --logfolder="legacy" uses the alternative layout, in which the logs of all runs are collected together in <study>/processing/logs. Studies that prefer this arrangement can make it permanent with the layout setting, so that it does not have to be repeated on every command line.

Runlogs#

A runlog is named after the command and the time the run started:

Log-<command>-<date>_<hour>.<minute>.<second>.<microsecond>.log

It begins with the QuNex version and the full call, written out one parameter per line, so that months later you can see exactly how a result was produced — and reproduce it by copying the call back out. The reports of the individual sessions follow, each added as that session finishes.

At the end comes the summary, which is the part most worth reading:

---> Final report for command hcp_fmri_volume
     3 run, 2 successful, 1 failed, 0 did not complete
     Successful:
     ... OP101 ---> HCP fMRI Volume completed [log: .../done_hcp_fmri_volume_OP101_....log]
     ... OP102 ---> HCP fMRI Volume completed [log: .../done_hcp_fmri_volume_OP102_....log]
     Failed:
     ... OP103 ---> HCP fMRI Volume failed [log: .../error_hcp_fmri_volume_OP103_....log]
---> Not all tasks completed fully!

The counts come first, then the sessions grouped by how they ended, and each line names the comlog that holds the detail. So the path from "something went wrong" to the file that explains it is a single glance.

The summary distinguishes three possible outcomes for a session. Successful and failed are self-explanatory. Did not complete covers the sessions that never reported back at all, which happens when a job is killed, runs out of memory, or loses its compute node — QuNex counts these separately, because "no news" is not good news.

Finally, a run in which one or more sessions fail ends with a non-zero exit status. This matters if you call QuNex from a shell script, a && chain, a CI step or a scheduler job: those can act on the exit status directly, without anyone having to read the log.

Comlogs#

A comlog is created the moment an external call starts. While the call is running, the file is prefixed tmp_:

tmp_<command>[_<logtag>]_<session id>_<date>_<hour>.<minute>.<microsecond>.log

QuNex prints the path as it starts the call, so you can watch the work as it happens:

# follow a running call as it writes its output
tail -f <study>/logs/<run folder>/comlogs/tmp_hcp_fmri_volume_OP101_*.log

When the call ends, the file is renamed to say how it went — which means you can see the state of a whole run from a directory listing alone:

Prefix

What it means

tmp_

the call is still running

done_

the call completed

error_

the call failed

incomplete_

the call was interrupted before it could finish

A command that makes several external calls in the course of processing one session collects all of them in a single comlog, named after the QuNex command. So one session processed by one command gives you one file to read, in the order the work happened.

Keeping and removing comlogs#

Comlogs are kept by default. Since a large study can accumulate a great many of them, --log="remove" deletes the comlog of any call that completed cleanly, leaving the failures behind for inspection. Two safeguards apply when you ask for that:

  • a comlog whose contents report errors is kept even if the call otherwise looked successful — this covers tools that report a problem and still exit with a success status;

  • the runlog records that the call ran and how it ended in any case, so the outcome survives the deletion of the file.

--keep_comlogs does the opposite and protects every comlog of a run, regardless of what individual commands ask for.

Comlogs when several sessions run at once#

With --parsessions greater than 1, several sessions are processed simultaneously, and each of them writes to its own comlog. Your terminal shows the announcement of each call as it starts (with the path to follow), the confirmation as each one finishes, and the closing summary. The detailed output of each session lives in that session's comlog, which keeps the terminal readable and each session's story in one piece.

Batchlogs#

When a command is submitted to a cluster queue with --scheduler, the standard and error output of each submitted job is captured into the run's batchlogs folder, named:

<batch system>_<command>_job<job number>.<date>_<hour>.<minute>.<microsecond>.log

These are the files to consult when a job never starts, or when the queueing system itself has something to say.

Logging parameters#

The following parameters can be added to any qunex command:

--logfolder       ... A path to place this run's logs in, or "legacy" for the
                      <study>/processing/logs layout [<study>/logs/<run folder>].
--logging         ... What to write: "full" (runlogs and comlogs), "runlog",
                      "comlog", or "none" [full].
--log             ... Whether a comlog is kept after its call succeeds: "keep"
                      or "remove" [keep].
--comlog_folders  ... Where comlogs are placed: "study", "session", "hcp", or a
                      path. Several can be given [study].
--keep_comlogs    ... Protects every comlog of this run from removal. Given bare
                      or as "yes"/"no" [no].
--runlog_content  ... Whether the runlog also repeats the report of each
                      non-processing command: "manifest" or "full" [manifest].
--logtag          ... An extra word placed in comlog names, useful for telling
                      several runs of the same command apart [].

Choosing where comlogs are placed#

--comlog_folders accepts one destination or several, separated by a comma or a pipe (|). When you give several, the comlog is placed in the first and copied to the others, so the same log can be found in more than one convenient place:

  • study — <study>/logs/<run folder>/comlogs, keeping all logs of a run together;

  • session — <sessions folder>/<session id>/logs/comlogs, so that a session's logs travel with the session if it is moved or archived;

  • hcp — <sessions folder>/<session id>/hcp/<session id>/logs/comlogs, next to the HCP results the call produced;

  • any path you provide.

Destination values passed to --log

--log also accepts the destination values study, session and hcp. When it is given one, QuNex reads it as --comlog_folders and prints a warning naming the values it moved, so that existing scripts keep working. Write new calls with --comlog_folders.

Setting the logging policy for a study#

Repeating the same logging parameters on every command line is tedious and easy to get wrong. Instead, the policy can be written once into a qunex_settings.yaml file. QuNex reads two of them:

  • a user file, from your home folder — ~/.qunex_settings.yaml (also accepted as ~/qunex_settings.yaml, ~/qunex/qunex_settings.yaml or ~/.qunex/qunex_settings.yaml);

  • a study file, at <study>/qunex_settings.yaml.

The study file takes precedence over your user file, and any parameter given on the command line takes precedence over both. So a study can set the house style while you keep your personal preferences elsewhere, and you can still override either for a single run.

logging:
    enabled: true             # write logs at all
    runlog: true              # write the per-run summary log
    comlog: true              # write the per-call raw output logs
    outside_study: home       # home | cwd | <path>, when the run has no study
    layout: default           # default | legacy | nested
    keep_comlogs: false       # protect every comlog from removal
    comlog_folders: [study]   # study | session | hcp | <path>, one or more
    runlog_content: manifest  # manifest | full
    skip_commands: []         # commands to run without logging
    log_commands: []          # commands to always log
    skip_types: []            # command types to run without logging, e.g. [utility]

Most of these mirror the command line parameters above. Three are worth a word of their own:

  • outside_study decides where a run with no study folder writes its logs: home (~/qunex_logs), cwd (the folder you are in), or a path of your choice;

  • layout chooses the folder arrangement: default gives each run its own folder under <study>/logs, legacy collects the logs of all runs in <study>/processing/logs, and nested is default plus a numbered subfolder for each step of a run_recipe run, which is handy when a recipe has many steps and you want each step's logs on their own;

  • runlog_content decides whether the runlog also repeats the report of each non-processing command. Under manifest the runlog carries the call, a status line naming the comlog, and the closing summary, while the report itself stays in the comlog. Choose full when you would rather have one self-contained file to read or to pass on to someone.

A worked example#

# run from the study folder
qunex hcp_fmri_volume \
    --batchfile="processing/batch.txt" \
    --sessionsfolder="sessions" \
    --parsessions="4" \
    --comlog_folders="study|session" \
    --log="keep"

The breakdown of the logging parameters is as follows:

  • --parsessions="4" ... four sessions are processed at the same time, so four comlogs are being written at once

  • --comlog_folders="study|session" ... each comlog is placed with the study's logs and copied into its own session's folder

  • --log="keep" ... comlogs of successful calls are kept (this is the default, spelled out here for clarity)

While the command runs, the terminal announces each session as it starts, names the comlog to follow, and confirms each session as it finishes. When it is done, this is what has been written:

<study>
├── logs
│   └── 2026-08-11_10.02.10.104512_hcp_fmri_volume
│       ├── Log-hcp_fmri_volume-2026-08-11_10.02.10.104512.log
│       └── comlogs
│           ├── done_hcp_fmri_volume_OP101_2026-08-11_10.02.30.118324.log
│           ├── done_hcp_fmri_volume_OP102_2026-08-11_10.19.14.902551.log
│           └── error_hcp_fmri_volume_OP103_2026-08-11_10.44.51.663018.log
└── sessions
    ├── OP101
    │   └── logs
    │       └── comlogs
    │           └── done_hcp_fmri_volume_OP101_2026-08-11_10.02.30.118324.log
    ├── OP102
    │   └── logs
    │       └── comlogs
    │           └── done_hcp_fmri_volume_OP102_2026-08-11_10.19.14.902551.log
    └── OP103
        └── logs
            └── comlogs
                └── error_hcp_fmri_volume_OP103_2026-08-11_10.44.51.663018.log

Reading that back: the runlog gives you the whole run at a glance and tells you that OP103 failed; the error_ file named in its summary line holds the output of the call that failed; and because session was among the comlog destinations, each session also carries a copy of its own log. The command itself ended with a non-zero exit status, so a script wrapping this call would have noticed the failure without reading anything at all.