Build branch v0.3 with version v0.3.8 (dafe2d8)

Build pipeline: viash-hub.demultiplex.v0.3-brf8v Source commit: dafe2d8290 Source message: Bump version to v0.3.8
Build branch v0.3 with version v0.3.7 (e177226)
2025-03-27 16:16:22 +00:00 · 2025-03-20 20:56:15 +00:00 · 2025-03-20 20:00:43 +00:00 · 2025-03-04 13:36:18 +00:00 · 2024-12-20 12:50:35 +00:00 · 2024-12-19 15:56:52 +00:00
56 changed files with 2383 additions and 412 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,114 @@
+# demultiplex v0.3.8
+
+## Bug fixes
+
+* Provide a proper error when a FASTQ file is empty after demultiplexing (PR #40).
+
+# demultiplex v0.3.7
+
+## Minor updates
+
+* Ignore lines starting with '#' when parsing run information CSV (PR #39).
+
+# demultiplex v0.3.6
+
+## Minor updates
+
+* Allow letter case variants for headers when looking for sample information in run information CSV (PR #38).
+
+# demultiplex v0.3.5
+
+## Breaking changes
+
+* The `demultiplex` workflow now outputs a list of directories
+  for the `output_falco` argument (one for each barcode) instead of one directory
+  for the complete run. The output from the `runner` workflow remained
+  unchanged (PR #33).
+
+## Minor updates
+
+* In case Illumina data is detected in the input folder, check for the presence of the 'copyComplete.txt' file.
+  This check can be disabled using `--skip_copycomplete_check` (PR #34).
+
+# demultiplex v0.3.4
+
+## Minor updates
+
+* Resource labels are now automatically included during build (PR #32).
+
+# demultiplex v0.3.3
+
+## Breaking change
+
+- The `runner` defines the output differently now:
+
+  - The last part of the `--input` path is expected to be the run ID and this run ID is used to create the output directory.
+  - If the input is `file.tar.gz` instead of a directory, the `file` part is used as the run ID.
+
+- The output structure is then as follows:
+
+    ```
+    $publish_dir/<run_id>/<date_time_stamp>_demultiplex_<version>/
+    ```
+
+    For instance:
+
+    ```
+    $publish_dir
+    └── 200624_A00834_0183_BHMTFYDRXX
+        └── 20241217_051404_demultiplex_v1.2
+            ├── run_information.csv
+            ├── fastq
+            │   ├── Sample1_S1_L001_R1_001.fastq.gz
+            │   ├── Sample23_S3_L001_R1_001.fastq.gz
+            │   ├── SampleA_S2_L001_R1_001.fastq.gz
+            │   ├── Undetermined_S0_L001_R1_001.fastq.gz
+            │   └── sampletest_S4_L001_R1_001.fastq.gz
+            └── qc
+                ├── fastqc
+                │   ├── Sample1_S1_L001_R1_001.fastq.gz_fastqc_data.txt
+                │   ├── Sample1_S1_L001_R1_001.fastq.gz_fastqc_report.html
+                │   ├── Sample1_S1_L001_R1_001.fastq.gz_summary.txt
+                │   ├── Sample23_S3_L001_R1_001.fastq.gz_fastqc_data.txt
+                │   ├── Sample23_S3_L001_R1_001.fastq.gz_fastqc_report.html
+                │   ├── Sample23_S3_L001_R1_001.fastq.gz_summary.txt
+                │   ├── SampleA_S2_L001_R1_001.fastq.gz_fastqc_data.txt
+                │   ├── SampleA_S2_L001_R1_001.fastq.gz_fastqc_report.html
+                │   ├── SampleA_S2_L001_R1_001.fastq.gz_summary.txt
+                │   ├── Undetermined_S0_L001_R1_001.fastq.gz_fastqc_data.txt
+                │   ├── Undetermined_S0_L001_R1_001.fastq.gz_fastqc_report.html
+                │   ├── Undetermined_S0_L001_R1_001.fastq.gz_summary.txt
+                │   ├── sampletest_S4_L001_R1_001.fastq.gz_fastqc_data.txt
+                │   ├── sampletest_S4_L001_R1_001.fastq.gz_fastqc_report.html
+                │   └── sampletest_S4_L001_R1_001.fastq.gz_summary.txt
+                └── multiqc_report.html
+
+    ```
+
+- This logic can be avoided by providing the flag `--plain_output`.
+
+# Minor updates
+
+* Added `output_run_information` argument that copies the run information file to the output (PR #31).
+
+# demultiplex v0.3.2
+
+# Bug fixes
+
+* Ignore empty CSV entries when parsing sample information (PR #29).
+
+# demultiplex v0.3.1
+
+# Minor updates
+
+* Add `--run_information` and `--demultiplexer` arguments to `runner` workflow (PR #27).
+
+# Bug fixes
+
+* Fix detection of sample IDs from Illumina V2 sample sheets (PR #28).
+
+* Provide a clear error message when `--run_information` is provided but not `--demultiplexer` (PR #27).
+
 # demultiplex v0.3.0

 ## Major updates
--- a/21
+++ b/21
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Data Intuitive
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
--- a/README.md
+++ b/README.md
@@ -0,0 +1,100 @@
+# Demultiplex.vsh
+
+Demultiplex.vsh is a workflow for demultiplexing of raw sequencing data. Currently data from Illumina and Element Biosciences sequencers are supported.
+
+[![ViashHub](https://img.shields.io/badge/ViashHub-demultiplex-7a4baa.svg)](https://web.viash-hub.com/packages/demultiplex)
+[![GitHub](https://img.shields.io/badge/GitHub-viash--hub%2Fdemultiplex-blue.svg)](https://github.com/viash-hub/demultiplex)
+[![GitHub
+License](https://img.shields.io/github/license/viash-hub/demultiplex.svg)](https://github.com/viash-hub/demultiplex/blob/main/LICENSE)
+[![GitHub
+Issues](https://img.shields.io/github/issues/viash-hub/demultiplex.svg)](https://github.com/viash-hub/demultiplex/issues)
+[![Viash
+version](https://img.shields.io/badge/Viash-v0.9.1-blue)](https://viash.io)
+
+## Workflow Overview
+The workflow executes the following steps: 
+1. Unpacking the input data (when a TAR archive is provided)
+2. Run `bclconvert` or `bases2fastq`
+3. Run `falco` and convert Illumina InterOp information to csv
+4. Run `multiqc` to generate a report
+
+## Usage
+
+Two variants of the same workflow are provided, depending on the flexibility in the ouput structure required:
+
+* The `runner` workflow provides a predifined output structure. It requires the minimal amount of parameters to be provided, at the cost of being less flexible. It is located at `target/nextflow/runner/main.nf`
+* The `demultiplex` workflow (`target/nextflow/demultiplex/main.nf`) allows for more fine-grained tuning, but required more parameters to be provided.
+
+### Test data
+
+We have provided test data at `gs://viash-hub-test-data/demultiplex/v3/demultiplex_htrnaseq_meta/SingleCell-RNA_P3_2`, but please feel free to bring your own. The URL of the test data can be provided as-is to the workflow, or you can download everything and specify a local path.
+
+### Setup
+
+In order to use the workflows in this package, you'll need to do the following:
+* Install [nextflow](https://www.nextflow.io/docs/latest/install.html)
+* Install a nextflow compatible executor. This workflow provides a profile for [docker](https://docs.docker.com/get-started/).
+
+### Setting up SCM
+
+In order to let nextflow use the viash-hub workflows, you need to setup a [SCM](https://www.nextflow.io/docs/latest/git.html#git-configuration) file. This can be done once by creating `$HOME/.nextflow/scm` and adding the following:
+```
+providers {
+   vsh {
+    platform = 'gitlab'
+    server = "packages.viash-hub.com"
+  }
+}
+```
+
+Alternatively, a custom location for the SCM file can be specified using the `NXF_SCM_FILE` environment variable.
+
+You can check if everything is working by getting the `--help` for a workflow:
+```bash
+nextflow run \
+vsh/demultiplex \
+-r v0.3.4 \
+--help
+```
+
+### (Optional) Resource usage tuning
+
+Nextflow's labels can be used to specify the amount of resources a process can use. This workflow uses the following labels for CPU and memory:
+* `verylowmem`, `lowmem`, `midmem`, `highmem`
+* `verylowcpu`, `lowcpu`, `midcpu`, `highcpu`
+
+The defaults for these labels can be found at `src/config/labels.config`. Nextflow checks that the specified resources for a process do not exceed what is available on the machine and will not start if it does. Create your own config file to tune the labels to your needs, for example:
+
+```
+// Resource labels
+withLabel: verylowcpu { cpus = 2 }
+withLabel: lowcpu { cpus = 8 }
+withLabel: midcpu { cpus = 16 }
+withLabel: highcpu { cpus = 16 }
+
+withLabel: verylowmem { memory = 4.GB }
+withLabel: lowmem { memory = 8.GB }
+withLabel: midmem { memory = 8.GB }
+withLabel: highmem { memory = 8.GB }
+```
+
+When starting nextflow using the CLI, you can use `-c` to provide the file to nextflow and overwrite the defaults.
+
+### Example
+
+```bash
+nextflow run vsh/demultiplex \
+-r v0.3.4 \
+-main-script target/nextflow/runner/main.nf \
+--input "gs://viash-hub-test-data/demultiplex/v3/demultiplex_htrnaseq_meta/SingleCell-RNA_P3_2"  \
+--demultiplexer bclconvert \
+--publish_dir example_output/ \
+-profile docker \
+-c labels.config
+```
+
+## Acknowledgements
+
+Developed in collaboration with Data Intuitive and Open Analytics.
+
+
--- a/_viash.yaml
+++ b/_viash.yaml
@@ -1,5 +1,5 @@
 name: demultiplex
-version: v0.3.0
+version: v0.3.8
 description: |
  Demultiplexing pipeline
 license: MIT
@@ -17,3 +17,5 @@ viash_version: 0.9.0
 config_mods: |
  .requirements.commands := ['ps']
  .runners[.type == 'nextflow'].directives.tag := '$id'
+  .resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}
+  .runners[.type == 'nextflow'].config.script := 'includeConfig("nextflow_labels.config")'
--- a/src/dataflow/combine_samples/config.vsh.yaml
+++ b/src/dataflow/combine_samples/config.vsh.yaml
@@ -16,6 +16,9 @@ argument_groups:
        type: file
        required: false
        multiple: true
+      - name: "--falco_dir"
+        type: file
+        required: true
  - name: Output arguments
    arguments:
      - name: --output_forward
@@ -28,6 +31,11 @@ argument_groups:
        direction: output
        multiple: true
        required: false
+      - name: "--output_falco"
+        type: file
+        direction: output
+        required: true
+        multiple: true
 resources:
  - type: nextflow_script
    path: main.nf
--- a/src/dataflow/combine_samples/main.nf
+++ b/src/dataflow/combine_samples/main.nf
@@ -13,10 +13,12 @@ workflow run_wf {
        // Gather the following state for all samples
        def forward_fastqs = states.collect{it.forward_input}.flatten()
        def reverse_fastqs = states.collect{it.reverse_input}.findAll{it != null}.flatten()
+        def falco_dirs = states.collect{it.falco_dir}
        
        def resultState = [
          "output_forward": forward_fastqs,
          "output_reverse": reverse_fastqs,
+          "output_falco": falco_dirs,
          // The join ID is the same across all samples from the same run
          "_meta": ["join_id": states[0]._meta.join_id]
        ]
--- a/src/dataflow/gather_fastqs_and_validate/main.nf
+++ b/src/dataflow/gather_fastqs_and_validate/main.nf
@@ -1,3 +1,26 @@
+import java.util.zip.GZIPInputStream
+import java.nio.file.Files
+import java.io.BufferedInputStream
+
+def is_empty(file_to_check){
+  /* 
+  Checks if a file has content
+  */
+  if (file_to_check.size() == 0) {
+    return true
+  }
+  def input_stream = Files.newInputStream(file_to_check)
+  def gzInputStream
+  try {
+    gzInputStream = new GZIPInputStream(new BufferedInputStream(input_stream))
+  } catch (java.io.EOFException ex) {
+    // This is not a gzipfile...
+    return false
+  }
+  def read_one_byte = gzInputStream.read()
+  return read_one_byte == -1
+}
+
 workflow run_wf {
  take:
    input_ch
@@ -17,8 +40,8 @@ workflow run_wf {
        println "Processing run information file ${sample_sheet}"
        csv_lines = sample_sheet.splitCsv(header: false, sep: ',')
        csv_lines.any { csv_items ->
-          if (csv_items.isEmpty()) {
-            // skip empty line
+          if (csv_items.isEmpty() || csv_items[0].startsWith("#")) {
+            // skip empty or commented line 
            return
          }
          def possible_header = csv_items[0]
@@ -29,9 +52,9 @@ workflow run_wf {
              println "Encountered next header '[${start_parsing}]', stopping parsing."
              return true
            }
-            // [Data] for illumina
-            // [Samples] for Element Biosciences
-            if (header in ["Data", "Samples"]) {
+            // [Data], [BCLConvert_Data] for illumina
+            // [Samples] or sometimes [SAMPLES] for Element Biosciences
+            if (header.toLowerCase() in ["data", "samples", "bclconvert_data"]) {
              println "Found header [${header}], start parsing."
              start_parsing = true
              return
@@ -50,13 +73,16 @@ workflow run_wf {
              println "Found sample names column '${csv_items[sample_id_column_index]}'."
              return
            }
-            samples += csv_items[sample_id_column_index]
+            def candidate_sample_id = csv_items[sample_id_column_index]
+            if (candidate_sample_id?.trim()) { // Don't add empty csv entries.
+              samples += csv_items[sample_id_column_index]
+            }
          }
          // This return is important! (If 'true' is returned, the parsing stops.)
          return 
        }
        assert start_parsing: 
-          "Sample information file does not contain [Data] or [Samples] header!"
+          "Sample information file does not contain [Data], [Samples] or [BCLConvert_Data] header!"
        assert samples.size() > 1:
          "Sample information file does not seem to contain any information about the samples!"
        println "Finished processing run information file, found samples: ${samples}."
@@ -75,6 +101,9 @@ workflow run_wf {
            "Found forward: ${forward_fastq} and reverse: ${reverse_fastq}."
          println "Found ${forward_fastq.size()} forward and ${reverse_fastq.size()} reverse " +
            "fastq files for sample ${sample_id}"
+
+          assert forward_fastq.every{!is_empty(it)} && reverse_fastq.every{!is_empty(it)}:
+            "A fastq file for sample '${sample_id}' appears to be empty!"
          def fastqs_state = [
            "fastq_forward": forward_fastq,
            "fastq_reverse": reverse_fastq,
--- a/src/demultiplex/config.vsh.yaml
+++ b/src/demultiplex/config.vsh.yaml
@@ -41,6 +41,7 @@ argument_groups:
        type: file
        direction: output
        required: false
+        multiple: true
        default: "$id/qc/fastqc"
      - name: "--output_multiqc"
        description: Directory to write falco output to
@@ -48,6 +49,19 @@ argument_groups:
        direction: output
        required: false
        default: "$id/qc/multiqc_report.html"
+      - name: "--output_run_information"
+        type: file
+        direction: "output"
+        required: true
+        default: "$id/run_information.csv"
+  - name: "Other arguments"
+    arguments:
+      - name: --skip_copycomplete_check
+        type: boolean_true
+        description: |
+          Disable the check for the presence of a "CopyComplete.txt" file in input
+          directory in case of Illumina data.
+        
 resources:
  - type: nextflow_script
    path: main.nf
--- a/src/demultiplex/main.nf
+++ b/src/demultiplex/main.nf
@@ -25,6 +25,11 @@ workflow run_wf {
      | map {id, state ->
        def newState = [:]
        println("Provided run information: ${state.run_information} and demultiplexer: ${state.demultiplexer}")
+        // No auto-detection of run information file (it is user provided),
+        // in this case the demultiplexer should also be specified.
+        assert (!state.run_information || state.demultiplexer): "When setting --run_information, " +
+          "you must also provide a demultiplexer"
+
        if (!state.run_information) {
          println("Run information was not specified, auto-detecting...")
          // The supported_platforms hashmap must be a 1-on-1 mapping
@@ -62,7 +67,7 @@ workflow run_wf {
            "found in input directory."

          // When autodetecting, the demultiplexer must be set if the run information was found
-          assert demultiplexer, "State error: the demultiplexer should have been autodetected. " +
+          assert demultiplexer: "State error: the demultiplexer should have been autodetected. " +
            "Please report this as a bug."

          // When autodetecting, the found demultiplexer must match
@@ -82,13 +87,17 @@ workflow run_wf {
            "run_information": run_information,
            "demultiplexer": demultiplexer,
          ]
-        }
+        } // end auto-detection logic

        if (newState.demultiplexer in ["bclconvert"]) {
          // Do not add InterOp to state because we generate the summary csv's in the next
          // step based on the run dir, not the InterOp dir.
          def interop_dir = state.input.resolve("InterOp")
          assert interop_dir.isDirectory(): "Expected InterOp directory to be present."
+
+          def copycomplete_file = state.input.resolve("CopyComplete.txt")
+          assert (copycomplete_file.isFile() || state.skip_copycomplete_check): 
+            "'CopyComplete.txt' file was not found!"
        }

        def resultState = state + newState
@@ -164,28 +173,12 @@ workflow run_wf {
      )

    output_ch = samples_ch 
-
-
-      | combine_samples.run(
-        fromState: { id, state ->
-          [
-            "id": state.run_id,
-            "forward_input": state.fastq_forward,
-            "reverse_input": state.fastq_reverse, 
-          ]
-        },
-        toState: [
-          "forward_fastqs": "output_forward",
-          "reverse_fastqs": "output_reverse",
-        ]
-      )
      | falco.run(
-        directives: [label: ["lowcpu", "lowmem"]],
+        directives: [label: ["verylowcpu", "lowmem"]],
        fromState: {id, state ->
-          reverse_fastqs_list = state.reverse_fastqs ? state.reverse_fastqs : []
          [
-            "input": state.forward_fastqs + reverse_fastqs_list,
-            "outdir": "${state.output_falco}",
+            "input": [state.fastq_forward, state.fastq_reverse],
+            "outdir": "$id/qc/falco",
            "summary_filename": null,
            "report_filename": null,
            "data_filename": null,
@@ -195,11 +188,28 @@ workflow run_wf {
          state + [ "output_falco" : result.outdir ]
        }
      )
+
+      | combine_samples.run(
+        fromState: { id, state ->
+          [
+            "id": state.run_id,
+            "forward_input": state.fastq_forward,
+            "reverse_input": state.fastq_reverse, 
+            "falco_dir": state.output_falco,
+          ]
+        },
+        toState: [
+          "forward_fastqs": "output_forward",
+          "reverse_fastqs": "output_reverse",
+          "output_falco": "output_falco",
+        ]
+      )
+
      | multiqc.run(
        directives: [label: ["midcpu", "midmem"]],
        fromState: {id, state ->
          def new_state = [
-            "input": [state.output_falco],
+            "input": state.output_falco,
            "output_report": state.output_multiqc,
            "cl_config": 'sp: {fastqc/data: {fn: "*_fastqc_data.txt"}}'
          ]
@@ -220,7 +230,8 @@ workflow run_wf {
          //"_meta": "_meta",
          "output": "output_demultiplexer",
          "output_falco": "output_falco",
-          "output_multiqc": "output_multiqc"
+          "output_multiqc": "output_multiqc",
+          "output_run_information": "run_information",
        ]
      )

--- a/src/demultiplex/test.nf
+++ b/src/demultiplex/test.nf
@@ -25,7 +25,9 @@ workflow test_illumina {
      }
    | map {id, state ->
      assert state.output.isDirectory(): "Expected bclconvert output to be a directory"
-      assert state.output_falco.isDirectory(): "Expected falco output to be a directory"
+      state.output_falco.each{
+         assert it.isDirectory(): "Expected falco output to be a directory"
+      }
      assert state.output_multiqc.isFile(): "Expected multiQC output to be a file"
      fastq_files = state.output.listFiles().collect{it.name}
      assert ["Undetermined_S0_L001_R1_001.fastq.gz", "Sample23_S3_L001_R1_001.fastq.gz", 
@@ -35,6 +37,26 @@ workflow test_illumina {
      fastq_files.each{
        assert it.length() != 0: "Expected FASTQ file to not be empty"
      }
+      assert state.output_run_information.isFile(): "Expected output run information to be a file"
+      expected_run_information = """[Header]
+                                   |Date,6/24/2020
+                                   |Application,Illumina DRAGEN COVIDSeq Test Pipeline
+                                   |Instrument Type,NovaSeq6000
+                                   |Assay,Illumina COVIDSeq Test
+                                   |Index Adapters,IDT-ILMN DNA-RNA UDP Indexes
+                                   |Chemistry,Amplicon
+
+                                   |[Settings]
+                                   |AdapterRead1,CTGTCTCTTATACACATCT
+
+                                   |[Data]
+                                   |Lane,Sample_ID,Sample_Type,Index_ID,Index,Index2
+                                   |1,Sample1,PatientSample,UDP0001,GAACTGAGCG,TCGTGGAGCG
+                                   |1,SampleA,PatientSample,UDP0002,AGGTCAGATA,CTACAAGATA
+                                   |1,Sample23,PatientSample,UDP0003,CGTCTCATAT,TATAGTAGCT
+                                   |1,sampletest,PatientSample,UDP0004,ATTCCATAAG,TGCCTGGTGG
+                                   |""".stripMargin()
+      assert state.output_run_information.text.replaceAll("\r\n", "\n") == expected_run_information
    }
 }

@@ -56,7 +78,7 @@ workflow test_bases2fastq {
      }
    | map {id, state ->
      assert state.output.isDirectory(): "Expected bases2fastq output to be a directory"
-      assert state.output_falco.isDirectory(): "Expected falco output to be a directory"
+      state.output_falco.each{assert it.isDirectory(): "Expected falco output to be a directory"}
      assert state.output_multiqc.isFile(): "Expected multiQC output to be a file"
    }
 }
--- a/src/io/interop_summary_to_csv/config.vsh.yaml
+++ b/src/io/interop_summary_to_csv/config.vsh.yaml
@@ -18,7 +18,7 @@ argument_groups:
        direction: output
        required: true
 requirements:
-  - commands: ["summary", "index-summary"]
+  commands: ["summary", "index-summary"]
 resources:
  - type: bash_script
    path: script.sh
@@ -38,4 +38,4 @@ engines:

 runners:
  - type: executable
-  - type: nextflow
+  - type: nextflow
--- a/src/io/publish/code.sh
+++ b/src/io/publish/code.sh
@@ -1,30 +1,33 @@
 #!/bin/bash

-echo "Publishing $par_input -> $par_output"
-echo "Publishing $par_input_falco -> $par_output_falco"
-echo "Publishing $par_input_multiqc -> $par_output_multiqc"
+set -eo pipefail

-echo
-echo "Creating directory if it does not exist:"
-mkdir -p $(dirname "$par_output") && echo "Containing directory $par_output created"
-mkdir -p $(dirname "$par_output_falco") && echo "Containing directory $par_output_falco created"
-mkdir -p $(dirname "$par_output_multiqc") && echo "Containing directory $par_output_multiqc created"
+declare -A input_output_mapping=(["par_input"]="par_output" 
+                                 ["par_input_multiqc"]="par_output_multiqc"
+                                 ["par_input_run_information"]="par_output_run_information"
+                                )

-echo
-echo "Copying files..."
-cp -rL "$par_input" "$par_output"
-cp -rL "$par_input_falco" "$par_output_falco"
-cp -rL "$par_input_multiqc" "$par_output_multiqc"
+for input_argument_name in "${!input_output_mapping[@]}"
+do
+    input_location="${!input_argument_name}"
+    output_argument_name="${input_output_mapping[$input_argument_name]}"
+    output_location="${!output_argument_name}"
+    echo "Publishing $input_location -> $output_location"

-echo
-echo "Output files:"
-echo "par_output:"
-ls "$par_output"
+    echo "Creating directory if it does not exist."
+    mkdir -p $(dirname "$output_location") && echo "Containing directory $output_location created"

-echo
-echo "par_output_falco:"
-ls "$par_output_falco"
+    echo "Copying files..."
+    cp -rL "$input_location" "$output_location"

-echo
-echo "par_output_multiqc:"
-ls "$par_output_multiqc"
+    echo "Output files for $output_location:"
+    ls "$output_location"
+done
+
+echo "Grouping output from $par_input_falco into $par_output_falco"
+mkdir -p "$par_output_falco"
+IFS=";" read -ra falco_inputs <<< $par_input_falco
+for falco_dir in "${falco_inputs[@]}"; do
+    echo "Copying contents of $falco_dir"
+    find -H -D exec "$falco_dir" -type f -maxdepth 1 -exec cp -t "$par_output_falco" {} +
+done
--- a/src/io/publish/config.vsh.yaml
+++ b/src/io/publish/config.vsh.yaml
@@ -12,8 +12,13 @@ argument_groups:
        description: Directory to write falco output to
        type: file
        required: true
+        multiple: true
      - name: "--input_multiqc"
-        description: Directory to write falco output to
+        description: Location where to write the MultiQC report to.
+        type: file
+        required: true
+      - name: "--input_run_information"
+        description: "Location where to write the run information to."
        type: file
        required: true
  - name: Output arguments
@@ -30,6 +35,10 @@ argument_groups:
        type: file
        direction: output
        default: "qc/multiqc_report.html"
+      - name: --output_run_information
+        type: file
+        direction: output
+        default: run_information.csv

 resources:
  - type: bash_script
--- a/src/runner/config.vsh.yaml
+++ b/src/runner/config.vsh.yaml
@@ -4,20 +4,36 @@ argument_groups:
  - name: Input arguments
    arguments:
      - name: --input
-        description: Base directory of the form `s3:/<bucket>/Sequencing/<Sequencer>/<RunID>/`
+        description: |
+          Base directory of the canonical form `s3://<bucket>/<path>/<RunID>/`.
+          A tarball (tar.gz, .tgz, .tar) containing run information can be provided in which
+          case the RunID is set to the name of the tarball without the extension.
        type: file
        required: true
+      - name: --run_information
+        description: |
+          CSV file containing sample information, which will be used as 
+          input for the demultiplexer. Canonically called 'SampleSheet.csv' (Illumina)
+          or 'RunManifest.csv' (Element Biosciences). If not specified,
+          will try to autodetect the sample sheet in the input directory.
+          Requires --demultiplexer to be set.
+        type: file
+        required: false
+      - name: "--demultiplexer"
+        type: string
+        required: false
+        choices: ["bases2fastq", "bclconvert"]
+        description: |
+          Demultiplexer to use, choice depends on the provider
+          of the instrument that was used to generate the data.
+          When not using --sample_sheet, specifying this argument is not
+          required.
  - name: Annotation flags
    arguments:
-      - name: --add_date_time
+      - name: --plain_output
        description: |
-          Add date and time to the output directory name. This is useful
-          when running the same pipeline multiple times on the same input
-          directory.
-        type: boolean_true
-      - name: --add_workflow_id
-        description: |
-          Add a workflow identifier to the output directory name.
+          Flag to indicate that the output should be stored directly under $publish_dir rather than
+          under a subdirectory structure runID/<date_time>_demultiplex_<version>/.
        type: boolean_true
  - name: Output arguments
    arguments:
@@ -33,7 +49,13 @@ argument_groups:
        type: file
        direction: output
        default: "qc/multiqc_report.html"
-
+  - name: "Other arguments"
+    arguments:
+      - name: --skip_copycomplete_check
+        type: boolean_true
+        description: |
+          Disable the check for the presence of a "CopyComplete.txt" file in input
+          directory in case of Illumina data.
 resources:
  - type: nextflow_script
    path: main.nf
--- a/src/runner/main.nf
+++ b/src/runner/main.nf
@@ -9,27 +9,47 @@ workflow run_wf {

  main:
    output_ch = input_ch
+      // Extract the ID from the input.
+      // If the input is a tarball, strip the suffix.
+      | map{ id, state ->
+        def id_with_suffix = state.input.getFileName().toString()
+        [
+          id,
+          state + [ run_id: id_with_suffix - ~/\.(tar.gz|tgz|tar)$/ ]
+        ]
+      }
      | demultiplex.run(
-        fromState: [
-          "input": "input",
-          "output": "fastq",
-          "output_falco": "qc/fastqc",
-          "output_multiqc": "qc/multiqc_report.html",
-        ],
+        fromState: { id, state ->
+          def state_to_pass = [
+            "input": state.input,
+            "run_information": state.run_information,
+            "demultiplexer": state.demultiplexer,
+            "skip_copycomplete_check": state.skip_copycomplete_check,
+            "output": "$id/fastq",
+            "output_falco": "$id/qc/fastqc",
+            "output_multiqc": "$id/qc/multiqc_report.html",
+          ]
+          if (state.run_information) {
+            state_to_pass += ["output_run_information": state.run_information.getName()] 
+          }
+          state_to_pass
+        },
        toState: { id, result, state ->
          state + result
        },
      )
      | publish.run(
        fromState: { id, state ->
-          def id1 = (params.add_date_time) ? "${id}_${date}" : id
-          def id2 = (params.add_workflow_id) ? "${id1}_demultiplex_${version}" : id1
+          println(state.plain_output)
+          def id1 = (state.plain_output) ? id : "${state.run_id}/${date}"
+          def id2 = (state.plain_output) ? id : "${id1}_demultiplex_${version}"

-          def fastq_output_1 = (id == "run") ? state.fastq_output : "${id2}/" + state.fastq_output
-          def falco_output_1 = (id == "run") ? state.falco_output : "${id2}/" + state.falco_output
-          def multiqc_output_1 = (id == "run") ? state.multiqc_output : "${id2}/" + state.multiqc_output
+          def fastq_output_1 = (id2 == "run") ? state.fastq_output : "${id2}/" + state.fastq_output
+          def falco_output_1 = (id2 == "run") ? state.falco_output : "${id2}/" + state.falco_output
+          def multiqc_output_1 = (id2 == "run") ? state.multiqc_output : "${id2}/" + state.multiqc_output
+          def run_information_output_1 = (id2 == "run") ? "${state.output_run_information.getName()}" : "${id2}/${state.output_run_information.getName()}"

-          if (id == "run") {
+          if (id2 == "run") {
            println("Publising to ${params.publish_dir}")
          } else {
            println("Publising to ${params.publish_dir}/${id2}")
@@ -39,9 +59,11 @@ workflow run_wf {
            input: state.output,
            input_falco: state.output_falco,
            input_multiqc: state.output_multiqc,
+            input_run_information: state.output_run_information,
            output: fastq_output_1,
            output_falco: falco_output_1,
-            output_multiqc: multiqc_output_1
+            output_multiqc: multiqc_output_1,
+            output_run_information: run_information_output_1,
          ]
        },
        toState: { id, result, state -> [:] },
--- a/target/executable/io/interop_summary_to_csv/.config.vsh.yaml
+++ b/target/executable/io/interop_summary_to_csv/.config.vsh.yaml
@@ -1,6 +1,6 @@
 name: "interop_summary_to_csv"
 namespace: "io"
-version: "v0.3.0"
+version: "v0.3.8"
 argument_groups:
 - name: "Input arguments"
  arguments:
@@ -38,6 +38,9 @@ resources:
 - type: "bash_script"
  path: "script.sh"
  is_executable: true
+- type: "file"
+  path: "nextflow_labels.config"
+  dest: "nextflow_labels.config"
 info: null
 status: "enabled"
 requirements:
@@ -109,6 +112,8 @@ runners:
      cpu200: "cpus = 200"
      cpu500: "cpus = 500"
      cpu1000: "cpus = 1000"
+    script:
+    - "includeConfig(\"nextflow_labels.config\")"
  debug: false
  container: "docker"
 engines:
@@ -116,7 +121,7 @@ engines:
  id: "docker"
  image: "debian:stable-slim"
  target_registry: "images.viash-hub.com"
-  target_tag: "v0.3.0"
+  target_tag: "v0.3.8"
  namespace_separator: "/"
  setup:
  - type: "apt"
@@ -141,12 +146,12 @@ build_info:
  output: "target/executable/io/interop_summary_to_csv"
  executable: "target/executable/io/interop_summary_to_csv/interop_summary_to_csv"
  viash_version: "0.9.0"
-  git_commit: "c9cf5854796b31767c70848bcaf9aa42648d0bd7"
-  git_remote: "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex"
-  git_tag: "v0.1.1-9-gc9cf585"
+  git_commit: "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742"
+  git_remote: "https://github.com/viash-hub/demultiplex"
+  git_tag: "v0.3.5-9-gdafe2d8"
 package_config:
  name: "demultiplex"
-  version: "v0.3.0"
+  version: "v0.3.8"
  description: "Demultiplexing pipeline\n"
  info:
    test_resources:
@@ -157,10 +162,12 @@ package_config:
  target: "target"
  config_mods:
  - ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag\
-    \ := '$id'\n"
+    \ := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n\
+    .runners[.type == 'nextflow'].config.script := 'includeConfig(\"nextflow_labels.config\"\
+    )'\n"
  - ".engines += { type: \"native\" }"
  - ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
-  - ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+  - ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
  keywords:
  - "bioinformatics"
  - "sequence"
--- a/target/executable/io/interop_summary_to_csv/interop_summary_to_csv
+++ b/target/executable/io/interop_summary_to_csv/interop_summary_to_csv
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash

-# interop_summary_to_csv v0.3.0
+# interop_summary_to_csv v0.3.8
 # 
 # This wrapper script is auto-generated by viash 0.9.0 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
@@ -171,7 +171,7 @@ VIASH_META_TEMP_DIR="$VIASH_TEMP"

 # ViashHelp: Display helpful explanation about this executable
 function ViashHelp {
-  echo "interop_summary_to_csv v0.3.0"
+  echo "interop_summary_to_csv v0.3.8"
  echo ""
  echo "Input arguments:"
  echo "    --input"
@@ -470,10 +470,10 @@ tar -C /tmp/ --no-same-owner --no-same-permissions -xvf /tmp/interop.tar.gz && \
 mv /tmp/interop-1.3.1-Linux-GNU/bin/index-summary /tmp/interop-1.3.1-Linux-GNU/bin/summary /usr/local/bin/

 LABEL org.opencontainers.image.description="Companion container for running component io interop_summary_to_csv"
-LABEL org.opencontainers.image.created="2024-12-11T09:19:38Z"
+LABEL org.opencontainers.image.created="2025-03-27T16:00:55Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/demultiplex"
-LABEL org.opencontainers.image.revision="c9cf5854796b31767c70848bcaf9aa42648d0bd7"
-LABEL org.opencontainers.image.version="v0.3.0"
+LABEL org.opencontainers.image.revision="dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742"
+LABEL org.opencontainers.image.version="v0.3.8"

 VIASHDOCKER
  fi
@@ -609,7 +609,7 @@ while [[ $# -gt 0 ]]; do
            shift 1
            ;;
        --version)
-            echo "interop_summary_to_csv v0.3.0"
+            echo "interop_summary_to_csv v0.3.8"
            exit
            ;;
        --input)
@@ -733,7 +733,7 @@ if [[ "$VIASH_ENGINE_TYPE" == "docker" ]]; then

  # determine docker image id
  if [[ "$VIASH_ENGINE_ID" == 'docker' ]]; then
-    VIASH_DOCKER_IMAGE_ID='images.viash-hub.com/vsh/demultiplex/io/interop_summary_to_csv:v0.3.0'
+    VIASH_DOCKER_IMAGE_ID='images.viash-hub.com/vsh/demultiplex/io/interop_summary_to_csv:v0.3.8'
  fi

  # print dockerfile
--- a/target/executable/io/interop_summary_to_csv/nextflow_labels.config
+++ b/target/executable/io/interop_summary_to_csv/nextflow_labels.config
@@ -0,0 +1,98 @@
+process {
+  container = 'nextflow/bash:latest'
+  
+  // default resources
+  memory = { 8.Gb * task.attempt }
+  cpus = 8
+  maxForks = 36
+
+  // Retry for exit codes that have something to do with memory issues
+  errorStrategy = { task.exitStatus in 137..140 ? 'retry' : 'terminate' }
+  maxRetries = 3
+  maxMemory = 192.GB
+
+  // Resource labels
+  withLabel: verylowcpu { cpus = 2 }
+  withLabel: lowcpu { cpus = 8 }
+  withLabel: midcpu { cpus = 16 }
+  withLabel: highcpu { cpus = 32 }
+  
+  withLabel: verylowmem { memory = { get_memory( 4.GB * task.attempt ) } }
+  withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+  withLabel: midmem { memory = { get_memory( 16.GB * task.attempt ) } }
+  withLabel: highmem { memory = { get_memory( 64.GB * task.attempt ) } }
+
+}
+
+profiles {
+  // detect tempdir
+  tempDir = java.nio.file.Paths.get(
+    System.getenv('NXF_TEMP') ?:
+      System.getenv('VIASH_TEMP') ?: 
+      System.getenv('TEMPDIR') ?: 
+      System.getenv('TMPDIR') ?: 
+      '/tmp'
+  ).toAbsolutePath()
+
+  mount_temp {
+    docker.temp            = tempDir
+    podman.temp            = tempDir
+    charliecloud.temp      = tempDir
+  }
+
+  no_publish {
+    process {
+      withName: '.*' {
+        publishDir = [
+          enabled: false
+        ]
+      }
+    }
+  }
+
+  docker {
+    docker.fixOwnership    = true
+    docker.enabled         = true
+    // docker.userEmulation   = true
+    singularity.enabled    = false
+    podman.enabled         = false
+    shifter.enabled        = false
+    charliecloud.enabled   = false
+  }
+
+  local {
+    // This config is for local processing.
+    process {
+        maxMemory = 25.GB
+        withLabel: verylowcpu { cpus = 2 }
+        withLabel: lowcpu { cpus = 4 }
+        withLabel: midcpu { cpus = 6 }
+        withLabel: highcpu { cpus = 12 }
+  
+        withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+        withLabel: midmem { memory = { get_memory( 12.GB * task.attempt ) } }
+        withLabel: highmem { memory = { get_memory( 20.GB * task.attempt ) } }
+    }
+  }
+}
+
+def get_memory(to_compare) {
+    if (!process.containsKey("maxMemory") || !process.maxMemory) {
+      return to_compare
+    }
+
+    try {
+      if (process.containsKey("maxRetries") && process.maxRetries && task.attempt == (process.maxRetries as int)) {
+        return process.maxMemory
+      }
+      else if (to_compare.compareTo(process.maxMemory as nextflow.util.MemoryUnit) == 1) {
+        return max_memory as nextflow.util.MemoryUnit
+      }
+      else {
+        return to_compare
+      }  
+    } catch (all) {
+          println "Error processing memory resources. Please check that process.maxMemory '${process.maxMemory}' and process.maxRetries '${process.maxRetries}' are valid!"
+          System.exit(1)
+    }
+  }
--- a/target/executable/io/publish/.config.vsh.yaml
+++ b/target/executable/io/publish/.config.vsh.yaml
@@ -1,6 +1,6 @@
 name: "publish"
 namespace: "io"
-version: "v0.3.0"
+version: "v0.3.8"
 argument_groups:
 - name: "Input arguments"
  arguments:
@@ -22,11 +22,21 @@ argument_groups:
    create_parent: true
    required: true
    direction: "input"
-    multiple: false
+    multiple: true
    multiple_sep: ";"
  - type: "file"
    name: "--input_multiqc"
-    description: "Directory to write falco output to"
+    description: "Location where to write the MultiQC report to."
+    info: null
+    must_exist: true
+    create_parent: true
+    required: true
+    direction: "input"
+    multiple: false
+    multiple_sep: ";"
+  - type: "file"
+    name: "--input_run_information"
+    description: "Location where to write the run information to."
    info: null
    must_exist: true
    create_parent: true
@@ -69,10 +79,24 @@ argument_groups:
    direction: "output"
    multiple: false
    multiple_sep: ";"
+  - type: "file"
+    name: "--output_run_information"
+    info: null
+    default:
+    - "run_information.csv"
+    must_exist: true
+    create_parent: true
+    required: false
+    direction: "output"
+    multiple: false
+    multiple_sep: ";"
 resources:
 - type: "bash_script"
  path: "code.sh"
  is_executable: true
+- type: "file"
+  path: "nextflow_labels.config"
+  dest: "nextflow_labels.config"
 description: "Publish the processed results of the run"
 info: null
 status: "enabled"
@@ -145,6 +169,8 @@ runners:
      cpu200: "cpus = 200"
      cpu500: "cpus = 500"
      cpu1000: "cpus = 1000"
+    script:
+    - "includeConfig(\"nextflow_labels.config\")"
  debug: false
  container: "docker"
 engines:
@@ -152,7 +178,7 @@ engines:
  id: "docker"
  image: "debian:stable-slim"
  target_registry: "images.viash-hub.com"
-  target_tag: "v0.3.0"
+  target_tag: "v0.3.8"
  namespace_separator: "/"
  setup:
  - type: "apt"
@@ -170,12 +196,12 @@ build_info:
  output: "target/executable/io/publish"
  executable: "target/executable/io/publish/publish"
  viash_version: "0.9.0"
-  git_commit: "c9cf5854796b31767c70848bcaf9aa42648d0bd7"
-  git_remote: "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex"
-  git_tag: "v0.1.1-9-gc9cf585"
+  git_commit: "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742"
+  git_remote: "https://github.com/viash-hub/demultiplex"
+  git_tag: "v0.3.5-9-gdafe2d8"
 package_config:
  name: "demultiplex"
-  version: "v0.3.0"
+  version: "v0.3.8"
  description: "Demultiplexing pipeline\n"
  info:
    test_resources:
@@ -186,10 +212,12 @@ package_config:
  target: "target"
  config_mods:
  - ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag\
-    \ := '$id'\n"
+    \ := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n\
+    .runners[.type == 'nextflow'].config.script := 'includeConfig(\"nextflow_labels.config\"\
+    )'\n"
  - ".engines += { type: \"native\" }"
  - ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
-  - ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+  - ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
  keywords:
  - "bioinformatics"
  - "sequence"
--- a/target/executable/io/publish/nextflow_labels.config
+++ b/target/executable/io/publish/nextflow_labels.config
@@ -0,0 +1,98 @@
+process {
+  container = 'nextflow/bash:latest'
+  
+  // default resources
+  memory = { 8.Gb * task.attempt }
+  cpus = 8
+  maxForks = 36
+
+  // Retry for exit codes that have something to do with memory issues
+  errorStrategy = { task.exitStatus in 137..140 ? 'retry' : 'terminate' }
+  maxRetries = 3
+  maxMemory = 192.GB
+
+  // Resource labels
+  withLabel: verylowcpu { cpus = 2 }
+  withLabel: lowcpu { cpus = 8 }
+  withLabel: midcpu { cpus = 16 }
+  withLabel: highcpu { cpus = 32 }
+  
+  withLabel: verylowmem { memory = { get_memory( 4.GB * task.attempt ) } }
+  withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+  withLabel: midmem { memory = { get_memory( 16.GB * task.attempt ) } }
+  withLabel: highmem { memory = { get_memory( 64.GB * task.attempt ) } }
+
+}
+
+profiles {
+  // detect tempdir
+  tempDir = java.nio.file.Paths.get(
+    System.getenv('NXF_TEMP') ?:
+      System.getenv('VIASH_TEMP') ?: 
+      System.getenv('TEMPDIR') ?: 
+      System.getenv('TMPDIR') ?: 
+      '/tmp'
+  ).toAbsolutePath()
+
+  mount_temp {
+    docker.temp            = tempDir
+    podman.temp            = tempDir
+    charliecloud.temp      = tempDir
+  }
+
+  no_publish {
+    process {
+      withName: '.*' {
+        publishDir = [
+          enabled: false
+        ]
+      }
+    }
+  }
+
+  docker {
+    docker.fixOwnership    = true
+    docker.enabled         = true
+    // docker.userEmulation   = true
+    singularity.enabled    = false
+    podman.enabled         = false
+    shifter.enabled        = false
+    charliecloud.enabled   = false
+  }
+
+  local {
+    // This config is for local processing.
+    process {
+        maxMemory = 25.GB
+        withLabel: verylowcpu { cpus = 2 }
+        withLabel: lowcpu { cpus = 4 }
+        withLabel: midcpu { cpus = 6 }
+        withLabel: highcpu { cpus = 12 }
+  
+        withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+        withLabel: midmem { memory = { get_memory( 12.GB * task.attempt ) } }
+        withLabel: highmem { memory = { get_memory( 20.GB * task.attempt ) } }
+    }
+  }
+}
+
+def get_memory(to_compare) {
+    if (!process.containsKey("maxMemory") || !process.maxMemory) {
+      return to_compare
+    }
+
+    try {
+      if (process.containsKey("maxRetries") && process.maxRetries && task.attempt == (process.maxRetries as int)) {
+        return process.maxMemory
+      }
+      else if (to_compare.compareTo(process.maxMemory as nextflow.util.MemoryUnit) == 1) {
+        return max_memory as nextflow.util.MemoryUnit
+      }
+      else {
+        return to_compare
+      }  
+    } catch (all) {
+          println "Error processing memory resources. Please check that process.maxMemory '${process.maxMemory}' and process.maxRetries '${process.maxRetries}' are valid!"
+          System.exit(1)
+    }
+  }
--- a/target/executable/io/publish/publish
+++ b/target/executable/io/publish/publish
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash

-# publish v0.3.0
+# publish v0.3.8
 # 
 # This wrapper script is auto-generated by viash 0.9.0 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
@@ -171,7 +171,7 @@ VIASH_META_TEMP_DIR="$VIASH_TEMP"

 # ViashHelp: Display helpful explanation about this executable
 function ViashHelp {
-  echo "publish v0.3.0"
+  echo "publish v0.3.8"
  echo ""
  echo "Publish the processed results of the run"
  echo ""
@@ -181,12 +181,16 @@ function ViashHelp {
  echo "        Directory to write fastq data to"
  echo ""
  echo "    --input_falco"
-  echo "        type: file, required parameter, file must exist"
+  echo "        type: file, required parameter, multiple values allowed, file must exist"
  echo "        Directory to write falco output to"
  echo ""
  echo "    --input_multiqc"
  echo "        type: file, required parameter, file must exist"
-  echo "        Directory to write falco output to"
+  echo "        Location where to write the MultiQC report to."
+  echo ""
+  echo "    --input_run_information"
+  echo "        type: file, required parameter, file must exist"
+  echo "        Location where to write the run information to."
  echo ""
  echo "Output arguments:"
  echo "    --output"
@@ -200,6 +204,10 @@ function ViashHelp {
  echo "    --output_multiqc"
  echo "        type: file, output, file must exist"
  echo "        default: qc/multiqc_report.html"
+  echo ""
+  echo "    --output_run_information"
+  echo "        type: file, output, file must exist"
+  echo "        default: run_information.csv"
 }

 # initialise variables
@@ -482,10 +490,10 @@ RUN apt-get update && \
  rm -rf /var/lib/apt/lists/*

 LABEL org.opencontainers.image.description="Companion container for running component io publish"
-LABEL org.opencontainers.image.created="2024-12-11T09:19:38Z"
+LABEL org.opencontainers.image.created="2025-03-27T16:00:54Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/demultiplex"
-LABEL org.opencontainers.image.revision="c9cf5854796b31767c70848bcaf9aa42648d0bd7"
-LABEL org.opencontainers.image.version="v0.3.0"
+LABEL org.opencontainers.image.revision="dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742"
+LABEL org.opencontainers.image.version="v0.3.8"

 VIASHDOCKER
  fi
@@ -621,7 +629,7 @@ while [[ $# -gt 0 ]]; do
            shift 1
            ;;
        --version)
-            echo "publish v0.3.0"
+            echo "publish v0.3.8"
            exit
            ;;
        --input)
@@ -636,14 +644,20 @@ while [[ $# -gt 0 ]]; do
            shift 1
            ;;
        --input_falco)
-            [ -n "$VIASH_PAR_INPUT_FALCO" ] && ViashError Bad arguments for option \'--input_falco\': \'$VIASH_PAR_INPUT_FALCO\' \& \'$2\' - you should provide exactly one argument for this option. && exit 1
-            VIASH_PAR_INPUT_FALCO="$2"
+            if [ -z "$VIASH_PAR_INPUT_FALCO" ]; then
+              VIASH_PAR_INPUT_FALCO="$2"
+            else
+              VIASH_PAR_INPUT_FALCO="$VIASH_PAR_INPUT_FALCO;""$2"
+            fi
            [ $# -lt 2 ] && ViashError Not enough arguments passed to --input_falco. Use "--help" to get more information on the parameters. && exit 1
            shift 2
            ;;
        --input_falco=*)
-            [ -n "$VIASH_PAR_INPUT_FALCO" ] && ViashError Bad arguments for option \'--input_falco=*\': \'$VIASH_PAR_INPUT_FALCO\' \& \'$2\' - you should provide exactly one argument for this option. && exit 1
-            VIASH_PAR_INPUT_FALCO=$(ViashRemoveFlags "$1")
+            if [ -z "$VIASH_PAR_INPUT_FALCO" ]; then
+              VIASH_PAR_INPUT_FALCO=$(ViashRemoveFlags "$1")
+            else
+              VIASH_PAR_INPUT_FALCO="$VIASH_PAR_INPUT_FALCO;"$(ViashRemoveFlags "$1")
+            fi
            shift 1
            ;;
        --input_multiqc)
@@ -657,6 +671,17 @@ while [[ $# -gt 0 ]]; do
            VIASH_PAR_INPUT_MULTIQC=$(ViashRemoveFlags "$1")
            shift 1
            ;;
+        --input_run_information)
+            [ -n "$VIASH_PAR_INPUT_RUN_INFORMATION" ] && ViashError Bad arguments for option \'--input_run_information\': \'$VIASH_PAR_INPUT_RUN_INFORMATION\' \& \'$2\' - you should provide exactly one argument for this option. && exit 1
+            VIASH_PAR_INPUT_RUN_INFORMATION="$2"
+            [ $# -lt 2 ] && ViashError Not enough arguments passed to --input_run_information. Use "--help" to get more information on the parameters. && exit 1
+            shift 2
+            ;;
+        --input_run_information=*)
+            [ -n "$VIASH_PAR_INPUT_RUN_INFORMATION" ] && ViashError Bad arguments for option \'--input_run_information=*\': \'$VIASH_PAR_INPUT_RUN_INFORMATION\' \& \'$2\' - you should provide exactly one argument for this option. && exit 1
+            VIASH_PAR_INPUT_RUN_INFORMATION=$(ViashRemoveFlags "$1")
+            shift 1
+            ;;
        --output)
            [ -n "$VIASH_PAR_OUTPUT" ] && ViashError Bad arguments for option \'--output\': \'$VIASH_PAR_OUTPUT\' \& \'$2\' - you should provide exactly one argument for this option. && exit 1
            VIASH_PAR_OUTPUT="$2"
@@ -690,6 +715,17 @@ while [[ $# -gt 0 ]]; do
            VIASH_PAR_OUTPUT_MULTIQC=$(ViashRemoveFlags "$1")
            shift 1
            ;;
+        --output_run_information)
+            [ -n "$VIASH_PAR_OUTPUT_RUN_INFORMATION" ] && ViashError Bad arguments for option \'--output_run_information\': \'$VIASH_PAR_OUTPUT_RUN_INFORMATION\' \& \'$2\' - you should provide exactly one argument for this option. && exit 1
+            VIASH_PAR_OUTPUT_RUN_INFORMATION="$2"
+            [ $# -lt 2 ] && ViashError Not enough arguments passed to --output_run_information. Use "--help" to get more information on the parameters. && exit 1
+            shift 2
+            ;;
+        --output_run_information=*)
+            [ -n "$VIASH_PAR_OUTPUT_RUN_INFORMATION" ] && ViashError Bad arguments for option \'--output_run_information=*\': \'$VIASH_PAR_OUTPUT_RUN_INFORMATION\' \& \'$2\' - you should provide exactly one argument for this option. && exit 1
+            VIASH_PAR_OUTPUT_RUN_INFORMATION=$(ViashRemoveFlags "$1")
+            shift 1
+            ;;
        ---engine)
            VIASH_ENGINE_ID="$2"
            shift 2
@@ -778,7 +814,7 @@ if [[ "$VIASH_ENGINE_TYPE" == "docker" ]]; then

  # determine docker image id
  if [[ "$VIASH_ENGINE_ID" == 'docker' ]]; then
-    VIASH_DOCKER_IMAGE_ID='images.viash-hub.com/vsh/demultiplex/io/publish:v0.3.0'
+    VIASH_DOCKER_IMAGE_ID='images.viash-hub.com/vsh/demultiplex/io/publish:v0.3.8'
  fi

  # print dockerfile
@@ -874,6 +910,10 @@ if [ -z ${VIASH_PAR_INPUT_MULTIQC+x} ]; then
  ViashError '--input_multiqc' is a required argument. Use "--help" to get more information on the parameters.
  exit 1
 fi
+if [ -z ${VIASH_PAR_INPUT_RUN_INFORMATION+x} ]; then
+  ViashError '--input_run_information' is a required argument. Use "--help" to get more information on the parameters.
+  exit 1
+fi
 if [ -z ${VIASH_META_NAME+x} ]; then
  ViashError 'name' is a required argument. Use "--help" to get more information on the parameters.
  exit 1
@@ -909,20 +949,35 @@ fi
 if [ -z ${VIASH_PAR_OUTPUT_MULTIQC+x} ]; then
  VIASH_PAR_OUTPUT_MULTIQC="qc/multiqc_report.html"
 fi
+if [ -z ${VIASH_PAR_OUTPUT_RUN_INFORMATION+x} ]; then
+  VIASH_PAR_OUTPUT_RUN_INFORMATION="run_information.csv"
+fi

 # check whether required files exist
 if [ ! -z "$VIASH_PAR_INPUT" ] && [ ! -e "$VIASH_PAR_INPUT" ]; then
  ViashError "Input file '$VIASH_PAR_INPUT' does not exist."
  exit 1
 fi
-if [ ! -z "$VIASH_PAR_INPUT_FALCO" ] && [ ! -e "$VIASH_PAR_INPUT_FALCO" ]; then
-  ViashError "Input file '$VIASH_PAR_INPUT_FALCO' does not exist."
-  exit 1
+if [ ! -z "$VIASH_PAR_INPUT_FALCO" ]; then
+  IFS=';'
+  set -f
+  for file in $VIASH_PAR_INPUT_FALCO; do
+    unset IFS
+    if [ ! -e "$file" ]; then
+      ViashError "Input file '$file' does not exist."
+      exit 1
+    fi
+  done
+  set +f
 fi
 if [ ! -z "$VIASH_PAR_INPUT_MULTIQC" ] && [ ! -e "$VIASH_PAR_INPUT_MULTIQC" ]; then
  ViashError "Input file '$VIASH_PAR_INPUT_MULTIQC' does not exist."
  exit 1
 fi
+if [ ! -z "$VIASH_PAR_INPUT_RUN_INFORMATION" ] && [ ! -e "$VIASH_PAR_INPUT_RUN_INFORMATION" ]; then
+  ViashError "Input file '$VIASH_PAR_INPUT_RUN_INFORMATION' does not exist."
+  exit 1
+fi

 # check whether parameters values are of the right type
 if [[ -n "$VIASH_META_CPUS" ]]; then
@@ -1008,6 +1063,9 @@ fi
 if [ ! -z "$VIASH_PAR_OUTPUT_MULTIQC" ] && [ ! -d "$(dirname "$VIASH_PAR_OUTPUT_MULTIQC")" ]; then
  mkdir -p "$(dirname "$VIASH_PAR_OUTPUT_MULTIQC")"
 fi
+if [ ! -z "$VIASH_PAR_OUTPUT_RUN_INFORMATION" ] && [ ! -d "$(dirname "$VIASH_PAR_OUTPUT_RUN_INFORMATION")" ]; then
+  mkdir -p "$(dirname "$VIASH_PAR_OUTPUT_RUN_INFORMATION")"
+fi

 if  [ "$VIASH_ENGINE_ID" == "native" ]  ; then
  if [ "$VIASH_MODE" == "run" ]; then
@@ -1026,13 +1084,24 @@ if [ ! -z "$VIASH_PAR_INPUT" ]; then
  VIASH_PAR_INPUT=$(ViashDockerAutodetectMount "$VIASH_PAR_INPUT")
 fi
 if [ ! -z "$VIASH_PAR_INPUT_FALCO" ]; then
-  VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$VIASH_PAR_INPUT_FALCO")" )
-  VIASH_PAR_INPUT_FALCO=$(ViashDockerAutodetectMount "$VIASH_PAR_INPUT_FALCO")
+  VIASH_TEST_INPUT_FALCO=()
+  IFS=';'
+  for var in $VIASH_PAR_INPUT_FALCO; do
+    unset IFS
+    VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$var")" )
+    var=$(ViashDockerAutodetectMount "$var")
+    VIASH_TEST_INPUT_FALCO+=( "$var" )
+  done
+  VIASH_PAR_INPUT_FALCO=$(IFS=';' ; echo "${VIASH_TEST_INPUT_FALCO[*]}")
 fi
 if [ ! -z "$VIASH_PAR_INPUT_MULTIQC" ]; then
  VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$VIASH_PAR_INPUT_MULTIQC")" )
  VIASH_PAR_INPUT_MULTIQC=$(ViashDockerAutodetectMount "$VIASH_PAR_INPUT_MULTIQC")
 fi
+if [ ! -z "$VIASH_PAR_INPUT_RUN_INFORMATION" ]; then
+  VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$VIASH_PAR_INPUT_RUN_INFORMATION")" )
+  VIASH_PAR_INPUT_RUN_INFORMATION=$(ViashDockerAutodetectMount "$VIASH_PAR_INPUT_RUN_INFORMATION")
+fi
 if [ ! -z "$VIASH_PAR_OUTPUT" ]; then
  VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$VIASH_PAR_OUTPUT")" )
  VIASH_PAR_OUTPUT=$(ViashDockerAutodetectMount "$VIASH_PAR_OUTPUT")
@@ -1048,6 +1117,11 @@ if [ ! -z "$VIASH_PAR_OUTPUT_MULTIQC" ]; then
  VIASH_PAR_OUTPUT_MULTIQC=$(ViashDockerAutodetectMount "$VIASH_PAR_OUTPUT_MULTIQC")
  VIASH_CHOWN_VARS+=( "$VIASH_PAR_OUTPUT_MULTIQC" )
 fi
+if [ ! -z "$VIASH_PAR_OUTPUT_RUN_INFORMATION" ]; then
+  VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$VIASH_PAR_OUTPUT_RUN_INFORMATION")" )
+  VIASH_PAR_OUTPUT_RUN_INFORMATION=$(ViashDockerAutodetectMount "$VIASH_PAR_OUTPUT_RUN_INFORMATION")
+  VIASH_CHOWN_VARS+=( "$VIASH_PAR_OUTPUT_RUN_INFORMATION" )
+fi
 if [ ! -z "$VIASH_META_RESOURCES_DIR" ]; then
  VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$VIASH_META_RESOURCES_DIR")" )
  VIASH_META_RESOURCES_DIR=$(ViashDockerAutodetectMount "$VIASH_META_RESOURCES_DIR")
@@ -1120,9 +1194,11 @@ cat > "\$tempscript" << 'VIASHMAIN'
 $( if [ ! -z ${VIASH_PAR_INPUT+x} ]; then echo "${VIASH_PAR_INPUT}" | sed "s#'#'\"'\"'#g;s#.*#par_input='&'#" ; else echo "# par_input="; fi )
 $( if [ ! -z ${VIASH_PAR_INPUT_FALCO+x} ]; then echo "${VIASH_PAR_INPUT_FALCO}" | sed "s#'#'\"'\"'#g;s#.*#par_input_falco='&'#" ; else echo "# par_input_falco="; fi )
 $( if [ ! -z ${VIASH_PAR_INPUT_MULTIQC+x} ]; then echo "${VIASH_PAR_INPUT_MULTIQC}" | sed "s#'#'\"'\"'#g;s#.*#par_input_multiqc='&'#" ; else echo "# par_input_multiqc="; fi )
+$( if [ ! -z ${VIASH_PAR_INPUT_RUN_INFORMATION+x} ]; then echo "${VIASH_PAR_INPUT_RUN_INFORMATION}" | sed "s#'#'\"'\"'#g;s#.*#par_input_run_information='&'#" ; else echo "# par_input_run_information="; fi )
 $( if [ ! -z ${VIASH_PAR_OUTPUT+x} ]; then echo "${VIASH_PAR_OUTPUT}" | sed "s#'#'\"'\"'#g;s#.*#par_output='&'#" ; else echo "# par_output="; fi )
 $( if [ ! -z ${VIASH_PAR_OUTPUT_FALCO+x} ]; then echo "${VIASH_PAR_OUTPUT_FALCO}" | sed "s#'#'\"'\"'#g;s#.*#par_output_falco='&'#" ; else echo "# par_output_falco="; fi )
 $( if [ ! -z ${VIASH_PAR_OUTPUT_MULTIQC+x} ]; then echo "${VIASH_PAR_OUTPUT_MULTIQC}" | sed "s#'#'\"'\"'#g;s#.*#par_output_multiqc='&'#" ; else echo "# par_output_multiqc="; fi )
+$( if [ ! -z ${VIASH_PAR_OUTPUT_RUN_INFORMATION+x} ]; then echo "${VIASH_PAR_OUTPUT_RUN_INFORMATION}" | sed "s#'#'\"'\"'#g;s#.*#par_output_run_information='&'#" ; else echo "# par_output_run_information="; fi )
 $( if [ ! -z ${VIASH_META_NAME+x} ]; then echo "${VIASH_META_NAME}" | sed "s#'#'\"'\"'#g;s#.*#meta_name='&'#" ; else echo "# meta_name="; fi )
 $( if [ ! -z ${VIASH_META_FUNCTIONALITY_NAME+x} ]; then echo "${VIASH_META_FUNCTIONALITY_NAME}" | sed "s#'#'\"'\"'#g;s#.*#meta_functionality_name='&'#" ; else echo "# meta_functionality_name="; fi )
 $( if [ ! -z ${VIASH_META_RESOURCES_DIR+x} ]; then echo "${VIASH_META_RESOURCES_DIR}" | sed "s#'#'\"'\"'#g;s#.*#meta_resources_dir='&'#" ; else echo "# meta_resources_dir="; fi )
@@ -1145,34 +1221,37 @@ $( if [ ! -z ${VIASH_META_MEMORY_PIB+x} ]; then echo "${VIASH_META_MEMORY_PIB}"
 ## VIASH END
 #!/bin/bash

-echo "Publishing \$par_input -> \$par_output"
-echo "Publishing \$par_input_falco -> \$par_output_falco"
-echo "Publishing \$par_input_multiqc -> \$par_output_multiqc"
+set -eo pipefail

-echo
-echo "Creating directory if it does not exist:"
-mkdir -p \$(dirname "\$par_output") && echo "Containing directory \$par_output created"
-mkdir -p \$(dirname "\$par_output_falco") && echo "Containing directory \$par_output_falco created"
-mkdir -p \$(dirname "\$par_output_multiqc") && echo "Containing directory \$par_output_multiqc created"
+declare -A input_output_mapping=(["par_input"]="par_output" 
+                                 ["par_input_multiqc"]="par_output_multiqc"
+                                 ["par_input_run_information"]="par_output_run_information"
+                                )

-echo
-echo "Copying files..."
-cp -rL "\$par_input" "\$par_output"
-cp -rL "\$par_input_falco" "\$par_output_falco"
-cp -rL "\$par_input_multiqc" "\$par_output_multiqc"
+for input_argument_name in "\${!input_output_mapping[@]}"
+do
+    input_location="\${!input_argument_name}"
+    output_argument_name="\${input_output_mapping[\$input_argument_name]}"
+    output_location="\${!output_argument_name}"
+    echo "Publishing \$input_location -> \$output_location"

-echo
-echo "Output files:"
-echo "par_output:"
-ls "\$par_output"
+    echo "Creating directory if it does not exist."
+    mkdir -p \$(dirname "\$output_location") && echo "Containing directory \$output_location created"

-echo
-echo "par_output_falco:"
-ls "\$par_output_falco"
+    echo "Copying files..."
+    cp -rL "\$input_location" "\$output_location"

-echo
-echo "par_output_multiqc:"
-ls "\$par_output_multiqc"
+    echo "Output files for \$output_location:"
+    ls "\$output_location"
+done
+
+echo "Grouping output from \$par_input_falco into \$par_output_falco"
+mkdir -p "\$par_output_falco"
+IFS=";" read -ra falco_inputs <<< \$par_input_falco
+for falco_dir in "\${falco_inputs[@]}"; do
+    echo "Copying contents of \$falco_dir"
+    find -H -D exec "\$falco_dir" -type f -maxdepth 1 -exec cp -t "\$par_output_falco" {} +
+done
 VIASHMAIN
 bash "\$tempscript" &
 wait "\$!"
@@ -1187,11 +1266,24 @@ if [[ "$VIASH_ENGINE_TYPE" == "docker" ]]; then
    VIASH_PAR_INPUT=$(ViashDockerStripAutomount "$VIASH_PAR_INPUT")
  fi
  if [ ! -z "$VIASH_PAR_INPUT_FALCO" ]; then
-    VIASH_PAR_INPUT_FALCO=$(ViashDockerStripAutomount "$VIASH_PAR_INPUT_FALCO")
+    unset VIASH_TEST_INPUT_FALCO
+    IFS=';'
+    for var in $VIASH_PAR_INPUT_FALCO; do
+      unset IFS
+      if [ -z "$VIASH_TEST_INPUT_FALCO" ]; then
+      VIASH_TEST_INPUT_FALCO="$(ViashDockerStripAutomount "$var")"
+    else
+      VIASH_TEST_INPUT_FALCO="$VIASH_TEST_INPUT_FALCO;""$(ViashDockerStripAutomount "$var")"
+    fi
+    done
+    VIASH_PAR_INPUT_FALCO="$VIASH_TEST_INPUT_FALCO"
  fi
  if [ ! -z "$VIASH_PAR_INPUT_MULTIQC" ]; then
    VIASH_PAR_INPUT_MULTIQC=$(ViashDockerStripAutomount "$VIASH_PAR_INPUT_MULTIQC")
  fi
+  if [ ! -z "$VIASH_PAR_INPUT_RUN_INFORMATION" ]; then
+    VIASH_PAR_INPUT_RUN_INFORMATION=$(ViashDockerStripAutomount "$VIASH_PAR_INPUT_RUN_INFORMATION")
+  fi
  if [ ! -z "$VIASH_PAR_OUTPUT" ]; then
    VIASH_PAR_OUTPUT=$(ViashDockerStripAutomount "$VIASH_PAR_OUTPUT")
  fi
@@ -1201,6 +1293,9 @@ if [[ "$VIASH_ENGINE_TYPE" == "docker" ]]; then
  if [ ! -z "$VIASH_PAR_OUTPUT_MULTIQC" ]; then
    VIASH_PAR_OUTPUT_MULTIQC=$(ViashDockerStripAutomount "$VIASH_PAR_OUTPUT_MULTIQC")
  fi
+  if [ ! -z "$VIASH_PAR_OUTPUT_RUN_INFORMATION" ]; then
+    VIASH_PAR_OUTPUT_RUN_INFORMATION=$(ViashDockerStripAutomount "$VIASH_PAR_OUTPUT_RUN_INFORMATION")
+  fi
  if [ ! -z "$VIASH_META_RESOURCES_DIR" ]; then
    VIASH_META_RESOURCES_DIR=$(ViashDockerStripAutomount "$VIASH_META_RESOURCES_DIR")
  fi
@@ -1229,6 +1324,10 @@ if [ ! -z "$VIASH_PAR_OUTPUT_MULTIQC" ] && [ ! -e "$VIASH_PAR_OUTPUT_MULTIQC" ];
  ViashError "Output file '$VIASH_PAR_OUTPUT_MULTIQC' does not exist."
  exit 1
 fi
+if [ ! -z "$VIASH_PAR_OUTPUT_RUN_INFORMATION" ] && [ ! -e "$VIASH_PAR_OUTPUT_RUN_INFORMATION" ]; then
+  ViashError "Output file '$VIASH_PAR_OUTPUT_RUN_INFORMATION' does not exist."
+  exit 1
+fi


 exit 0
--- a/target/executable/io/untar/.config.vsh.yaml
+++ b/target/executable/io/untar/.config.vsh.yaml
@@ -1,6 +1,6 @@
 name: "untar"
 namespace: "io"
-version: "v0.3.0"
+version: "v0.3.8"
 argument_groups:
 - name: "Input arguments"
  arguments:
@@ -45,6 +45,9 @@ resources:
 - type: "bash_script"
  path: "script.sh"
  is_executable: true
+- type: "file"
+  path: "nextflow_labels.config"
+  dest: "nextflow_labels.config"
 description: "Unpack a .tar file. When the contents of the .tar file is just a single\
  \ directory,\nput the contents of the directory into the output folder instead of\
  \ that directory.\n"
@@ -123,6 +126,8 @@ runners:
      cpu200: "cpus = 200"
      cpu500: "cpus = 500"
      cpu1000: "cpus = 1000"
+    script:
+    - "includeConfig(\"nextflow_labels.config\")"
  debug: false
  container: "docker"
 engines:
@@ -130,7 +135,7 @@ engines:
  id: "docker"
  image: "debian:stable-slim"
  target_registry: "images.viash-hub.com"
-  target_tag: "v0.3.0"
+  target_tag: "v0.3.8"
  namespace_separator: "/"
  setup:
  - type: "apt"
@@ -148,12 +153,12 @@ build_info:
  output: "target/executable/io/untar"
  executable: "target/executable/io/untar/untar"
  viash_version: "0.9.0"
-  git_commit: "c9cf5854796b31767c70848bcaf9aa42648d0bd7"
-  git_remote: "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex"
-  git_tag: "v0.1.1-9-gc9cf585"
+  git_commit: "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742"
+  git_remote: "https://github.com/viash-hub/demultiplex"
+  git_tag: "v0.3.5-9-gdafe2d8"
 package_config:
  name: "demultiplex"
-  version: "v0.3.0"
+  version: "v0.3.8"
  description: "Demultiplexing pipeline\n"
  info:
    test_resources:
@@ -164,10 +169,12 @@ package_config:
  target: "target"
  config_mods:
  - ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag\
-    \ := '$id'\n"
+    \ := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n\
+    .runners[.type == 'nextflow'].config.script := 'includeConfig(\"nextflow_labels.config\"\
+    )'\n"
  - ".engines += { type: \"native\" }"
  - ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
-  - ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+  - ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
  keywords:
  - "bioinformatics"
  - "sequence"
--- a/target/executable/io/untar/nextflow_labels.config
+++ b/target/executable/io/untar/nextflow_labels.config
@@ -0,0 +1,98 @@
+process {
+  container = 'nextflow/bash:latest'
+  
+  // default resources
+  memory = { 8.Gb * task.attempt }
+  cpus = 8
+  maxForks = 36
+
+  // Retry for exit codes that have something to do with memory issues
+  errorStrategy = { task.exitStatus in 137..140 ? 'retry' : 'terminate' }
+  maxRetries = 3
+  maxMemory = 192.GB
+
+  // Resource labels
+  withLabel: verylowcpu { cpus = 2 }
+  withLabel: lowcpu { cpus = 8 }
+  withLabel: midcpu { cpus = 16 }
+  withLabel: highcpu { cpus = 32 }
+  
+  withLabel: verylowmem { memory = { get_memory( 4.GB * task.attempt ) } }
+  withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+  withLabel: midmem { memory = { get_memory( 16.GB * task.attempt ) } }
+  withLabel: highmem { memory = { get_memory( 64.GB * task.attempt ) } }
+
+}
+
+profiles {
+  // detect tempdir
+  tempDir = java.nio.file.Paths.get(
+    System.getenv('NXF_TEMP') ?:
+      System.getenv('VIASH_TEMP') ?: 
+      System.getenv('TEMPDIR') ?: 
+      System.getenv('TMPDIR') ?: 
+      '/tmp'
+  ).toAbsolutePath()
+
+  mount_temp {
+    docker.temp            = tempDir
+    podman.temp            = tempDir
+    charliecloud.temp      = tempDir
+  }
+
+  no_publish {
+    process {
+      withName: '.*' {
+        publishDir = [
+          enabled: false
+        ]
+      }
+    }
+  }
+
+  docker {
+    docker.fixOwnership    = true
+    docker.enabled         = true
+    // docker.userEmulation   = true
+    singularity.enabled    = false
+    podman.enabled         = false
+    shifter.enabled        = false
+    charliecloud.enabled   = false
+  }
+
+  local {
+    // This config is for local processing.
+    process {
+        maxMemory = 25.GB
+        withLabel: verylowcpu { cpus = 2 }
+        withLabel: lowcpu { cpus = 4 }
+        withLabel: midcpu { cpus = 6 }
+        withLabel: highcpu { cpus = 12 }
+  
+        withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+        withLabel: midmem { memory = { get_memory( 12.GB * task.attempt ) } }
+        withLabel: highmem { memory = { get_memory( 20.GB * task.attempt ) } }
+    }
+  }
+}
+
+def get_memory(to_compare) {
+    if (!process.containsKey("maxMemory") || !process.maxMemory) {
+      return to_compare
+    }
+
+    try {
+      if (process.containsKey("maxRetries") && process.maxRetries && task.attempt == (process.maxRetries as int)) {
+        return process.maxMemory
+      }
+      else if (to_compare.compareTo(process.maxMemory as nextflow.util.MemoryUnit) == 1) {
+        return max_memory as nextflow.util.MemoryUnit
+      }
+      else {
+        return to_compare
+      }  
+    } catch (all) {
+          println "Error processing memory resources. Please check that process.maxMemory '${process.maxMemory}' and process.maxRetries '${process.maxRetries}' are valid!"
+          System.exit(1)
+    }
+  }
--- a/target/executable/io/untar/untar
+++ b/target/executable/io/untar/untar
@@ -1,6 +1,6 @@
 #!/usr/bin/env bash

-# untar v0.3.0
+# untar v0.3.8
 # 
 # This wrapper script is auto-generated by viash 0.9.0 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
@@ -171,7 +171,7 @@ VIASH_META_TEMP_DIR="$VIASH_TEMP"

 # ViashHelp: Display helpful explanation about this executable
 function ViashHelp {
-  echo "untar v0.3.0"
+  echo "untar v0.3.8"
  echo ""
  echo "Unpack a .tar file. When the contents of the .tar file is just a single"
  echo "directory,"
@@ -476,10 +476,10 @@ RUN apt-get update && \
  rm -rf /var/lib/apt/lists/*

 LABEL org.opencontainers.image.description="Companion container for running component io untar"
-LABEL org.opencontainers.image.created="2024-12-11T09:19:39Z"
+LABEL org.opencontainers.image.created="2025-03-27T16:00:54Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/demultiplex"
-LABEL org.opencontainers.image.revision="c9cf5854796b31767c70848bcaf9aa42648d0bd7"
-LABEL org.opencontainers.image.version="v0.3.0"
+LABEL org.opencontainers.image.revision="dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742"
+LABEL org.opencontainers.image.version="v0.3.8"

 VIASHDOCKER
  fi
@@ -615,7 +615,7 @@ while [[ $# -gt 0 ]]; do
            shift 1
            ;;
        --version)
-            echo "untar v0.3.0"
+            echo "untar v0.3.8"
            exit
            ;;
        --input)
@@ -745,7 +745,7 @@ if [[ "$VIASH_ENGINE_TYPE" == "docker" ]]; then

  # determine docker image id
  if [[ "$VIASH_ENGINE_ID" == 'docker' ]]; then
-    VIASH_DOCKER_IMAGE_ID='images.viash-hub.com/vsh/demultiplex/io/untar:v0.3.0'
+    VIASH_DOCKER_IMAGE_ID='images.viash-hub.com/vsh/demultiplex/io/untar:v0.3.8'
  fi

  # print dockerfile
--- a/target/nextflow/dataflow/combine_samples/.config.vsh.yaml
+++ b/target/nextflow/dataflow/combine_samples/.config.vsh.yaml
@@ -1,6 +1,6 @@
 name: "combine_samples"
 namespace: "dataflow"
-version: "v0.3.0"
+version: "v0.3.8"
 argument_groups:
 - name: "Input arguments"
  arguments:
@@ -30,6 +30,15 @@ argument_groups:
    direction: "input"
    multiple: true
    multiple_sep: ";"
+  - type: "file"
+    name: "--falco_dir"
+    info: null
+    must_exist: true
+    create_parent: true
+    required: true
+    direction: "input"
+    multiple: false
+    multiple_sep: ";"
 - name: "Output arguments"
  arguments:
  - type: "file"
@@ -50,11 +59,23 @@ argument_groups:
    direction: "output"
    multiple: true
    multiple_sep: ";"
+  - type: "file"
+    name: "--output_falco"
+    info: null
+    must_exist: true
+    create_parent: true
+    required: true
+    direction: "output"
+    multiple: true
+    multiple_sep: ";"
 resources:
 - type: "nextflow_script"
  path: "main.nf"
  is_executable: true
  entrypoint: "run_wf"
+- type: "file"
+  path: "nextflow_labels.config"
+  dest: "nextflow_labels.config"
 description: "Combine fastq files from across samples into one event with a list of\
  \ fastq files per orientation."
 info: null
@@ -125,6 +146,8 @@ runners:
      cpu200: "cpus = 200"
      cpu500: "cpus = 500"
      cpu1000: "cpus = 1000"
+    script:
+    - "includeConfig(\"nextflow_labels.config\")"
  debug: false
  container: "docker"
 engines:
@@ -139,12 +162,12 @@ build_info:
  output: "target/nextflow/dataflow/combine_samples"
  executable: "target/nextflow/dataflow/combine_samples/main.nf"
  viash_version: "0.9.0"
-  git_commit: "c9cf5854796b31767c70848bcaf9aa42648d0bd7"
-  git_remote: "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex"
-  git_tag: "v0.1.1-9-gc9cf585"
+  git_commit: "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742"
+  git_remote: "https://github.com/viash-hub/demultiplex"
+  git_tag: "v0.3.5-9-gdafe2d8"
 package_config:
  name: "demultiplex"
-  version: "v0.3.0"
+  version: "v0.3.8"
  description: "Demultiplexing pipeline\n"
  info:
    test_resources:
@@ -155,10 +178,12 @@ package_config:
  target: "target"
  config_mods:
  - ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag\
-    \ := '$id'\n"
+    \ := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n\
+    .runners[.type == 'nextflow'].config.script := 'includeConfig(\"nextflow_labels.config\"\
+    )'\n"
  - ".engines += { type: \"native\" }"
  - ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
-  - ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+  - ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
  keywords:
  - "bioinformatics"
  - "sequence"
--- a/target/nextflow/dataflow/combine_samples/main.nf
+++ b/target/nextflow/dataflow/combine_samples/main.nf
@@ -1,4 +1,4 @@
-// combine_samples v0.3.0
+// combine_samples v0.3.8
 // 
 // This wrapper script is auto-generated by viash 0.9.0 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
@@ -2806,7 +2806,7 @@ meta = [
  "config": processConfig(readJsonBlob('''{
  "name" : "combine_samples",
  "namespace" : "dataflow",
-  "version" : "v0.3.0",
+  "version" : "v0.3.8",
  "argument_groups" : [
    {
      "name" : "Input arguments",
@@ -2839,6 +2839,16 @@ meta = [
          "direction" : "input",
          "multiple" : true,
          "multiple_sep" : ";"
+        },
+        {
+          "type" : "file",
+          "name" : "--falco_dir",
+          "must_exist" : true,
+          "create_parent" : true,
+          "required" : true,
+          "direction" : "input",
+          "multiple" : false,
+          "multiple_sep" : ";"
        }
      ]
    },
@@ -2864,6 +2874,16 @@ meta = [
          "direction" : "output",
          "multiple" : true,
          "multiple_sep" : ";"
+        },
+        {
+          "type" : "file",
+          "name" : "--output_falco",
+          "must_exist" : true,
+          "create_parent" : true,
+          "required" : true,
+          "direction" : "output",
+          "multiple" : true,
+          "multiple_sep" : ";"
        }
      ]
    }
@@ -2874,6 +2894,11 @@ meta = [
      "path" : "main.nf",
      "is_executable" : true,
      "entrypoint" : "run_wf"
+    },
+    {
+      "type" : "file",
+      "path" : "/src/config/labels.config",
+      "dest" : "nextflow_labels.config"
    }
  ],
  "description" : "Combine fastq files from across samples into one event with a list of fastq files per orientation.",
@@ -2950,7 +2975,10 @@ meta = [
          "cpu200" : "cpus = 200",
          "cpu500" : "cpus = 500",
          "cpu1000" : "cpus = 1000"
-        }
+        },
+        "script" : [
+          "includeConfig(\\"nextflow_labels.config\\")"
+        ]
      },
      "debug" : false,
      "container" : "docker"
@@ -2972,13 +3000,13 @@ meta = [
    "engine" : "native|native",
    "output" : "target/nextflow/dataflow/combine_samples",
    "viash_version" : "0.9.0",
-    "git_commit" : "c9cf5854796b31767c70848bcaf9aa42648d0bd7",
-    "git_remote" : "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex",
-    "git_tag" : "v0.1.1-9-gc9cf585"
+    "git_commit" : "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742",
+    "git_remote" : "https://github.com/viash-hub/demultiplex",
+    "git_tag" : "v0.3.5-9-gdafe2d8"
  },
  "package_config" : {
    "name" : "demultiplex",
-    "version" : "v0.3.0",
+    "version" : "v0.3.8",
    "description" : "Demultiplexing pipeline\n",
    "info" : {
      "test_resources" : [
@@ -2992,10 +3020,10 @@ meta = [
    "source" : "src",
    "target" : "target",
    "config_mods" : [
-      ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag := '$id'\n",
+      ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n.runners[.type == 'nextflow'].config.script := 'includeConfig(\\"nextflow_labels.config\\")'\n",
      ".engines += { type: \\"native\\" }",
      ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'",
-      ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+      ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
    ],
    "keywords" : [
      "bioinformatics",
@@ -3033,10 +3061,12 @@ workflow run_wf {
        // Gather the following state for all samples
        def forward_fastqs = states.collect{it.forward_input}.flatten()
        def reverse_fastqs = states.collect{it.reverse_input}.findAll{it != null}.flatten()
+        def falco_dirs = states.collect{it.falco_dir}
        
        def resultState = [
          "output_forward": forward_fastqs,
          "output_reverse": reverse_fastqs,
+          "output_falco": falco_dirs,
          // The join ID is the same across all samples from the same run
          "_meta": ["join_id": states[0]._meta.join_id]
        ]
--- a/target/nextflow/dataflow/combine_samples/nextflow.config
+++ b/target/nextflow/dataflow/combine_samples/nextflow.config
@@ -2,7 +2,7 @@ manifest {
  name = 'dataflow/combine_samples'
  mainScript = 'main.nf'
  nextflowVersion = '!>=20.12.1-edge'
-  version = 'v0.3.0'
+  version = 'v0.3.8'
  description = 'Combine fastq files from across samples into one event with a list of fastq files per orientation.'
 }

@@ -122,4 +122,4 @@ process{
  withLabel: cpu1000 { cpus = 1000 }
 }

-
+includeConfig("nextflow_labels.config")
--- a/target/nextflow/dataflow/combine_samples/nextflow_labels.config
+++ b/target/nextflow/dataflow/combine_samples/nextflow_labels.config
@@ -0,0 +1,98 @@
+process {
+  container = 'nextflow/bash:latest'
+  
+  // default resources
+  memory = { 8.Gb * task.attempt }
+  cpus = 8
+  maxForks = 36
+
+  // Retry for exit codes that have something to do with memory issues
+  errorStrategy = { task.exitStatus in 137..140 ? 'retry' : 'terminate' }
+  maxRetries = 3
+  maxMemory = 192.GB
+
+  // Resource labels
+  withLabel: verylowcpu { cpus = 2 }
+  withLabel: lowcpu { cpus = 8 }
+  withLabel: midcpu { cpus = 16 }
+  withLabel: highcpu { cpus = 32 }
+  
+  withLabel: verylowmem { memory = { get_memory( 4.GB * task.attempt ) } }
+  withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+  withLabel: midmem { memory = { get_memory( 16.GB * task.attempt ) } }
+  withLabel: highmem { memory = { get_memory( 64.GB * task.attempt ) } }
+
+}
+
+profiles {
+  // detect tempdir
+  tempDir = java.nio.file.Paths.get(
+    System.getenv('NXF_TEMP') ?:
+      System.getenv('VIASH_TEMP') ?: 
+      System.getenv('TEMPDIR') ?: 
+      System.getenv('TMPDIR') ?: 
+      '/tmp'
+  ).toAbsolutePath()
+
+  mount_temp {
+    docker.temp            = tempDir
+    podman.temp            = tempDir
+    charliecloud.temp      = tempDir
+  }
+
+  no_publish {
+    process {
+      withName: '.*' {
+        publishDir = [
+          enabled: false
+        ]
+      }
+    }
+  }
+
+  docker {
+    docker.fixOwnership    = true
+    docker.enabled         = true
+    // docker.userEmulation   = true
+    singularity.enabled    = false
+    podman.enabled         = false
+    shifter.enabled        = false
+    charliecloud.enabled   = false
+  }
+
+  local {
+    // This config is for local processing.
+    process {
+        maxMemory = 25.GB
+        withLabel: verylowcpu { cpus = 2 }
+        withLabel: lowcpu { cpus = 4 }
+        withLabel: midcpu { cpus = 6 }
+        withLabel: highcpu { cpus = 12 }
+  
+        withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+        withLabel: midmem { memory = { get_memory( 12.GB * task.attempt ) } }
+        withLabel: highmem { memory = { get_memory( 20.GB * task.attempt ) } }
+    }
+  }
+}
+
+def get_memory(to_compare) {
+    if (!process.containsKey("maxMemory") || !process.maxMemory) {
+      return to_compare
+    }
+
+    try {
+      if (process.containsKey("maxRetries") && process.maxRetries && task.attempt == (process.maxRetries as int)) {
+        return process.maxMemory
+      }
+      else if (to_compare.compareTo(process.maxMemory as nextflow.util.MemoryUnit) == 1) {
+        return max_memory as nextflow.util.MemoryUnit
+      }
+      else {
+        return to_compare
+      }  
+    } catch (all) {
+          println "Error processing memory resources. Please check that process.maxMemory '${process.maxMemory}' and process.maxRetries '${process.maxRetries}' are valid!"
+          System.exit(1)
+    }
+  }
--- a/target/nextflow/dataflow/combine_samples/nextflow_schema.json
+++ b/target/nextflow/dataflow/combine_samples/nextflow_schema.json
@@ -43,6 +43,16 @@
            }
    

+        ,
+                "falco_dir": {
+                "type":
+                "string",
+                "description": "Type: `file`, required. ",
+                "help_text": "Type: `file`, required. "
+            
+            }
+    
+
 }
 },
    
@@ -75,6 +85,17 @@
            }
    

+        ,
+                "output_falco": {
+                "type":
+                "string",
+                "description": "Type: List of `file`, required, default: `$id.$key.output_falco_*.output_falco_*`, multiple_sep: `\";\"`. ",
+                "help_text": "Type: List of `file`, required, default: `$id.$key.output_falco_*.output_falco_*`, multiple_sep: `\";\"`. "
+            ,
+                "default":"$id.$key.output_falco_*.output_falco_*"
+            }
+    
+
 }
 },
    
--- a/target/nextflow/dataflow/gather_fastqs_and_validate/.config.vsh.yaml
+++ b/target/nextflow/dataflow/gather_fastqs_and_validate/.config.vsh.yaml
@@ -1,6 +1,6 @@
 name: "gather_fastqs_and_validate"
 namespace: "dataflow"
-version: "v0.3.0"
+version: "v0.3.8"
 argument_groups:
 - name: "Input arguments"
  arguments:
@@ -49,6 +49,9 @@ resources:
  path: "main.nf"
  is_executable: true
  entrypoint: "run_wf"
+- type: "file"
+  path: "nextflow_labels.config"
+  dest: "nextflow_labels.config"
 description: "From a directory containing fastq files, gather the files per sample\
  \ \nand validate according to the contents of the sample sheet.\n"
 info: null
@@ -119,6 +122,8 @@ runners:
      cpu200: "cpus = 200"
      cpu500: "cpus = 500"
      cpu1000: "cpus = 1000"
+    script:
+    - "includeConfig(\"nextflow_labels.config\")"
  debug: false
  container: "docker"
 engines:
@@ -133,12 +138,12 @@ build_info:
  output: "target/nextflow/dataflow/gather_fastqs_and_validate"
  executable: "target/nextflow/dataflow/gather_fastqs_and_validate/main.nf"
  viash_version: "0.9.0"
-  git_commit: "c9cf5854796b31767c70848bcaf9aa42648d0bd7"
-  git_remote: "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex"
-  git_tag: "v0.1.1-9-gc9cf585"
+  git_commit: "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742"
+  git_remote: "https://github.com/viash-hub/demultiplex"
+  git_tag: "v0.3.5-9-gdafe2d8"
 package_config:
  name: "demultiplex"
-  version: "v0.3.0"
+  version: "v0.3.8"
  description: "Demultiplexing pipeline\n"
  info:
    test_resources:
@@ -149,10 +154,12 @@ package_config:
  target: "target"
  config_mods:
  - ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag\
-    \ := '$id'\n"
+    \ := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n\
+    .runners[.type == 'nextflow'].config.script := 'includeConfig(\"nextflow_labels.config\"\
+    )'\n"
  - ".engines += { type: \"native\" }"
  - ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
-  - ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+  - ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
  keywords:
  - "bioinformatics"
  - "sequence"
--- a/target/nextflow/dataflow/gather_fastqs_and_validate/main.nf
+++ b/target/nextflow/dataflow/gather_fastqs_and_validate/main.nf
@@ -1,4 +1,4 @@
-// gather_fastqs_and_validate v0.3.0
+// gather_fastqs_and_validate v0.3.8
 // 
 // This wrapper script is auto-generated by viash 0.9.0 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
@@ -2806,7 +2806,7 @@ meta = [
  "config": processConfig(readJsonBlob('''{
  "name" : "gather_fastqs_and_validate",
  "namespace" : "dataflow",
-  "version" : "v0.3.0",
+  "version" : "v0.3.8",
  "argument_groups" : [
    {
      "name" : "Input arguments",
@@ -2867,6 +2867,11 @@ meta = [
      "path" : "main.nf",
      "is_executable" : true,
      "entrypoint" : "run_wf"
+    },
+    {
+      "type" : "file",
+      "path" : "/src/config/labels.config",
+      "dest" : "nextflow_labels.config"
    }
  ],
  "description" : "From a directory containing fastq files, gather the files per sample \nand validate according to the contents of the sample sheet.\n",
@@ -2943,7 +2948,10 @@ meta = [
          "cpu200" : "cpus = 200",
          "cpu500" : "cpus = 500",
          "cpu1000" : "cpus = 1000"
-        }
+        },
+        "script" : [
+          "includeConfig(\\"nextflow_labels.config\\")"
+        ]
      },
      "debug" : false,
      "container" : "docker"
@@ -2965,13 +2973,13 @@ meta = [
    "engine" : "native|native",
    "output" : "target/nextflow/dataflow/gather_fastqs_and_validate",
    "viash_version" : "0.9.0",
-    "git_commit" : "c9cf5854796b31767c70848bcaf9aa42648d0bd7",
-    "git_remote" : "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex",
-    "git_tag" : "v0.1.1-9-gc9cf585"
+    "git_commit" : "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742",
+    "git_remote" : "https://github.com/viash-hub/demultiplex",
+    "git_tag" : "v0.3.5-9-gdafe2d8"
  },
  "package_config" : {
    "name" : "demultiplex",
-    "version" : "v0.3.0",
+    "version" : "v0.3.8",
    "description" : "Demultiplexing pipeline\n",
    "info" : {
      "test_resources" : [
@@ -2985,10 +2993,10 @@ meta = [
    "source" : "src",
    "target" : "target",
    "config_mods" : [
-      ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag := '$id'\n",
+      ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n.runners[.type == 'nextflow'].config.script := 'includeConfig(\\"nextflow_labels.config\\")'\n",
      ".engines += { type: \\"native\\" }",
      ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'",
-      ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+      ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
    ],
    "keywords" : [
      "bioinformatics",
@@ -3011,6 +3019,29 @@ meta = [

 // inner workflow
 // user-provided Nextflow code
+import java.util.zip.GZIPInputStream
+import java.nio.file.Files
+import java.io.BufferedInputStream
+
+def is_empty(file_to_check){
+  /* 
+  Checks if a file has content
+  */
+  if (file_to_check.size() == 0) {
+    return true
+  }
+  def input_stream = Files.newInputStream(file_to_check)
+  def gzInputStream
+  try {
+    gzInputStream = new GZIPInputStream(new BufferedInputStream(input_stream))
+  } catch (java.io.EOFException ex) {
+    // This is not a gzipfile...
+    return false
+  }
+  def read_one_byte = gzInputStream.read()
+  return read_one_byte == -1
+}
+
 workflow run_wf {
  take:
    input_ch
@@ -3030,8 +3061,8 @@ workflow run_wf {
        println "Processing run information file ${sample_sheet}"
        csv_lines = sample_sheet.splitCsv(header: false, sep: ',')
        csv_lines.any { csv_items ->
-          if (csv_items.isEmpty()) {
-            // skip empty line
+          if (csv_items.isEmpty() || csv_items[0].startsWith("#")) {
+            // skip empty or commented line 
            return
          }
          def possible_header = csv_items[0]
@@ -3042,9 +3073,9 @@ workflow run_wf {
              println "Encountered next header '[${start_parsing}]', stopping parsing."
              return true
            }
-            // [Data] for illumina
-            // [Samples] for Element Biosciences
-            if (header in ["Data", "Samples"]) {
+            // [Data], [BCLConvert_Data] for illumina
+            // [Samples] or sometimes [SAMPLES] for Element Biosciences
+            if (header.toLowerCase() in ["data", "samples", "bclconvert_data"]) {
              println "Found header [${header}], start parsing."
              start_parsing = true
              return
@@ -3063,13 +3094,16 @@ workflow run_wf {
              println "Found sample names column '${csv_items[sample_id_column_index]}'."
              return
            }
-            samples += csv_items[sample_id_column_index]
+            def candidate_sample_id = csv_items[sample_id_column_index]
+            if (candidate_sample_id?.trim()) { // Don't add empty csv entries.
+              samples += csv_items[sample_id_column_index]
+            }
          }
          // This return is important! (If 'true' is returned, the parsing stops.)
          return 
        }
        assert start_parsing: 
-          "Sample information file does not contain [Data] or [Samples] header!"
+          "Sample information file does not contain [Data], [Samples] or [BCLConvert_Data] header!"
        assert samples.size() > 1:
          "Sample information file does not seem to contain any information about the samples!"
        println "Finished processing run information file, found samples: ${samples}."
@@ -3088,6 +3122,9 @@ workflow run_wf {
            "Found forward: ${forward_fastq} and reverse: ${reverse_fastq}."
          println "Found ${forward_fastq.size()} forward and ${reverse_fastq.size()} reverse " +
            "fastq files for sample ${sample_id}"
+
+          assert forward_fastq.every{!is_empty(it)} && reverse_fastq.every{!is_empty(it)}:
+            "A fastq file for sample '${sample_id}' appears to be empty!"
          def fastqs_state = [
            "fastq_forward": forward_fastq,
            "fastq_reverse": reverse_fastq,
--- a/target/nextflow/dataflow/gather_fastqs_and_validate/nextflow.config
+++ b/target/nextflow/dataflow/gather_fastqs_and_validate/nextflow.config
@@ -2,7 +2,7 @@ manifest {
  name = 'dataflow/gather_fastqs_and_validate'
  mainScript = 'main.nf'
  nextflowVersion = '!>=20.12.1-edge'
-  version = 'v0.3.0'
+  version = 'v0.3.8'
  description = 'From a directory containing fastq files, gather the files per sample \nand validate according to the contents of the sample sheet.\n'
 }

@@ -122,4 +122,4 @@ process{
  withLabel: cpu1000 { cpus = 1000 }
 }

-
+includeConfig("nextflow_labels.config")
--- a/target/nextflow/dataflow/gather_fastqs_and_validate/nextflow_labels.config
+++ b/target/nextflow/dataflow/gather_fastqs_and_validate/nextflow_labels.config
@@ -0,0 +1,98 @@
+process {
+  container = 'nextflow/bash:latest'
+  
+  // default resources
+  memory = { 8.Gb * task.attempt }
+  cpus = 8
+  maxForks = 36
+
+  // Retry for exit codes that have something to do with memory issues
+  errorStrategy = { task.exitStatus in 137..140 ? 'retry' : 'terminate' }
+  maxRetries = 3
+  maxMemory = 192.GB
+
+  // Resource labels
+  withLabel: verylowcpu { cpus = 2 }
+  withLabel: lowcpu { cpus = 8 }
+  withLabel: midcpu { cpus = 16 }
+  withLabel: highcpu { cpus = 32 }
+  
+  withLabel: verylowmem { memory = { get_memory( 4.GB * task.attempt ) } }
+  withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+  withLabel: midmem { memory = { get_memory( 16.GB * task.attempt ) } }
+  withLabel: highmem { memory = { get_memory( 64.GB * task.attempt ) } }
+
+}
+
+profiles {
+  // detect tempdir
+  tempDir = java.nio.file.Paths.get(
+    System.getenv('NXF_TEMP') ?:
+      System.getenv('VIASH_TEMP') ?: 
+      System.getenv('TEMPDIR') ?: 
+      System.getenv('TMPDIR') ?: 
+      '/tmp'
+  ).toAbsolutePath()
+
+  mount_temp {
+    docker.temp            = tempDir
+    podman.temp            = tempDir
+    charliecloud.temp      = tempDir
+  }
+
+  no_publish {
+    process {
+      withName: '.*' {
+        publishDir = [
+          enabled: false
+        ]
+      }
+    }
+  }
+
+  docker {
+    docker.fixOwnership    = true
+    docker.enabled         = true
+    // docker.userEmulation   = true
+    singularity.enabled    = false
+    podman.enabled         = false
+    shifter.enabled        = false
+    charliecloud.enabled   = false
+  }
+
+  local {
+    // This config is for local processing.
+    process {
+        maxMemory = 25.GB
+        withLabel: verylowcpu { cpus = 2 }
+        withLabel: lowcpu { cpus = 4 }
+        withLabel: midcpu { cpus = 6 }
+        withLabel: highcpu { cpus = 12 }
+  
+        withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+        withLabel: midmem { memory = { get_memory( 12.GB * task.attempt ) } }
+        withLabel: highmem { memory = { get_memory( 20.GB * task.attempt ) } }
+    }
+  }
+}
+
+def get_memory(to_compare) {
+    if (!process.containsKey("maxMemory") || !process.maxMemory) {
+      return to_compare
+    }
+
+    try {
+      if (process.containsKey("maxRetries") && process.maxRetries && task.attempt == (process.maxRetries as int)) {
+        return process.maxMemory
+      }
+      else if (to_compare.compareTo(process.maxMemory as nextflow.util.MemoryUnit) == 1) {
+        return max_memory as nextflow.util.MemoryUnit
+      }
+      else {
+        return to_compare
+      }  
+    } catch (all) {
+          println "Error processing memory resources. Please check that process.maxMemory '${process.maxMemory}' and process.maxRetries '${process.maxRetries}' are valid!"
+          System.exit(1)
+    }
+  }
--- a/target/nextflow/demultiplex/.config.vsh.yaml
+++ b/target/nextflow/demultiplex/.config.vsh.yaml
@@ -1,5 +1,5 @@
 name: "demultiplex"
-version: "v0.3.0"
+version: "v0.3.8"
 argument_groups:
 - name: "Input arguments"
  arguments:
@@ -71,7 +71,7 @@ argument_groups:
    create_parent: true
    required: false
    direction: "output"
-    multiple: false
+    multiple: true
    multiple_sep: ";"
  - type: "file"
    name: "--output_multiqc"
@@ -85,11 +85,33 @@ argument_groups:
    direction: "output"
    multiple: false
    multiple_sep: ";"
+  - type: "file"
+    name: "--output_run_information"
+    info: null
+    default:
+    - "$id/run_information.csv"
+    must_exist: true
+    create_parent: true
+    required: true
+    direction: "output"
+    multiple: false
+    multiple_sep: ";"
+- name: "Other arguments"
+  arguments:
+  - type: "boolean_true"
+    name: "--skip_copycomplete_check"
+    description: "Disable the check for the presence of a \"CopyComplete.txt\" file\
+      \ in input\ndirectory in case of Illumina data.\n"
+    info: null
+    direction: "input"
 resources:
 - type: "nextflow_script"
  path: "main.nf"
  is_executable: true
  entrypoint: "run_wf"
+- type: "file"
+  path: "nextflow_labels.config"
+  dest: "nextflow_labels.config"
 description: "Demultiplexing of raw sequencing data"
 test_resources:
 - type: "nextflow_script"
@@ -206,6 +228,8 @@ runners:
      cpu200: "cpus = 200"
      cpu500: "cpus = 500"
      cpu1000: "cpus = 1000"
+    script:
+    - "includeConfig(\"nextflow_labels.config\")"
  debug: false
  container: "docker"
 engines:
@@ -220,9 +244,9 @@ build_info:
  output: "target/nextflow/demultiplex"
  executable: "target/nextflow/demultiplex/main.nf"
  viash_version: "0.9.0"
-  git_commit: "c9cf5854796b31767c70848bcaf9aa42648d0bd7"
-  git_remote: "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex"
-  git_tag: "v0.1.1-9-gc9cf585"
+  git_commit: "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742"
+  git_remote: "https://github.com/viash-hub/demultiplex"
+  git_tag: "v0.3.5-9-gdafe2d8"
  dependencies:
  - "target/nextflow/io/untar"
  - "target/nextflow/dataflow/gather_fastqs_and_validate"
@@ -234,7 +258,7 @@ build_info:
  - "target/dependencies/vsh/vsh/biobox/v0.3.0/nextflow/multiqc"
 package_config:
  name: "demultiplex"
-  version: "v0.3.0"
+  version: "v0.3.8"
  description: "Demultiplexing pipeline\n"
  info:
    test_resources:
@@ -245,10 +269,12 @@ package_config:
  target: "target"
  config_mods:
  - ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag\
-    \ := '$id'\n"
+    \ := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n\
+    .runners[.type == 'nextflow'].config.script := 'includeConfig(\"nextflow_labels.config\"\
+    )'\n"
  - ".engines += { type: \"native\" }"
  - ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
-  - ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+  - ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
  keywords:
  - "bioinformatics"
  - "sequence"
--- a/target/nextflow/demultiplex/main.nf
+++ b/target/nextflow/demultiplex/main.nf
@@ -1,4 +1,4 @@
-// demultiplex v0.3.0
+// demultiplex v0.3.8
 // 
 // This wrapper script is auto-generated by viash 0.9.0 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
@@ -2805,7 +2805,7 @@ meta = [
  "resources_dir": moduleDir.toRealPath().normalize(),
  "config": processConfig(readJsonBlob('''{
  "name" : "demultiplex",
-  "version" : "v0.3.0",
+  "version" : "v0.3.8",
  "argument_groups" : [
    {
      "name" : "Input arguments",
@@ -2884,7 +2884,7 @@ meta = [
          "create_parent" : true,
          "required" : false,
          "direction" : "output",
-          "multiple" : false,
+          "multiple" : true,
          "multiple_sep" : ";"
        },
        {
@@ -2900,6 +2900,30 @@ meta = [
          "direction" : "output",
          "multiple" : false,
          "multiple_sep" : ";"
+        },
+        {
+          "type" : "file",
+          "name" : "--output_run_information",
+          "default" : [
+            "$id/run_information.csv"
+          ],
+          "must_exist" : true,
+          "create_parent" : true,
+          "required" : true,
+          "direction" : "output",
+          "multiple" : false,
+          "multiple_sep" : ";"
+        }
+      ]
+    },
+    {
+      "name" : "Other arguments",
+      "arguments" : [
+        {
+          "type" : "boolean_true",
+          "name" : "--skip_copycomplete_check",
+          "description" : "Disable the check for the presence of a \\"CopyComplete.txt\\" file in input\ndirectory in case of Illumina data.\n",
+          "direction" : "input"
        }
      ]
    }
@@ -2910,6 +2934,11 @@ meta = [
      "path" : "main.nf",
      "is_executable" : true,
      "entrypoint" : "run_wf"
+    },
+    {
+      "type" : "file",
+      "path" : "/src/config/labels.config",
+      "dest" : "nextflow_labels.config"
    }
  ],
  "description" : "Demultiplexing of raw sequencing data",
@@ -3066,7 +3095,10 @@ meta = [
          "cpu200" : "cpus = 200",
          "cpu500" : "cpus = 500",
          "cpu1000" : "cpus = 1000"
-        }
+        },
+        "script" : [
+          "includeConfig(\\"nextflow_labels.config\\")"
+        ]
      },
      "debug" : false,
      "container" : "docker"
@@ -3088,13 +3120,13 @@ meta = [
    "engine" : "native|native",
    "output" : "target/nextflow/demultiplex",
    "viash_version" : "0.9.0",
-    "git_commit" : "c9cf5854796b31767c70848bcaf9aa42648d0bd7",
-    "git_remote" : "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex",
-    "git_tag" : "v0.1.1-9-gc9cf585"
+    "git_commit" : "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742",
+    "git_remote" : "https://github.com/viash-hub/demultiplex",
+    "git_tag" : "v0.3.5-9-gdafe2d8"
  },
  "package_config" : {
    "name" : "demultiplex",
-    "version" : "v0.3.0",
+    "version" : "v0.3.8",
    "description" : "Demultiplexing pipeline\n",
    "info" : {
      "test_resources" : [
@@ -3108,10 +3140,10 @@ meta = [
    "source" : "src",
    "target" : "target",
    "config_mods" : [
-      ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag := '$id'\n",
+      ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n.runners[.type == 'nextflow'].config.script := 'includeConfig(\\"nextflow_labels.config\\")'\n",
      ".engines += { type: \\"native\\" }",
      ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'",
-      ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+      ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
    ],
    "keywords" : [
      "bioinformatics",
@@ -3169,6 +3201,11 @@ workflow run_wf {
      | map {id, state ->
        def newState = [:]
        println("Provided run information: ${state.run_information} and demultiplexer: ${state.demultiplexer}")
+        // No auto-detection of run information file (it is user provided),
+        // in this case the demultiplexer should also be specified.
+        assert (!state.run_information || state.demultiplexer): "When setting --run_information, " +
+          "you must also provide a demultiplexer"
+
        if (!state.run_information) {
          println("Run information was not specified, auto-detecting...")
          // The supported_platforms hashmap must be a 1-on-1 mapping
@@ -3206,7 +3243,7 @@ workflow run_wf {
            "found in input directory."

          // When autodetecting, the demultiplexer must be set if the run information was found
-          assert demultiplexer, "State error: the demultiplexer should have been autodetected. " +
+          assert demultiplexer: "State error: the demultiplexer should have been autodetected. " +
            "Please report this as a bug."

          // When autodetecting, the found demultiplexer must match
@@ -3226,13 +3263,17 @@ workflow run_wf {
            "run_information": run_information,
            "demultiplexer": demultiplexer,
          ]
-        }
+        } // end auto-detection logic

        if (newState.demultiplexer in ["bclconvert"]) {
          // Do not add InterOp to state because we generate the summary csv's in the next
          // step based on the run dir, not the InterOp dir.
          def interop_dir = state.input.resolve("InterOp")
          assert interop_dir.isDirectory(): "Expected InterOp directory to be present."
+
+          def copycomplete_file = state.input.resolve("CopyComplete.txt")
+          assert (copycomplete_file.isFile() || state.skip_copycomplete_check): 
+            "'CopyComplete.txt' file was not found!"
        }

        def resultState = state + newState
@@ -3308,28 +3349,12 @@ workflow run_wf {
      )

    output_ch = samples_ch 
-
-
-      | combine_samples.run(
-        fromState: { id, state ->
-          [
-            "id": state.run_id,
-            "forward_input": state.fastq_forward,
-            "reverse_input": state.fastq_reverse, 
-          ]
-        },
-        toState: [
-          "forward_fastqs": "output_forward",
-          "reverse_fastqs": "output_reverse",
-        ]
-      )
      | falco.run(
-        directives: [label: ["lowcpu", "lowmem"]],
+        directives: [label: ["verylowcpu", "lowmem"]],
        fromState: {id, state ->
-          reverse_fastqs_list = state.reverse_fastqs ? state.reverse_fastqs : []
          [
-            "input": state.forward_fastqs + reverse_fastqs_list,
-            "outdir": "${state.output_falco}",
+            "input": [state.fastq_forward, state.fastq_reverse],
+            "outdir": "$id/qc/falco",
            "summary_filename": null,
            "report_filename": null,
            "data_filename": null,
@@ -3339,11 +3364,28 @@ workflow run_wf {
          state + [ "output_falco" : result.outdir ]
        }
      )
+
+      | combine_samples.run(
+        fromState: { id, state ->
+          [
+            "id": state.run_id,
+            "forward_input": state.fastq_forward,
+            "reverse_input": state.fastq_reverse, 
+            "falco_dir": state.output_falco,
+          ]
+        },
+        toState: [
+          "forward_fastqs": "output_forward",
+          "reverse_fastqs": "output_reverse",
+          "output_falco": "output_falco",
+        ]
+      )
+
      | multiqc.run(
        directives: [label: ["midcpu", "midmem"]],
        fromState: {id, state ->
          def new_state = [
-            "input": [state.output_falco],
+            "input": state.output_falco,
            "output_report": state.output_multiqc,
            "cl_config": 'sp: {fastqc/data: {fn: "*_fastqc_data.txt"}}'
          ]
@@ -3364,7 +3406,8 @@ workflow run_wf {
          //"_meta": "_meta",
          "output": "output_demultiplexer",
          "output_falco": "output_falco",
-          "output_multiqc": "output_multiqc"
+          "output_multiqc": "output_multiqc",
+          "output_run_information": "run_information",
        ]
      )

--- a/target/nextflow/demultiplex/nextflow.config
+++ b/target/nextflow/demultiplex/nextflow.config
@@ -2,7 +2,7 @@ manifest {
  name = 'demultiplex'
  mainScript = 'main.nf'
  nextflowVersion = '!>=20.12.1-edge'
-  version = 'v0.3.0'
+  version = 'v0.3.8'
  description = 'Demultiplexing of raw sequencing data'
 }

@@ -122,4 +122,4 @@ process{
  withLabel: cpu1000 { cpus = 1000 }
 }

-
+includeConfig("nextflow_labels.config")
--- a/target/nextflow/demultiplex/nextflow_labels.config
+++ b/target/nextflow/demultiplex/nextflow_labels.config
@@ -0,0 +1,98 @@
+process {
+  container = 'nextflow/bash:latest'
+  
+  // default resources
+  memory = { 8.Gb * task.attempt }
+  cpus = 8
+  maxForks = 36
+
+  // Retry for exit codes that have something to do with memory issues
+  errorStrategy = { task.exitStatus in 137..140 ? 'retry' : 'terminate' }
+  maxRetries = 3
+  maxMemory = 192.GB
+
+  // Resource labels
+  withLabel: verylowcpu { cpus = 2 }
+  withLabel: lowcpu { cpus = 8 }
+  withLabel: midcpu { cpus = 16 }
+  withLabel: highcpu { cpus = 32 }
+  
+  withLabel: verylowmem { memory = { get_memory( 4.GB * task.attempt ) } }
+  withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+  withLabel: midmem { memory = { get_memory( 16.GB * task.attempt ) } }
+  withLabel: highmem { memory = { get_memory( 64.GB * task.attempt ) } }
+
+}
+
+profiles {
+  // detect tempdir
+  tempDir = java.nio.file.Paths.get(
+    System.getenv('NXF_TEMP') ?:
+      System.getenv('VIASH_TEMP') ?: 
+      System.getenv('TEMPDIR') ?: 
+      System.getenv('TMPDIR') ?: 
+      '/tmp'
+  ).toAbsolutePath()
+
+  mount_temp {
+    docker.temp            = tempDir
+    podman.temp            = tempDir
+    charliecloud.temp      = tempDir
+  }
+
+  no_publish {
+    process {
+      withName: '.*' {
+        publishDir = [
+          enabled: false
+        ]
+      }
+    }
+  }
+
+  docker {
+    docker.fixOwnership    = true
+    docker.enabled         = true
+    // docker.userEmulation   = true
+    singularity.enabled    = false
+    podman.enabled         = false
+    shifter.enabled        = false
+    charliecloud.enabled   = false
+  }
+
+  local {
+    // This config is for local processing.
+    process {
+        maxMemory = 25.GB
+        withLabel: verylowcpu { cpus = 2 }
+        withLabel: lowcpu { cpus = 4 }
+        withLabel: midcpu { cpus = 6 }
+        withLabel: highcpu { cpus = 12 }
+  
+        withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+        withLabel: midmem { memory = { get_memory( 12.GB * task.attempt ) } }
+        withLabel: highmem { memory = { get_memory( 20.GB * task.attempt ) } }
+    }
+  }
+}
+
+def get_memory(to_compare) {
+    if (!process.containsKey("maxMemory") || !process.maxMemory) {
+      return to_compare
+    }
+
+    try {
+      if (process.containsKey("maxRetries") && process.maxRetries && task.attempt == (process.maxRetries as int)) {
+        return process.maxMemory
+      }
+      else if (to_compare.compareTo(process.maxMemory as nextflow.util.MemoryUnit) == 1) {
+        return max_memory as nextflow.util.MemoryUnit
+      }
+      else {
+        return to_compare
+      }  
+    } catch (all) {
+          println "Error processing memory resources. Please check that process.maxMemory '${process.maxMemory}' and process.maxRetries '${process.maxRetries}' are valid!"
+          System.exit(1)
+    }
+  }
--- a/target/nextflow/demultiplex/nextflow_schema.json
+++ b/target/nextflow/demultiplex/nextflow_schema.json
@@ -80,10 +80,10 @@
                "output_falco": {
                "type":
                "string",
-                "description": "Type: `file`, default: `$id.$key.output_falco.output_falco`. Directory to write falco output to",
-                "help_text": "Type: `file`, default: `$id.$key.output_falco.output_falco`. Directory to write falco output to"
+                "description": "Type: List of `file`, default: `$id.$key.output_falco_*.output_falco_*`, multiple_sep: `\";\"`. Directory to write falco output to",
+                "help_text": "Type: List of `file`, default: `$id.$key.output_falco_*.output_falco_*`, multiple_sep: `\";\"`. Directory to write falco output to"
            ,
-                "default":"$id.$key.output_falco.output_falco"
+                "default":"$id.$key.output_falco_*.output_falco_*"
            }
    

@@ -98,6 +98,38 @@
            }
    

+        ,
+                "output_run_information": {
+                "type":
+                "string",
+                "description": "Type: `file`, required, default: `$id.$key.output_run_information.csv`. ",
+                "help_text": "Type: `file`, required, default: `$id.$key.output_run_information.csv`. "
+            ,
+                "default":"$id.$key.output_run_information.csv"
+            }
+    
+
+}
+},
+    
+    
+    "other arguments" : {
+    "title": "Other arguments",
+    "type": "object",
+    "description": "No description",
+    "properties": {
+    
+        
+                "skip_copycomplete_check": {
+                "type":
+                "boolean",
+                "description": "Type: `boolean_true`, default: `false`. Disable the check for the presence of a \"CopyComplete",
+                "help_text": "Type: `boolean_true`, default: `false`. Disable the check for the presence of a \"CopyComplete.txt\" file in input\ndirectory in case of Illumina data.\n"
+            ,
+                "default":false
+            }
+    
+
 }
 },
    
@@ -142,6 +174,10 @@
    "$ref": "#/definitions/output arguments"
    },

+    {
+    "$ref": "#/definitions/other arguments"
+    },
+
    {
    "$ref": "#/definitions/nextflow input-output arguments"
    }
--- a/target/nextflow/io/interop_summary_to_csv/.config.vsh.yaml
+++ b/target/nextflow/io/interop_summary_to_csv/.config.vsh.yaml
@@ -1,6 +1,6 @@
 name: "interop_summary_to_csv"
 namespace: "io"
-version: "v0.3.0"
+version: "v0.3.8"
 argument_groups:
 - name: "Input arguments"
  arguments:
@@ -38,6 +38,9 @@ resources:
 - type: "bash_script"
  path: "script.sh"
  is_executable: true
+- type: "file"
+  path: "nextflow_labels.config"
+  dest: "nextflow_labels.config"
 info: null
 status: "enabled"
 requirements:
@@ -109,6 +112,8 @@ runners:
      cpu200: "cpus = 200"
      cpu500: "cpus = 500"
      cpu1000: "cpus = 1000"
+    script:
+    - "includeConfig(\"nextflow_labels.config\")"
  debug: false
  container: "docker"
 engines:
@@ -116,7 +121,7 @@ engines:
  id: "docker"
  image: "debian:stable-slim"
  target_registry: "images.viash-hub.com"
-  target_tag: "v0.3.0"
+  target_tag: "v0.3.8"
  namespace_separator: "/"
  setup:
  - type: "apt"
@@ -141,12 +146,12 @@ build_info:
  output: "target/nextflow/io/interop_summary_to_csv"
  executable: "target/nextflow/io/interop_summary_to_csv/main.nf"
  viash_version: "0.9.0"
-  git_commit: "c9cf5854796b31767c70848bcaf9aa42648d0bd7"
-  git_remote: "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex"
-  git_tag: "v0.1.1-9-gc9cf585"
+  git_commit: "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742"
+  git_remote: "https://github.com/viash-hub/demultiplex"
+  git_tag: "v0.3.5-9-gdafe2d8"
 package_config:
  name: "demultiplex"
-  version: "v0.3.0"
+  version: "v0.3.8"
  description: "Demultiplexing pipeline\n"
  info:
    test_resources:
@@ -157,10 +162,12 @@ package_config:
  target: "target"
  config_mods:
  - ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag\
-    \ := '$id'\n"
+    \ := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n\
+    .runners[.type == 'nextflow'].config.script := 'includeConfig(\"nextflow_labels.config\"\
+    )'\n"
  - ".engines += { type: \"native\" }"
  - ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
-  - ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+  - ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
  keywords:
  - "bioinformatics"
  - "sequence"
--- a/target/nextflow/io/interop_summary_to_csv/main.nf
+++ b/target/nextflow/io/interop_summary_to_csv/main.nf
@@ -1,4 +1,4 @@
-// interop_summary_to_csv v0.3.0
+// interop_summary_to_csv v0.3.8
 // 
 // This wrapper script is auto-generated by viash 0.9.0 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
@@ -2806,7 +2806,7 @@ meta = [
  "config": processConfig(readJsonBlob('''{
  "name" : "interop_summary_to_csv",
  "namespace" : "io",
-  "version" : "v0.3.0",
+  "version" : "v0.3.8",
  "argument_groups" : [
    {
      "name" : "Input arguments",
@@ -2855,6 +2855,11 @@ meta = [
      "type" : "bash_script",
      "path" : "script.sh",
      "is_executable" : true
+    },
+    {
+      "type" : "file",
+      "path" : "/src/config/labels.config",
+      "dest" : "nextflow_labels.config"
    }
  ],
  "status" : "enabled",
@@ -2935,7 +2940,10 @@ meta = [
          "cpu200" : "cpus = 200",
          "cpu500" : "cpus = 500",
          "cpu1000" : "cpus = 1000"
-        }
+        },
+        "script" : [
+          "includeConfig(\\"nextflow_labels.config\\")"
+        ]
      },
      "debug" : false,
      "container" : "docker"
@@ -2947,7 +2955,7 @@ meta = [
      "id" : "docker",
      "image" : "debian:stable-slim",
      "target_registry" : "images.viash-hub.com",
-      "target_tag" : "v0.3.0",
+      "target_tag" : "v0.3.8",
      "namespace_separator" : "/",
      "setup" : [
        {
@@ -2977,13 +2985,13 @@ meta = [
    "engine" : "docker|native",
    "output" : "target/nextflow/io/interop_summary_to_csv",
    "viash_version" : "0.9.0",
-    "git_commit" : "c9cf5854796b31767c70848bcaf9aa42648d0bd7",
-    "git_remote" : "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex",
-    "git_tag" : "v0.1.1-9-gc9cf585"
+    "git_commit" : "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742",
+    "git_remote" : "https://github.com/viash-hub/demultiplex",
+    "git_tag" : "v0.3.5-9-gdafe2d8"
  },
  "package_config" : {
    "name" : "demultiplex",
-    "version" : "v0.3.0",
+    "version" : "v0.3.8",
    "description" : "Demultiplexing pipeline\n",
    "info" : {
      "test_resources" : [
@@ -2997,10 +3005,10 @@ meta = [
    "source" : "src",
    "target" : "target",
    "config_mods" : [
-      ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag := '$id'\n",
+      ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n.runners[.type == 'nextflow'].config.script := 'includeConfig(\\"nextflow_labels.config\\")'\n",
      ".engines += { type: \\"native\\" }",
      ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'",
-      ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+      ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
    ],
    "keywords" : [
      "bioinformatics",
@@ -3423,7 +3431,7 @@ meta["defaults"] = [
  "container" : {
    "registry" : "images.viash-hub.com",
    "image" : "vsh/demultiplex/io/interop_summary_to_csv",
-    "tag" : "v0.3.0"
+    "tag" : "v0.3.8"
  },
  "tag" : "$id"
 }'''),
--- a/target/nextflow/io/interop_summary_to_csv/nextflow.config
+++ b/target/nextflow/io/interop_summary_to_csv/nextflow.config
@@ -2,7 +2,7 @@ manifest {
  name = 'io/interop_summary_to_csv'
  mainScript = 'main.nf'
  nextflowVersion = '!>=20.12.1-edge'
-  version = 'v0.3.0'
+  version = 'v0.3.8'
 }

 process.container = 'nextflow/bash:latest'
@@ -121,4 +121,4 @@ process{
  withLabel: cpu1000 { cpus = 1000 }
 }

-
+includeConfig("nextflow_labels.config")
--- a/target/nextflow/io/interop_summary_to_csv/nextflow_labels.config
+++ b/target/nextflow/io/interop_summary_to_csv/nextflow_labels.config
@@ -0,0 +1,98 @@
+process {
+  container = 'nextflow/bash:latest'
+  
+  // default resources
+  memory = { 8.Gb * task.attempt }
+  cpus = 8
+  maxForks = 36
+
+  // Retry for exit codes that have something to do with memory issues
+  errorStrategy = { task.exitStatus in 137..140 ? 'retry' : 'terminate' }
+  maxRetries = 3
+  maxMemory = 192.GB
+
+  // Resource labels
+  withLabel: verylowcpu { cpus = 2 }
+  withLabel: lowcpu { cpus = 8 }
+  withLabel: midcpu { cpus = 16 }
+  withLabel: highcpu { cpus = 32 }
+  
+  withLabel: verylowmem { memory = { get_memory( 4.GB * task.attempt ) } }
+  withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+  withLabel: midmem { memory = { get_memory( 16.GB * task.attempt ) } }
+  withLabel: highmem { memory = { get_memory( 64.GB * task.attempt ) } }
+
+}
+
+profiles {
+  // detect tempdir
+  tempDir = java.nio.file.Paths.get(
+    System.getenv('NXF_TEMP') ?:
+      System.getenv('VIASH_TEMP') ?: 
+      System.getenv('TEMPDIR') ?: 
+      System.getenv('TMPDIR') ?: 
+      '/tmp'
+  ).toAbsolutePath()
+
+  mount_temp {
+    docker.temp            = tempDir
+    podman.temp            = tempDir
+    charliecloud.temp      = tempDir
+  }
+
+  no_publish {
+    process {
+      withName: '.*' {
+        publishDir = [
+          enabled: false
+        ]
+      }
+    }
+  }
+
+  docker {
+    docker.fixOwnership    = true
+    docker.enabled         = true
+    // docker.userEmulation   = true
+    singularity.enabled    = false
+    podman.enabled         = false
+    shifter.enabled        = false
+    charliecloud.enabled   = false
+  }
+
+  local {
+    // This config is for local processing.
+    process {
+        maxMemory = 25.GB
+        withLabel: verylowcpu { cpus = 2 }
+        withLabel: lowcpu { cpus = 4 }
+        withLabel: midcpu { cpus = 6 }
+        withLabel: highcpu { cpus = 12 }
+  
+        withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+        withLabel: midmem { memory = { get_memory( 12.GB * task.attempt ) } }
+        withLabel: highmem { memory = { get_memory( 20.GB * task.attempt ) } }
+    }
+  }
+}
+
+def get_memory(to_compare) {
+    if (!process.containsKey("maxMemory") || !process.maxMemory) {
+      return to_compare
+    }
+
+    try {
+      if (process.containsKey("maxRetries") && process.maxRetries && task.attempt == (process.maxRetries as int)) {
+        return process.maxMemory
+      }
+      else if (to_compare.compareTo(process.maxMemory as nextflow.util.MemoryUnit) == 1) {
+        return max_memory as nextflow.util.MemoryUnit
+      }
+      else {
+        return to_compare
+      }  
+    } catch (all) {
+          println "Error processing memory resources. Please check that process.maxMemory '${process.maxMemory}' and process.maxRetries '${process.maxRetries}' are valid!"
+          System.exit(1)
+    }
+  }
--- a/target/nextflow/io/publish/.config.vsh.yaml
+++ b/target/nextflow/io/publish/.config.vsh.yaml
@@ -1,6 +1,6 @@
 name: "publish"
 namespace: "io"
-version: "v0.3.0"
+version: "v0.3.8"
 argument_groups:
 - name: "Input arguments"
  arguments:
@@ -22,11 +22,21 @@ argument_groups:
    create_parent: true
    required: true
    direction: "input"
-    multiple: false
+    multiple: true
    multiple_sep: ";"
  - type: "file"
    name: "--input_multiqc"
-    description: "Directory to write falco output to"
+    description: "Location where to write the MultiQC report to."
+    info: null
+    must_exist: true
+    create_parent: true
+    required: true
+    direction: "input"
+    multiple: false
+    multiple_sep: ";"
+  - type: "file"
+    name: "--input_run_information"
+    description: "Location where to write the run information to."
    info: null
    must_exist: true
    create_parent: true
@@ -69,10 +79,24 @@ argument_groups:
    direction: "output"
    multiple: false
    multiple_sep: ";"
+  - type: "file"
+    name: "--output_run_information"
+    info: null
+    default:
+    - "run_information.csv"
+    must_exist: true
+    create_parent: true
+    required: false
+    direction: "output"
+    multiple: false
+    multiple_sep: ";"
 resources:
 - type: "bash_script"
  path: "code.sh"
  is_executable: true
+- type: "file"
+  path: "nextflow_labels.config"
+  dest: "nextflow_labels.config"
 description: "Publish the processed results of the run"
 info: null
 status: "enabled"
@@ -145,6 +169,8 @@ runners:
      cpu200: "cpus = 200"
      cpu500: "cpus = 500"
      cpu1000: "cpus = 1000"
+    script:
+    - "includeConfig(\"nextflow_labels.config\")"
  debug: false
  container: "docker"
 engines:
@@ -152,7 +178,7 @@ engines:
  id: "docker"
  image: "debian:stable-slim"
  target_registry: "images.viash-hub.com"
-  target_tag: "v0.3.0"
+  target_tag: "v0.3.8"
  namespace_separator: "/"
  setup:
  - type: "apt"
@@ -170,12 +196,12 @@ build_info:
  output: "target/nextflow/io/publish"
  executable: "target/nextflow/io/publish/main.nf"
  viash_version: "0.9.0"
-  git_commit: "c9cf5854796b31767c70848bcaf9aa42648d0bd7"
-  git_remote: "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex"
-  git_tag: "v0.1.1-9-gc9cf585"
+  git_commit: "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742"
+  git_remote: "https://github.com/viash-hub/demultiplex"
+  git_tag: "v0.3.5-9-gdafe2d8"
 package_config:
  name: "demultiplex"
-  version: "v0.3.0"
+  version: "v0.3.8"
  description: "Demultiplexing pipeline\n"
  info:
    test_resources:
@@ -186,10 +212,12 @@ package_config:
  target: "target"
  config_mods:
  - ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag\
-    \ := '$id'\n"
+    \ := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n\
+    .runners[.type == 'nextflow'].config.script := 'includeConfig(\"nextflow_labels.config\"\
+    )'\n"
  - ".engines += { type: \"native\" }"
  - ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
-  - ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+  - ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
  keywords:
  - "bioinformatics"
  - "sequence"
--- a/target/nextflow/io/publish/main.nf
+++ b/target/nextflow/io/publish/main.nf
@@ -1,4 +1,4 @@
-// publish v0.3.0
+// publish v0.3.8
 // 
 // This wrapper script is auto-generated by viash 0.9.0 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
@@ -2806,7 +2806,7 @@ meta = [
  "config": processConfig(readJsonBlob('''{
  "name" : "publish",
  "namespace" : "io",
-  "version" : "v0.3.0",
+  "version" : "v0.3.8",
  "argument_groups" : [
    {
      "name" : "Input arguments",
@@ -2830,13 +2830,24 @@ meta = [
          "create_parent" : true,
          "required" : true,
          "direction" : "input",
-          "multiple" : false,
+          "multiple" : true,
          "multiple_sep" : ";"
        },
        {
          "type" : "file",
          "name" : "--input_multiqc",
-          "description" : "Directory to write falco output to",
+          "description" : "Location where to write the MultiQC report to.",
+          "must_exist" : true,
+          "create_parent" : true,
+          "required" : true,
+          "direction" : "input",
+          "multiple" : false,
+          "multiple_sep" : ";"
+        },
+        {
+          "type" : "file",
+          "name" : "--input_run_information",
+          "description" : "Location where to write the run information to.",
          "must_exist" : true,
          "create_parent" : true,
          "required" : true,
@@ -2887,6 +2898,19 @@ meta = [
          "direction" : "output",
          "multiple" : false,
          "multiple_sep" : ";"
+        },
+        {
+          "type" : "file",
+          "name" : "--output_run_information",
+          "default" : [
+            "run_information.csv"
+          ],
+          "must_exist" : true,
+          "create_parent" : true,
+          "required" : false,
+          "direction" : "output",
+          "multiple" : false,
+          "multiple_sep" : ";"
        }
      ]
    }
@@ -2896,6 +2920,11 @@ meta = [
      "type" : "bash_script",
      "path" : "./code.sh",
      "is_executable" : true
+    },
+    {
+      "type" : "file",
+      "path" : "/src/config/labels.config",
+      "dest" : "nextflow_labels.config"
    }
  ],
  "description" : "Publish the processed results of the run",
@@ -2977,7 +3006,10 @@ meta = [
          "cpu200" : "cpus = 200",
          "cpu500" : "cpus = 500",
          "cpu1000" : "cpus = 1000"
-        }
+        },
+        "script" : [
+          "includeConfig(\\"nextflow_labels.config\\")"
+        ]
      },
      "debug" : false,
      "container" : "docker"
@@ -2989,7 +3021,7 @@ meta = [
      "id" : "docker",
      "image" : "debian:stable-slim",
      "target_registry" : "images.viash-hub.com",
-      "target_tag" : "v0.3.0",
+      "target_tag" : "v0.3.8",
      "namespace_separator" : "/",
      "setup" : [
        {
@@ -3012,13 +3044,13 @@ meta = [
    "engine" : "docker|native",
    "output" : "target/nextflow/io/publish",
    "viash_version" : "0.9.0",
-    "git_commit" : "c9cf5854796b31767c70848bcaf9aa42648d0bd7",
-    "git_remote" : "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex",
-    "git_tag" : "v0.1.1-9-gc9cf585"
+    "git_commit" : "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742",
+    "git_remote" : "https://github.com/viash-hub/demultiplex",
+    "git_tag" : "v0.3.5-9-gdafe2d8"
  },
  "package_config" : {
    "name" : "demultiplex",
-    "version" : "v0.3.0",
+    "version" : "v0.3.8",
    "description" : "Demultiplexing pipeline\n",
    "info" : {
      "test_resources" : [
@@ -3032,10 +3064,10 @@ meta = [
    "source" : "src",
    "target" : "target",
    "config_mods" : [
-      ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag := '$id'\n",
+      ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n.runners[.type == 'nextflow'].config.script := 'includeConfig(\\"nextflow_labels.config\\")'\n",
      ".engines += { type: \\"native\\" }",
      ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'",
-      ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+      ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
    ],
    "keywords" : [
      "bioinformatics",
@@ -3067,9 +3099,11 @@ cat > "$tempscript" << VIASHMAIN
 $( if [ ! -z ${VIASH_PAR_INPUT+x} ]; then echo "${VIASH_PAR_INPUT}" | sed "s#'#'\\"'\\"'#g;s#.*#par_input='&'#" ; else echo "# par_input="; fi )
 $( if [ ! -z ${VIASH_PAR_INPUT_FALCO+x} ]; then echo "${VIASH_PAR_INPUT_FALCO}" | sed "s#'#'\\"'\\"'#g;s#.*#par_input_falco='&'#" ; else echo "# par_input_falco="; fi )
 $( if [ ! -z ${VIASH_PAR_INPUT_MULTIQC+x} ]; then echo "${VIASH_PAR_INPUT_MULTIQC}" | sed "s#'#'\\"'\\"'#g;s#.*#par_input_multiqc='&'#" ; else echo "# par_input_multiqc="; fi )
+$( if [ ! -z ${VIASH_PAR_INPUT_RUN_INFORMATION+x} ]; then echo "${VIASH_PAR_INPUT_RUN_INFORMATION}" | sed "s#'#'\\"'\\"'#g;s#.*#par_input_run_information='&'#" ; else echo "# par_input_run_information="; fi )
 $( if [ ! -z ${VIASH_PAR_OUTPUT+x} ]; then echo "${VIASH_PAR_OUTPUT}" | sed "s#'#'\\"'\\"'#g;s#.*#par_output='&'#" ; else echo "# par_output="; fi )
 $( if [ ! -z ${VIASH_PAR_OUTPUT_FALCO+x} ]; then echo "${VIASH_PAR_OUTPUT_FALCO}" | sed "s#'#'\\"'\\"'#g;s#.*#par_output_falco='&'#" ; else echo "# par_output_falco="; fi )
 $( if [ ! -z ${VIASH_PAR_OUTPUT_MULTIQC+x} ]; then echo "${VIASH_PAR_OUTPUT_MULTIQC}" | sed "s#'#'\\"'\\"'#g;s#.*#par_output_multiqc='&'#" ; else echo "# par_output_multiqc="; fi )
+$( if [ ! -z ${VIASH_PAR_OUTPUT_RUN_INFORMATION+x} ]; then echo "${VIASH_PAR_OUTPUT_RUN_INFORMATION}" | sed "s#'#'\\"'\\"'#g;s#.*#par_output_run_information='&'#" ; else echo "# par_output_run_information="; fi )
 $( if [ ! -z ${VIASH_META_NAME+x} ]; then echo "${VIASH_META_NAME}" | sed "s#'#'\\"'\\"'#g;s#.*#meta_name='&'#" ; else echo "# meta_name="; fi )
 $( if [ ! -z ${VIASH_META_FUNCTIONALITY_NAME+x} ]; then echo "${VIASH_META_FUNCTIONALITY_NAME}" | sed "s#'#'\\"'\\"'#g;s#.*#meta_functionality_name='&'#" ; else echo "# meta_functionality_name="; fi )
 $( if [ ! -z ${VIASH_META_RESOURCES_DIR+x} ]; then echo "${VIASH_META_RESOURCES_DIR}" | sed "s#'#'\\"'\\"'#g;s#.*#meta_resources_dir='&'#" ; else echo "# meta_resources_dir="; fi )
@@ -3092,34 +3126,37 @@ $( if [ ! -z ${VIASH_META_MEMORY_PIB+x} ]; then echo "${VIASH_META_MEMORY_PIB}"
 ## VIASH END
 #!/bin/bash

-echo "Publishing \\$par_input -> \\$par_output"
-echo "Publishing \\$par_input_falco -> \\$par_output_falco"
-echo "Publishing \\$par_input_multiqc -> \\$par_output_multiqc"
+set -eo pipefail

-echo
-echo "Creating directory if it does not exist:"
-mkdir -p \\$(dirname "\\$par_output") && echo "Containing directory \\$par_output created"
-mkdir -p \\$(dirname "\\$par_output_falco") && echo "Containing directory \\$par_output_falco created"
-mkdir -p \\$(dirname "\\$par_output_multiqc") && echo "Containing directory \\$par_output_multiqc created"
+declare -A input_output_mapping=(["par_input"]="par_output" 
+                                 ["par_input_multiqc"]="par_output_multiqc"
+                                 ["par_input_run_information"]="par_output_run_information"
+                                )

-echo
-echo "Copying files..."
-cp -rL "\\$par_input" "\\$par_output"
-cp -rL "\\$par_input_falco" "\\$par_output_falco"
-cp -rL "\\$par_input_multiqc" "\\$par_output_multiqc"
+for input_argument_name in "\\${!input_output_mapping[@]}"
+do
+    input_location="\\${!input_argument_name}"
+    output_argument_name="\\${input_output_mapping[\\$input_argument_name]}"
+    output_location="\\${!output_argument_name}"
+    echo "Publishing \\$input_location -> \\$output_location"

-echo
-echo "Output files:"
-echo "par_output:"
-ls "\\$par_output"
+    echo "Creating directory if it does not exist."
+    mkdir -p \\$(dirname "\\$output_location") && echo "Containing directory \\$output_location created"

-echo
-echo "par_output_falco:"
-ls "\\$par_output_falco"
+    echo "Copying files..."
+    cp -rL "\\$input_location" "\\$output_location"

-echo
-echo "par_output_multiqc:"
-ls "\\$par_output_multiqc"
+    echo "Output files for \\$output_location:"
+    ls "\\$output_location"
+done
+
+echo "Grouping output from \\$par_input_falco into \\$par_output_falco"
+mkdir -p "\\$par_output_falco"
+IFS=";" read -ra falco_inputs <<< \\$par_input_falco
+for falco_dir in "\\${falco_inputs[@]}"; do
+    echo "Copying contents of \\$falco_dir"
+    find -H -D exec "\\$falco_dir" -type f -maxdepth 1 -exec cp -t "\\$par_output_falco" {} +
+done
 VIASHMAIN
 bash "$tempscript"
 '''
@@ -3481,7 +3518,7 @@ meta["defaults"] = [
  "container" : {
    "registry" : "images.viash-hub.com",
    "image" : "vsh/demultiplex/io/publish",
-    "tag" : "v0.3.0"
+    "tag" : "v0.3.8"
  },
  "tag" : "$id"
 }'''),
--- a/target/nextflow/io/publish/nextflow.config
+++ b/target/nextflow/io/publish/nextflow.config
@@ -2,7 +2,7 @@ manifest {
  name = 'io/publish'
  mainScript = 'main.nf'
  nextflowVersion = '!>=20.12.1-edge'
-  version = 'v0.3.0'
+  version = 'v0.3.8'
  description = 'Publish the processed results of the run'
 }

@@ -122,4 +122,4 @@ process{
  withLabel: cpu1000 { cpus = 1000 }
 }

-
+includeConfig("nextflow_labels.config")
--- a/target/nextflow/io/publish/nextflow_labels.config
+++ b/target/nextflow/io/publish/nextflow_labels.config
@@ -0,0 +1,98 @@
+process {
+  container = 'nextflow/bash:latest'
+  
+  // default resources
+  memory = { 8.Gb * task.attempt }
+  cpus = 8
+  maxForks = 36
+
+  // Retry for exit codes that have something to do with memory issues
+  errorStrategy = { task.exitStatus in 137..140 ? 'retry' : 'terminate' }
+  maxRetries = 3
+  maxMemory = 192.GB
+
+  // Resource labels
+  withLabel: verylowcpu { cpus = 2 }
+  withLabel: lowcpu { cpus = 8 }
+  withLabel: midcpu { cpus = 16 }
+  withLabel: highcpu { cpus = 32 }
+  
+  withLabel: verylowmem { memory = { get_memory( 4.GB * task.attempt ) } }
+  withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+  withLabel: midmem { memory = { get_memory( 16.GB * task.attempt ) } }
+  withLabel: highmem { memory = { get_memory( 64.GB * task.attempt ) } }
+
+}
+
+profiles {
+  // detect tempdir
+  tempDir = java.nio.file.Paths.get(
+    System.getenv('NXF_TEMP') ?:
+      System.getenv('VIASH_TEMP') ?: 
+      System.getenv('TEMPDIR') ?: 
+      System.getenv('TMPDIR') ?: 
+      '/tmp'
+  ).toAbsolutePath()
+
+  mount_temp {
+    docker.temp            = tempDir
+    podman.temp            = tempDir
+    charliecloud.temp      = tempDir
+  }
+
+  no_publish {
+    process {
+      withName: '.*' {
+        publishDir = [
+          enabled: false
+        ]
+      }
+    }
+  }
+
+  docker {
+    docker.fixOwnership    = true
+    docker.enabled         = true
+    // docker.userEmulation   = true
+    singularity.enabled    = false
+    podman.enabled         = false
+    shifter.enabled        = false
+    charliecloud.enabled   = false
+  }
+
+  local {
+    // This config is for local processing.
+    process {
+        maxMemory = 25.GB
+        withLabel: verylowcpu { cpus = 2 }
+        withLabel: lowcpu { cpus = 4 }
+        withLabel: midcpu { cpus = 6 }
+        withLabel: highcpu { cpus = 12 }
+  
+        withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+        withLabel: midmem { memory = { get_memory( 12.GB * task.attempt ) } }
+        withLabel: highmem { memory = { get_memory( 20.GB * task.attempt ) } }
+    }
+  }
+}
+
+def get_memory(to_compare) {
+    if (!process.containsKey("maxMemory") || !process.maxMemory) {
+      return to_compare
+    }
+
+    try {
+      if (process.containsKey("maxRetries") && process.maxRetries && task.attempt == (process.maxRetries as int)) {
+        return process.maxMemory
+      }
+      else if (to_compare.compareTo(process.maxMemory as nextflow.util.MemoryUnit) == 1) {
+        return max_memory as nextflow.util.MemoryUnit
+      }
+      else {
+        return to_compare
+      }  
+    } catch (all) {
+          println "Error processing memory resources. Please check that process.maxMemory '${process.maxMemory}' and process.maxRetries '${process.maxRetries}' are valid!"
+          System.exit(1)
+    }
+  }
--- a/target/nextflow/io/publish/nextflow_schema.json
+++ b/target/nextflow/io/publish/nextflow_schema.json
@@ -27,8 +27,8 @@
                "input_falco": {
                "type":
                "string",
-                "description": "Type: `file`, required. Directory to write falco output to",
-                "help_text": "Type: `file`, required. Directory to write falco output to"
+                "description": "Type: List of `file`, required, multiple_sep: `\";\"`. Directory to write falco output to",
+                "help_text": "Type: List of `file`, required, multiple_sep: `\";\"`. Directory to write falco output to"
            
            }
    
@@ -37,8 +37,18 @@
                "input_multiqc": {
                "type":
                "string",
-                "description": "Type: `file`, required. Directory to write falco output to",
-                "help_text": "Type: `file`, required. Directory to write falco output to"
+                "description": "Type: `file`, required. Location where to write the MultiQC report to",
+                "help_text": "Type: `file`, required. Location where to write the MultiQC report to."
+            
+            }
+    
+
+        ,
+                "input_run_information": {
+                "type":
+                "string",
+                "description": "Type: `file`, required. Location where to write the run information to",
+                "help_text": "Type: `file`, required. Location where to write the run information to."
            
            }
    
@@ -86,6 +96,17 @@
            }
    

+        ,
+                "output_run_information": {
+                "type":
+                "string",
+                "description": "Type: `file`, default: `$id.$key.output_run_information.csv`. ",
+                "help_text": "Type: `file`, default: `$id.$key.output_run_information.csv`. "
+            ,
+                "default":"$id.$key.output_run_information.csv"
+            }
+    
+
 }
 },
    
--- a/target/nextflow/io/untar/.config.vsh.yaml
+++ b/target/nextflow/io/untar/.config.vsh.yaml
@@ -1,6 +1,6 @@
 name: "untar"
 namespace: "io"
-version: "v0.3.0"
+version: "v0.3.8"
 argument_groups:
 - name: "Input arguments"
  arguments:
@@ -45,6 +45,9 @@ resources:
 - type: "bash_script"
  path: "script.sh"
  is_executable: true
+- type: "file"
+  path: "nextflow_labels.config"
+  dest: "nextflow_labels.config"
 description: "Unpack a .tar file. When the contents of the .tar file is just a single\
  \ directory,\nput the contents of the directory into the output folder instead of\
  \ that directory.\n"
@@ -123,6 +126,8 @@ runners:
      cpu200: "cpus = 200"
      cpu500: "cpus = 500"
      cpu1000: "cpus = 1000"
+    script:
+    - "includeConfig(\"nextflow_labels.config\")"
  debug: false
  container: "docker"
 engines:
@@ -130,7 +135,7 @@ engines:
  id: "docker"
  image: "debian:stable-slim"
  target_registry: "images.viash-hub.com"
-  target_tag: "v0.3.0"
+  target_tag: "v0.3.8"
  namespace_separator: "/"
  setup:
  - type: "apt"
@@ -148,12 +153,12 @@ build_info:
  output: "target/nextflow/io/untar"
  executable: "target/nextflow/io/untar/main.nf"
  viash_version: "0.9.0"
-  git_commit: "c9cf5854796b31767c70848bcaf9aa42648d0bd7"
-  git_remote: "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex"
-  git_tag: "v0.1.1-9-gc9cf585"
+  git_commit: "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742"
+  git_remote: "https://github.com/viash-hub/demultiplex"
+  git_tag: "v0.3.5-9-gdafe2d8"
 package_config:
  name: "demultiplex"
-  version: "v0.3.0"
+  version: "v0.3.8"
  description: "Demultiplexing pipeline\n"
  info:
    test_resources:
@@ -164,10 +169,12 @@ package_config:
  target: "target"
  config_mods:
  - ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag\
-    \ := '$id'\n"
+    \ := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n\
+    .runners[.type == 'nextflow'].config.script := 'includeConfig(\"nextflow_labels.config\"\
+    )'\n"
  - ".engines += { type: \"native\" }"
  - ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
-  - ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+  - ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
  keywords:
  - "bioinformatics"
  - "sequence"
--- a/target/nextflow/io/untar/main.nf
+++ b/target/nextflow/io/untar/main.nf
@@ -1,4 +1,4 @@
-// untar v0.3.0
+// untar v0.3.8
 // 
 // This wrapper script is auto-generated by viash 0.9.0 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
@@ -2806,7 +2806,7 @@ meta = [
  "config": processConfig(readJsonBlob('''{
  "name" : "untar",
  "namespace" : "io",
-  "version" : "v0.3.0",
+  "version" : "v0.3.8",
  "argument_groups" : [
    {
      "name" : "Input arguments",
@@ -2866,6 +2866,11 @@ meta = [
      "type" : "bash_script",
      "path" : "script.sh",
      "is_executable" : true
+    },
+    {
+      "type" : "file",
+      "path" : "/src/config/labels.config",
+      "dest" : "nextflow_labels.config"
    }
  ],
  "description" : "Unpack a .tar file. When the contents of the .tar file is just a single directory,\nput the contents of the directory into the output folder instead of that directory.\n",
@@ -2954,7 +2959,10 @@ meta = [
          "cpu200" : "cpus = 200",
          "cpu500" : "cpus = 500",
          "cpu1000" : "cpus = 1000"
-        }
+        },
+        "script" : [
+          "includeConfig(\\"nextflow_labels.config\\")"
+        ]
      },
      "debug" : false,
      "container" : "docker"
@@ -2966,7 +2974,7 @@ meta = [
      "id" : "docker",
      "image" : "debian:stable-slim",
      "target_registry" : "images.viash-hub.com",
-      "target_tag" : "v0.3.0",
+      "target_tag" : "v0.3.8",
      "namespace_separator" : "/",
      "setup" : [
        {
@@ -2989,13 +2997,13 @@ meta = [
    "engine" : "docker|native",
    "output" : "target/nextflow/io/untar",
    "viash_version" : "0.9.0",
-    "git_commit" : "c9cf5854796b31767c70848bcaf9aa42648d0bd7",
-    "git_remote" : "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex",
-    "git_tag" : "v0.1.1-9-gc9cf585"
+    "git_commit" : "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742",
+    "git_remote" : "https://github.com/viash-hub/demultiplex",
+    "git_tag" : "v0.3.5-9-gdafe2d8"
  },
  "package_config" : {
    "name" : "demultiplex",
-    "version" : "v0.3.0",
+    "version" : "v0.3.8",
    "description" : "Demultiplexing pipeline\n",
    "info" : {
      "test_resources" : [
@@ -3009,10 +3017,10 @@ meta = [
    "source" : "src",
    "target" : "target",
    "config_mods" : [
-      ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag := '$id'\n",
+      ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n.runners[.type == 'nextflow'].config.script := 'includeConfig(\\"nextflow_labels.config\\")'\n",
      ".engines += { type: \\"native\\" }",
      ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'",
-      ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+      ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
    ],
    "keywords" : [
      "bioinformatics",
@@ -3465,7 +3473,7 @@ meta["defaults"] = [
  "container" : {
    "registry" : "images.viash-hub.com",
    "image" : "vsh/demultiplex/io/untar",
-    "tag" : "v0.3.0"
+    "tag" : "v0.3.8"
  },
  "tag" : "$id"
 }'''),
--- a/target/nextflow/io/untar/nextflow.config
+++ b/target/nextflow/io/untar/nextflow.config
@@ -2,7 +2,7 @@ manifest {
  name = 'io/untar'
  mainScript = 'main.nf'
  nextflowVersion = '!>=20.12.1-edge'
-  version = 'v0.3.0'
+  version = 'v0.3.8'
  description = 'Unpack a .tar file. When the contents of the .tar file is just a single directory,\nput the contents of the directory into the output folder instead of that directory.\n'
 }

@@ -122,4 +122,4 @@ process{
  withLabel: cpu1000 { cpus = 1000 }
 }

-
+includeConfig("nextflow_labels.config")
--- a/target/nextflow/io/untar/nextflow_labels.config
+++ b/target/nextflow/io/untar/nextflow_labels.config
@@ -0,0 +1,98 @@
+process {
+  container = 'nextflow/bash:latest'
+  
+  // default resources
+  memory = { 8.Gb * task.attempt }
+  cpus = 8
+  maxForks = 36
+
+  // Retry for exit codes that have something to do with memory issues
+  errorStrategy = { task.exitStatus in 137..140 ? 'retry' : 'terminate' }
+  maxRetries = 3
+  maxMemory = 192.GB
+
+  // Resource labels
+  withLabel: verylowcpu { cpus = 2 }
+  withLabel: lowcpu { cpus = 8 }
+  withLabel: midcpu { cpus = 16 }
+  withLabel: highcpu { cpus = 32 }
+  
+  withLabel: verylowmem { memory = { get_memory( 4.GB * task.attempt ) } }
+  withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+  withLabel: midmem { memory = { get_memory( 16.GB * task.attempt ) } }
+  withLabel: highmem { memory = { get_memory( 64.GB * task.attempt ) } }
+
+}
+
+profiles {
+  // detect tempdir
+  tempDir = java.nio.file.Paths.get(
+    System.getenv('NXF_TEMP') ?:
+      System.getenv('VIASH_TEMP') ?: 
+      System.getenv('TEMPDIR') ?: 
+      System.getenv('TMPDIR') ?: 
+      '/tmp'
+  ).toAbsolutePath()
+
+  mount_temp {
+    docker.temp            = tempDir
+    podman.temp            = tempDir
+    charliecloud.temp      = tempDir
+  }
+
+  no_publish {
+    process {
+      withName: '.*' {
+        publishDir = [
+          enabled: false
+        ]
+      }
+    }
+  }
+
+  docker {
+    docker.fixOwnership    = true
+    docker.enabled         = true
+    // docker.userEmulation   = true
+    singularity.enabled    = false
+    podman.enabled         = false
+    shifter.enabled        = false
+    charliecloud.enabled   = false
+  }
+
+  local {
+    // This config is for local processing.
+    process {
+        maxMemory = 25.GB
+        withLabel: verylowcpu { cpus = 2 }
+        withLabel: lowcpu { cpus = 4 }
+        withLabel: midcpu { cpus = 6 }
+        withLabel: highcpu { cpus = 12 }
+  
+        withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+        withLabel: midmem { memory = { get_memory( 12.GB * task.attempt ) } }
+        withLabel: highmem { memory = { get_memory( 20.GB * task.attempt ) } }
+    }
+  }
+}
+
+def get_memory(to_compare) {
+    if (!process.containsKey("maxMemory") || !process.maxMemory) {
+      return to_compare
+    }
+
+    try {
+      if (process.containsKey("maxRetries") && process.maxRetries && task.attempt == (process.maxRetries as int)) {
+        return process.maxMemory
+      }
+      else if (to_compare.compareTo(process.maxMemory as nextflow.util.MemoryUnit) == 1) {
+        return max_memory as nextflow.util.MemoryUnit
+      }
+      else {
+        return to_compare
+      }  
+    } catch (all) {
+          println "Error processing memory resources. Please check that process.maxMemory '${process.maxMemory}' and process.maxRetries '${process.maxRetries}' are valid!"
+          System.exit(1)
+    }
+  }
--- a/target/nextflow/runner/.config.vsh.yaml
+++ b/target/nextflow/runner/.config.vsh.yaml
@@ -1,11 +1,13 @@
 name: "runner"
-version: "v0.3.0"
+version: "v0.3.8"
 argument_groups:
 - name: "Input arguments"
  arguments:
  - type: "file"
    name: "--input"
-    description: "Base directory of the form `s3:/<bucket>/Sequencing/<Sequencer>/<RunID>/`"
+    description: "Base directory of the canonical form `s3://<bucket>/<path>/<RunID>/`.\n\
+      A tarball (tar.gz, .tgz, .tar) containing run information can be provided in\
+      \ which\ncase the RunID is set to the name of the tarball without the extension.\n"
    info: null
    must_exist: true
    create_parent: true
@@ -13,17 +15,38 @@ argument_groups:
    direction: "input"
    multiple: false
    multiple_sep: ";"
+  - type: "file"
+    name: "--run_information"
+    description: "CSV file containing sample information, which will be used as \n\
+      input for the demultiplexer. Canonically called 'SampleSheet.csv' (Illumina)\n\
+      or 'RunManifest.csv' (Element Biosciences). If not specified,\nwill try to autodetect\
+      \ the sample sheet in the input directory.\nRequires --demultiplexer to be set.\n"
+    info: null
+    must_exist: true
+    create_parent: true
+    required: false
+    direction: "input"
+    multiple: false
+    multiple_sep: ";"
+  - type: "string"
+    name: "--demultiplexer"
+    description: "Demultiplexer to use, choice depends on the provider\nof the instrument\
+      \ that was used to generate the data.\nWhen not using --sample_sheet, specifying\
+      \ this argument is not\nrequired.\n"
+    info: null
+    required: false
+    choices:
+    - "bases2fastq"
+    - "bclconvert"
+    direction: "input"
+    multiple: false
+    multiple_sep: ";"
 - name: "Annotation flags"
  arguments:
  - type: "boolean_true"
-    name: "--add_date_time"
-    description: "Add date and time to the output directory name. This is useful\n\
-      when running the same pipeline multiple times on the same input\ndirectory.\n"
-    info: null
-    direction: "input"
-  - type: "boolean_true"
-    name: "--add_workflow_id"
-    description: "Add a workflow identifier to the output directory name.\n"
+    name: "--plain_output"
+    description: "Flag to indicate that the output should be stored directly under\
+      \ $publish_dir rather than\nunder a subdirectory structure runID/<date_time>_demultiplex_<version>/.\n"
    info: null
    direction: "input"
 - name: "Output arguments"
@@ -61,11 +84,22 @@ argument_groups:
    direction: "output"
    multiple: false
    multiple_sep: ";"
+- name: "Other arguments"
+  arguments:
+  - type: "boolean_true"
+    name: "--skip_copycomplete_check"
+    description: "Disable the check for the presence of a \"CopyComplete.txt\" file\
+      \ in input\ndirectory in case of Illumina data.\n"
+    info: null
+    direction: "input"
 resources:
 - type: "nextflow_script"
  path: "main.nf"
  is_executable: true
  entrypoint: "run_wf"
+- type: "file"
+  path: "nextflow_labels.config"
+  dest: "nextflow_labels.config"
 description: "Runner for demultiplexing of raw sequencing data"
 info: null
 status: "enabled"
@@ -142,6 +176,8 @@ runners:
      cpu200: "cpus = 200"
      cpu500: "cpus = 500"
      cpu1000: "cpus = 1000"
+    script:
+    - "includeConfig(\"nextflow_labels.config\")"
  debug: false
  container: "docker"
 engines:
@@ -156,15 +192,15 @@ build_info:
  output: "target/nextflow/runner"
  executable: "target/nextflow/runner/main.nf"
  viash_version: "0.9.0"
-  git_commit: "c9cf5854796b31767c70848bcaf9aa42648d0bd7"
-  git_remote: "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex"
-  git_tag: "v0.1.1-9-gc9cf585"
+  git_commit: "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742"
+  git_remote: "https://github.com/viash-hub/demultiplex"
+  git_tag: "v0.3.5-9-gdafe2d8"
  dependencies:
  - "target/nextflow/demultiplex"
  - "target/nextflow/io/publish"
 package_config:
  name: "demultiplex"
-  version: "v0.3.0"
+  version: "v0.3.8"
  description: "Demultiplexing pipeline\n"
  info:
    test_resources:
@@ -175,10 +211,12 @@ package_config:
  target: "target"
  config_mods:
  - ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag\
-    \ := '$id'\n"
+    \ := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n\
+    .runners[.type == 'nextflow'].config.script := 'includeConfig(\"nextflow_labels.config\"\
+    )'\n"
  - ".engines += { type: \"native\" }"
  - ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
-  - ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+  - ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
  keywords:
  - "bioinformatics"
  - "sequence"
--- a/target/nextflow/runner/main.nf
+++ b/target/nextflow/runner/main.nf
@@ -1,4 +1,4 @@
-// runner v0.3.0
+// runner v0.3.8
 // 
 // This wrapper script is auto-generated by viash 0.9.0 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
@@ -2805,7 +2805,7 @@ meta = [
  "resources_dir": moduleDir.toRealPath().normalize(),
  "config": processConfig(readJsonBlob('''{
  "name" : "runner",
-  "version" : "v0.3.0",
+  "version" : "v0.3.8",
  "argument_groups" : [
    {
      "name" : "Input arguments",
@@ -2813,13 +2813,37 @@ meta = [
        {
          "type" : "file",
          "name" : "--input",
-          "description" : "Base directory of the form `s3:/<bucket>/Sequencing/<Sequencer>/<RunID>/`",
+          "description" : "Base directory of the canonical form `s3://<bucket>/<path>/<RunID>/`.\nA tarball (tar.gz, .tgz, .tar) containing run information can be provided in which\ncase the RunID is set to the name of the tarball without the extension.\n",
          "must_exist" : true,
          "create_parent" : true,
          "required" : true,
          "direction" : "input",
          "multiple" : false,
          "multiple_sep" : ";"
+        },
+        {
+          "type" : "file",
+          "name" : "--run_information",
+          "description" : "CSV file containing sample information, which will be used as \ninput for the demultiplexer. Canonically called 'SampleSheet.csv' (Illumina)\nor 'RunManifest.csv' (Element Biosciences). If not specified,\nwill try to autodetect the sample sheet in the input directory.\nRequires --demultiplexer to be set.\n",
+          "must_exist" : true,
+          "create_parent" : true,
+          "required" : false,
+          "direction" : "input",
+          "multiple" : false,
+          "multiple_sep" : ";"
+        },
+        {
+          "type" : "string",
+          "name" : "--demultiplexer",
+          "description" : "Demultiplexer to use, choice depends on the provider\nof the instrument that was used to generate the data.\nWhen not using --sample_sheet, specifying this argument is not\nrequired.\n",
+          "required" : false,
+          "choices" : [
+            "bases2fastq",
+            "bclconvert"
+          ],
+          "direction" : "input",
+          "multiple" : false,
+          "multiple_sep" : ";"
        }
      ]
    },
@@ -2828,14 +2852,8 @@ meta = [
      "arguments" : [
        {
          "type" : "boolean_true",
-          "name" : "--add_date_time",
-          "description" : "Add date and time to the output directory name. This is useful\nwhen running the same pipeline multiple times on the same input\ndirectory.\n",
-          "direction" : "input"
-        },
-        {
-          "type" : "boolean_true",
-          "name" : "--add_workflow_id",
-          "description" : "Add a workflow identifier to the output directory name.\n",
+          "name" : "--plain_output",
+          "description" : "Flag to indicate that the output should be stored directly under $publish_dir rather than\nunder a subdirectory structure runID/<date_time>_demultiplex_<version>/.\n",
          "direction" : "input"
        }
      ]
@@ -2883,6 +2901,17 @@ meta = [
          "multiple_sep" : ";"
        }
      ]
+    },
+    {
+      "name" : "Other arguments",
+      "arguments" : [
+        {
+          "type" : "boolean_true",
+          "name" : "--skip_copycomplete_check",
+          "description" : "Disable the check for the presence of a \\"CopyComplete.txt\\" file in input\ndirectory in case of Illumina data.\n",
+          "direction" : "input"
+        }
+      ]
    }
  ],
  "resources" : [
@@ -2891,6 +2920,11 @@ meta = [
      "path" : "main.nf",
      "is_executable" : true,
      "entrypoint" : "run_wf"
+    },
+    {
+      "type" : "file",
+      "path" : "/src/config/labels.config",
+      "dest" : "nextflow_labels.config"
    }
  ],
  "description" : "Runner for demultiplexing of raw sequencing data",
@@ -2981,7 +3015,10 @@ meta = [
          "cpu200" : "cpus = 200",
          "cpu500" : "cpus = 500",
          "cpu1000" : "cpus = 1000"
-        }
+        },
+        "script" : [
+          "includeConfig(\\"nextflow_labels.config\\")"
+        ]
      },
      "debug" : false,
      "container" : "docker"
@@ -3003,13 +3040,13 @@ meta = [
    "engine" : "native|native",
    "output" : "target/nextflow/runner",
    "viash_version" : "0.9.0",
-    "git_commit" : "c9cf5854796b31767c70848bcaf9aa42648d0bd7",
-    "git_remote" : "https://x-access-token:ghs_PoDtMpilpQnn1Nk8WgE3ZAlxTrfk790lUeZy@github.com/viash-hub/demultiplex",
-    "git_tag" : "v0.1.1-9-gc9cf585"
+    "git_commit" : "dafe2d829079bd5a9f5eb4e7cf63e92edbf1a742",
+    "git_remote" : "https://github.com/viash-hub/demultiplex",
+    "git_tag" : "v0.3.5-9-gdafe2d8"
  },
  "package_config" : {
    "name" : "demultiplex",
-    "version" : "v0.3.0",
+    "version" : "v0.3.8",
    "description" : "Demultiplexing pipeline\n",
    "info" : {
      "test_resources" : [
@@ -3023,10 +3060,10 @@ meta = [
    "source" : "src",
    "target" : "target",
    "config_mods" : [
-      ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag := '$id'\n",
+      ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].directives.tag := '$id'\n.resources += {path: '/src/config/labels.config', dest: 'nextflow_labels.config'}\n.runners[.type == 'nextflow'].config.script := 'includeConfig(\\"nextflow_labels.config\\")'\n",
      ".engines += { type: \\"native\\" }",
      ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'",
-      ".engines[.type == 'docker'].target_tag := 'v0.3.0'"
+      ".engines[.type == 'docker'].target_tag := 'v0.3.8'"
    ],
    "keywords" : [
      "bioinformatics",
@@ -3062,27 +3099,47 @@ workflow run_wf {

  main:
    output_ch = input_ch
+      // Extract the ID from the input.
+      // If the input is a tarball, strip the suffix.
+      | map{ id, state ->
+        def id_with_suffix = state.input.getFileName().toString()
+        [
+          id,
+          state + [ run_id: id_with_suffix - ~/\.(tar.gz|tgz|tar)$/ ]
+        ]
+      }
      | demultiplex.run(
-        fromState: [
-          "input": "input",
-          "output": "fastq",
-          "output_falco": "qc/fastqc",
-          "output_multiqc": "qc/multiqc_report.html",
-        ],
+        fromState: { id, state ->
+          def state_to_pass = [
+            "input": state.input,
+            "run_information": state.run_information,
+            "demultiplexer": state.demultiplexer,
+            "skip_copycomplete_check": state.skip_copycomplete_check,
+            "output": "$id/fastq",
+            "output_falco": "$id/qc/fastqc",
+            "output_multiqc": "$id/qc/multiqc_report.html",
+          ]
+          if (state.run_information) {
+            state_to_pass += ["output_run_information": state.run_information.getName()] 
+          }
+          state_to_pass
+        },
        toState: { id, result, state ->
          state + result
        },
      )
      | publish.run(
        fromState: { id, state ->
-          def id1 = (params.add_date_time) ? "${id}_${date}" : id
-          def id2 = (params.add_workflow_id) ? "${id1}_demultiplex_${version}" : id1
+          println(state.plain_output)
+          def id1 = (state.plain_output) ? id : "${state.run_id}/${date}"
+          def id2 = (state.plain_output) ? id : "${id1}_demultiplex_${version}"

-          def fastq_output_1 = (id == "run") ? state.fastq_output : "${id2}/" + state.fastq_output
-          def falco_output_1 = (id == "run") ? state.falco_output : "${id2}/" + state.falco_output
-          def multiqc_output_1 = (id == "run") ? state.multiqc_output : "${id2}/" + state.multiqc_output
+          def fastq_output_1 = (id2 == "run") ? state.fastq_output : "${id2}/" + state.fastq_output
+          def falco_output_1 = (id2 == "run") ? state.falco_output : "${id2}/" + state.falco_output
+          def multiqc_output_1 = (id2 == "run") ? state.multiqc_output : "${id2}/" + state.multiqc_output
+          def run_information_output_1 = (id2 == "run") ? "${state.output_run_information.getName()}" : "${id2}/${state.output_run_information.getName()}"

-          if (id == "run") {
+          if (id2 == "run") {
            println("Publising to ${params.publish_dir}")
          } else {
            println("Publising to ${params.publish_dir}/${id2}")
@@ -3092,9 +3149,11 @@ workflow run_wf {
            input: state.output,
            input_falco: state.output_falco,
            input_multiqc: state.output_multiqc,
+            input_run_information: state.output_run_information,
            output: fastq_output_1,
            output_falco: falco_output_1,
-            output_multiqc: multiqc_output_1
+            output_multiqc: multiqc_output_1,
+            output_run_information: run_information_output_1,
          ]
        },
        toState: { id, result, state -> [:] },
--- a/target/nextflow/runner/nextflow.config
+++ b/target/nextflow/runner/nextflow.config
@@ -2,7 +2,7 @@ manifest {
  name = 'runner'
  mainScript = 'main.nf'
  nextflowVersion = '!>=20.12.1-edge'
-  version = 'v0.3.0'
+  version = 'v0.3.8'
  description = 'Runner for demultiplexing of raw sequencing data'
 }

@@ -122,4 +122,4 @@ process{
  withLabel: cpu1000 { cpus = 1000 }
 }

-
+includeConfig("nextflow_labels.config")
--- a/target/nextflow/runner/nextflow_labels.config
+++ b/target/nextflow/runner/nextflow_labels.config
@@ -0,0 +1,98 @@
+process {
+  container = 'nextflow/bash:latest'
+  
+  // default resources
+  memory = { 8.Gb * task.attempt }
+  cpus = 8
+  maxForks = 36
+
+  // Retry for exit codes that have something to do with memory issues
+  errorStrategy = { task.exitStatus in 137..140 ? 'retry' : 'terminate' }
+  maxRetries = 3
+  maxMemory = 192.GB
+
+  // Resource labels
+  withLabel: verylowcpu { cpus = 2 }
+  withLabel: lowcpu { cpus = 8 }
+  withLabel: midcpu { cpus = 16 }
+  withLabel: highcpu { cpus = 32 }
+  
+  withLabel: verylowmem { memory = { get_memory( 4.GB * task.attempt ) } }
+  withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+  withLabel: midmem { memory = { get_memory( 16.GB * task.attempt ) } }
+  withLabel: highmem { memory = { get_memory( 64.GB * task.attempt ) } }
+
+}
+
+profiles {
+  // detect tempdir
+  tempDir = java.nio.file.Paths.get(
+    System.getenv('NXF_TEMP') ?:
+      System.getenv('VIASH_TEMP') ?: 
+      System.getenv('TEMPDIR') ?: 
+      System.getenv('TMPDIR') ?: 
+      '/tmp'
+  ).toAbsolutePath()
+
+  mount_temp {
+    docker.temp            = tempDir
+    podman.temp            = tempDir
+    charliecloud.temp      = tempDir
+  }
+
+  no_publish {
+    process {
+      withName: '.*' {
+        publishDir = [
+          enabled: false
+        ]
+      }
+    }
+  }
+
+  docker {
+    docker.fixOwnership    = true
+    docker.enabled         = true
+    // docker.userEmulation   = true
+    singularity.enabled    = false
+    podman.enabled         = false
+    shifter.enabled        = false
+    charliecloud.enabled   = false
+  }
+
+  local {
+    // This config is for local processing.
+    process {
+        maxMemory = 25.GB
+        withLabel: verylowcpu { cpus = 2 }
+        withLabel: lowcpu { cpus = 4 }
+        withLabel: midcpu { cpus = 6 }
+        withLabel: highcpu { cpus = 12 }
+  
+        withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+        withLabel: midmem { memory = { get_memory( 12.GB * task.attempt ) } }
+        withLabel: highmem { memory = { get_memory( 20.GB * task.attempt ) } }
+    }
+  }
+}
+
+def get_memory(to_compare) {
+    if (!process.containsKey("maxMemory") || !process.maxMemory) {
+      return to_compare
+    }
+
+    try {
+      if (process.containsKey("maxRetries") && process.maxRetries && task.attempt == (process.maxRetries as int)) {
+        return process.maxMemory
+      }
+      else if (to_compare.compareTo(process.maxMemory as nextflow.util.MemoryUnit) == 1) {
+        return max_memory as nextflow.util.MemoryUnit
+      }
+      else {
+        return to_compare
+      }  
+    } catch (all) {
+          println "Error processing memory resources. Please check that process.maxMemory '${process.maxMemory}' and process.maxRetries '${process.maxRetries}' are valid!"
+          System.exit(1)
+    }
+  }
--- a/target/nextflow/runner/nextflow_schema.json
+++ b/target/nextflow/runner/nextflow_schema.json
@@ -17,8 +17,30 @@
                "input": {
                "type":
                "string",
-                "description": "Type: `file`, required. Base directory of the form `s3:/\u003cbucket\u003e/Sequencing/\u003cSequencer\u003e/\u003cRunID\u003e/`",
-                "help_text": "Type: `file`, required. Base directory of the form `s3:/\u003cbucket\u003e/Sequencing/\u003cSequencer\u003e/\u003cRunID\u003e/`"
+                "description": "Type: `file`, required. Base directory of the canonical form `s3://\u003cbucket\u003e/\u003cpath\u003e/\u003cRunID\u003e/`",
+                "help_text": "Type: `file`, required. Base directory of the canonical form `s3://\u003cbucket\u003e/\u003cpath\u003e/\u003cRunID\u003e/`.\nA tarball (tar.gz, .tgz, .tar) containing run information can be provided in which\ncase the RunID is set to the name of the tarball without the extension.\n"
+            
+            }
+    
+
+        ,
+                "run_information": {
+                "type":
+                "string",
+                "description": "Type: `file`. CSV file containing sample information, which will be used as \ninput for the demultiplexer",
+                "help_text": "Type: `file`. CSV file containing sample information, which will be used as \ninput for the demultiplexer. Canonically called \u0027SampleSheet.csv\u0027 (Illumina)\nor \u0027RunManifest.csv\u0027 (Element Biosciences). If not specified,\nwill try to autodetect the sample sheet in the input directory.\nRequires --demultiplexer to be set.\n"
+            
+            }
+    
+
+        ,
+                "demultiplexer": {
+                "type":
+                "string",
+                "description": "Type: `string`, choices: ``bases2fastq`, `bclconvert``. Demultiplexer to use, choice depends on the provider\nof the instrument that was used to generate the data",
+                "help_text": "Type: `string`, choices: ``bases2fastq`, `bclconvert``. Demultiplexer to use, choice depends on the provider\nof the instrument that was used to generate the data.\nWhen not using --sample_sheet, specifying this argument is not\nrequired.\n",
+                "enum": ["bases2fastq", "bclconvert"]
+            
            
            }
    
@@ -34,22 +56,11 @@
    "properties": {
    
        
-                "add_date_time": {
+                "plain_output": {
                "type":
                "boolean",
-                "description": "Type: `boolean_true`, default: `false`. Add date and time to the output directory name",
-                "help_text": "Type: `boolean_true`, default: `false`. Add date and time to the output directory name. This is useful\nwhen running the same pipeline multiple times on the same input\ndirectory.\n"
-            ,
-                "default":false
-            }
-    
-
-        ,
-                "add_workflow_id": {
-                "type":
-                "boolean",
-                "description": "Type: `boolean_true`, default: `false`. Add a workflow identifier to the output directory name",
-                "help_text": "Type: `boolean_true`, default: `false`. Add a workflow identifier to the output directory name.\n"
+                "description": "Type: `boolean_true`, default: `false`. Flag to indicate that the output should be stored directly under $publish_dir rather than\nunder a subdirectory structure runID/\u003cdate_time\u003e_demultiplex_\u003cversion\u003e/",
+                "help_text": "Type: `boolean_true`, default: `false`. Flag to indicate that the output should be stored directly under $publish_dir rather than\nunder a subdirectory structure runID/\u003cdate_time\u003e_demultiplex_\u003cversion\u003e/.\n"
            ,
                "default":false
            }
@@ -98,6 +109,27 @@
            }
    

+}
+},
+    
+    
+    "other arguments" : {
+    "title": "Other arguments",
+    "type": "object",
+    "description": "No description",
+    "properties": {
+    
+        
+                "skip_copycomplete_check": {
+                "type":
+                "boolean",
+                "description": "Type: `boolean_true`, default: `false`. Disable the check for the presence of a \"CopyComplete",
+                "help_text": "Type: `boolean_true`, default: `false`. Disable the check for the presence of a \"CopyComplete.txt\" file in input\ndirectory in case of Illumina data.\n"
+            ,
+                "default":false
+            }
+    
+
 }
 },
    
@@ -146,6 +178,10 @@
    "$ref": "#/definitions/output arguments"
    },

+    {
+    "$ref": "#/definitions/other arguments"
+    },
+
    {
    "$ref": "#/definitions/nextflow input-output arguments"
    }
Author	SHA1	Message	Date
CI	cd1815354d	Build branch v0.3 with version v0.3.8 (dafe2d8) Build pipeline: viash-hub.demultiplex.v0.3-brf8v Source commit: `dafe2d8290` Source message: Bump version to v0.3.8	2025-03-27 16:16:22 +00:00
CI	e9dce89f32	Build branch v0.3 with version v0.3.7 (e177226) Build pipeline: viash-hub.demultiplex.v0.3-dc7wn Source commit: `e177226c52` Source message: Bump version to v0.3.7	2025-03-20 20:56:15 +00:00
CI	0cbd5c00cb	Build branch v0.3 with version v0.3.6 (3a985c8) Build pipeline: viash-hub.demultiplex.v0.3-2vbxf Source commit: `3a985c8386` Source message: Bump version to v0.3.6	2025-03-20 20:00:43 +00:00
CI	78f3a8b65e	Build branch v0.3 with version v0.3.5 (18e092d) Build pipeline: viash-hub.demultiplex.v0.3-ldxlr Source commit: `18e092d169` Source message: Bump version to 0.3.5	2025-03-04 13:36:18 +00:00
CI	4d7688483b	Build branch v0.3 with version v0.3.4 (e6f8b20) Build pipeline: viash-hub.demultiplex.v0.3-ck9ss Source commit: `e6f8b20d70` Source message: Bump version to v0.3.4	2024-12-20 12:50:35 +00:00
CI	e1af283efe	Build branch v0.3 with version v0.3.3 (e297e8f) Build pipeline: viash-hub.demultiplex.v0.3-dd5mz Source commit: `e297e8ff45` Source message: Bump version to v0.3.3	2024-12-19 15:56:52 +00:00
CI	e5ae6d3bb5	Build branch v0.3 with version v0.3.2 (7ad45f3) Build pipeline: viash-hub.demultiplex.v0.3-92c55 Source commit: `7ad45f3597` Source message: Bump version to 0.3.2	2024-12-11 18:07:49 +00:00
CI	09a049c60c	Build branch v0.3 with version v0.3.1 (35e1ab8) Build pipeline: viash-hub.demultiplex.v0.3-jknqb Source commit: `35e1ab8495` Source message: Bump version to 0.3.1	2024-12-11 15:46:01 +00:00