Build branch update-resources with version update-resources (f1f0c98)

Build pipeline: viash-hub.htrnaseq.update-resources-2j4gn

Source commit: f1f0c98bba

Source message: Merge branch 'main' into update-resources

CHANGELOG.md

@@ -1,9 +1,82 @@
-# htrnaseq v0.x.y
+# htrnaseq v0.8.0
 
 ## Under the hood
 
-- Bumped the version of Viash to v0.9.2
-- Moved the test resources to their new location
+* Moved the test resources to their new location (PR 47)
+
+# htrnaseq v0.7.2
+
+## Documentation
+
+* Update README (PR #54)
+
+# htrnaseq v0.7.1
+
+## Bug fixes
+
+* Bump viash version to `0.9.4`. This adds support for nextflow versions starting major version 25.01 and 
+fixes an issue where an integer being passed to a argument with `type: double` resulted in an error (PR #51).
+
+* `reporting`: updated default colour mapping (PR #50).
+
+## Minor changes
+
+* `create_report`: bump bioconductor version to 3.21 in order to accommodate R version 4.5 (PR #52).
+
+# htrnaseq v0.7.0
+
+## Breaking changes
+
+The `runner` and `htrnaseq` workflow now output FASTQ files corresponding to the barcodes per input ID (per sequencing run). 
+Previously, when multiple input folders or multiple input FASTQ files were provided
+(for the `runner` and `htrnaseq` workflows respectively), the demultiplexed FASTQ files for these inputs were concatenated
+and provided as output. For the `htrnaseq` workflow, reads can still be combined by using a newly added `sampleID` argument.
+This means that two lists of FASTQ files can be provided for a single sample, and by assigning the same `sampleID`,
+these reads will be joined. For example, with other arguments are left out for brevity:
+
+```yaml
+- id: sample1_run1
+  input_r1: [sample_1_L001_1_R1.fastq, sample_1_L002_1_R1.fastq]
+  input_r2: [sample_1_L001_1_R2.fastq, sample_1_L002_1_R2.fastq]
+  sampleID: "sample_1"
+- id: sample1_run2
+  input_r1: [sample_1_L001_1_R1.fastq, sample_1_L002_1_R1.fastq]
+  input_r2: [sample_1_L001_1_R2.fastq, sample_1_L002_1_R2.fastq]
+  sampleID: "sample_1"
+- id: sample_2
+  input_r1: [sample_2_L001_1_R1.fastq, sample_2_L002_1_R1.fastq]
+  input_r2: [sample_2_L001_1_R2.fastq, sample_2_L002_1_R2.fastq]
+```
+
+For the runner, concatenation of data across samples is automatically inferred. Previously, multiple IDs (events) could be
+provided which were processed in parallel. This is no longer possible, as providing multiple will cause the matching
+samples for these runs to be concatenated.
+
+
+For example, the following old parameter yaml
+```yaml
+- id: run1
+  input: ["run_folder_1/", run_folder_2/]
+```
+should now be provided as: 
+```yaml
+- id: run1
+  input: "run_folder_1/"
+- id: run2
+  input: run_folder_2/
+```
+
+## Minor changes
+
+* Updated viash to `0.9.2` (PR #49)
+
+# htrnaseq v0.6.0
+
+## Breaking changes
+
+* `runner`: a subdirectory `data_processed` is now added to the output structure, in between
+  the experiment ID and the directory with the workflow date and version (PR #45).
+
 # htrnaseq v0.5.5
 
 ## New functionality

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2021 OpenPipelines
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,129 +1,207 @@
-# HT-RNAseq - A pipeline for processing high-throughput RNA-seq data
+
+
+# HT-RNAseq
+
+[![ViashHub](https://img.shields.io/badge/ViashHub-htrnaseq-7a4baa.svg)](https://www.viash-hub.com/packages/htrnaseq)
+[![GitHub](https://img.shields.io/badge/GitHub-viash--hub%2Fhtrnaseq-blue.svg)](https://github.com/viash-hub/htrnaseq)
+[![GitHub
+License](https://img.shields.io/github/license/viash-hub/htrnaseq.svg)](https://github.com/viash-hub/htrnaseq/blob/main/LICENSE)
+[![GitHub
+Issues](https://img.shields.io/github/issues/viash-hub/htrnaseq.svg)](https://github.com/viash-hub/htrnaseq/issues)
+[![Viash
+version](https://img.shields.io/badge/Viash-v0.9.2-blue.svg)](https://viash.io)
 
 ## Introduction
-__TODO__: Add a description of the pipeline here.
 
-## Test data
+This workflow is designed to process high-throughput RNA-seq data, where
+every well of a microarray plate is a sample. A fasta file provided as
+input defines the mapping between sample barcodes and wells.
 
-As test data, we use [a DRUGseq dataset](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE176150) from the [NCBI Sequence Read Archive](https://www.ncbi.nlm.nih.gov/sra).
+The workflow is built in a modular fashion, where most of the base
+functionality is provided by components from
+[`biobox`](https://www.viash-hub.com/packages/biobox/latest)
+supplemented by custom base components and workflow components in this
+package.
 
-The original data has been (partly) subsampled to reduce the test runtime. We used [seqtk](https://github.com/lh3/seqtk) for this with a seed of 1, e.g.:
+The full workflow is split in two major subworkflows that can be run
+independently:
 
-```bash
+- **Well-demultiplexing:** Split the input (plate/pool level) fastq
+  files per well.
+- **Mapping, counting and QC:** Run per-well mapping, counting and
+  generate QC reports.
+
+Each of those can be started individually, or the full workflow can be
+run in two ways:
+
+1.  Run the [main
+    workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)
+    containing the main functionality.
+2.  Run the [(opinionated)
+    `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)
+    where a number of choices (input/output structure and location) have
+    been made.
+
+Input for the workflow has to be `fastq` files (zipped or not). For bcl
+or other formats, please consider running
+[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.
+
+``` mermaid lang="mermaid"
+flowchart TB
+  subgraph runner [runner]
+    direction TB
+    subgraph htrnaseq [HT-RNAseq]
+      direction LR
+      demultiplex[Well demultiplexing]
+      map
+      report
+      eset
+    end
+  end
+
+  demultiplex --> map --> report --> eset
+
+  class runner container
+  class htrnaseq container
+  class demultiplex container-inner
+  class map container-inner
+  class report container-inner
+  class eset container-inner
+
+  class demultiplex node
+  class map node
+  class report node
+  class eset node
+```
+
+## Example usage
+
+### Test and example data
+
+If you want to explore this workflow, it’s possible to the use data we
+use as test data: [a DRUGseq
+dataset](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE176150)
+from the [NCBI Sequence Read Archive](https://www.ncbi.nlm.nih.gov/sra).
+For the unit and integration tests, this data has been (partly)
+subsampled to reduce the test runtime. We used
+[seqtk](https://github.com/lh3/seqtk) for this with a seed of 1, e.g.:
+
+``` bash
 seqtk sample -s1 orig/SRR14730302/VH02001614_S8_R1_001.fastq.gz 10000 > 10k/SRR14730302/VH02001614_S8_R1_001.fastq.gz
 ```
 
-The data is available at: `gs://viash-hub-test-data/htrnaseq/v1/`:
+This data is available at: `gs://viash-hub-resources/htrnaseq/v1/`.
 
-```
-❯ gcstree -f viash-hub-test-data/htrnaseq/v1/
-viash-hub-test-data
-└── htrnaseq
-    └── v1
-        ├── [  48]  2-wells.fasta
-        ├── [465.3K]  GSE176150_metadata.csv
-        ├── 100k
-        │   ├── SRR14730301
-        │   │   ├── [8.5M]  VH02001612_S9_R1_001.fastq
-        │   │   └── [14.9M]  VH02001612_S9_R2_001.fastq
-        │   └── SRR14730302
-        │       ├── [8.5M]  VH02001614_S8_R1_001.fastq.gz
-        │       └── [14.9M]  VH02001614_S8_R2_001.fastq.gz
-        ├── 10k
-        │   ├── SRR14730301
-        │   │   ├── [845.4K]  VH02001612_S9_R1_001.fastq
-        │   │   └── [1.5M]  VH02001612_S9_R2_001.fastq
-        │   └── SRR14730302
-        │       ├── [845.3K]  VH02001614_S8_R1_001.fastq.gz
-        │       └── [1.5M]  VH02001614_S8_R2_001.fastq.gz
-        └── orig
-            ├── [20.4G]  SRR14730301
-            │   └── [20.4G]  SRR14730301
-            ├── SRR14730301
-            │   ├── [9.1G]  VH02001612_S9_R1_001.fastq.gz
-            │   └── [22.0G]  VH02001612_S9_R2_001.fastq.gz
-            ├── [16.9G]  SRR14730302
-            │   └── [16.9G]  SRR14730302
-            ├── SRR14730302
-            │   ├── [7.6G]  VH02001614_S8_R1_001.fastq.gz
-            │   └── [18.0G]  VH02001614_S8_R2_001.fastq.gz
-            ├── [18.0G]  SRR14730303
-            │   └── [18.0G]  SRR14730303
-            ├── SRR14730303
-            │   ├── [8.1G]  VH02001618_S7_R1_001.fastq.gz
-            │   └── [19.2G]  VH02001618_S7_R2_001.fastq.gz
-            ├── [16.5G]  SRR14730304
-            │   └── [16.5G]  SRR14730304
-            ├── SRR14730304
-            │   ├── [7.5G]  VH02001700_S6_R1_001.fastq.gz
-            │   └── [17.8G]  VH02001700_S6_R2_001.fastq.gz
-            ├── [19.0G]  SRR14730305
-            │   └── [19.0G]  SRR14730305
-            ├── SRR14730305
-            │   ├── [8.4G]  VH02001702_S5_R1_001.fastq.gz
-            │   └── [20.6G]  VH02001702_S5_R2_001.fastq.gz
-            ├── [14.6G]  SRR14730306
-            │   └── [14.6G]  SRR14730306
-            ├── SRR14730306
-            │   ├── [6.6G]  VH02001704_S4_R1_001.fastq.gz
-            │   └── [16.0G]  VH02001704_S4_R2_001.fastq.gz
-            ├── [21.5G]  SRR14730307
-            │   └── [21.5G]  SRR14730307
-            ├── SRR14730307
-            │   ├── [9.6G]  VH02001708_S3_R1_001.fastq.gz
-            │   └── [23.2G]  VH02001708_S3_R2_001.fastq.gz
-            ├── [20.7G]  SRR14730308
-            │   └── [20.7G]  SRR14730308
-            ├── SRR14730308
-            │   ├── [9.3G]  VH02001710_S2_R1_001.fastq.gz
-            │   └── [22.1G]  VH02001710_S2_R2_001.fastq.gz
-            ├── [15.8G]  SRR14730309
-            │   └── [15.8G]  SRR14730309
-            └── SRR14730309
-                ├── [7.2G]  VH02001712_S1_R1_001.fastq.gz
-                └── [16.9G]  VH02001712_S1_R2_001.fastq.gz
+### Run from Viash Hub
 
-18 directories, 37 files
+Open [Viash Hub](https://www.viash-hub.com) and browse to the [htrnaseq
+component](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq).
+Press the ‘Launch’ button and follow the instructions.
+
+![](assets/htrnaseq-launch-small.png)
+
+We will start an example run loading just one input and using a barcodes
+fasta file containing only 2 wells.
+
+In the first step, we add the `local` profile to the list of profiles in
+order to limit the cpu and memory requirements of the workflow steps:
+
+![](assets/launch-parameters-1-small.png)
+
+In the next step, we provide the paramters as follows:
+
+- `input_r1`:
+  `gs://viash-hub-test-data/htrnaseq/v1/100k/SRR14730301/VH02001612_S9_R1_001.fastq`
+- `input_r2`:
+  `gs://viash-hub-test-data/htrnaseq/v1/100k/SRR14730301/VH02001612_S9_R2_001.fastq`
+- `genomeDir`:
+  `gs://viash-hub-test-data/htrnaseq/v1/genomeDir/subset/Homo_sapiens/v0.0.3/`
+- `barcodesFasta`:
+  `gs://viash-hub-test-data/htrnaseq/v1/2-wells-with-ids.fasta`
+- `annotation`:
+  `gs://viash-hub-test-data/htrnaseq/v1/genomeDir/gencode.v41.annotation.gtf.gz`
+
+Please note that both `input_r1` and `input_r2` can take multiple
+values. This means that one has to press ENTER after pasting the input
+path.
+
+![](assets/launch-parameters-2-small.png)
+
+Press the ‘Launch’ button at the end to get the instructions on how to
+run the workflow from the CLI.
+
+### Run using NF-Tower / Seqera Cloud
+
+It’s possible to run the workflow directly from [Seqera
+Cloud](https://cloud.seqera.io). The necessary [Nextflow schema
+file](https://nextflow-io.github.io/nf-schema/latest/nextflow_schema/nextflow_schema_specification/)
+has been built and provided with the workflows in order to use the
+form-based input. However, Seqera Cloud can not deal with multiple-value
+parameters when using the form-based input. Therefore, it’s better to
+use Viash Hub also here:
+
+First, select the option to run the workflow using Seqera Cloud. You
+will need to create an API token for your account. Once this token is
+filled in in the corresponding field, you will get the option to select
+a ‘Workspace’ and a ‘Compute environment’.
+
+![](assets/launch-parameters-3-small.png)
+
+Next, we need to fill in the parameters for the run. This is similar to
+before:
+
+![](assets/launch-parameters-4-small.png)
+
+In the next screen, pressing the ‘Launch’ button will actually start the
+workflow on Seqera Cloud. A message is shown when the submit was
+successful.
+
+![](assets/launch-parameters-5-small.png)
+
+### Run from the CLI
+
+Running from the CLI directly without using Viash hub is possible. The
+easiest is to just use the integrated help functionality, for instance
+using the following:
+
+``` bash
+ nextflow run https://packages.viash-hub.com/vsh/htrnaseq.git \
+  -revision v0.3.0 \
+  -main-script target/nextflow/workflows/runner/main.nf \
+  --help
 ```
 
+### (Optional) Resource usage tuning
 
-The `orig` directory contains the original fastq files. The fastq files are available for 10k and 100k subsamples in the `10k` and `100k` directories, respectively.
+Nextflow’s labels can be used to specify the amount of resources a
+process can use. This workflow uses the following labels for CPU and
+memory:
 
-The `2-wells.fasta` file contains the barcodes for 2 wells.
+- `verylowmem`, `lowmem`, `midmem`, `highmem`
+- `verylowcpu`, `lowcpu`, `midcpu`, `highcpu`
 
-## Test run
+The defaults for these labels can be found at
+`src/config/labels.config`. Nextflow checks that the specified resources
+for a process do not exceed what is available on the machine and will
+not start if it does. Create your own config file to tune the labels to
+your needs, for example:
 
-The pipeline can be run by creating a `params.yaml` file like this:
+    // Resource labels
+    withLabel: verylowcpu { cpus = 2 }
+    withLabel: lowcpu { cpus = 8 }
+    withLabel: midcpu { cpus = 16 }
+    withLabel: highcpu { cpus = 32 }
 
-```yaml
-param_list:
-  - input_r1: "gs://viash-hub-test-data/htrnaseq/v1/100k/SRR14730301/VH02001612_S9_R1_001.fastq"
-    input_r2: "gs://viash-hub-test-data/htrnaseq/v1/100k/SRR14730301/VH02001612_S9_R2_001.fastq"
-    genomeDir: "gs://viash-hub-test-data/htrnaseq/v1/genomeDir/gencode.v41.star.sparse"
-    barcodesFasta: "gs://viash-hub-test-data/htrnaseq/v1/2-wells.fasta"
-    id: sample_one
-  - input_r1: "gs://viash-hub-test-data/htrnaseq/v1/100k/SRR14730302/VH02001614_S8_R1_001.fastq"
-    input_r2: "gs://viash-hub-test-data/htrnaseq/v1/100k/SRR14730302/VH02001614_S8_R2_001.fastq"
-    genomeDir: "gs://viash-hub-test-data/htrnaseq/v1/genomeDir/gencode.v41.star.sparse"
-    barcodesFasta: "gs://viash-hub-test-data/htrnaseq/v1/2-wells.fasta"
-    id: sample_two
-```
+    withLabel: verylowmem { memory = { get_memory( 4.GB * task.attempt ) } }
+    withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+    withLabel: midmem { memory = { get_memory( 16.GB * task.attempt ) } }
+    withLabel: highmem { memory = { get_memory( 64.GB * task.attempt ) } }
 
-and then:
+When starting nextflow using the CLI, you can use `-c` to provide the
+file to nextflow and overwrite the defaults.
 
-```bash
-viash ns build --setup cb
-nextflow run . -main-script target/nextflow/workflows/htrnaseq/main.nf \
-  -profile docker \
-  -c target/nextflow/workflows/htrnaseq/nextflow.config \
-  -params-file params.yaml \
-  -resume \
-  --publish_dir output
-```
+## Contributions
 
-Or, by running `src/workflows/htrnaseq/integration_test.sh`.
+Developed in collaboration with Data Intuitive and Open Analytics.
 
-
-# Special Thanks
-
-Developed in collaboration with Data Intuitive and Open Analytics.
+Other contributions are welcome.

README.qmd

@@ -0,0 +1,149 @@
+---
+format: gfm
+---
+
+```{r setup, include=FALSE}
+project <- yaml::read_yaml("_viash.yaml")
+license <- paste0(project$links$repository, "/blob/main/LICENSE")
+contributing <- paste0(project$links$repository, "/blob/main/CONTRIBUTING.md")
+```
+
+# HT-RNAseq
+
+[![ViashHub](https://img.shields.io/badge/ViashHub-`r project$name`-7a4baa.svg)](https://www.viash-hub.com/packages/`r project$name`) 
+[![GitHub](https://img.shields.io/badge/GitHub-viash--hub%2F`r project$name`-blue.svg)](`r project$links$repository`) 
+[![GitHub License](https://img.shields.io/github/license/viash-hub/`r project$name`.svg)](`r license`) 
+[![GitHub Issues](https://img.shields.io/github/issues/viash-hub/`r project$name`.svg)](`r project$links$issue_tracker`) 
+[![Viash version](https://img.shields.io/badge/Viash-v`r gsub("-", "--", project$viash_version)`-blue.svg)](https://viash.io)
+
+## Introduction
+
+`r project$description`
+
+
+```{mermaid lang='mermaid'}
+flowchart TB
+  subgraph runner [runner]
+    direction TB
+    subgraph htrnaseq [HT-RNAseq]
+      direction LR
+      demultiplex[Well demultiplexing]
+      map
+      report
+      eset
+    end
+  end
+
+  demultiplex --> map --> report --> eset
+
+  class runner container
+  class htrnaseq container
+  class demultiplex container-inner
+  class map container-inner
+  class report container-inner
+  class eset container-inner
+
+  class demultiplex node
+  class map node
+  class report node
+  class eset node
+```
+
+
+## Example usage
+
+### Test and example data
+
+If you want to explore this workflow, it's possible to the use data we use as test data: [a DRUGseq dataset](https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE176150) from the [NCBI Sequence Read Archive](https://www.ncbi.nlm.nih.gov/sra). For the unit and integration tests, this data has been (partly) subsampled to reduce the test runtime. We used [seqtk](https://github.com/lh3/seqtk) for this with a seed of 1, e.g.:
+
+```bash
+seqtk sample -s1 orig/SRR14730302/VH02001614_S8_R1_001.fastq.gz 10000 > 10k/SRR14730302/VH02001614_S8_R1_001.fastq.gz
+```
+
+This data is available at: `gs://viash-hub-test-data/htrnaseq/v1/`.
+
+### Run from Viash Hub
+
+Open [Viash Hub](https://www.viash-hub.com) and browse to the [htrnaseq component](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq). Press the 'Launch' button and follow the instructions.
+
+
+![](assets/htrnaseq-launch-small.png)
+
+We will start an example run loading just one input and using a barcodes fasta file containing only 2 wells.
+
+In the first step, we add the `local` profile to the list of profiles in order to limit the cpu and memory requirements of the workflow steps:
+
+
+![](assets/launch-parameters-1-small.png)
+
+In the next step, we provide the paramters as follows:
+
+- `input_r1`: `gs://viash-hub-test-data/htrnaseq/v1/100k/SRR14730301/VH02001612_S9_R1_001.fastq`
+- `input_r2`: `gs://viash-hub-test-data/htrnaseq/v1/100k/SRR14730301/VH02001612_S9_R2_001.fastq`
+- `genomeDir`: `gs://viash-hub-test-data/htrnaseq/v1/genomeDir/subset/Homo_sapiens/v0.0.3/`
+- `barcodesFasta`: `gs://viash-hub-test-data/htrnaseq/v1/2-wells-with-ids.fasta`
+- `annotation`: `gs://viash-hub-test-data/htrnaseq/v1/genomeDir/gencode.v41.annotation.gtf.gz`
+
+Please note that both `input_r1` and `input_r2` can take multiple values. This means that one has to press ENTER after pasting the input path.
+
+![](assets/launch-parameters-2-small.png)
+
+Press the 'Launch' button at the end to get the instructions on how to run the workflow from the CLI.
+
+
+### Run using NF-Tower / Seqera Cloud
+
+It's possible to run the workflow directly from [Seqera Cloud](https://cloud.seqera.io). The necessary [Nextflow schema file](https://nextflow-io.github.io/nf-schema/latest/nextflow_schema/nextflow_schema_specification/) has been built and provided with the workflows in order to use the form-based input. However, Seqera Cloud can not deal with multiple-value parameters when using the form-based input. Therefore, it's better to use Viash Hub also here:
+
+First, select the option to run the workflow using Seqera Cloud. You will need to create an API token for your account. Once this token is filled in in the corresponding field, you will get the option to select a 'Workspace' and a 'Compute environment'. 
+
+![](assets/launch-parameters-3-small.png)
+
+Next, we need to fill in the parameters for the run. This is similar to before:
+
+![](assets/launch-parameters-4-small.png)
+
+In the next screen, pressing the 'Launch' button will actually start the workflow on Seqera Cloud. A message is shown when the submit was successful.
+
+![](assets/launch-parameters-5-small.png)
+
+### Run from the CLI
+
+Running from the CLI directly without using Viash hub is possible. The easiest is to just use the integrated help functionality, for instance using the following:
+
+```bash
+ nextflow run https://packages.viash-hub.com/vsh/htrnaseq.git \
+  -revision v0.3.0 \
+  -main-script target/nextflow/workflows/runner/main.nf \
+  --help
+```
+
+### (Optional) Resource usage tuning
+
+Nextflow's labels can be used to specify the amount of resources a process can use. This workflow uses the following labels for CPU and memory:
+
+* `verylowmem`, `lowmem`, `midmem`, `highmem`
+* `verylowcpu`, `lowcpu`, `midcpu`, `highcpu`
+
+The defaults for these labels can be found at `src/config/labels.config`. Nextflow checks that the specified resources for a process do not exceed what is available on the machine and will not start if it does. Create your own config file to tune the labels to your needs, for example:
+
+```
+// Resource labels
+withLabel: verylowcpu { cpus = 2 }
+withLabel: lowcpu { cpus = 8 }
+withLabel: midcpu { cpus = 16 }
+withLabel: highcpu { cpus = 32 }
+
+withLabel: verylowmem { memory = { get_memory( 4.GB * task.attempt ) } }
+withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+withLabel: midmem { memory = { get_memory( 16.GB * task.attempt ) } }
+withLabel: highmem { memory = { get_memory( 64.GB * task.attempt ) } }
+```
+
+When starting nextflow using the CLI, you can use `-c` to provide the file to nextflow and overwrite the defaults.
+
+## Contributions
+
+Developed in collaboration with Data Intuitive and Open Analytics.
+
+Other contributions are welcome.

_viash.yaml

@@ -1,13 +1,47 @@
 name: htrnaseq
+summary: |
+  A workflow for high-throughput RNA-seq data analyses.
 description: |
-  High-throughput pipeline [WIP]
+  This workflow is designed to process high-throughput RNA-seq data, where every
+  well of a microarray plate is a sample. A fasta file provided as input
+  defines the mapping between sample barcodes and wells.
+
+  The workflow is built in a modular fashion, where most of the base functionality
+  is provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)
+  supplemented by custom base components and workflow components in this package.
+
+  The full workflow is split in two major subworkflows that can be run independently:
+
+  * **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.
+  * **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.
+
+  Each of those can be started individually, or the full workflow can be run in two ways:
+
+  1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) 
+  containing the main functionality.
+  2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a
+  number of choices (input/output structure and location) have been made.
+
+  Input for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running
+  [demultiplex](https://www.viash-hub.com/packages/demultiplex) first.
+
 license: MIT
-keywords: [bioinformatics, sequence, high-throughput, mapping, counting, pipeline]
+keywords:
+  [
+    bioinformatics,
+    sequencing,
+    high-throughput,
+    RNAseq,
+    mapping,
+    counting,
+    pipeline,
+    workflow,
+  ]
 links:
   issue_tracker: https://github.com/viash-hub/htrnaseq/issues
   repository: https://github.com/viash-hub/htrnaseq
 
-viash_version: 0.9.2
+viash_version: 0.9.4
 
 info:
   test_resources:

src/io/publish_fastqs/code.sh

@@ -8,13 +8,8 @@ mkdir -p "$par_output" && echo "$par_output created"
 
 echo
 echo "Copying files..."
-IFS=";" read -ra input_r1 <<<$par_input_r1
-IFS=";" read -ra input_r2 <<<$par_input_r2
+IFS=";" read -ra input <<<$par_input
 
-for i in "${input_r1[@]}"; do
+for i in "${input[@]}"; do
   cp -rL "$i" "$par_output/"
-done
-
-for i in "${input_r2[@]}"; do
-  cp -rL "$i" "$par_output/"
-done
+done

src/io/publish_fastqs/config.vsh.yaml

@@ -4,13 +4,8 @@ description: "Publish the fastq files per well"
 argument_groups:
   - name: Input arguments
     arguments:
-      - name: --input_r1
-        description: Directory to write R1 fastq data to
-        type: file
-        multiple: true
-        required: true
-      - name: --input_r2
-        description: Directory to write R2 fastq data to
+      - name: --input
+        description: Directory to write fastq data to
         type: file
         multiple: true
         required: true

src/parallel_map/script.sh

@@ -114,7 +114,8 @@ for barcode_index in "${!barcodes[@]}"; do
     fi
   done
   echo "Did not find FASTQ files files for well ${well_id}! "\
-    "Make sure that the input files have the correct file name format."
+    "Make sure that the input files have the correct file name format."\
+    "Input files: ${input_r1[@]}"
   exit 1
 done
 

src/report/config.vsh.yaml

@@ -40,6 +40,10 @@ engines:
         packages:
           - procps
           - pandoc
+      - type: r
+        script:
+          - install.packages("BiocManager")
+          - BiocManager::install(version = "3.21", type = "source", checkBuilt = TRUE)
       - type: r
         bioc:
           - Biobase

src/report/plateLayouts.R

@@ -283,15 +283,31 @@ plateLayout <- function(
 
   if (is.null(colours)) {
     colours <- tryCatch({
-      colorRamp2(
+      circlize::colorRamp2(
         breaks = breaks,
         colors = brewer.pal(length(breaks), "Purples")
       )
     },
-    error = function(cond) {
-      return(c("#9370DB", "white"))
+    error = function(cond){
+      
+      message("Recomputed breaks for proper colour mapping")
+      
+      breakValues <- plateValues$values
+      breakValues[which(is.na(breakValues))] <- 0
+      if (all(breakValues >= 0)) {
+        breaks <- computeBreaks(7, max(plateValues$values, na.rm = TRUE))
+      } else {
+        breaks <- quantile(plateValues$values,  probs = seq(0, 1, 0.125))
+      }
+      
+      circlize::colorRamp2(
+        breaks = breaks,
+        colors = brewer.pal(length(breaks), "Purples")
+      )
+      
     })
   }
+  
   ht <- Heatmap(
     plateValues$values,
     column_title = mainTitle, column_title_side = "top",
@@ -425,6 +441,7 @@ computeBreaks <- function(nBreaks, variable) {
     )
     coefExp <- c(exp(coefSystem[1]), coefSystem[2])
     breaks <- coefExp[1] * exp((1:(nBreaks - 1)) * coefExp[2])
+    breaks <- unique(c(0, breaks))
   }
-  return(c(0, breaks))
-}
+  return(breaks)
+}

src/utils/concatRuns/config.vsh.yaml

@@ -0,0 +1,43 @@
+name: concatRuns
+namespace: utils
+description: |
+  Concatenate well FASTQ files from different runs in order to increase sequencing depth.
+arguments:
+  - name: "--input_r1"
+    type: file
+    required: true
+    multiple: true
+  - name: "--input_r2"
+    type: file
+    required: true
+    multiple: true
+  - name: "--sample_id"
+    type: string
+    required: true
+  - name: "--output_r1"
+    type: file
+    multiple: true
+    description: Path to read 1 fastq/fasta file
+    direction: output
+  - name: "--output_r2"
+    type: file
+    multiple: true
+    description: Path to read 2 fastq/fasta file
+    direction: output
+resources:
+  - type: nextflow_script
+    path: main.nf
+    entrypoint: run_wf
+dependencies:
+    - name: concat_text
+      repository: cb
+repositories:
+  - name: cb
+    type: vsh
+    repo: craftbox
+    tag: v0.1.0
+runners:
+  - type: nextflow
+
+engines:
+  - type: native

src/utils/concatRuns/main.nf

@@ -0,0 +1,128 @@
+workflow run_wf {
+
+  take:
+    input_ch
+
+  main:
+    // Count the number of input events per sample
+    // Results from events with the same sample ID need to be concatenated.
+    event_counts_ch = input_ch
+      | map {id, state ->
+        def new_state = state + ["event_id": id]
+        def new_event = [state.sample_id, new_state]
+        return new_event
+      }
+      | groupTuple(by: 0)
+      | flatMap { id, states ->
+          def orig_event_ids = states.collect{it.event_id}
+          def new_events = orig_event_ids.collect{ orig_event_id ->
+            [orig_event_id, ["n_events": states.size()]]
+           }
+          return new_events
+      }
+
+
+    // The number of events per sample needs is passed number to `groupTuple()`
+    // so that it can emit the sample as soon as it is ready. This makes sure
+    // that the samples are processed asynchronously. 
+    output_ch = input_ch.join(event_counts_ch)
+      | flatMap {id, state_demultiplex, state_event_counts ->
+          assert state_demultiplex.input_r1.size() == state_demultiplex.input_r2.size(),
+            "Expected output from well demultiplexing to contain equal amount or forward and reverse FASTQ files."
+          def new_states = [state_demultiplex.input_r1, state_demultiplex.input_r2].transpose().collect{ fastq_files ->
+            def (r1_file, r2_file) = fastq_files
+            def regex = ~/^(\w+)_R[12]{1}_001\.fastq(\.gz)?$/
+            def parsed_file_name = r1_file.name =~ regex
+            def parsed_file_name_r2 = r2_file.name =~ regex
+            def well_id = parsed_file_name[0][1]
+            def well_id_r2 = parsed_file_name_r2[0][1]
+  
+            assert (well_id.length() != 0) && (well_id == well_id_r2)
+            def new_state = state_demultiplex + [
+              "input_r1": r1_file,
+              "input_r2": r2_file,
+              "event_id": id,
+            ]
+            def group_settings = groupKey("${state_demultiplex.sample_id}_${well_id}", state_event_counts.n_events)
+            return [group_settings, new_state]
+
+          }
+        return new_states 
+      }
+      | groupTuple(by: 0, sort: "hash", remainder: true)
+      | map {group_settings, sample_states -> 
+        def input_r1 = sample_states.collect{it.input_r1}.flatten()
+        def input_r2 = sample_states.collect{it.input_r2}.flatten()
+        def event_ids = sample_states.collect{it.event_id}
+        def sample_id_list = sample_states.collect{it.sample_id}.unique()
+        assert sample_id_list.size() == 1
+        def sample_id = sample_id_list[0]
+        assert input_r1.size() == input_r2.size()  
+        
+        def new_state = [
+          "input_r1": input_r1, 
+          "input_r2": input_r2,
+          "event_id": event_ids,
+          "sample_id": sample_id,
+        ]
+        return [group_settings.target, new_state]
+      } 
+      | concat_text.run(
+        directives: [label: ["lowmem", "lowcpu"]],
+        key: "concat_samples_r1",
+        runIf: {id, state -> state.input_r1.size() > 1},
+        fromState: { id, state ->
+          def output_file_name = state.input_r1[0].name
+          [
+            input: state.input_r1,
+            gzip_output: false,
+            output: output_file_name
+          ]
+        },
+        toState: { id, result, state ->
+          def newState = state + [ input_r1: [ result.output ] ]
+          return newState
+        }
+      )
+      | concat_text.run(
+        directives: [label: ["lowmem", "lowcpu"]],
+        key: "concat_samples_r2",
+        runIf: {id, state -> state.input_r2.size() > 1},
+        fromState: { id, state ->
+          def output_file_name = state.input_r2[0].name
+          [
+            input: state.input_r2,
+            gzip_output: false,
+            output: output_file_name
+          ]
+        },
+        toState: { id, result, state ->
+          def newState = state + [ input_r2: [ result.output ] ]
+          return newState
+        }
+      )
+      | map {id, state ->
+          def new_state = [state.sample_id, state]
+          return new_state
+      }
+      | groupTuple(by: 0, sort: 'hash')
+      | map {id, states ->
+        def new_state = [
+          "input_r1": states.collect{it.input_r1}.flatten(),
+          "input_r2": states.collect{it.input_r2}.flatten(),
+          "_meta": ["join_id": states[0].event_id[0]]
+        ]
+        return [id, new_state]
+      }
+      | setState(
+        [
+          "output_r1": "input_r1",
+          "output_r2": "input_r2",
+          "_meta": "_meta"
+        ]
+      )
+
+  emit: 
+    output_ch
+
+}

src/workflows/htrnaseq/config.vsh.yaml

@@ -8,14 +8,14 @@ argument_groups:
     arguments:
       - name: --input_r1
         description: |
-          Forward reads in FASTQ format. Multiple files can be provided which will
+          Forward reads in FASTQ format. Multiple files corresponding to different lanes can be provided which will
           be demultiplexed separately before joining the results for each individual well.
         type: file
         required: true
         multiple: true
       - name: --input_r2
         description: |
-          Reverse reads in FASTQ format. Multiple files can be provided which will
+          Reverse reads in FASTQ format. Multiple files corresponding to different lanes can be provided which will
           be demultiplexed separately before joining the results for each individual well.
         type: file
         required: true
@@ -35,22 +35,22 @@ argument_groups:
       - name: --annotation
         type: file
         required: true
+      - name: --sample_id
+        type: string
+        required: false
+        description: |
+          Sample ID for the provided input files. If not provided, the value of --id
+          will be used. Input files will allways be demultiplexed separately,
+          but the FASTQs for wells with matching sample IDs will be concatenated before mapping.
   - name: Output arguments
     arguments:
-      - name: --fastq_output_r1
-        description: List of demultiplexed fastq files
+      - name: "--fastq_output"
+        description: "Directory containing output fastq files"
         type: file
-        direction: output
         multiple: true
         required: true
-        default: "fastq/*_R1_001.fastq"
-      - name: --fastq_output_r2
-        description: List of demultiplexed fastq files
-        type: file
+        default: "fastq/*"
         direction: output
-        multiple: true
-        required: true
-        default: "fastq/*_R2_001.fastq"
       - name: --star_output
         description: Output from mapping with STAR
         type: file
@@ -120,6 +120,8 @@ dependencies:
     repository: local
   - name: report/create_report
     repository: local
+  - name: utils/concatRuns
+    repository: local
 repositories:
   - name: local
     type: local

src/workflows/htrnaseq/main.nf

@@ -1,33 +1,116 @@
 workflow run_wf {
   take:
-    input_ch
+    raw_ch
 
   main:
+    input_ch = raw_ch
+      // Use the event ID as the default for the sample ID
+      | map {id, state ->
+        def sample_id = state.sample_id ?: id 
+        def newState = state + ["sample_id": sample_id, "run_id": id]
+        return [id, newState]
+      }
+
     // The featureData only has one requirement: the genome annotation.
-    // It can be generated straight away.
+    // It can be generated straight away. Most of the time, there is one shared 
+    // annotation for all of the inputs and the fData should only be calculated once.
+    // The state is manpulated in such a way that there is one event created per unique
+    // input annotation file. In turn, the featureData file can joined into the original input
+    // channel which allows it to be shared across events if required.
     f_data_ch = input_ch
+      | toSortedList()
+      | flatMap {ids_and_states ->
+        def annotation_files = ids_and_states.inject([:]){ old_state, id_and_state ->
+          def (id, state) = id_and_state
+          def annotation_file = state.annotation
+          def new_state = old_state + [(annotation_file): (old_state.getOrDefault(annotation_file, []) + [id])]
+          return new_state
+        }
+        def file_names = annotation_files.keySet().collect{it.name}
+        assert (file_names.toSet().size() == file_names.size()), 
+          "Please make sure that the annotation files have unique file names."
+        def new_states = annotation_files.collect{annotation_file, value ->
+          def new_state = [annotation_file.name , ["annotation": annotation_file, "event_ids": value]]
+          return new_state
+        }
+        return new_states 
+      }
       | create_fdata.run(
         directives: [label: ["lowmem", "lowcpu"]],
         fromState: [
           "gtf": "annotation",
           "output": "f_data"
         ],
-        toState: {id, result, state -> ["f_data": result.output]}
+        toState: ["f_data": "output"]
       )
+      | flatMap {_, state -> 
+          def new_states = state.event_ids.collect{event_id ->
+            [event_id, ["f_data": state.f_data]]
+          }
+          return new_states
+      }
 
     // Perform mapping of each well.
-    mapping_ch = input_ch
+    demultiplex_ch = input_ch
       | well_demultiplex.run(
         fromState: [
             "input_r1": "input_r1",
             "input_r2": "input_r2",
             "barcodesFasta": "barcodesFasta",
         ],
-        toState: [
-          "input_r1": "output_r1",
-          "input_r2": "output_r2",
-        ]
+        toState: {id, result, state ->
+          def all_fastq = result.output_r1 + result.output_r2
+          def output_dir = all_fastq.collect{it.parent}.unique()
+          assert output_dir.size() == 1, "Expected output from well demultiplexing to reside into one directory."
+          def new_state = state + [
+            "input_r1": result.output_r1,
+            "input_r2": result.output_r2,
+            "fastq_output_directory": output_dir[0],
+          ]
+          return new_state
+        }
       )
+
+    fastq_output_directory_ch = demultiplex_ch
+      | map {id, state ->
+        def new_event = [state.sample_id, state]
+        return new_event
+      }
+      | groupTuple(by: 0, sort: "hash")
+      | map {id, states ->
+        def fastq_output_dirs = states.collect{it.fastq_output_directory}
+        def new_state = ["fastq_output_directory": fastq_output_dirs]
+        def new_event = [id, new_state]
+        return [id, new_state]
+      }
+
+
+    concat_samples_ch = demultiplex_ch.join(f_data_ch)
+      | map {id, demutliplex_state, f_data_state ->
+        def newState = demutliplex_state + ["f_data": f_data_state["f_data"]]
+        [id, newState]
+      }
+      | concatRuns.run(
+        fromState: [
+          "input_r1": "input_r1",
+          "input_r2": "input_r2",
+          "sample_id": "sample_id",
+        ],
+        toState: {id, result, state ->
+          def state_overwite = [
+            "input_r1": result.output_r1,
+            "input_r2": result.output_r2,
+            "_meta": ["join_id": state.run_id]
+          ]
+          return state + state_overwite
+        }
+      )
+
+    pool_ch = concat_samples_ch.join(fastq_output_directory_ch)
+      | map {id, demux_state, fastq_output_directory_state ->
+        def new_state = demux_state + fastq_output_directory_state
+        return [id, new_state]
+      } 
       | parallel_map.run(
         directives: ["label": ["highmem", "lowcpu"]],
         fromState: {id, state ->
@@ -44,9 +127,6 @@ workflow run_wf {
           "star_output": "output",
         ]
       )
-
-    // From the mapped wells, create statistics based on the BAM files.
-    pool_ch = mapping_ch
       // Split the events from 1 event per pool into events per well
       // and add extra metadata about the wells to the state.
       | well_metadata.run(
@@ -167,7 +247,7 @@ workflow run_wf {
         ]
       )
     
-    p_data_ch = star_logs_ch.join(pool_statistics_ch, remainder: true)
+    eset_ch = star_logs_ch.join(pool_statistics_ch, remainder: true)
       | map {id, star_logs_state, pool_statistics_state ->
         def newState = star_logs_state + ["nrReadsNrGenesPerChromPool": pool_statistics_state.nrReadsNrGenesPerChromPool]
         return [id, newState]
@@ -181,12 +261,6 @@ workflow run_wf {
         ],
         toState: ["p_data": "output"],
       )
-
-    eset_ch = p_data_ch.join(f_data_ch, remainder: true)
-      | map {id, p_data_state, f_data_state ->
-        def newState = p_data_state + ["f_data": f_data_state["f_data"]]
-        [id, newState]
-      }
       | create_eset.run(
         directives: [label: ["lowmem", "lowcpu"]],
         fromState: [
@@ -228,13 +302,14 @@ workflow run_wf {
 
     output_ch = eset_ch.join(report_channel)
       | map {id, state_eset, state_report ->
-        def new_state = state_eset + ["html_report": state_report.html_report]
+        def new_state = state_eset + [
+          "html_report": state_report.html_report,
+        ]
         [id, new_state]
       }
       | setState([
-        "star_output": "star_output", 
-        "fastq_output_r1": "input_r1",
-        "fastq_output_r2": "input_r2",
+        "star_output": "star_output",
+        "fastq_output": "fastq_output_directory",
         "star_output": "star_output",
         "nrReadsNrGenesPerChrom": "nrReadsNrGenesPerChromPool",
         "star_qc_metrics": "star_qc_metrics",
@@ -242,6 +317,7 @@ workflow run_wf {
         "f_data": "f_data",
         "p_data": "p_data",
         "html_report": "html_report",
+        "_meta": "_meta",
       ])
 
 

src/workflows/runner/config.vsh.yaml

@@ -6,7 +6,6 @@ argument_groups:
     arguments:
       - name: --input
         description: Base directory of the form `s3:/<bucket>/Sequencing/<Sequencer>/<RunID>/<demultiplex_dir>`
-        multiple: true
         type: file
         required: true
       - name: --barcodesFasta

src/workflows/runner/main.nf

@@ -8,19 +8,13 @@ workflow run_wf {
     input_ch
 
   main:
-    output_ch = input_ch
-      // Multiple runs can be provided, and the reads for these runs will
-      // be concatenated. Here, we gather the FASTQ files from each input directory first.
-      | flatMap {id, state ->
-        // Create an input event per input directory
-        def new_state = state.input.withIndex().collect{input_dir, id_index ->
-          def state_item = state + ["input": input_dir, "index": id_index, "run_id": id]
-          return ["${id}_${id_index}".toString(), state_item]
-        }
-        return new_state
-      }
+    htrnaseq_ch = input_ch
       // List the FASTQ files per input directory
       // Be careful: an event per lane is created!
+      | map {id, state ->
+        def new_state = state + ["run_id": id]
+        return [id, new_state]
+      }
       | listInputDir.run(
         fromState: [
           "input": "input",
@@ -38,13 +32,11 @@ workflow run_wf {
       // there might be multiple FASTQs for a single sample that correspond to the
       // lanes. So the fastq files must be gathered across lanes and input folders
       // in order to create an input lists for R1 and R2.
-      | map {id, state -> [state.sample_id, state]}
-      | groupTuple(by: 0, sort: { state1, state2 ->
-        if (state1.index == state2.index) {
-          return state1.lane <=> state2.lane
-        }
-        return state1.index <=> state2.index
-      })
+      // The ID of the event here is important! It determines the name of the output
+      // folders for the FASTQ files and these folders are published as-is later.
+      // The folder where the FASTQ files are stored in should be named after the run ID.
+      | map {id, state -> ["${state.sample_id}/${state.run_id}".toString(), state]}
+      | groupTuple(by: 0, sort: "hash")
       | map {id, states ->
         def new_r1 = states.collect{it.r1_output}
         def new_r2 = states.collect{it.r2_output}
@@ -53,7 +45,7 @@ workflow run_wf {
         // TODO: this can be asserted.
         def new_state = states[0] + [
           "r1": new_r1,
-          "r2": new_r2
+          "r2": new_r2,
         ]
         return [id, new_state]
       }
@@ -62,8 +54,7 @@ workflow run_wf {
           f_data: 'fData/$id.txt',
           p_data: 'pData/$id.txt',
           star_output: 'star_output/$id/*',
-          fastq_output_r1: 'fastq/*_R1_001.fastq',
-          fastq_output_r2: 'fastq/*_R1_001.fastq',
+          fastq_output: 'fastq/*',
           eset: 'esets/$id.rds',
           nrReadsNrGenesPerChrom: 'nrReadsNrGenesPerChrom/$id.txt',
           star_qc_metrics: 'starLogs/$id.txt',
@@ -76,32 +67,32 @@ workflow run_wf {
           genomeDir: "genomeDir",
           annotation: "annotation",
           umi_length: "umi_length",
+          sample_id: "sample_id",
         ],
         toState: { id, result, state -> state + result }
       )
+
       // The HT-RNAseq workflow outputs multiple events, one per 'pool' (usually a plate)
       // but for publishing the results, this is not handy because we want to use the $id
       // variable as a pointer to the target data.
       //
       // So, we should combine everything together
       //
-      // project_id / experiment_id / date_workflow
-
+      // project_id / experiment_id / "data_processed" / date_workflow
+    grouped_ch = htrnaseq_ch
       | toSortedList
-
       | map{ vs ->
           def all_fastqs
           [
             vs[0][1].run_id, // The original ID
             [
               star_output: reduce_paths(vs.collect{ it[1].star_output }.flatten()),
-              fastq_output_r1: reduce_paths(vs.collect{ it[1].fastq_output_r1 }.flatten(), 1),
-              fastq_output_r2: reduce_paths(vs.collect{ it[1].fastq_output_r2 }.flatten(), 1),
               nrReadsNrGenesPerChrom: reduce_paths(vs.collect{ it[1].nrReadsNrGenesPerChrom }),
               star_qc_metrics: reduce_paths(vs.collect{ it[1].star_qc_metrics }),
               eset: reduce_paths(vs.collect{ it[1].eset }),
               f_data: reduce_paths(vs.collect{ it[1].f_data }),
               p_data: reduce_paths(vs.collect{ it[1].p_data }),
+              fastq_output: vs.collect{ it[1].fastq_output }.flatten().unique(),
               html_report: vs.collect{ it[1].html_report }[0],  // The report is for all pools
               plain_output: vs.collect{ it[1].plain_output }[0],
               project_id: vs.collect{ it[1].project_id }[0],
@@ -110,12 +101,13 @@ workflow run_wf {
           ]
         }
 
+    results_publish_ch = grouped_ch
       | publish_results.run(
         fromState: { id, state ->
           def project = (state.plain_output) ? id : "${state.project_id}"
           def experiment = (state.plain_output) ? id : "${state.experiment_id}"
           def id0 = "${project}/${experiment}"
-          def id1 = (state.plain_output) ? id : "${id0}/${date}"
+          def id1 = (state.plain_output) ? id : "${id0}/data_processed/${date}"
           def id2 = (state.plain_output) ? id : "${id1}_htrnaseq_${version}"
 
           if (id == id2) {
@@ -146,14 +138,24 @@ workflow run_wf {
         ]
       )
 
+    fastq_publish_ch = grouped_ch
+      | flatMap{id, state ->
+        def new_states = state.fastq_output.collect{fastq_dir -> 
+          def new_id = fastq_dir.name // The folder name corresponds to the run
+          def fastq_files = fastq_dir.listFiles()
+          def new_state = [
+            "fastq_output": fastq_files
+          ]
+          return [new_id, new_state]
+        }
+        return new_states
+      }
       | publish_fastqs.run(
         fromState: { id, state ->
           def id0 = "${id}"
           def id1 = (state.plain_output) ? id : "${id0}/${date}"
           def id2 = (state.plain_output) ? id : "${id1}_htrnaseq_${version}"
 
-          println(state.plain_output)
-
           if (id == id2) {
             println("Publising fastqs to ${params.fastq_publish_dir}")
           } else {
@@ -161,8 +163,7 @@ workflow run_wf {
           }
 
           [
-            input_r1: state.fastq_output_r1,
-            input_r2: state.fastq_output_r2,
+            input: state.fastq_output,
             output: "${id2}",
           ]
         },
@@ -177,7 +178,7 @@ workflow run_wf {
       )
 
   emit:
-    output_ch
+    grouped_ch
       | map{ id, state -> [ id, [ _meta: [ join_id: state.run_id ] ] ] }
 }
 

src/workflows/well_demultiplex/main.nf

@@ -60,6 +60,8 @@ workflow run_wf {
             output: new_output,
             error_rate: 0.10,
             demultiplex_mode: "single",
+            output_r1: state.output_r1,
+            output_r2: state.output_r2,
           ]
         },
         toState: { id, result, state ->

target/executable/eset/create_eset/.config.vsh.yaml

@@ -202,18 +202,35 @@ build_info:
   engine: "docker|native"
   output: "target/executable/eset/create_eset"
   executable: "target/executable/eset/create_eset/create_eset"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -225,11 +242,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/executable/eset/create_eset/create_eset

@@ -2,7 +2,7 @@
 
 # create_eset update-resources
 # 
-# This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+# This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 # Intuitive.
 # 
@@ -456,9 +456,9 @@ RUN Rscript -e 'options(warn = 2); if (!requireNamespace("remotes", quietly = TR
 
 LABEL org.opencontainers.image.authors="Dries Schaumont, Marijke Van Moerbeke"
 LABEL org.opencontainers.image.description="Companion container for running component eset create_eset"
-LABEL org.opencontainers.image.created="2025-04-25T07:44:05Z"
+LABEL org.opencontainers.image.created="2025-05-08T12:58:14Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/htrnaseq"
-LABEL org.opencontainers.image.revision="d157606b49b157cd2955acf9124f9043fbd0ca5a"
+LABEL org.opencontainers.image.revision="f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
 LABEL org.opencontainers.image.version="update-resources"
 
 VIASHDOCKER

target/executable/eset/create_fdata/.config.vsh.yaml

@@ -179,18 +179,35 @@ build_info:
   engine: "docker|native"
   output: "target/executable/eset/create_fdata"
   executable: "target/executable/eset/create_fdata/create_fdata"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -202,11 +219,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/executable/eset/create_fdata/create_fdata

@@ -2,7 +2,7 @@
 
 # create_fdata update-resources
 # 
-# This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+# This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 # Intuitive.
 # 
@@ -458,9 +458,9 @@ RUN pip install --upgrade pip && \
 
 LABEL org.opencontainers.image.authors="Dries Schaumont, Marijke Van Moerbeke"
 LABEL org.opencontainers.image.description="Companion container for running component eset create_fdata"
-LABEL org.opencontainers.image.created="2025-04-25T07:44:05Z"
+LABEL org.opencontainers.image.created="2025-05-08T12:58:14Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/htrnaseq"
-LABEL org.opencontainers.image.revision="d157606b49b157cd2955acf9124f9043fbd0ca5a"
+LABEL org.opencontainers.image.revision="f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
 LABEL org.opencontainers.image.version="update-resources"
 
 VIASHDOCKER

target/executable/eset/create_pdata/.config.vsh.yaml

@@ -193,18 +193,35 @@ build_info:
   engine: "docker|native"
   output: "target/executable/eset/create_pdata"
   executable: "target/executable/eset/create_pdata/create_pdata"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -216,11 +233,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/executable/eset/create_pdata/create_pdata

@@ -2,7 +2,7 @@
 
 # create_pdata update-resources
 # 
-# This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+# This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 # Intuitive.
 # 
@@ -458,9 +458,9 @@ RUN pip install --upgrade pip && \
 
 LABEL org.opencontainers.image.authors="Dries Schaumont, Marijke Van Moerbeke"
 LABEL org.opencontainers.image.description="Companion container for running component eset create_pdata"
-LABEL org.opencontainers.image.created="2025-04-25T07:44:06Z"
+LABEL org.opencontainers.image.created="2025-05-08T12:58:15Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/htrnaseq"
-LABEL org.opencontainers.image.revision="d157606b49b157cd2955acf9124f9043fbd0ca5a"
+LABEL org.opencontainers.image.revision="f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
 LABEL org.opencontainers.image.version="update-resources"
 
 VIASHDOCKER

target/executable/integration_test_components/htrnaseq/check_eset/.config.vsh.yaml

@@ -151,18 +151,35 @@ build_info:
   engine: "docker|native"
   output: "target/executable/integration_test_components/htrnaseq/check_eset"
   executable: "target/executable/integration_test_components/htrnaseq/check_eset/check_eset"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -174,11 +191,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/executable/integration_test_components/htrnaseq/check_eset/check_eset

@@ -2,7 +2,7 @@
 
 # check_eset update-resources
 # 
-# This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+# This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 # Intuitive.
 # 
@@ -455,9 +455,9 @@ RUN Rscript -e 'options(warn = 2); if (!requireNamespace("remotes", quietly = TR
 
 LABEL org.opencontainers.image.authors="Dries Schaumont"
 LABEL org.opencontainers.image.description="Companion container for running component integration_test_components/htrnaseq check_eset"
-LABEL org.opencontainers.image.created="2025-04-25T07:44:05Z"
+LABEL org.opencontainers.image.created="2025-05-08T12:58:14Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/htrnaseq"
-LABEL org.opencontainers.image.revision="d157606b49b157cd2955acf9124f9043fbd0ca5a"
+LABEL org.opencontainers.image.revision="f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
 LABEL org.opencontainers.image.version="update-resources"
 
 VIASHDOCKER

target/executable/integration_test_components/well_demultiplexing/check_cutadapt_output/.config.vsh.yaml

@@ -160,18 +160,35 @@ build_info:
   engine: "docker|native"
   output: "target/executable/integration_test_components/well_demultiplexing/check_cutadapt_output"
   executable: "target/executable/integration_test_components/well_demultiplexing/check_cutadapt_output/check_cutadapt_output"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -183,11 +200,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/executable/integration_test_components/well_demultiplexing/check_cutadapt_output/check_cutadapt_output

@@ -2,7 +2,7 @@
 
 # check_cutadapt_output update-resources
 # 
-# This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+# This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 # Intuitive.
 # 
@@ -457,9 +457,9 @@ RUN pip install --upgrade pip && \
 
 LABEL org.opencontainers.image.authors="Dries Schaumont"
 LABEL org.opencontainers.image.description="Companion container for running component integration_test_components/well_demultiplexing check_cutadapt_output"
-LABEL org.opencontainers.image.created="2025-04-25T07:44:04Z"
+LABEL org.opencontainers.image.created="2025-05-08T12:58:15Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/htrnaseq"
-LABEL org.opencontainers.image.revision="d157606b49b157cd2955acf9124f9043fbd0ca5a"
+LABEL org.opencontainers.image.revision="f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
 LABEL org.opencontainers.image.version="update-resources"
 
 VIASHDOCKER

target/executable/io/publish_fastqs/.config.vsh.yaml

@@ -5,18 +5,8 @@ argument_groups:
 - name: "Input arguments"
   arguments:
   - type: "file"
-    name: "--input_r1"
-    description: "Directory to write R1 fastq data to"
-    info: null
-    must_exist: true
-    create_parent: true
-    required: true
-    direction: "input"
-    multiple: true
-    multiple_sep: ";"
-  - type: "file"
-    name: "--input_r2"
-    description: "Directory to write R2 fastq data to"
+    name: "--input"
+    description: "Directory to write fastq data to"
     info: null
     must_exist: true
     create_parent: true
@@ -145,18 +135,35 @@ build_info:
   engine: "docker|native"
   output: "target/executable/io/publish_fastqs"
   executable: "target/executable/io/publish_fastqs/publish_fastqs"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -168,11 +175,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/executable/io/publish_fastqs/publish_fastqs

@@ -2,7 +2,7 @@
 
 # publish_fastqs update-resources
 # 
-# This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+# This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 # Intuitive.
 # 
@@ -450,9 +450,9 @@ RUN apt-get update && \
   rm -rf /var/lib/apt/lists/*
 
 LABEL org.opencontainers.image.description="Companion container for running component io publish_fastqs"
-LABEL org.opencontainers.image.created="2025-04-25T07:44:05Z"
+LABEL org.opencontainers.image.created="2025-05-08T12:58:14Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/htrnaseq"
-LABEL org.opencontainers.image.revision="d157606b49b157cd2955acf9124f9043fbd0ca5a"
+LABEL org.opencontainers.image.revision="f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
 LABEL org.opencontainers.image.version="update-resources"
 
 VIASHDOCKER
@@ -575,13 +575,9 @@ function ViashHelp {
   echo "Publish the fastq files per well"
   echo ""
   echo "Input arguments:"
-  echo "    --input_r1"
+  echo "    --input"
   echo "        type: file, required parameter, multiple values allowed, file must exist"
-  echo "        Directory to write R1 fastq data to"
-  echo ""
-  echo "    --input_r2"
-  echo "        type: file, required parameter, multiple values allowed, file must exist"
-  echo "        Directory to write R2 fastq data to"
+  echo "        Directory to write fastq data to"
   echo ""
   echo "Output arguments:"
   echo "    --output"
@@ -638,37 +634,20 @@ while [[ $# -gt 0 ]]; do
             echo "publish_fastqs update-resources"
             exit
             ;;
-        --input_r1)
-            if [ -z "$VIASH_PAR_INPUT_R1" ]; then
-              VIASH_PAR_INPUT_R1="$2"
+        --input)
+            if [ -z "$VIASH_PAR_INPUT" ]; then
+              VIASH_PAR_INPUT="$2"
             else
-              VIASH_PAR_INPUT_R1="$VIASH_PAR_INPUT_R1;""$2"
+              VIASH_PAR_INPUT="$VIASH_PAR_INPUT;""$2"
             fi
-            [ $# -lt 2 ] && ViashError Not enough arguments passed to --input_r1. Use "--help" to get more information on the parameters. && exit 1
+            [ $# -lt 2 ] && ViashError Not enough arguments passed to --input. Use "--help" to get more information on the parameters. && exit 1
             shift 2
             ;;
-        --input_r1=*)
-            if [ -z "$VIASH_PAR_INPUT_R1" ]; then
-              VIASH_PAR_INPUT_R1=$(ViashRemoveFlags "$1")
+        --input=*)
+            if [ -z "$VIASH_PAR_INPUT" ]; then
+              VIASH_PAR_INPUT=$(ViashRemoveFlags "$1")
             else
-              VIASH_PAR_INPUT_R1="$VIASH_PAR_INPUT_R1;"$(ViashRemoveFlags "$1")
-            fi
-            shift 1
-            ;;
-        --input_r2)
-            if [ -z "$VIASH_PAR_INPUT_R2" ]; then
-              VIASH_PAR_INPUT_R2="$2"
-            else
-              VIASH_PAR_INPUT_R2="$VIASH_PAR_INPUT_R2;""$2"
-            fi
-            [ $# -lt 2 ] && ViashError Not enough arguments passed to --input_r2. Use "--help" to get more information on the parameters. && exit 1
-            shift 2
-            ;;
-        --input_r2=*)
-            if [ -z "$VIASH_PAR_INPUT_R2" ]; then
-              VIASH_PAR_INPUT_R2=$(ViashRemoveFlags "$1")
-            else
-              VIASH_PAR_INPUT_R2="$VIASH_PAR_INPUT_R2;"$(ViashRemoveFlags "$1")
+              VIASH_PAR_INPUT="$VIASH_PAR_INPUT;"$(ViashRemoveFlags "$1")
             fi
             shift 1
             ;;
@@ -855,12 +834,8 @@ fi
 
 
 # check whether required parameters exist
-if [ -z ${VIASH_PAR_INPUT_R1+x} ]; then
-  ViashError '--input_r1' is a required argument. Use "--help" to get more information on the parameters.
-  exit 1
-fi
-if [ -z ${VIASH_PAR_INPUT_R2+x} ]; then
-  ViashError '--input_r2' is a required argument. Use "--help" to get more information on the parameters.
+if [ -z ${VIASH_PAR_INPUT+x} ]; then
+  ViashError '--input' is a required argument. Use "--help" to get more information on the parameters.
   exit 1
 fi
 if [ -z ${VIASH_META_NAME+x} ]; then
@@ -894,22 +869,10 @@ if [ -z ${VIASH_PAR_OUTPUT+x} ]; then
 fi
 
 # check whether required files exist
-if [ ! -z "$VIASH_PAR_INPUT_R1" ]; then
+if [ ! -z "$VIASH_PAR_INPUT" ]; then
   IFS=';'
   set -f
-  for file in $VIASH_PAR_INPUT_R1; do
-    unset IFS
-    if [ ! -e "$file" ]; then
-      ViashError "Input file '$file' does not exist."
-      exit 1
-    fi
-  done
-  set +f
-fi
-if [ ! -z "$VIASH_PAR_INPUT_R2" ]; then
-  IFS=';'
-  set -f
-  for file in $VIASH_PAR_INPUT_R2; do
+  for file in $VIASH_PAR_INPUT; do
     unset IFS
     if [ ! -e "$file" ]; then
       ViashError "Input file '$file' does not exist."
@@ -1010,27 +973,16 @@ fi
 if [[ "$VIASH_ENGINE_TYPE" == "docker" ]]; then
   # detect volumes from file arguments
   VIASH_CHOWN_VARS=()
-if [ ! -z "$VIASH_PAR_INPUT_R1" ]; then
-  VIASH_TEST_INPUT_R1=()
+if [ ! -z "$VIASH_PAR_INPUT" ]; then
+  VIASH_TEST_INPUT=()
   IFS=';'
-  for var in $VIASH_PAR_INPUT_R1; do
+  for var in $VIASH_PAR_INPUT; do
     unset IFS
     VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$var")" )
     var=$(ViashDockerAutodetectMount "$var")
-    VIASH_TEST_INPUT_R1+=( "$var" )
+    VIASH_TEST_INPUT+=( "$var" )
   done
-  VIASH_PAR_INPUT_R1=$(IFS=';' ; echo "${VIASH_TEST_INPUT_R1[*]}")
-fi
-if [ ! -z "$VIASH_PAR_INPUT_R2" ]; then
-  VIASH_TEST_INPUT_R2=()
-  IFS=';'
-  for var in $VIASH_PAR_INPUT_R2; do
-    unset IFS
-    VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$var")" )
-    var=$(ViashDockerAutodetectMount "$var")
-    VIASH_TEST_INPUT_R2+=( "$var" )
-  done
-  VIASH_PAR_INPUT_R2=$(IFS=';' ; echo "${VIASH_TEST_INPUT_R2[*]}")
+  VIASH_PAR_INPUT=$(IFS=';' ; echo "${VIASH_TEST_INPUT[*]}")
 fi
 if [ ! -z "$VIASH_PAR_OUTPUT" ]; then
   VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$VIASH_PAR_OUTPUT")" )
@@ -1106,8 +1058,7 @@ trap interrupt INT SIGINT
 cat > "\$tempscript" << 'VIASHMAIN'
 ## VIASH START
 # The following code has been auto-generated by Viash.
-$( if [ ! -z ${VIASH_PAR_INPUT_R1+x} ]; then echo "${VIASH_PAR_INPUT_R1}" | sed "s#'#'\"'\"'#g;s#.*#par_input_r1='&'#" ; else echo "# par_input_r1="; fi )
-$( if [ ! -z ${VIASH_PAR_INPUT_R2+x} ]; then echo "${VIASH_PAR_INPUT_R2}" | sed "s#'#'\"'\"'#g;s#.*#par_input_r2='&'#" ; else echo "# par_input_r2="; fi )
+$( if [ ! -z ${VIASH_PAR_INPUT+x} ]; then echo "${VIASH_PAR_INPUT}" | sed "s#'#'\"'\"'#g;s#.*#par_input='&'#" ; else echo "# par_input="; fi )
 $( if [ ! -z ${VIASH_PAR_OUTPUT+x} ]; then echo "${VIASH_PAR_OUTPUT}" | sed "s#'#'\"'\"'#g;s#.*#par_output='&'#" ; else echo "# par_output="; fi )
 $( if [ ! -z ${VIASH_META_NAME+x} ]; then echo "${VIASH_META_NAME}" | sed "s#'#'\"'\"'#g;s#.*#meta_name='&'#" ; else echo "# meta_name="; fi )
 $( if [ ! -z ${VIASH_META_FUNCTIONALITY_NAME+x} ]; then echo "${VIASH_META_FUNCTIONALITY_NAME}" | sed "s#'#'\"'\"'#g;s#.*#meta_functionality_name='&'#" ; else echo "# meta_functionality_name="; fi )
@@ -1139,14 +1090,9 @@ mkdir -p "\$par_output" && echo "\$par_output created"
 
 echo
 echo "Copying files..."
-IFS=";" read -ra input_r1 <<<\$par_input_r1
-IFS=";" read -ra input_r2 <<<\$par_input_r2
+IFS=";" read -ra input <<<\$par_input
 
-for i in "\${input_r1[@]}"; do
-  cp -rL "\$i" "\$par_output/"
-done
-
-for i in "\${input_r2[@]}"; do
+for i in "\${input[@]}"; do
   cp -rL "\$i" "\$par_output/"
 done
 VIASHMAIN
@@ -1159,31 +1105,18 @@ VIASHEOF
 if [[ "$VIASH_ENGINE_TYPE" == "docker" ]]; then
   # strip viash automount from file paths
   
-  if [ ! -z "$VIASH_PAR_INPUT_R1" ]; then
-    unset VIASH_TEST_INPUT_R1
+  if [ ! -z "$VIASH_PAR_INPUT" ]; then
+    unset VIASH_TEST_INPUT
     IFS=';'
-    for var in $VIASH_PAR_INPUT_R1; do
+    for var in $VIASH_PAR_INPUT; do
       unset IFS
-      if [ -z "$VIASH_TEST_INPUT_R1" ]; then
-      VIASH_TEST_INPUT_R1="$(ViashDockerStripAutomount "$var")"
+      if [ -z "$VIASH_TEST_INPUT" ]; then
+      VIASH_TEST_INPUT="$(ViashDockerStripAutomount "$var")"
     else
-      VIASH_TEST_INPUT_R1="$VIASH_TEST_INPUT_R1;""$(ViashDockerStripAutomount "$var")"
+      VIASH_TEST_INPUT="$VIASH_TEST_INPUT;""$(ViashDockerStripAutomount "$var")"
     fi
     done
-    VIASH_PAR_INPUT_R1="$VIASH_TEST_INPUT_R1"
-  fi
-  if [ ! -z "$VIASH_PAR_INPUT_R2" ]; then
-    unset VIASH_TEST_INPUT_R2
-    IFS=';'
-    for var in $VIASH_PAR_INPUT_R2; do
-      unset IFS
-      if [ -z "$VIASH_TEST_INPUT_R2" ]; then
-      VIASH_TEST_INPUT_R2="$(ViashDockerStripAutomount "$var")"
-    else
-      VIASH_TEST_INPUT_R2="$VIASH_TEST_INPUT_R2;""$(ViashDockerStripAutomount "$var")"
-    fi
-    done
-    VIASH_PAR_INPUT_R2="$VIASH_TEST_INPUT_R2"
+    VIASH_PAR_INPUT="$VIASH_TEST_INPUT"
   fi
   if [ ! -z "$VIASH_PAR_OUTPUT" ]; then
     VIASH_PAR_OUTPUT=$(ViashDockerStripAutomount "$VIASH_PAR_OUTPUT")

target/executable/io/publish_results/.config.vsh.yaml

@@ -189,18 +189,35 @@ build_info:
   engine: "docker|native"
   output: "target/executable/io/publish_results"
   executable: "target/executable/io/publish_results/publish_results"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -212,11 +229,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/executable/io/publish_results/publish_results

@@ -2,7 +2,7 @@
 
 # publish_results update-resources
 # 
-# This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+# This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 # Intuitive.
 # 
@@ -450,9 +450,9 @@ RUN apt-get update && \
   rm -rf /var/lib/apt/lists/*
 
 LABEL org.opencontainers.image.description="Companion container for running component io publish_results"
-LABEL org.opencontainers.image.created="2025-04-25T07:44:05Z"
+LABEL org.opencontainers.image.created="2025-05-08T12:58:14Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/htrnaseq"
-LABEL org.opencontainers.image.revision="d157606b49b157cd2955acf9124f9043fbd0ca5a"
+LABEL org.opencontainers.image.revision="f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
 LABEL org.opencontainers.image.version="update-resources"
 
 VIASHDOCKER

target/executable/parallel_map/.config.vsh.yaml

@@ -281,18 +281,35 @@ build_info:
   engine: "docker|native"
   output: "target/executable/parallel_map"
   executable: "target/executable/parallel_map/parallel_map"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -304,11 +321,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/executable/parallel_map/parallel_map

@@ -2,7 +2,7 @@
 
 # parallel_map update-resources
 # 
-# This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+# This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 # Intuitive.
 # 
@@ -461,9 +461,9 @@ ENV STAR_BINARY=STAR
 COPY STAR /usr/local/bin/$STAR_BINARY
 LABEL org.opencontainers.image.authors="Dries Schaumont, Toni Verbeiren"
 LABEL org.opencontainers.image.description="Companion container for running component parallel_map"
-LABEL org.opencontainers.image.created="2025-04-25T07:44:06Z"
+LABEL org.opencontainers.image.created="2025-05-08T12:58:15Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/htrnaseq"
-LABEL org.opencontainers.image.revision="d157606b49b157cd2955acf9124f9043fbd0ca5a"
+LABEL org.opencontainers.image.revision="f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
 LABEL org.opencontainers.image.version="update-resources"
 
 VIASHDOCKER
@@ -1447,7 +1447,8 @@ for barcode_index in "\${!barcodes[@]}"; do
     fi
   done
   echo "Did not find FASTQ files files for well \${well_id}! "\\
-    "Make sure that the input files have the correct file name format."
+    "Make sure that the input files have the correct file name format."\\
+    "Input files: \${input_r1[@]}"
   exit 1
 done
 

target/executable/report/create_report/.config.vsh.yaml

@@ -164,6 +164,12 @@ engines:
     - "procps"
     - "pandoc"
     interactive: false
+  - type: "r"
+    script:
+    - "install.packages(\"BiocManager\")"
+    - "BiocManager::install(version = \"3.21\", type = \"source\", checkBuilt = TRUE)"
+    bioc_force_install: false
+    warnings_as_errors: true
   - type: "r"
     cran:
     - "ggplot2"
@@ -205,18 +211,35 @@ build_info:
   engine: "docker|native"
   output: "target/executable/report/create_report"
   executable: "target/executable/report/create_report/create_report"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -228,11 +251,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/executable/report/create_report/create_report

@@ -2,7 +2,7 @@
 
 # create_report update-resources
 # 
-# This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+# This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 # Intuitive.
 # 
@@ -453,6 +453,9 @@ RUN apt-get update && \
   DEBIAN_FRONTEND=noninteractive apt-get install -y procps pandoc && \
   rm -rf /var/lib/apt/lists/*
 
+RUN Rscript -e 'options(warn = 2); install.packages("BiocManager")' && \
+  Rscript -e 'options(warn = 2); BiocManager::install(version = "3.21", type = "source", checkBuilt = TRUE)'
+
 RUN Rscript -e 'options(warn = 2); if (!requireNamespace("remotes", quietly = TRUE)) install.packages("remotes")' && \
   Rscript -e 'options(warn = 2); if (!requireNamespace("BiocManager", quietly = TRUE)) install.packages("BiocManager")' && \
   Rscript -e 'options(warn = 2); if (!requireNamespace("Biobase", quietly = TRUE)) BiocManager::install("Biobase")' && \
@@ -462,9 +465,9 @@ RUN Rscript -e 'options(warn = 2); if (!requireNamespace("remotes", quietly = TR
 
 LABEL org.opencontainers.image.authors="Dries Schaumont, Marijke Van Moerbeke"
 LABEL org.opencontainers.image.description="Companion container for running component report create_report"
-LABEL org.opencontainers.image.created="2025-04-25T07:44:06Z"
+LABEL org.opencontainers.image.created="2025-05-08T12:58:15Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/htrnaseq"
-LABEL org.opencontainers.image.revision="d157606b49b157cd2955acf9124f9043fbd0ca5a"
+LABEL org.opencontainers.image.revision="f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
 LABEL org.opencontainers.image.version="update-resources"
 
 VIASHDOCKER

target/executable/report/create_report/plateLayouts.R

@@ -283,15 +283,31 @@ plateLayout <- function(
 
   if (is.null(colours)) {
     colours <- tryCatch({
-      colorRamp2(
+      circlize::colorRamp2(
         breaks = breaks,
         colors = brewer.pal(length(breaks), "Purples")
       )
     },
-    error = function(cond) {
-      return(c("#9370DB", "white"))
+    error = function(cond){
+      
+      message("Recomputed breaks for proper colour mapping")
+      
+      breakValues <- plateValues$values
+      breakValues[which(is.na(breakValues))] <- 0
+      if (all(breakValues >= 0)) {
+        breaks <- computeBreaks(7, max(plateValues$values, na.rm = TRUE))
+      } else {
+        breaks <- quantile(plateValues$values,  probs = seq(0, 1, 0.125))
+      }
+      
+      circlize::colorRamp2(
+        breaks = breaks,
+        colors = brewer.pal(length(breaks), "Purples")
+      )
+      
     })
   }
+  
   ht <- Heatmap(
     plateValues$values,
     column_title = mainTitle, column_title_side = "top",
@@ -425,6 +441,7 @@ computeBreaks <- function(nBreaks, variable) {
     )
     coefExp <- c(exp(coefSystem[1]), coefSystem[2])
     breaks <- coefExp[1] * exp((1:(nBreaks - 1)) * coefExp[2])
+    breaks <- unique(c(0, breaks))
   }
-  return(c(0, breaks))
-}
+  return(breaks)
+}

target/executable/stats/combine_star_logs/.config.vsh.yaml

@@ -200,18 +200,35 @@ build_info:
   engine: "docker|native"
   output: "target/executable/stats/combine_star_logs"
   executable: "target/executable/stats/combine_star_logs/combine_star_logs"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -223,11 +240,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/executable/stats/combine_star_logs/combine_star_logs

@@ -2,7 +2,7 @@
 
 # combine_star_logs update-resources
 # 
-# This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+# This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 # Intuitive.
 # 
@@ -457,9 +457,9 @@ RUN pip install --upgrade pip && \
 
 LABEL org.opencontainers.image.authors="Dries Schaumont"
 LABEL org.opencontainers.image.description="Companion container for running component stats combine_star_logs"
-LABEL org.opencontainers.image.created="2025-04-25T07:44:04Z"
+LABEL org.opencontainers.image.created="2025-05-08T12:58:13Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/htrnaseq"
-LABEL org.opencontainers.image.revision="d157606b49b157cd2955acf9124f9043fbd0ca5a"
+LABEL org.opencontainers.image.revision="f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
 LABEL org.opencontainers.image.version="update-resources"
 
 VIASHDOCKER

target/executable/stats/generate_pool_statistics/.config.vsh.yaml

@@ -184,18 +184,35 @@ build_info:
   engine: "docker|native"
   output: "target/executable/stats/generate_pool_statistics"
   executable: "target/executable/stats/generate_pool_statistics/generate_pool_statistics"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -207,11 +224,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/executable/stats/generate_pool_statistics/generate_pool_statistics

@@ -2,7 +2,7 @@
 
 # generate_pool_statistics update-resources
 # 
-# This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+# This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 # Intuitive.
 # 
@@ -458,9 +458,9 @@ RUN pip install --upgrade pip && \
 
 LABEL org.opencontainers.image.authors="Dries Schaumont, Marijke Van Moerbeke"
 LABEL org.opencontainers.image.description="Companion container for running component stats generate_pool_statistics"
-LABEL org.opencontainers.image.created="2025-04-25T07:44:05Z"
+LABEL org.opencontainers.image.created="2025-05-08T12:58:14Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/htrnaseq"
-LABEL org.opencontainers.image.revision="d157606b49b157cd2955acf9124f9043fbd0ca5a"
+LABEL org.opencontainers.image.revision="f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
 LABEL org.opencontainers.image.version="update-resources"
 
 VIASHDOCKER

target/executable/stats/generate_well_statistics/.config.vsh.yaml

@@ -266,18 +266,35 @@ build_info:
   engine: "docker|native"
   output: "target/executable/stats/generate_well_statistics"
   executable: "target/executable/stats/generate_well_statistics/generate_well_statistics"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -289,11 +306,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/executable/stats/generate_well_statistics/generate_well_statistics

@@ -2,7 +2,7 @@
 
 # generate_well_statistics update-resources
 # 
-# This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+# This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 # work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 # Intuitive.
 # 
@@ -461,9 +461,9 @@ RUN pip install --upgrade pip && \
 
 LABEL org.opencontainers.image.authors="Dries Schaumont, Marijke Van Moerbeke"
 LABEL org.opencontainers.image.description="Companion container for running component stats generate_well_statistics"
-LABEL org.opencontainers.image.created="2025-04-25T07:44:05Z"
+LABEL org.opencontainers.image.created="2025-05-08T12:58:14Z"
 LABEL org.opencontainers.image.source="https://github.com/viash-hub/htrnaseq"
-LABEL org.opencontainers.image.revision="d157606b49b157cd2955acf9124f9043fbd0ca5a"
+LABEL org.opencontainers.image.revision="f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
 LABEL org.opencontainers.image.version="update-resources"
 
 VIASHDOCKER

target/nextflow/eset/create_eset/.config.vsh.yaml

@@ -202,18 +202,35 @@ build_info:
   engine: "docker|native"
   output: "target/nextflow/eset/create_eset"
   executable: "target/nextflow/eset/create_eset/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -225,11 +242,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/eset/create_eset/main.nf

@@ -1,6 +1,6 @@
 // create_eset update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -86,64 +86,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -155,10 +147,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3308,14 +3303,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "docker|native",
     "output" : "target/nextflow/eset/create_eset",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3324,7 +3321,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3335,11 +3332,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",
@@ -3358,7 +3357,7 @@ meta = [
 // inner workflow hook
 def innerWorkflowFactory(args) {
   def rawScript = '''set -e
-tempscript=".viash_script.sh"
+tempscript=".viash_script.R"
 cat > "$tempscript" << VIASHMAIN
 library(Biobase)
 library(data.table)
@@ -4156,7 +4155,7 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   // create process from temp file
   def binding = new nextflow.script.ScriptBinding([:])
   def session = nextflow.Nextflow.getSession()
-  def parser = new nextflow.script.ScriptParser(session)
+  def parser = _getScriptLoader(session)
     .setModule(true)
     .setBinding(binding)
   def moduleScript = parser.runScript(tempFile)
@@ -4170,6 +4169,27 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   return scriptMeta.getProcess(procKey)
 }
 
+// use Reflection to get a ScriptParser / ScriptLoader
+//   <25.02.0-edge: new nextflow.script.ScriptParser(session)
+//   >=25.02.0-edge: nextflow.script.ScriptLoaderFactory.create(session)
+def _getScriptLoader(nextflow.Session session) {
+  // try using the old method
+  try {
+    Class<?> scriptParserClass = Class.forName('nextflow.script.ScriptParser')
+    return scriptParserClass.getDeclaredConstructor(nextflow.Session).newInstance(session)
+  } catch (ClassNotFoundException e) {
+    // else try with the new method
+    try {
+      Class<?> scriptLoaderFactoryClass = Class.forName('nextflow.script.ScriptLoaderFactory')
+      def createMethod = scriptLoaderFactoryClass.getDeclaredMethod('create', nextflow.Session)
+      return createMethod.invoke(null, session) // null because create is static
+    } catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | java.lang.reflect.InvocationTargetException e2) {
+      // Handle the case where neither class is found
+      throw new Exception("Neither nextflow.script.ScriptParser nor nextflow.script.ScriptLoaderFactory could be found. Is this a compatible Nextflow version?", e2)
+    }
+  }
+}
+
 // defaults
 meta["defaults"] = [
   // key to be used to trace the process and determine output names

target/nextflow/eset/create_eset/nextflow_schema.json

@@ -57,10 +57,10 @@
                 "output": {
                 "type":
                 "string",
-                "description": "Type: `file`, required, default: `$id.$key.output.rds`. ",
-                "help_text": "Type: `file`, required, default: `$id.$key.output.rds`. "
+                "description": "Type: `file`, required, default: `eset.$id.rds`. ",
+                "help_text": "Type: `file`, required, default: `eset.$id.rds`. "
             ,
-                "default":"$id.$key.output.rds"
+                "default":"eset.$id.rds"
             }
     
 

target/nextflow/eset/create_fdata/.config.vsh.yaml

@@ -179,18 +179,35 @@ build_info:
   engine: "docker|native"
   output: "target/nextflow/eset/create_fdata"
   executable: "target/nextflow/eset/create_fdata/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -202,11 +219,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/eset/create_fdata/main.nf

@@ -1,6 +1,6 @@
 // create_fdata update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -86,64 +86,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -155,10 +147,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3278,14 +3273,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "docker|native",
     "output" : "target/nextflow/eset/create_fdata",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3294,7 +3291,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3305,11 +3302,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",
@@ -3328,7 +3327,7 @@ meta = [
 // inner workflow hook
 def innerWorkflowFactory(args) {
   def rawScript = '''set -e
-tempscript=".viash_script.sh"
+tempscript=".viash_script.py"
 cat > "$tempscript" << VIASHMAIN
 import logging
 import pandas as pd
@@ -3821,7 +3820,7 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   // create process from temp file
   def binding = new nextflow.script.ScriptBinding([:])
   def session = nextflow.Nextflow.getSession()
-  def parser = new nextflow.script.ScriptParser(session)
+  def parser = _getScriptLoader(session)
     .setModule(true)
     .setBinding(binding)
   def moduleScript = parser.runScript(tempFile)
@@ -3835,6 +3834,27 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   return scriptMeta.getProcess(procKey)
 }
 
+// use Reflection to get a ScriptParser / ScriptLoader
+//   <25.02.0-edge: new nextflow.script.ScriptParser(session)
+//   >=25.02.0-edge: nextflow.script.ScriptLoaderFactory.create(session)
+def _getScriptLoader(nextflow.Session session) {
+  // try using the old method
+  try {
+    Class<?> scriptParserClass = Class.forName('nextflow.script.ScriptParser')
+    return scriptParserClass.getDeclaredConstructor(nextflow.Session).newInstance(session)
+  } catch (ClassNotFoundException e) {
+    // else try with the new method
+    try {
+      Class<?> scriptLoaderFactoryClass = Class.forName('nextflow.script.ScriptLoaderFactory')
+      def createMethod = scriptLoaderFactoryClass.getDeclaredMethod('create', nextflow.Session)
+      return createMethod.invoke(null, session) // null because create is static
+    } catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | java.lang.reflect.InvocationTargetException e2) {
+      // Handle the case where neither class is found
+      throw new Exception("Neither nextflow.script.ScriptParser nor nextflow.script.ScriptLoaderFactory could be found. Is this a compatible Nextflow version?", e2)
+    }
+  }
+}
+
 // defaults
 meta["defaults"] = [
   // key to be used to trace the process and determine output names

target/nextflow/eset/create_fdata/nextflow_schema.json

@@ -27,10 +27,10 @@
                 "output": {
                 "type":
                 "string",
-                "description": "Type: `file`, default: `$id.$key.output.txt`. Tab-delimited text file containing information about the \u0027gene\u0027 or \u0027transcript\u0027\nentries from the input GTF file",
-                "help_text": "Type: `file`, default: `$id.$key.output.txt`. Tab-delimited text file containing information about the \u0027gene\u0027 or \u0027transcript\u0027\nentries from the input GTF file. The \u0027transcript\u0027 entries are used in case the source\nof the GTF was \u0027refGene\u0027 or \u0027ncbiRefSeq\u0027. \n"
+                "description": "Type: `file`, default: `fData.$id.txt`. Tab-delimited text file containing information about the \u0027gene\u0027 or \u0027transcript\u0027\nentries from the input GTF file",
+                "help_text": "Type: `file`, default: `fData.$id.txt`. Tab-delimited text file containing information about the \u0027gene\u0027 or \u0027transcript\u0027\nentries from the input GTF file. The \u0027transcript\u0027 entries are used in case the source\nof the GTF was \u0027refGene\u0027 or \u0027ncbiRefSeq\u0027. \n"
             ,
-                "default":"$id.$key.output.txt"
+                "default":"fData.$id.txt"
             }
     
 

target/nextflow/eset/create_pdata/.config.vsh.yaml

@@ -193,18 +193,35 @@ build_info:
   engine: "docker|native"
   output: "target/nextflow/eset/create_pdata"
   executable: "target/nextflow/eset/create_pdata/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -216,11 +233,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/eset/create_pdata/main.nf

@@ -1,6 +1,6 @@
 // create_pdata update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -86,64 +86,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -155,10 +147,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3292,14 +3287,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "docker|native",
     "output" : "target/nextflow/eset/create_pdata",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3308,7 +3305,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3319,11 +3316,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",
@@ -3342,7 +3341,7 @@ meta = [
 // inner workflow hook
 def innerWorkflowFactory(args) {
   def rawScript = '''set -e
-tempscript=".viash_script.sh"
+tempscript=".viash_script.py"
 cat > "$tempscript" << VIASHMAIN
 from itertools import batched
 import pandas as pd
@@ -3761,7 +3760,7 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   // create process from temp file
   def binding = new nextflow.script.ScriptBinding([:])
   def session = nextflow.Nextflow.getSession()
-  def parser = new nextflow.script.ScriptParser(session)
+  def parser = _getScriptLoader(session)
     .setModule(true)
     .setBinding(binding)
   def moduleScript = parser.runScript(tempFile)
@@ -3775,6 +3774,27 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   return scriptMeta.getProcess(procKey)
 }
 
+// use Reflection to get a ScriptParser / ScriptLoader
+//   <25.02.0-edge: new nextflow.script.ScriptParser(session)
+//   >=25.02.0-edge: nextflow.script.ScriptLoaderFactory.create(session)
+def _getScriptLoader(nextflow.Session session) {
+  // try using the old method
+  try {
+    Class<?> scriptParserClass = Class.forName('nextflow.script.ScriptParser')
+    return scriptParserClass.getDeclaredConstructor(nextflow.Session).newInstance(session)
+  } catch (ClassNotFoundException e) {
+    // else try with the new method
+    try {
+      Class<?> scriptLoaderFactoryClass = Class.forName('nextflow.script.ScriptLoaderFactory')
+      def createMethod = scriptLoaderFactoryClass.getDeclaredMethod('create', nextflow.Session)
+      return createMethod.invoke(null, session) // null because create is static
+    } catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | java.lang.reflect.InvocationTargetException e2) {
+      // Handle the case where neither class is found
+      throw new Exception("Neither nextflow.script.ScriptParser nor nextflow.script.ScriptLoaderFactory could be found. Is this a compatible Nextflow version?", e2)
+    }
+  }
+}
+
 // defaults
 meta["defaults"] = [
   // key to be used to trace the process and determine output names

target/nextflow/eset/create_pdata/nextflow_schema.json

@@ -37,10 +37,10 @@
                 "output": {
                 "type":
                 "string",
-                "description": "Type: `file`, default: `$id.$key.output.txt`. ",
-                "help_text": "Type: `file`, default: `$id.$key.output.txt`. "
+                "description": "Type: `file`, default: `pData.$id.txt`. ",
+                "help_text": "Type: `file`, default: `pData.$id.txt`. "
             ,
-                "default":"$id.$key.output.txt"
+                "default":"pData.$id.txt"
             }
     
 

target/nextflow/integration_test_components/htrnaseq/check_eset/.config.vsh.yaml

@@ -151,18 +151,35 @@ build_info:
   engine: "docker|native"
   output: "target/nextflow/integration_test_components/htrnaseq/check_eset"
   executable: "target/nextflow/integration_test_components/htrnaseq/check_eset/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -174,11 +191,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/integration_test_components/htrnaseq/check_eset/main.nf

@@ -1,6 +1,6 @@
 // check_eset update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -85,64 +85,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -154,10 +146,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3232,14 +3227,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "docker|native",
     "output" : "target/nextflow/integration_test_components/htrnaseq/check_eset",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3248,7 +3245,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3259,11 +3256,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",
@@ -3282,7 +3281,7 @@ meta = [
 // inner workflow hook
 def innerWorkflowFactory(args) {
   def rawScript = '''set -e
-tempscript=".viash_script.sh"
+tempscript=".viash_script.R"
 cat > "$tempscript" << VIASHMAIN
 ## VIASH START
 # The following code has been auto-generated by Viash.
@@ -3855,7 +3854,7 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   // create process from temp file
   def binding = new nextflow.script.ScriptBinding([:])
   def session = nextflow.Nextflow.getSession()
-  def parser = new nextflow.script.ScriptParser(session)
+  def parser = _getScriptLoader(session)
     .setModule(true)
     .setBinding(binding)
   def moduleScript = parser.runScript(tempFile)
@@ -3869,6 +3868,27 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   return scriptMeta.getProcess(procKey)
 }
 
+// use Reflection to get a ScriptParser / ScriptLoader
+//   <25.02.0-edge: new nextflow.script.ScriptParser(session)
+//   >=25.02.0-edge: nextflow.script.ScriptLoaderFactory.create(session)
+def _getScriptLoader(nextflow.Session session) {
+  // try using the old method
+  try {
+    Class<?> scriptParserClass = Class.forName('nextflow.script.ScriptParser')
+    return scriptParserClass.getDeclaredConstructor(nextflow.Session).newInstance(session)
+  } catch (ClassNotFoundException e) {
+    // else try with the new method
+    try {
+      Class<?> scriptLoaderFactoryClass = Class.forName('nextflow.script.ScriptLoaderFactory')
+      def createMethod = scriptLoaderFactoryClass.getDeclaredMethod('create', nextflow.Session)
+      return createMethod.invoke(null, session) // null because create is static
+    } catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | java.lang.reflect.InvocationTargetException e2) {
+      // Handle the case where neither class is found
+      throw new Exception("Neither nextflow.script.ScriptParser nor nextflow.script.ScriptLoaderFactory could be found. Is this a compatible Nextflow version?", e2)
+    }
+  }
+}
+
 // defaults
 meta["defaults"] = [
   // key to be used to trace the process and determine output names

target/nextflow/integration_test_components/well_demultiplexing/check_cutadapt_output/.config.vsh.yaml

@@ -160,18 +160,35 @@ build_info:
   engine: "docker|native"
   output: "target/nextflow/integration_test_components/well_demultiplexing/check_cutadapt_output"
   executable: "target/nextflow/integration_test_components/well_demultiplexing/check_cutadapt_output/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -183,11 +200,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/integration_test_components/well_demultiplexing/check_cutadapt_output/main.nf

@@ -1,6 +1,6 @@
 // check_cutadapt_output update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -85,64 +85,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -154,10 +146,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3243,14 +3238,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "docker|native",
     "output" : "target/nextflow/integration_test_components/well_demultiplexing/check_cutadapt_output",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3259,7 +3256,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3270,11 +3267,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",
@@ -3293,7 +3292,7 @@ meta = [
 // inner workflow hook
 def innerWorkflowFactory(args) {
   def rawScript = '''set -e
-tempscript=".viash_script.sh"
+tempscript=".viash_script.py"
 cat > "$tempscript" << VIASHMAIN
 import dnaio
 from operator import itemgetter
@@ -3735,7 +3734,7 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   // create process from temp file
   def binding = new nextflow.script.ScriptBinding([:])
   def session = nextflow.Nextflow.getSession()
-  def parser = new nextflow.script.ScriptParser(session)
+  def parser = _getScriptLoader(session)
     .setModule(true)
     .setBinding(binding)
   def moduleScript = parser.runScript(tempFile)
@@ -3749,6 +3748,27 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   return scriptMeta.getProcess(procKey)
 }
 
+// use Reflection to get a ScriptParser / ScriptLoader
+//   <25.02.0-edge: new nextflow.script.ScriptParser(session)
+//   >=25.02.0-edge: nextflow.script.ScriptLoaderFactory.create(session)
+def _getScriptLoader(nextflow.Session session) {
+  // try using the old method
+  try {
+    Class<?> scriptParserClass = Class.forName('nextflow.script.ScriptParser')
+    return scriptParserClass.getDeclaredConstructor(nextflow.Session).newInstance(session)
+  } catch (ClassNotFoundException e) {
+    // else try with the new method
+    try {
+      Class<?> scriptLoaderFactoryClass = Class.forName('nextflow.script.ScriptLoaderFactory')
+      def createMethod = scriptLoaderFactoryClass.getDeclaredMethod('create', nextflow.Session)
+      return createMethod.invoke(null, session) // null because create is static
+    } catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | java.lang.reflect.InvocationTargetException e2) {
+      // Handle the case where neither class is found
+      throw new Exception("Neither nextflow.script.ScriptParser nor nextflow.script.ScriptLoaderFactory could be found. Is this a compatible Nextflow version?", e2)
+    }
+  }
+}
+
 // defaults
 meta["defaults"] = [
   // key to be used to trace the process and determine output names

target/nextflow/io/publish_fastqs/.config.vsh.yaml

@@ -5,18 +5,8 @@ argument_groups:
 - name: "Input arguments"
   arguments:
   - type: "file"
-    name: "--input_r1"
-    description: "Directory to write R1 fastq data to"
-    info: null
-    must_exist: true
-    create_parent: true
-    required: true
-    direction: "input"
-    multiple: true
-    multiple_sep: ";"
-  - type: "file"
-    name: "--input_r2"
-    description: "Directory to write R2 fastq data to"
+    name: "--input"
+    description: "Directory to write fastq data to"
     info: null
     must_exist: true
     create_parent: true
@@ -145,18 +135,35 @@ build_info:
   engine: "docker|native"
   output: "target/nextflow/io/publish_fastqs"
   executable: "target/nextflow/io/publish_fastqs/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -168,11 +175,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/io/publish_fastqs/main.nf

@@ -1,6 +1,6 @@
 // publish_fastqs update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -82,64 +82,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -151,10 +143,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3044,19 +3039,8 @@ meta = [
       "arguments" : [
         {
           "type" : "file",
-          "name" : "--input_r1",
-          "description" : "Directory to write R1 fastq data to",
-          "must_exist" : true,
-          "create_parent" : true,
-          "required" : true,
-          "direction" : "input",
-          "multiple" : true,
-          "multiple_sep" : ";"
-        },
-        {
-          "type" : "file",
-          "name" : "--input_r2",
-          "description" : "Directory to write R2 fastq data to",
+          "name" : "--input",
+          "description" : "Directory to write fastq data to",
           "must_exist" : true,
           "create_parent" : true,
           "required" : true,
@@ -3217,14 +3201,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "docker|native",
     "output" : "target/nextflow/io/publish_fastqs",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3233,7 +3219,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3244,11 +3230,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",
@@ -3271,8 +3259,7 @@ tempscript=".viash_script.sh"
 cat > "$tempscript" << VIASHMAIN
 ## VIASH START
 # The following code has been auto-generated by Viash.
-$( if [ ! -z ${VIASH_PAR_INPUT_R1+x} ]; then echo "${VIASH_PAR_INPUT_R1}" | sed "s#'#'\\"'\\"'#g;s#.*#par_input_r1='&'#" ; else echo "# par_input_r1="; fi )
-$( if [ ! -z ${VIASH_PAR_INPUT_R2+x} ]; then echo "${VIASH_PAR_INPUT_R2}" | sed "s#'#'\\"'\\"'#g;s#.*#par_input_r2='&'#" ; else echo "# par_input_r2="; fi )
+$( if [ ! -z ${VIASH_PAR_INPUT+x} ]; then echo "${VIASH_PAR_INPUT}" | sed "s#'#'\\"'\\"'#g;s#.*#par_input='&'#" ; else echo "# par_input="; fi )
 $( if [ ! -z ${VIASH_PAR_OUTPUT+x} ]; then echo "${VIASH_PAR_OUTPUT}" | sed "s#'#'\\"'\\"'#g;s#.*#par_output='&'#" ; else echo "# par_output="; fi )
 $( if [ ! -z ${VIASH_META_NAME+x} ]; then echo "${VIASH_META_NAME}" | sed "s#'#'\\"'\\"'#g;s#.*#meta_name='&'#" ; else echo "# meta_name="; fi )
 $( if [ ! -z ${VIASH_META_FUNCTIONALITY_NAME+x} ]; then echo "${VIASH_META_FUNCTIONALITY_NAME}" | sed "s#'#'\\"'\\"'#g;s#.*#meta_functionality_name='&'#" ; else echo "# meta_functionality_name="; fi )
@@ -3304,14 +3291,9 @@ mkdir -p "\\$par_output" && echo "\\$par_output created"
 
 echo
 echo "Copying files..."
-IFS=";" read -ra input_r1 <<<\\$par_input_r1
-IFS=";" read -ra input_r2 <<<\\$par_input_r2
+IFS=";" read -ra input <<<\\$par_input
 
-for i in "\\${input_r1[@]}"; do
-  cp -rL "\\$i" "\\$par_output/"
-done
-
-for i in "\\${input_r2[@]}"; do
+for i in "\\${input[@]}"; do
   cp -rL "\\$i" "\\$par_output/"
 done
 VIASHMAIN
@@ -3648,7 +3630,7 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   // create process from temp file
   def binding = new nextflow.script.ScriptBinding([:])
   def session = nextflow.Nextflow.getSession()
-  def parser = new nextflow.script.ScriptParser(session)
+  def parser = _getScriptLoader(session)
     .setModule(true)
     .setBinding(binding)
   def moduleScript = parser.runScript(tempFile)
@@ -3662,6 +3644,27 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   return scriptMeta.getProcess(procKey)
 }
 
+// use Reflection to get a ScriptParser / ScriptLoader
+//   <25.02.0-edge: new nextflow.script.ScriptParser(session)
+//   >=25.02.0-edge: nextflow.script.ScriptLoaderFactory.create(session)
+def _getScriptLoader(nextflow.Session session) {
+  // try using the old method
+  try {
+    Class<?> scriptParserClass = Class.forName('nextflow.script.ScriptParser')
+    return scriptParserClass.getDeclaredConstructor(nextflow.Session).newInstance(session)
+  } catch (ClassNotFoundException e) {
+    // else try with the new method
+    try {
+      Class<?> scriptLoaderFactoryClass = Class.forName('nextflow.script.ScriptLoaderFactory')
+      def createMethod = scriptLoaderFactoryClass.getDeclaredMethod('create', nextflow.Session)
+      return createMethod.invoke(null, session) // null because create is static
+    } catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | java.lang.reflect.InvocationTargetException e2) {
+      // Handle the case where neither class is found
+      throw new Exception("Neither nextflow.script.ScriptParser nor nextflow.script.ScriptLoaderFactory could be found. Is this a compatible Nextflow version?", e2)
+    }
+  }
+}
+
 // defaults
 meta["defaults"] = [
   // key to be used to trace the process and determine output names

target/nextflow/io/publish_fastqs/nextflow_schema.json

@@ -14,21 +14,11 @@
     "properties": {
     
         
-                "input_r1": {
+                "input": {
                 "type":
                 "string",
-                "description": "Type: List of `file`, required, multiple_sep: `\";\"`. Directory to write R1 fastq data to",
-                "help_text": "Type: List of `file`, required, multiple_sep: `\";\"`. Directory to write R1 fastq data to"
-            
-            }
-    
-
-        ,
-                "input_r2": {
-                "type":
-                "string",
-                "description": "Type: List of `file`, required, multiple_sep: `\";\"`. Directory to write R2 fastq data to",
-                "help_text": "Type: List of `file`, required, multiple_sep: `\";\"`. Directory to write R2 fastq data to"
+                "description": "Type: List of `file`, required, multiple_sep: `\";\"`. Directory to write fastq data to",
+                "help_text": "Type: List of `file`, required, multiple_sep: `\";\"`. Directory to write fastq data to"
             
             }
     
@@ -47,10 +37,10 @@
                 "output": {
                 "type":
                 "string",
-                "description": "Type: `file`, default: `$id.$key.output.output`. ",
-                "help_text": "Type: `file`, default: `$id.$key.output.output`. "
+                "description": "Type: `file`, default: `$id`. ",
+                "help_text": "Type: `file`, default: `$id`. "
             ,
-                "default":"$id.$key.output.output"
+                "default":"$id"
             }
     
 

target/nextflow/io/publish_results/.config.vsh.yaml

@@ -189,18 +189,35 @@ build_info:
   engine: "docker|native"
   output: "target/nextflow/io/publish_results"
   executable: "target/nextflow/io/publish_results/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -212,11 +229,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/io/publish_results/main.nf

@@ -1,6 +1,6 @@
 // publish_results update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -82,64 +82,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -151,10 +143,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3266,14 +3261,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "docker|native",
     "output" : "target/nextflow/io/publish_results",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3282,7 +3279,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3293,11 +3290,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",
@@ -3724,7 +3723,7 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   // create process from temp file
   def binding = new nextflow.script.ScriptBinding([:])
   def session = nextflow.Nextflow.getSession()
-  def parser = new nextflow.script.ScriptParser(session)
+  def parser = _getScriptLoader(session)
     .setModule(true)
     .setBinding(binding)
   def moduleScript = parser.runScript(tempFile)
@@ -3738,6 +3737,27 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   return scriptMeta.getProcess(procKey)
 }
 
+// use Reflection to get a ScriptParser / ScriptLoader
+//   <25.02.0-edge: new nextflow.script.ScriptParser(session)
+//   >=25.02.0-edge: nextflow.script.ScriptLoaderFactory.create(session)
+def _getScriptLoader(nextflow.Session session) {
+  // try using the old method
+  try {
+    Class<?> scriptParserClass = Class.forName('nextflow.script.ScriptParser')
+    return scriptParserClass.getDeclaredConstructor(nextflow.Session).newInstance(session)
+  } catch (ClassNotFoundException e) {
+    // else try with the new method
+    try {
+      Class<?> scriptLoaderFactoryClass = Class.forName('nextflow.script.ScriptLoaderFactory')
+      def createMethod = scriptLoaderFactoryClass.getDeclaredMethod('create', nextflow.Session)
+      return createMethod.invoke(null, session) // null because create is static
+    } catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | java.lang.reflect.InvocationTargetException e2) {
+      // Handle the case where neither class is found
+      throw new Exception("Neither nextflow.script.ScriptParser nor nextflow.script.ScriptLoaderFactory could be found. Is this a compatible Nextflow version?", e2)
+    }
+  }
+}
+
 // defaults
 meta["defaults"] = [
   // key to be used to trace the process and determine output names

target/nextflow/io/publish_results/nextflow_schema.json

@@ -97,10 +97,10 @@
                 "output": {
                 "type":
                 "string",
-                "description": "Type: `file`, default: `$id.$key.output.output`. ",
-                "help_text": "Type: `file`, default: `$id.$key.output.output`. "
+                "description": "Type: `file`, default: `$id`. ",
+                "help_text": "Type: `file`, default: `$id`. "
             ,
-                "default":"$id.$key.output.output"
+                "default":"$id"
             }
     
 

target/nextflow/parallel_map/.config.vsh.yaml

@@ -281,18 +281,35 @@ build_info:
   engine: "docker|native"
   output: "target/nextflow/parallel_map"
   executable: "target/nextflow/parallel_map/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -304,11 +321,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/parallel_map/main.nf

@@ -1,6 +1,6 @@
 // parallel_map update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -86,64 +86,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -155,10 +147,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3378,14 +3373,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "docker|native",
     "output" : "target/nextflow/parallel_map",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3394,7 +3391,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3405,11 +3402,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",
@@ -3566,7 +3565,8 @@ for barcode_index in "\\${!barcodes[@]}"; do
     fi
   done
   echo "Did not find FASTQ files files for well \\${well_id}! "\\\\
-    "Make sure that the input files have the correct file name format."
+    "Make sure that the input files have the correct file name format."\\\\
+    "Input files: \\${input_r1[@]}"
   exit 1
 done
 
@@ -4119,7 +4119,7 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   // create process from temp file
   def binding = new nextflow.script.ScriptBinding([:])
   def session = nextflow.Nextflow.getSession()
-  def parser = new nextflow.script.ScriptParser(session)
+  def parser = _getScriptLoader(session)
     .setModule(true)
     .setBinding(binding)
   def moduleScript = parser.runScript(tempFile)
@@ -4133,6 +4133,27 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   return scriptMeta.getProcess(procKey)
 }
 
+// use Reflection to get a ScriptParser / ScriptLoader
+//   <25.02.0-edge: new nextflow.script.ScriptParser(session)
+//   >=25.02.0-edge: nextflow.script.ScriptLoaderFactory.create(session)
+def _getScriptLoader(nextflow.Session session) {
+  // try using the old method
+  try {
+    Class<?> scriptParserClass = Class.forName('nextflow.script.ScriptParser')
+    return scriptParserClass.getDeclaredConstructor(nextflow.Session).newInstance(session)
+  } catch (ClassNotFoundException e) {
+    // else try with the new method
+    try {
+      Class<?> scriptLoaderFactoryClass = Class.forName('nextflow.script.ScriptLoaderFactory')
+      def createMethod = scriptLoaderFactoryClass.getDeclaredMethod('create', nextflow.Session)
+      return createMethod.invoke(null, session) // null because create is static
+    } catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | java.lang.reflect.InvocationTargetException e2) {
+      // Handle the case where neither class is found
+      throw new Exception("Neither nextflow.script.ScriptParser nor nextflow.script.ScriptLoaderFactory could be found. Is this a compatible Nextflow version?", e2)
+    }
+  }
+}
+
 // defaults
 meta["defaults"] = [
   // key to be used to trace the process and determine output names

target/nextflow/parallel_map/nextflow_schema.json

@@ -119,10 +119,10 @@
                 "output": {
                 "type":
                 "string",
-                "description": "Type: List of `file`, required, default: `$id.$key.output_*./*`, multiple_sep: `\";\"`. A list of output folders which are the result of using STAR to map each input FASTQ pair STAR to the reference genome",
-                "help_text": "Type: List of `file`, required, default: `$id.$key.output_*./*`, multiple_sep: `\";\"`. A list of output folders which are the result of using STAR to map each input FASTQ pair STAR to the reference genome.\nThe order of the items DO NOT match with the order of the entries in the barcodes FASTA file or the input FASTQ pairs. \n"
+                "description": "Type: List of `file`, required, default: `./*`, multiple_sep: `\";\"`. A list of output folders which are the result of using STAR to map each input FASTQ pair STAR to the reference genome",
+                "help_text": "Type: List of `file`, required, default: `./*`, multiple_sep: `\";\"`. A list of output folders which are the result of using STAR to map each input FASTQ pair STAR to the reference genome.\nThe order of the items DO NOT match with the order of the entries in the barcodes FASTA file or the input FASTQ pairs. \n"
             ,
-                "default":"$id.$key.output_*./*"
+                "default":"./*"
             }
     
 
@@ -130,10 +130,10 @@
                 "joblog": {
                 "type":
                 "string",
-                "description": "Type: `file`, default: `$id.$key.joblog.txt`. Where to store the log file listing all the jobs",
-                "help_text": "Type: `file`, default: `$id.$key.joblog.txt`. Where to store the log file listing all the jobs."
+                "description": "Type: `file`, default: `execution_log.txt`. Where to store the log file listing all the jobs",
+                "help_text": "Type: `file`, default: `execution_log.txt`. Where to store the log file listing all the jobs."
             ,
-                "default":"$id.$key.joblog.txt"
+                "default":"execution_log.txt"
             }
     
 

target/nextflow/report/create_report/.config.vsh.yaml

@@ -164,6 +164,12 @@ engines:
     - "procps"
     - "pandoc"
     interactive: false
+  - type: "r"
+    script:
+    - "install.packages(\"BiocManager\")"
+    - "BiocManager::install(version = \"3.21\", type = \"source\", checkBuilt = TRUE)"
+    bioc_force_install: false
+    warnings_as_errors: true
   - type: "r"
     cran:
     - "ggplot2"
@@ -205,18 +211,35 @@ build_info:
   engine: "docker|native"
   output: "target/nextflow/report/create_report"
   executable: "target/nextflow/report/create_report/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -228,11 +251,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/report/create_report/main.nf

@@ -1,6 +1,6 @@
 // create_report update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -86,64 +86,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -155,10 +147,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3262,6 +3257,15 @@ meta = [
           ],
           "interactive" : false
         },
+        {
+          "type" : "r",
+          "script" : [
+            "install.packages(\\"BiocManager\\")",
+            "BiocManager::install(version = \\"3.21\\", type = \\"source\\", checkBuilt = TRUE)"
+          ],
+          "bioc_force_install" : false,
+          "warnings_as_errors" : true
+        },
         {
           "type" : "r",
           "cran" : [
@@ -3313,14 +3317,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "docker|native",
     "output" : "target/nextflow/report/create_report",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3329,7 +3335,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3340,11 +3346,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",
@@ -3363,7 +3371,7 @@ meta = [
 // inner workflow hook
 def innerWorkflowFactory(args) {
   def rawScript = '''set -e
-tempscript=".viash_script.sh"
+tempscript=".viash_script.R"
 cat > "$tempscript" << VIASHMAIN
 ## VIASH START
 # The following code has been auto-generated by Viash.
@@ -3771,7 +3779,7 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   // create process from temp file
   def binding = new nextflow.script.ScriptBinding([:])
   def session = nextflow.Nextflow.getSession()
-  def parser = new nextflow.script.ScriptParser(session)
+  def parser = _getScriptLoader(session)
     .setModule(true)
     .setBinding(binding)
   def moduleScript = parser.runScript(tempFile)
@@ -3785,6 +3793,27 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   return scriptMeta.getProcess(procKey)
 }
 
+// use Reflection to get a ScriptParser / ScriptLoader
+//   <25.02.0-edge: new nextflow.script.ScriptParser(session)
+//   >=25.02.0-edge: nextflow.script.ScriptLoaderFactory.create(session)
+def _getScriptLoader(nextflow.Session session) {
+  // try using the old method
+  try {
+    Class<?> scriptParserClass = Class.forName('nextflow.script.ScriptParser')
+    return scriptParserClass.getDeclaredConstructor(nextflow.Session).newInstance(session)
+  } catch (ClassNotFoundException e) {
+    // else try with the new method
+    try {
+      Class<?> scriptLoaderFactoryClass = Class.forName('nextflow.script.ScriptLoaderFactory')
+      def createMethod = scriptLoaderFactoryClass.getDeclaredMethod('create', nextflow.Session)
+      return createMethod.invoke(null, session) // null because create is static
+    } catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | java.lang.reflect.InvocationTargetException e2) {
+      // Handle the case where neither class is found
+      throw new Exception("Neither nextflow.script.ScriptParser nor nextflow.script.ScriptLoaderFactory could be found. Is this a compatible Nextflow version?", e2)
+    }
+  }
+}
+
 // defaults
 meta["defaults"] = [
   // key to be used to trace the process and determine output names

target/nextflow/report/create_report/plateLayouts.R

@@ -283,15 +283,31 @@ plateLayout <- function(
 
   if (is.null(colours)) {
     colours <- tryCatch({
-      colorRamp2(
+      circlize::colorRamp2(
         breaks = breaks,
         colors = brewer.pal(length(breaks), "Purples")
       )
     },
-    error = function(cond) {
-      return(c("#9370DB", "white"))
+    error = function(cond){
+      
+      message("Recomputed breaks for proper colour mapping")
+      
+      breakValues <- plateValues$values
+      breakValues[which(is.na(breakValues))] <- 0
+      if (all(breakValues >= 0)) {
+        breaks <- computeBreaks(7, max(plateValues$values, na.rm = TRUE))
+      } else {
+        breaks <- quantile(plateValues$values,  probs = seq(0, 1, 0.125))
+      }
+      
+      circlize::colorRamp2(
+        breaks = breaks,
+        colors = brewer.pal(length(breaks), "Purples")
+      )
+      
     })
   }
+  
   ht <- Heatmap(
     plateValues$values,
     column_title = mainTitle, column_title_side = "top",
@@ -425,6 +441,7 @@ computeBreaks <- function(nBreaks, variable) {
     )
     coefExp <- c(exp(coefSystem[1]), coefSystem[2])
     breaks <- coefExp[1] * exp((1:(nBreaks - 1)) * coefExp[2])
+    breaks <- unique(c(0, breaks))
   }
-  return(c(0, breaks))
-}
+  return(breaks)
+}

target/nextflow/stats/combine_star_logs/.config.vsh.yaml

@@ -200,18 +200,35 @@ build_info:
   engine: "docker|native"
   output: "target/nextflow/stats/combine_star_logs"
   executable: "target/nextflow/stats/combine_star_logs/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -223,11 +240,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/stats/combine_star_logs/main.nf

@@ -1,6 +1,6 @@
 // combine_star_logs update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -85,64 +85,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -154,10 +146,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3294,14 +3289,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "docker|native",
     "output" : "target/nextflow/stats/combine_star_logs",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3310,7 +3307,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3321,11 +3318,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",
@@ -3344,7 +3343,7 @@ meta = [
 // inner workflow hook
 def innerWorkflowFactory(args) {
   def rawScript = '''set -e
-tempscript=".viash_script.sh"
+tempscript=".viash_script.py"
 cat > "$tempscript" << VIASHMAIN
 import logging
 import pandas as pd
@@ -3926,7 +3925,7 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   // create process from temp file
   def binding = new nextflow.script.ScriptBinding([:])
   def session = nextflow.Nextflow.getSession()
-  def parser = new nextflow.script.ScriptParser(session)
+  def parser = _getScriptLoader(session)
     .setModule(true)
     .setBinding(binding)
   def moduleScript = parser.runScript(tempFile)
@@ -3940,6 +3939,27 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   return scriptMeta.getProcess(procKey)
 }
 
+// use Reflection to get a ScriptParser / ScriptLoader
+//   <25.02.0-edge: new nextflow.script.ScriptParser(session)
+//   >=25.02.0-edge: nextflow.script.ScriptLoaderFactory.create(session)
+def _getScriptLoader(nextflow.Session session) {
+  // try using the old method
+  try {
+    Class<?> scriptParserClass = Class.forName('nextflow.script.ScriptParser')
+    return scriptParserClass.getDeclaredConstructor(nextflow.Session).newInstance(session)
+  } catch (ClassNotFoundException e) {
+    // else try with the new method
+    try {
+      Class<?> scriptLoaderFactoryClass = Class.forName('nextflow.script.ScriptLoaderFactory')
+      def createMethod = scriptLoaderFactoryClass.getDeclaredMethod('create', nextflow.Session)
+      return createMethod.invoke(null, session) // null because create is static
+    } catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | java.lang.reflect.InvocationTargetException e2) {
+      // Handle the case where neither class is found
+      throw new Exception("Neither nextflow.script.ScriptParser nor nextflow.script.ScriptLoaderFactory could be found. Is this a compatible Nextflow version?", e2)
+    }
+  }
+}
+
 // defaults
 meta["defaults"] = [
   // key to be used to trace the process and determine output names

target/nextflow/stats/combine_star_logs/nextflow_schema.json

@@ -57,10 +57,10 @@
                 "output": {
                 "type":
                 "string",
-                "description": "Type: `file`, default: `$id.$key.output.txt`. Tab-delimited file describing for each barcode (as the rows), the metrics (as columns)\ngathered from the different input files",
-                "help_text": "Type: `file`, default: `$id.$key.output.txt`. Tab-delimited file describing for each barcode (as the rows), the metrics (as columns)\ngathered from the different input files. \n"
+                "description": "Type: `file`, default: `starLogs.txt`. Tab-delimited file describing for each barcode (as the rows), the metrics (as columns)\ngathered from the different input files",
+                "help_text": "Type: `file`, default: `starLogs.txt`. Tab-delimited file describing for each barcode (as the rows), the metrics (as columns)\ngathered from the different input files. \n"
             ,
-                "default":"$id.$key.output.txt"
+                "default":"starLogs.txt"
             }
     
 

target/nextflow/stats/generate_pool_statistics/.config.vsh.yaml

@@ -184,18 +184,35 @@ build_info:
   engine: "docker|native"
   output: "target/nextflow/stats/generate_pool_statistics"
   executable: "target/nextflow/stats/generate_pool_statistics/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -207,11 +224,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/stats/generate_pool_statistics/main.nf

@@ -1,6 +1,6 @@
 // generate_pool_statistics update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -86,64 +86,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -155,10 +147,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3278,14 +3273,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "docker|native",
     "output" : "target/nextflow/stats/generate_pool_statistics",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3294,7 +3291,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3305,11 +3302,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",
@@ -3328,7 +3327,7 @@ meta = [
 // inner workflow hook
 def innerWorkflowFactory(args) {
   def rawScript = '''set -e
-tempscript=".viash_script.sh"
+tempscript=".viash_script.py"
 cat > "$tempscript" << VIASHMAIN
 import pandas as pd
 from pathlib import Path
@@ -3781,7 +3780,7 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   // create process from temp file
   def binding = new nextflow.script.ScriptBinding([:])
   def session = nextflow.Nextflow.getSession()
-  def parser = new nextflow.script.ScriptParser(session)
+  def parser = _getScriptLoader(session)
     .setModule(true)
     .setBinding(binding)
   def moduleScript = parser.runScript(tempFile)
@@ -3795,6 +3794,27 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   return scriptMeta.getProcess(procKey)
 }
 
+// use Reflection to get a ScriptParser / ScriptLoader
+//   <25.02.0-edge: new nextflow.script.ScriptParser(session)
+//   >=25.02.0-edge: nextflow.script.ScriptLoaderFactory.create(session)
+def _getScriptLoader(nextflow.Session session) {
+  // try using the old method
+  try {
+    Class<?> scriptParserClass = Class.forName('nextflow.script.ScriptParser')
+    return scriptParserClass.getDeclaredConstructor(nextflow.Session).newInstance(session)
+  } catch (ClassNotFoundException e) {
+    // else try with the new method
+    try {
+      Class<?> scriptLoaderFactoryClass = Class.forName('nextflow.script.ScriptLoaderFactory')
+      def createMethod = scriptLoaderFactoryClass.getDeclaredMethod('create', nextflow.Session)
+      return createMethod.invoke(null, session) // null because create is static
+    } catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | java.lang.reflect.InvocationTargetException e2) {
+      // Handle the case where neither class is found
+      throw new Exception("Neither nextflow.script.ScriptParser nor nextflow.script.ScriptLoaderFactory could be found. Is this a compatible Nextflow version?", e2)
+    }
+  }
+}
+
 // defaults
 meta["defaults"] = [
   // key to be used to trace the process and determine output names

target/nextflow/stats/generate_well_statistics/.config.vsh.yaml

@@ -266,18 +266,35 @@ build_info:
   engine: "docker|native"
   output: "target/nextflow/stats/generate_well_statistics"
   executable: "target/nextflow/stats/generate_well_statistics/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -289,11 +306,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/stats/generate_well_statistics/main.nf

@@ -1,6 +1,6 @@
 // generate_well_statistics update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -86,64 +86,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -155,10 +147,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3373,14 +3368,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "docker|native",
     "output" : "target/nextflow/stats/generate_well_statistics",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3389,7 +3386,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3400,11 +3397,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",
@@ -3423,7 +3422,7 @@ meta = [
 // inner workflow hook
 def innerWorkflowFactory(args) {
   def rawScript = '''set -e
-tempscript=".viash_script.sh"
+tempscript=".viash_script.py"
 cat > "$tempscript" << VIASHMAIN
 import pysam
 import pandas as pd
@@ -3867,7 +3866,7 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   // create process from temp file
   def binding = new nextflow.script.ScriptBinding([:])
   def session = nextflow.Nextflow.getSession()
-  def parser = new nextflow.script.ScriptParser(session)
+  def parser = _getScriptLoader(session)
     .setModule(true)
     .setBinding(binding)
   def moduleScript = parser.runScript(tempFile)
@@ -3881,6 +3880,27 @@ def _vdsl3ProcessFactory(Map workflowArgs, Map meta, String rawScript) {
   return scriptMeta.getProcess(procKey)
 }
 
+// use Reflection to get a ScriptParser / ScriptLoader
+//   <25.02.0-edge: new nextflow.script.ScriptParser(session)
+//   >=25.02.0-edge: nextflow.script.ScriptLoaderFactory.create(session)
+def _getScriptLoader(nextflow.Session session) {
+  // try using the old method
+  try {
+    Class<?> scriptParserClass = Class.forName('nextflow.script.ScriptParser')
+    return scriptParserClass.getDeclaredConstructor(nextflow.Session).newInstance(session)
+  } catch (ClassNotFoundException e) {
+    // else try with the new method
+    try {
+      Class<?> scriptLoaderFactoryClass = Class.forName('nextflow.script.ScriptLoaderFactory')
+      def createMethod = scriptLoaderFactoryClass.getDeclaredMethod('create', nextflow.Session)
+      return createMethod.invoke(null, session) // null because create is static
+    } catch (ClassNotFoundException | NoSuchMethodException | IllegalAccessException | java.lang.reflect.InvocationTargetException e2) {
+      // Handle the case where neither class is found
+      throw new Exception("Neither nextflow.script.ScriptParser nor nextflow.script.ScriptLoaderFactory could be found. Is this a compatible Nextflow version?", e2)
+    }
+  }
+}
+
 // defaults
 meta["defaults"] = [
   // key to be used to trace the process and determine output names

target/nextflow/stats/generate_well_statistics/nextflow_schema.json

@@ -47,10 +47,10 @@
                 "processedBAMFile": {
                 "type":
                 "string",
-                "description": "Type: `file`, default: `$id.$key.processedBAMFile.txt`. Path to a ",
-                "help_text": "Type: `file`, default: `$id.$key.processedBAMFile.txt`. Path to a .tsv file listing, per read in the BAM file,\nthe value for the \"CB\", \"UX\", \"GX\" and \"GN\" tag, together with the\nchromsome to which the read was mapped to.\n"
+                "description": "Type: `file`, default: `processedBamFile.txt`. Path to a ",
+                "help_text": "Type: `file`, default: `processedBamFile.txt`. Path to a .tsv file listing, per read in the BAM file,\nthe value for the \"CB\", \"UX\", \"GX\" and \"GN\" tag, together with the\nchromsome to which the read was mapped to.\n"
             ,
-                "default":"$id.$key.processedBAMFile.txt"
+                "default":"processedBamFile.txt"
             }
     
 
@@ -58,10 +58,10 @@
                 "nrReadsNrGenesPerChrom": {
                 "type":
                 "string",
-                "description": "Type: `file`, default: `$id.$key.nrReadsNrGenesPerChrom.txt`. Path to an output file that contains a ",
-                "help_text": "Type: `file`, default: `$id.$key.nrReadsNrGenesPerChrom.txt`. Path to an output file that contains a .tsv formatted table describing\nper chromosome the number of reads that were mapped to that chromosome (NumberOfReads\ncolumn) and the number of genes on that chromosome that had at least one\nread mapped to it (NumberOfGenes).\n"
+                "description": "Type: `file`, default: `nrReadsNrGenesPerChrom.txt`. Path to an output file that contains a ",
+                "help_text": "Type: `file`, default: `nrReadsNrGenesPerChrom.txt`. Path to an output file that contains a .tsv formatted table describing\nper chromosome the number of reads that were mapped to that chromosome (NumberOfReads\ncolumn) and the number of genes on that chromosome that had at least one\nread mapped to it (NumberOfGenes).\n"
             ,
-                "default":"$id.$key.nrReadsNrGenesPerChrom.txt"
+                "default":"nrReadsNrGenesPerChrom.txt"
             }
     
 
@@ -69,10 +69,10 @@
                 "nrReadsNrUMIsPerCB": {
                 "type":
                 "string",
-                "description": "Type: `file`, default: `$id.$key.nrReadsNrUMIsPerCB.txt`. Path to an output file that contains a ",
-                "help_text": "Type: `file`, default: `$id.$key.nrReadsNrUMIsPerCB.txt`. Path to an output file that contains a .tsv formatted table describing\nper barcode the number of UMI\u0027s (nrUMIs) and the total number of reads (NumberOfReads).\n"
+                "description": "Type: `file`, default: `nrReadsNrUMIsPerCB.txt`. Path to an output file that contains a ",
+                "help_text": "Type: `file`, default: `nrReadsNrUMIsPerCB.txt`. Path to an output file that contains a .tsv formatted table describing\nper barcode the number of UMI\u0027s (nrUMIs) and the total number of reads (NumberOfReads).\n"
             ,
-                "default":"$id.$key.nrReadsNrUMIsPerCB.txt"
+                "default":"nrReadsNrUMIsPerCB.txt"
             }
     
 
@@ -80,10 +80,10 @@
                 "umiFreqTop": {
                 "type":
                 "string",
-                "description": "Type: `file`, default: `$id.$key.umiFreqTop.txt`. Path to an output file that contains a ",
-                "help_text": "Type: `file`, default: `$id.$key.umiFreqTop.txt`. Path to an output file that contains a .tsv formatted table describing\nper UMI (column UB) the frequency at which they occur in the reads (column\nN). Only the top 100 UMIs are included.\n"
+                "description": "Type: `file`, default: `umiFreqTop100.txt`. Path to an output file that contains a ",
+                "help_text": "Type: `file`, default: `umiFreqTop100.txt`. Path to an output file that contains a .tsv formatted table describing\nper UMI (column UB) the frequency at which they occur in the reads (column\nN). Only the top 100 UMIs are included.\n"
             ,
-                "default":"$id.$key.umiFreqTop.txt"
+                "default":"umiFreqTop100.txt"
             }
     
 

target/nextflow/utils/concatRuns/.config.vsh.yaml

@@ -0,0 +1,212 @@
+name: "concatRuns"
+namespace: "utils"
+version: "update-resources"
+argument_groups:
+- name: "Arguments"
+  arguments:
+  - type: "file"
+    name: "--input_r1"
+    info: null
+    must_exist: true
+    create_parent: true
+    required: true
+    direction: "input"
+    multiple: true
+    multiple_sep: ";"
+  - type: "file"
+    name: "--input_r2"
+    info: null
+    must_exist: true
+    create_parent: true
+    required: true
+    direction: "input"
+    multiple: true
+    multiple_sep: ";"
+  - type: "string"
+    name: "--sample_id"
+    info: null
+    required: true
+    direction: "input"
+    multiple: false
+    multiple_sep: ";"
+  - type: "file"
+    name: "--output_r1"
+    description: "Path to read 1 fastq/fasta file"
+    info: null
+    must_exist: true
+    create_parent: true
+    required: false
+    direction: "output"
+    multiple: true
+    multiple_sep: ";"
+  - type: "file"
+    name: "--output_r2"
+    description: "Path to read 2 fastq/fasta file"
+    info: null
+    must_exist: true
+    create_parent: true
+    required: false
+    direction: "output"
+    multiple: true
+    multiple_sep: ";"
+resources:
+- type: "nextflow_script"
+  path: "main.nf"
+  is_executable: true
+  entrypoint: "run_wf"
+- type: "file"
+  path: "nextflow_labels.config"
+  dest: "nextflow_labels.config"
+description: "Concatenate well FASTQ files from different runs in order to increase\
+  \ sequencing depth.\n"
+info: null
+status: "enabled"
+scope:
+  image: "public"
+  target: "public"
+requirements:
+  commands:
+  - "ps"
+dependencies:
+- name: "concat_text"
+  repository:
+    type: "vsh"
+    repo: "craftbox"
+    tag: "v0.1.0"
+repositories:
+- type: "vsh"
+  name: "cb"
+  repo: "craftbox"
+  tag: "v0.1.0"
+license: "MIT"
+links:
+  repository: "https://github.com/viash-hub/htrnaseq"
+runners:
+- type: "nextflow"
+  id: "nextflow"
+  directives:
+    tag: "$id"
+  auto:
+    simplifyInput: true
+    simplifyOutput: false
+    transcript: false
+    publish: false
+  config:
+    labels:
+      mem1gb: "memory = 1000000000.B"
+      mem2gb: "memory = 2000000000.B"
+      mem5gb: "memory = 5000000000.B"
+      mem10gb: "memory = 10000000000.B"
+      mem20gb: "memory = 20000000000.B"
+      mem50gb: "memory = 50000000000.B"
+      mem100gb: "memory = 100000000000.B"
+      mem200gb: "memory = 200000000000.B"
+      mem500gb: "memory = 500000000000.B"
+      mem1tb: "memory = 1000000000000.B"
+      mem2tb: "memory = 2000000000000.B"
+      mem5tb: "memory = 5000000000000.B"
+      mem10tb: "memory = 10000000000000.B"
+      mem20tb: "memory = 20000000000000.B"
+      mem50tb: "memory = 50000000000000.B"
+      mem100tb: "memory = 100000000000000.B"
+      mem200tb: "memory = 200000000000000.B"
+      mem500tb: "memory = 500000000000000.B"
+      mem1gib: "memory = 1073741824.B"
+      mem2gib: "memory = 2147483648.B"
+      mem4gib: "memory = 4294967296.B"
+      mem8gib: "memory = 8589934592.B"
+      mem16gib: "memory = 17179869184.B"
+      mem32gib: "memory = 34359738368.B"
+      mem64gib: "memory = 68719476736.B"
+      mem128gib: "memory = 137438953472.B"
+      mem256gib: "memory = 274877906944.B"
+      mem512gib: "memory = 549755813888.B"
+      mem1tib: "memory = 1099511627776.B"
+      mem2tib: "memory = 2199023255552.B"
+      mem4tib: "memory = 4398046511104.B"
+      mem8tib: "memory = 8796093022208.B"
+      mem16tib: "memory = 17592186044416.B"
+      mem32tib: "memory = 35184372088832.B"
+      mem64tib: "memory = 70368744177664.B"
+      mem128tib: "memory = 140737488355328.B"
+      mem256tib: "memory = 281474976710656.B"
+      mem512tib: "memory = 562949953421312.B"
+      cpu1: "cpus = 1"
+      cpu2: "cpus = 2"
+      cpu5: "cpus = 5"
+      cpu10: "cpus = 10"
+      cpu20: "cpus = 20"
+      cpu50: "cpus = 50"
+      cpu100: "cpus = 100"
+      cpu200: "cpus = 200"
+      cpu500: "cpus = 500"
+      cpu1000: "cpus = 1000"
+    script:
+    - "includeConfig(\"nextflow_labels.config\")"
+  debug: false
+  container: "docker"
+engines:
+- type: "native"
+  id: "native"
+- type: "native"
+  id: "native"
+build_info:
+  config: "src/utils/concatRuns/config.vsh.yaml"
+  runner: "nextflow"
+  engine: "native|native"
+  output: "target/nextflow/utils/concatRuns"
+  executable: "target/nextflow/utils/concatRuns/main.nf"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
+  git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
+  dependencies:
+  - "target/dependencies/vsh/vsh/craftbox/v0.1.0/nextflow/concat_text"
+package_config:
+  name: "htrnaseq"
+  version: "update-resources"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
+  info:
+    test_resources:
+    - path: "gs://viash-hub-resources/htrnaseq/v1"
+      dest: "resources_test"
+  viash_version: "0.9.4"
+  source: "src"
+  target: "target"
+  config_mods:
+  - ".requirements.commands := ['ps']\n.runners[.type == 'nextflow'].config.script\
+    \ := 'includeConfig(\"nextflow_labels.config\")'\n.resources += {path: '/src/config/labels.config',\
+    \ dest: 'nextflow_labels.config'}\n"
+  - ".engines += { type: \"native\" }"
+  - ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
+  - ".engines[.type == 'docker'].target_tag := 'update-resources'"
+  keywords:
+  - "bioinformatics"
+  - "sequencing"
+  - "high-throughput"
+  - "RNAseq"
+  - "mapping"
+  - "counting"
+  - "pipeline"
+  - "workflow"
+  license: "MIT"
+  organization: "vsh"
+  links:
+    repository: "https://github.com/viash-hub/htrnaseq"
+    issue_tracker: "https://github.com/viash-hub/htrnaseq/issues"

target/nextflow/utils/concatRuns/nextflow.config

@@ -0,0 +1,125 @@
+manifest {
+  name = 'utils/concatRuns'
+  mainScript = 'main.nf'
+  nextflowVersion = '!>=20.12.1-edge'
+  version = 'update-resources'
+  description = 'Concatenate well FASTQ files from different runs in order to increase sequencing depth.\n'
+}
+
+process.container = 'nextflow/bash:latest'
+
+// detect tempdir
+tempDir = java.nio.file.Paths.get(
+  System.getenv('NXF_TEMP') ?:
+    System.getenv('VIASH_TEMP') ?: 
+    System.getenv('TEMPDIR') ?: 
+    System.getenv('TMPDIR') ?: 
+    '/tmp'
+).toAbsolutePath()
+
+profiles {
+  no_publish {
+    process {
+      withName: '.*' {
+        publishDir = [
+          enabled: false
+        ]
+      }
+    }
+  }
+  mount_temp {
+    docker.temp            = tempDir
+    podman.temp            = tempDir
+    charliecloud.temp      = tempDir
+  }
+  docker {
+    docker.enabled         = true
+    // docker.userEmulation   = true
+    singularity.enabled    = false
+    podman.enabled         = false
+    shifter.enabled        = false
+    charliecloud.enabled   = false
+  }
+  singularity {
+    singularity.enabled    = true
+    singularity.autoMounts = true
+    docker.enabled         = false
+    podman.enabled         = false
+    shifter.enabled        = false
+    charliecloud.enabled   = false
+  }
+  podman {
+    podman.enabled         = true
+    docker.enabled         = false
+    singularity.enabled    = false
+    shifter.enabled        = false
+    charliecloud.enabled   = false
+  }
+  shifter {
+    shifter.enabled        = true
+    docker.enabled         = false
+    singularity.enabled    = false
+    podman.enabled         = false
+    charliecloud.enabled   = false
+  }
+  charliecloud {
+    charliecloud.enabled   = true
+    docker.enabled         = false
+    singularity.enabled    = false
+    podman.enabled         = false
+    shifter.enabled        = false
+  }
+}
+
+process{
+  withLabel: mem1gb { memory = 1000000000.B }
+  withLabel: mem2gb { memory = 2000000000.B }
+  withLabel: mem5gb { memory = 5000000000.B }
+  withLabel: mem10gb { memory = 10000000000.B }
+  withLabel: mem20gb { memory = 20000000000.B }
+  withLabel: mem50gb { memory = 50000000000.B }
+  withLabel: mem100gb { memory = 100000000000.B }
+  withLabel: mem200gb { memory = 200000000000.B }
+  withLabel: mem500gb { memory = 500000000000.B }
+  withLabel: mem1tb { memory = 1000000000000.B }
+  withLabel: mem2tb { memory = 2000000000000.B }
+  withLabel: mem5tb { memory = 5000000000000.B }
+  withLabel: mem10tb { memory = 10000000000000.B }
+  withLabel: mem20tb { memory = 20000000000000.B }
+  withLabel: mem50tb { memory = 50000000000000.B }
+  withLabel: mem100tb { memory = 100000000000000.B }
+  withLabel: mem200tb { memory = 200000000000000.B }
+  withLabel: mem500tb { memory = 500000000000000.B }
+  withLabel: mem1gib { memory = 1073741824.B }
+  withLabel: mem2gib { memory = 2147483648.B }
+  withLabel: mem4gib { memory = 4294967296.B }
+  withLabel: mem8gib { memory = 8589934592.B }
+  withLabel: mem16gib { memory = 17179869184.B }
+  withLabel: mem32gib { memory = 34359738368.B }
+  withLabel: mem64gib { memory = 68719476736.B }
+  withLabel: mem128gib { memory = 137438953472.B }
+  withLabel: mem256gib { memory = 274877906944.B }
+  withLabel: mem512gib { memory = 549755813888.B }
+  withLabel: mem1tib { memory = 1099511627776.B }
+  withLabel: mem2tib { memory = 2199023255552.B }
+  withLabel: mem4tib { memory = 4398046511104.B }
+  withLabel: mem8tib { memory = 8796093022208.B }
+  withLabel: mem16tib { memory = 17592186044416.B }
+  withLabel: mem32tib { memory = 35184372088832.B }
+  withLabel: mem64tib { memory = 70368744177664.B }
+  withLabel: mem128tib { memory = 140737488355328.B }
+  withLabel: mem256tib { memory = 281474976710656.B }
+  withLabel: mem512tib { memory = 562949953421312.B }
+  withLabel: cpu1 { cpus = 1 }
+  withLabel: cpu2 { cpus = 2 }
+  withLabel: cpu5 { cpus = 5 }
+  withLabel: cpu10 { cpus = 10 }
+  withLabel: cpu20 { cpus = 20 }
+  withLabel: cpu50 { cpus = 50 }
+  withLabel: cpu100 { cpus = 100 }
+  withLabel: cpu200 { cpus = 200 }
+  withLabel: cpu500 { cpus = 500 }
+  withLabel: cpu1000 { cpus = 1000 }
+}
+
+includeConfig("nextflow_labels.config")

target/nextflow/utils/concatRuns/nextflow_labels.config

@@ -0,0 +1,108 @@
+executor {
+  $k8s {
+    submitRateLimit = '10sec'
+    pollInterval = '1 sec'
+  }
+}
+
+process {
+  container = 'nextflow/bash:latest'
+  
+  // default resources
+  memory = { 8.Gb * task.attempt }
+  cpus = 8
+  maxForks = 36
+
+  // Retry for exit codes that have something to do with memory issues
+  errorStrategy = { task.exitStatus in 137..140 ? 'retry' : 'terminate' }
+  maxRetries = 3
+  maxMemory = 192.GB
+
+  // Resource labels
+  withLabel: verylowcpu { cpus = 2 }
+  withLabel: lowcpu { cpus = 8 }
+  withLabel: midcpu { cpus = 16 }
+  withLabel: highcpu { cpus = 32 }
+  
+  withLabel: verylowmem { memory = { get_memory( 4.GB * task.attempt ) } }
+  withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+  withLabel: midmem { memory = { get_memory( 16.GB * task.attempt ) } }
+  withLabel: highmem { memory = { get_memory( 64.GB * task.attempt ) } }
+
+}
+
+profiles {
+  // detect tempdir
+  tempDir = java.nio.file.Paths.get(
+    System.getenv('NXF_TEMP') ?:
+      System.getenv('VIASH_TEMP') ?: 
+      System.getenv('TEMPDIR') ?: 
+      System.getenv('TMPDIR') ?: 
+      '/tmp'
+  ).toAbsolutePath()
+
+  mount_temp {
+    docker.temp            = tempDir
+    podman.temp            = tempDir
+    charliecloud.temp      = tempDir
+  }
+
+  no_publish {
+    process {
+      withName: '.*' {
+        publishDir = [
+          enabled: false
+        ]
+      }
+    }
+  }
+
+  docker {
+    docker.fixOwnership    = true
+    docker.enabled         = true
+    // docker.userEmulation   = true
+    singularity.enabled    = false
+    podman.enabled         = false
+    shifter.enabled        = false
+    charliecloud.enabled   = false
+  }
+
+  local {
+    // This config is for local processing.
+    process {
+        withName: ".*parallel_map_process" {
+          maxForks = 1
+        }
+        maxMemory = 25.GB
+        withLabel: verylowcpu { cpus = 2 }
+        withLabel: lowcpu { cpus = 4 }
+        withLabel: midcpu { cpus = 6 }
+        withLabel: highcpu { cpus = 8 }
+  
+        withLabel: lowmem { memory = { get_memory( 8.GB * task.attempt ) } }
+        withLabel: midmem { memory = { get_memory( 12.GB * task.attempt ) } }
+        withLabel: highmem { memory = { get_memory( 20.GB * task.attempt ) } }
+    }
+  }
+}
+
+def get_memory(to_compare) {
+    if (!process.containsKey("maxMemory") || !process.maxMemory) {
+      return to_compare
+    }
+
+    try {
+      if (process.containsKey("maxRetries") && process.maxRetries && task.attempt == (process.maxRetries as int)) {
+        return process.maxMemory
+      }
+      else if (to_compare.compareTo(process.maxMemory as nextflow.util.MemoryUnit) == 1) {
+        return max_memory as nextflow.util.MemoryUnit
+      }
+      else {
+        return to_compare
+      }  
+    } catch (all) {
+          println "Error processing memory resources. Please check that process.maxMemory '${process.maxMemory}' and process.maxRetries '${process.maxRetries}' are valid!"
+          System.exit(1)
+    }
+  }

target/nextflow/utils/concatRuns/nextflow_schema.json

@@ -0,0 +1,112 @@
+{
+"$schema": "http://json-schema.org/draft-07/schema",
+"title": "concatRuns",
+"description": "Concatenate well FASTQ files from different runs in order to increase sequencing depth.\n",
+"type": "object",
+"definitions": {
+
+    
+    
+    "arguments" : {
+    "title": "Arguments",
+    "type": "object",
+    "description": "No description",
+    "properties": {
+    
+        
+                "input_r1": {
+                "type":
+                "string",
+                "description": "Type: List of `file`, required, multiple_sep: `\";\"`. ",
+                "help_text": "Type: List of `file`, required, multiple_sep: `\";\"`. "
+            
+            }
+    
+
+        ,
+                "input_r2": {
+                "type":
+                "string",
+                "description": "Type: List of `file`, required, multiple_sep: `\";\"`. ",
+                "help_text": "Type: List of `file`, required, multiple_sep: `\";\"`. "
+            
+            }
+    
+
+        ,
+                "sample_id": {
+                "type":
+                "string",
+                "description": "Type: `string`, required. ",
+                "help_text": "Type: `string`, required. "
+            
+            }
+    
+
+        ,
+                "output_r1": {
+                "type":
+                "string",
+                "description": "Type: List of `file`, default: `$id.$key.output_r1_*`, multiple_sep: `\";\"`. Path to read 1 fastq/fasta file",
+                "help_text": "Type: List of `file`, default: `$id.$key.output_r1_*`, multiple_sep: `\";\"`. Path to read 1 fastq/fasta file"
+            ,
+                "default":"$id.$key.output_r1_*"
+            }
+    
+
+        ,
+                "output_r2": {
+                "type":
+                "string",
+                "description": "Type: List of `file`, default: `$id.$key.output_r2_*`, multiple_sep: `\";\"`. Path to read 2 fastq/fasta file",
+                "help_text": "Type: List of `file`, default: `$id.$key.output_r2_*`, multiple_sep: `\";\"`. Path to read 2 fastq/fasta file"
+            ,
+                "default":"$id.$key.output_r2_*"
+            }
+    
+
+}
+},
+    
+    
+    "nextflow input-output arguments" : {
+    "title": "Nextflow input-output arguments",
+    "type": "object",
+    "description": "Input/output parameters for Nextflow itself. Please note that both publishDir and publish_dir are supported but at least one has to be configured.",
+    "properties": {
+    
+        
+                "publish_dir": {
+                "type":
+                "string",
+                "description": "Type: `string`, required, example: `output/`. Path to an output directory",
+                "help_text": "Type: `string`, required, example: `output/`. Path to an output directory."
+            
+            }
+    
+
+        ,
+                "param_list": {
+                "type":
+                "string",
+                "description": "Type: `string`, example: `my_params.yaml`. Allows inputting multiple parameter sets to initialise a Nextflow channel",
+                "help_text": "Type: `string`, example: `my_params.yaml`. Allows inputting multiple parameter sets to initialise a Nextflow channel. A `param_list` can either be a list of maps, a csv file, a json file, a yaml file, or simply a yaml blob.\n\n* A list of maps (as-is) where the keys of each map corresponds to the arguments of the pipeline. Example: in a `nextflow.config` file: `param_list: [ [\u0027id\u0027: \u0027foo\u0027, \u0027input\u0027: \u0027foo.txt\u0027], [\u0027id\u0027: \u0027bar\u0027, \u0027input\u0027: \u0027bar.txt\u0027] ]`.\n* A csv file should have column names which correspond to the different arguments of this pipeline. Example: `--param_list data.csv` with columns `id,input`.\n* A json or a yaml file should be a list of maps, each of which has keys corresponding to the arguments of the pipeline. Example: `--param_list data.json` with contents `[ {\u0027id\u0027: \u0027foo\u0027, \u0027input\u0027: \u0027foo.txt\u0027}, {\u0027id\u0027: \u0027bar\u0027, \u0027input\u0027: \u0027bar.txt\u0027} ]`.\n* A yaml blob can also be passed directly as a string. Example: `--param_list \"[ {\u0027id\u0027: \u0027foo\u0027, \u0027input\u0027: \u0027foo.txt\u0027}, {\u0027id\u0027: \u0027bar\u0027, \u0027input\u0027: \u0027bar.txt\u0027} ]\"`.\n\nWhen passing a csv, json or yaml file, relative path names are relativized to the location of the parameter file. No relativation is performed when `param_list` is a list of maps (as-is) or a yaml blob.",
+                "hidden": true
+            
+            }
+    
+
+}
+}
+},
+"allOf": [
+
+    {
+    "$ref": "#/definitions/arguments"
+    },
+
+    {
+    "$ref": "#/definitions/nextflow input-output arguments"
+    }
+]
+}

target/nextflow/utils/listInputDir/.config.vsh.yaml

@@ -167,18 +167,35 @@ build_info:
   engine: "native|native"
   output: "target/nextflow/utils/listInputDir"
   executable: "target/nextflow/utils/listInputDir/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -190,11 +207,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/utils/listInputDir/main.nf

@@ -1,6 +1,6 @@
 // listInputDir update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -82,64 +82,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -151,10 +143,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3238,14 +3233,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "native|native",
     "output" : "target/nextflow/utils/listInputDir",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3254,7 +3251,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3265,11 +3262,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",

target/nextflow/utils/listInputDir/nextflow_schema.json

@@ -38,10 +38,10 @@
                 "r1_output": {
                 "type":
                 "string",
-                "description": "Type: `file`, default: `$id.$key.r1_output.r1_output`. Path to read 1 fastq/fasta file",
-                "help_text": "Type: `file`, default: `$id.$key.r1_output.r1_output`. Path to read 1 fastq/fasta file"
+                "description": "Type: `file`, default: `$id.$key.r1_output`. Path to read 1 fastq/fasta file",
+                "help_text": "Type: `file`, default: `$id.$key.r1_output`. Path to read 1 fastq/fasta file"
             ,
-                "default":"$id.$key.r1_output.r1_output"
+                "default":"$id.$key.r1_output"
             }
     
 
@@ -49,10 +49,10 @@
                 "r2_output": {
                 "type":
                 "string",
-                "description": "Type: `file`, default: `$id.$key.r2_output.r2_output`. Path to read 2 fastq/fasta file",
-                "help_text": "Type: `file`, default: `$id.$key.r2_output.r2_output`. Path to read 2 fastq/fasta file"
+                "description": "Type: `file`, default: `$id.$key.r2_output`. Path to read 2 fastq/fasta file",
+                "help_text": "Type: `file`, default: `$id.$key.r2_output`. Path to read 2 fastq/fasta file"
             ,
-                "default":"$id.$key.r2_output.r2_output"
+                "default":"$id.$key.r2_output"
             }
     
 

target/nextflow/workflows/htrnaseq/.config.vsh.yaml

@@ -20,9 +20,9 @@ argument_groups:
   arguments:
   - type: "file"
     name: "--input_r1"
-    description: "Forward reads in FASTQ format. Multiple files can be provided which\
-      \ will\nbe demultiplexed separately before joining the results for each individual\
-      \ well.\n"
+    description: "Forward reads in FASTQ format. Multiple files corresponding to different\
+      \ lanes can be provided which will\nbe demultiplexed separately before joining\
+      \ the results for each individual well.\n"
     info: null
     must_exist: true
     create_parent: true
@@ -32,9 +32,9 @@ argument_groups:
     multiple_sep: ";"
   - type: "file"
     name: "--input_r2"
-    description: "Reverse reads in FASTQ format. Multiple files can be provided which\
-      \ will\nbe demultiplexed separately before joining the results for each individual\
-      \ well.\n"
+    description: "Reverse reads in FASTQ format. Multiple files corresponding to different\
+      \ lanes can be provided which will\nbe demultiplexed separately before joining\
+      \ the results for each individual well.\n"
     info: null
     must_exist: true
     create_parent: true
@@ -80,26 +80,25 @@ argument_groups:
     direction: "input"
     multiple: false
     multiple_sep: ";"
+  - type: "string"
+    name: "--sample_id"
+    description: "Sample ID for the provided input files. If not provided, the value\
+      \ of --id\nwill be used. Input files will allways be demultiplexed separately,\n\
+      but the FASTQs for wells with matching sample IDs will be concatenated before\
+      \ mapping.\n"
+    info: null
+    required: false
+    direction: "input"
+    multiple: false
+    multiple_sep: ";"
 - name: "Output arguments"
   arguments:
   - type: "file"
-    name: "--fastq_output_r1"
-    description: "List of demultiplexed fastq files"
+    name: "--fastq_output"
+    description: "Directory containing output fastq files"
     info: null
     default:
-    - "fastq/*_R1_001.fastq"
-    must_exist: true
-    create_parent: true
-    required: true
-    direction: "output"
-    multiple: true
-    multiple_sep: ";"
-  - type: "file"
-    name: "--fastq_output_r2"
-    description: "List of demultiplexed fastq files"
-    info: null
-    default:
-    - "fastq/*_R2_001.fastq"
+    - "fastq/*"
     must_exist: true
     create_parent: true
     required: true
@@ -240,6 +239,9 @@ dependencies:
 - name: "report/create_report"
   repository:
     type: "local"
+- name: "utils/concatRuns"
+  repository:
+    type: "local"
 repositories:
 - type: "local"
   name: "local"
@@ -325,9 +327,10 @@ build_info:
   engine: "native|native"
   output: "target/nextflow/workflows/htrnaseq"
   executable: "target/nextflow/workflows/htrnaseq/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
   dependencies:
   - "target/nextflow/stats/combine_star_logs"
   - "target/nextflow/stats/generate_pool_statistics"
@@ -339,15 +342,32 @@ build_info:
   - "target/nextflow/eset/create_fdata"
   - "target/nextflow/eset/create_pdata"
   - "target/nextflow/report/create_report"
+  - "target/nextflow/utils/concatRuns"
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -359,11 +379,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/workflows/htrnaseq/main.nf

@@ -1,6 +1,6 @@
 // htrnaseq update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -85,64 +85,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -154,10 +146,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3071,7 +3066,7 @@ meta = [
         {
           "type" : "file",
           "name" : "--input_r1",
-          "description" : "Forward reads in FASTQ format. Multiple files can be provided which will\nbe demultiplexed separately before joining the results for each individual well.\n",
+          "description" : "Forward reads in FASTQ format. Multiple files corresponding to different lanes can be provided which will\nbe demultiplexed separately before joining the results for each individual well.\n",
           "must_exist" : true,
           "create_parent" : true,
           "required" : true,
@@ -3082,7 +3077,7 @@ meta = [
         {
           "type" : "file",
           "name" : "--input_r2",
-          "description" : "Reverse reads in FASTQ format. Multiple files can be provided which will\nbe demultiplexed separately before joining the results for each individual well.\n",
+          "description" : "Reverse reads in FASTQ format. Multiple files corresponding to different lanes can be provided which will\nbe demultiplexed separately before joining the results for each individual well.\n",
           "must_exist" : true,
           "create_parent" : true,
           "required" : true,
@@ -3132,6 +3127,15 @@ meta = [
           "direction" : "input",
           "multiple" : false,
           "multiple_sep" : ";"
+        },
+        {
+          "type" : "string",
+          "name" : "--sample_id",
+          "description" : "Sample ID for the provided input files. If not provided, the value of --id\nwill be used. Input files will allways be demultiplexed separately,\nbut the FASTQs for wells with matching sample IDs will be concatenated before mapping.\n",
+          "required" : false,
+          "direction" : "input",
+          "multiple" : false,
+          "multiple_sep" : ";"
         }
       ]
     },
@@ -3140,24 +3144,10 @@ meta = [
       "arguments" : [
         {
           "type" : "file",
-          "name" : "--fastq_output_r1",
-          "description" : "List of demultiplexed fastq files",
+          "name" : "--fastq_output",
+          "description" : "Directory containing output fastq files",
           "default" : [
-            "fastq/*_R1_001.fastq"
-          ],
-          "must_exist" : true,
-          "create_parent" : true,
-          "required" : true,
-          "direction" : "output",
-          "multiple" : true,
-          "multiple_sep" : ";"
-        },
-        {
-          "type" : "file",
-          "name" : "--fastq_output_r2",
-          "description" : "List of demultiplexed fastq files",
-          "default" : [
-            "fastq/*_R2_001.fastq"
+            "fastq/*"
           ],
           "must_exist" : true,
           "create_parent" : true,
@@ -3358,6 +3348,12 @@ meta = [
       "repository" : {
         "type" : "local"
       }
+    },
+    {
+      "name" : "utils/concatRuns",
+      "repository" : {
+        "type" : "local"
+      }
     }
   ],
   "repositories" : [
@@ -3463,14 +3459,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "native|native",
     "output" : "target/nextflow/workflows/htrnaseq",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3479,7 +3477,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3490,11 +3488,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",
@@ -3518,39 +3518,123 @@ include { create_eset } from "${meta.resources_dir}/../../../nextflow/eset/creat
 include { create_fdata } from "${meta.resources_dir}/../../../nextflow/eset/create_fdata/main.nf"
 include { create_pdata } from "${meta.resources_dir}/../../../nextflow/eset/create_pdata/main.nf"
 include { create_report } from "${meta.resources_dir}/../../../nextflow/report/create_report/main.nf"
+include { concatRuns } from "${meta.resources_dir}/../../../nextflow/utils/concatRuns/main.nf"
 
 // inner workflow
 // user-provided Nextflow code
 workflow run_wf {
   take:
-    input_ch
+    raw_ch
 
   main:
+    input_ch = raw_ch
+      // Use the event ID as the default for the sample ID
+      | map {id, state ->
+        def sample_id = state.sample_id ?: id 
+        def newState = state + ["sample_id": sample_id, "run_id": id]
+        return [id, newState]
+      }
+
     // The featureData only has one requirement: the genome annotation.
-    // It can be generated straight away.
+    // It can be generated straight away. Most of the time, there is one shared 
+    // annotation for all of the inputs and the fData should only be calculated once.
+    // The state is manpulated in such a way that there is one event created per unique
+    // input annotation file. In turn, the featureData file can joined into the original input
+    // channel which allows it to be shared across events if required.
     f_data_ch = input_ch
+      | toSortedList()
+      | flatMap {ids_and_states ->
+        def annotation_files = ids_and_states.inject([:]){ old_state, id_and_state ->
+          def (id, state) = id_and_state
+          def annotation_file = state.annotation
+          def new_state = old_state + [(annotation_file): (old_state.getOrDefault(annotation_file, []) + [id])]
+          return new_state
+        }
+        def file_names = annotation_files.keySet().collect{it.name}
+        assert (file_names.toSet().size() == file_names.size()), 
+          "Please make sure that the annotation files have unique file names."
+        def new_states = annotation_files.collect{annotation_file, value ->
+          def new_state = [annotation_file.name , ["annotation": annotation_file, "event_ids": value]]
+          return new_state
+        }
+        return new_states 
+      }
       | create_fdata.run(
         directives: [label: ["lowmem", "lowcpu"]],
         fromState: [
           "gtf": "annotation",
           "output": "f_data"
         ],
-        toState: {id, result, state -> ["f_data": result.output]}
+        toState: ["f_data": "output"]
       )
+      | flatMap {_, state -> 
+          def new_states = state.event_ids.collect{event_id ->
+            [event_id, ["f_data": state.f_data]]
+          }
+          return new_states
+      }
 
     // Perform mapping of each well.
-    mapping_ch = input_ch
+    demultiplex_ch = input_ch
       | well_demultiplex.run(
         fromState: [
             "input_r1": "input_r1",
             "input_r2": "input_r2",
             "barcodesFasta": "barcodesFasta",
         ],
-        toState: [
-          "input_r1": "output_r1",
-          "input_r2": "output_r2",
-        ]
+        toState: {id, result, state ->
+          def all_fastq = result.output_r1 + result.output_r2
+          def output_dir = all_fastq.collect{it.parent}.unique()
+          assert output_dir.size() == 1, "Expected output from well demultiplexing to reside into one directory."
+          def new_state = state + [
+            "input_r1": result.output_r1,
+            "input_r2": result.output_r2,
+            "fastq_output_directory": output_dir[0],
+          ]
+          return new_state
+        }
       )
+
+    fastq_output_directory_ch = demultiplex_ch
+      | map {id, state ->
+        def new_event = [state.sample_id, state]
+        return new_event
+      }
+      | groupTuple(by: 0, sort: "hash")
+      | map {id, states ->
+        def fastq_output_dirs = states.collect{it.fastq_output_directory}
+        def new_state = ["fastq_output_directory": fastq_output_dirs]
+        def new_event = [id, new_state]
+        return [id, new_state]
+      }
+
+
+    concat_samples_ch = demultiplex_ch.join(f_data_ch)
+      | map {id, demutliplex_state, f_data_state ->
+        def newState = demutliplex_state + ["f_data": f_data_state["f_data"]]
+        [id, newState]
+      }
+      | concatRuns.run(
+        fromState: [
+          "input_r1": "input_r1",
+          "input_r2": "input_r2",
+          "sample_id": "sample_id",
+        ],
+        toState: {id, result, state ->
+          def state_overwite = [
+            "input_r1": result.output_r1,
+            "input_r2": result.output_r2,
+            "_meta": ["join_id": state.run_id]
+          ]
+          return state + state_overwite
+        }
+      )
+
+    pool_ch = concat_samples_ch.join(fastq_output_directory_ch)
+      | map {id, demux_state, fastq_output_directory_state ->
+        def new_state = demux_state + fastq_output_directory_state
+        return [id, new_state]
+      } 
       | parallel_map.run(
         directives: ["label": ["highmem", "lowcpu"]],
         fromState: {id, state ->
@@ -3567,9 +3651,6 @@ workflow run_wf {
           "star_output": "output",
         ]
       )
-
-    // From the mapped wells, create statistics based on the BAM files.
-    pool_ch = mapping_ch
       // Split the events from 1 event per pool into events per well
       // and add extra metadata about the wells to the state.
       | well_metadata.run(
@@ -3690,7 +3771,7 @@ workflow run_wf {
         ]
       )
     
-    p_data_ch = star_logs_ch.join(pool_statistics_ch, remainder: true)
+    eset_ch = star_logs_ch.join(pool_statistics_ch, remainder: true)
       | map {id, star_logs_state, pool_statistics_state ->
         def newState = star_logs_state + ["nrReadsNrGenesPerChromPool": pool_statistics_state.nrReadsNrGenesPerChromPool]
         return [id, newState]
@@ -3704,12 +3785,6 @@ workflow run_wf {
         ],
         toState: ["p_data": "output"],
       )
-
-    eset_ch = p_data_ch.join(f_data_ch, remainder: true)
-      | map {id, p_data_state, f_data_state ->
-        def newState = p_data_state + ["f_data": f_data_state["f_data"]]
-        [id, newState]
-      }
       | create_eset.run(
         directives: [label: ["lowmem", "lowcpu"]],
         fromState: [
@@ -3751,13 +3826,14 @@ workflow run_wf {
 
     output_ch = eset_ch.join(report_channel)
       | map {id, state_eset, state_report ->
-        def new_state = state_eset + ["html_report": state_report.html_report]
+        def new_state = state_eset + [
+          "html_report": state_report.html_report,
+        ]
         [id, new_state]
       }
       | setState([
-        "star_output": "star_output", 
-        "fastq_output_r1": "input_r1",
-        "fastq_output_r2": "input_r2",
+        "star_output": "star_output",
+        "fastq_output": "fastq_output_directory",
         "star_output": "star_output",
         "nrReadsNrGenesPerChrom": "nrReadsNrGenesPerChromPool",
         "star_qc_metrics": "star_qc_metrics",
@@ -3765,6 +3841,7 @@ workflow run_wf {
         "f_data": "f_data",
         "p_data": "p_data",
         "html_report": "html_report",
+        "_meta": "_meta",
       ])
 
 

target/nextflow/workflows/htrnaseq/nextflow_schema.json

@@ -18,7 +18,7 @@
                 "type":
                 "string",
                 "description": "Type: List of `file`, required, multiple_sep: `\";\"`. Forward reads in FASTQ format",
-                "help_text": "Type: List of `file`, required, multiple_sep: `\";\"`. Forward reads in FASTQ format. Multiple files can be provided which will\nbe demultiplexed separately before joining the results for each individual well.\n"
+                "help_text": "Type: List of `file`, required, multiple_sep: `\";\"`. Forward reads in FASTQ format. Multiple files corresponding to different lanes can be provided which will\nbe demultiplexed separately before joining the results for each individual well.\n"
             
             }
     
@@ -28,7 +28,7 @@
                 "type":
                 "string",
                 "description": "Type: List of `file`, required, multiple_sep: `\";\"`. Reverse reads in FASTQ format",
-                "help_text": "Type: List of `file`, required, multiple_sep: `\";\"`. Reverse reads in FASTQ format. Multiple files can be provided which will\nbe demultiplexed separately before joining the results for each individual well.\n"
+                "help_text": "Type: List of `file`, required, multiple_sep: `\";\"`. Reverse reads in FASTQ format. Multiple files corresponding to different lanes can be provided which will\nbe demultiplexed separately before joining the results for each individual well.\n"
             
             }
     
@@ -74,6 +74,16 @@
             }
     
 
+        ,
+                "sample_id": {
+                "type":
+                "string",
+                "description": "Type: `string`. Sample ID for the provided input files",
+                "help_text": "Type: `string`. Sample ID for the provided input files. If not provided, the value of --id\nwill be used. Input files will allways be demultiplexed separately,\nbut the FASTQs for wells with matching sample IDs will be concatenated before mapping.\n"
+            
+            }
+    
+
 }
 },
     
@@ -85,24 +95,13 @@
     "properties": {
     
         
-                "fastq_output_r1": {
+                "fastq_output": {
                 "type":
                 "string",
-                "description": "Type: List of `file`, required, default: `$id.$key.fastq_output_r1_*.fastq`, multiple_sep: `\";\"`. List of demultiplexed fastq files",
-                "help_text": "Type: List of `file`, required, default: `$id.$key.fastq_output_r1_*.fastq`, multiple_sep: `\";\"`. List of demultiplexed fastq files"
+                "description": "Type: List of `file`, required, default: `fastq/*`, multiple_sep: `\";\"`. Directory containing output fastq files",
+                "help_text": "Type: List of `file`, required, default: `fastq/*`, multiple_sep: `\";\"`. Directory containing output fastq files"
             ,
-                "default":"$id.$key.fastq_output_r1_*.fastq"
-            }
-    
-
-        ,
-                "fastq_output_r2": {
-                "type":
-                "string",
-                "description": "Type: List of `file`, required, default: `$id.$key.fastq_output_r2_*.fastq`, multiple_sep: `\";\"`. List of demultiplexed fastq files",
-                "help_text": "Type: List of `file`, required, default: `$id.$key.fastq_output_r2_*.fastq`, multiple_sep: `\";\"`. List of demultiplexed fastq files"
-            ,
-                "default":"$id.$key.fastq_output_r2_*.fastq"
+                "default":"fastq/*"
             }
     
 
@@ -110,10 +109,10 @@
                 "star_output": {
                 "type":
                 "string",
-                "description": "Type: List of `file`, required, default: `$id.$key.star_output_*.$id/*`, multiple_sep: `\";\"`. Output from mapping with STAR",
-                "help_text": "Type: List of `file`, required, default: `$id.$key.star_output_*.$id/*`, multiple_sep: `\";\"`. Output from mapping with STAR"
+                "description": "Type: List of `file`, required, default: `star.$id/*`, multiple_sep: `\";\"`. Output from mapping with STAR",
+                "help_text": "Type: List of `file`, required, default: `star.$id/*`, multiple_sep: `\";\"`. Output from mapping with STAR"
             ,
-                "default":"$id.$key.star_output_*.$id/*"
+                "default":"star.$id/*"
             }
     
 
@@ -121,10 +120,10 @@
                 "nrReadsNrGenesPerChrom": {
                 "type":
                 "string",
-                "description": "Type: `file`, required, default: `$id.$key.nrReadsNrGenesPerChrom.txt`. ",
-                "help_text": "Type: `file`, required, default: `$id.$key.nrReadsNrGenesPerChrom.txt`. "
+                "description": "Type: `file`, required, default: `nrReadsNrGenesPerChrom.$id.txt`. ",
+                "help_text": "Type: `file`, required, default: `nrReadsNrGenesPerChrom.$id.txt`. "
             ,
-                "default":"$id.$key.nrReadsNrGenesPerChrom.txt"
+                "default":"nrReadsNrGenesPerChrom.$id.txt"
             }
     
 
@@ -132,10 +131,10 @@
                 "star_qc_metrics": {
                 "type":
                 "string",
-                "description": "Type: `file`, required, default: `$id.$key.star_qc_metrics.txt`. ",
-                "help_text": "Type: `file`, required, default: `$id.$key.star_qc_metrics.txt`. "
+                "description": "Type: `file`, required, default: `starLogs.$id.txt`. ",
+                "help_text": "Type: `file`, required, default: `starLogs.$id.txt`. "
             ,
-                "default":"$id.$key.star_qc_metrics.txt"
+                "default":"starLogs.$id.txt"
             }
     
 
@@ -143,10 +142,10 @@
                 "eset": {
                 "type":
                 "string",
-                "description": "Type: `file`, required, default: `$id.$key.eset.rds`. ",
-                "help_text": "Type: `file`, required, default: `$id.$key.eset.rds`. "
+                "description": "Type: `file`, required, default: `eset.$id.rds`. ",
+                "help_text": "Type: `file`, required, default: `eset.$id.rds`. "
             ,
-                "default":"$id.$key.eset.rds"
+                "default":"eset.$id.rds"
             }
     
 
@@ -154,10 +153,10 @@
                 "f_data": {
                 "type":
                 "string",
-                "description": "Type: `file`, required, default: `$id.$key.f_data.tsv`. ",
-                "help_text": "Type: `file`, required, default: `$id.$key.f_data.tsv`. "
+                "description": "Type: `file`, required, default: `fData.$id.tsv`. ",
+                "help_text": "Type: `file`, required, default: `fData.$id.tsv`. "
             ,
-                "default":"$id.$key.f_data.tsv"
+                "default":"fData.$id.tsv"
             }
     
 
@@ -165,10 +164,10 @@
                 "p_data": {
                 "type":
                 "string",
-                "description": "Type: `file`, required, default: `$id.$key.p_data.tsv`. ",
-                "help_text": "Type: `file`, required, default: `$id.$key.p_data.tsv`. "
+                "description": "Type: `file`, required, default: `pData.$id.tsv`. ",
+                "help_text": "Type: `file`, required, default: `pData.$id.tsv`. "
             ,
-                "default":"$id.$key.p_data.tsv"
+                "default":"pData.$id.tsv"
             }
     
 
@@ -176,10 +175,10 @@
                 "html_report": {
                 "type":
                 "string",
-                "description": "Type: `file`, required, default: `$id.$key.html_report.html`. ",
-                "help_text": "Type: `file`, required, default: `$id.$key.html_report.html`. "
+                "description": "Type: `file`, required, default: `report.$id.html`. ",
+                "help_text": "Type: `file`, required, default: `report.$id.html`. "
             ,
-                "default":"$id.$key.html_report.html"
+                "default":"report.$id.html"
             }
     
 

target/nextflow/workflows/runner/.config.vsh.yaml

@@ -12,7 +12,7 @@ argument_groups:
     create_parent: true
     required: true
     direction: "input"
-    multiple: true
+    multiple: false
     multiple_sep: ";"
   - type: "file"
     name: "--barcodesFasta"
@@ -220,9 +220,10 @@ build_info:
   engine: "native|native"
   output: "target/nextflow/workflows/runner"
   executable: "target/nextflow/workflows/runner/main.nf"
-  viash_version: "0.9.2"
-  git_commit: "d157606b49b157cd2955acf9124f9043fbd0ca5a"
+  viash_version: "0.9.4"
+  git_commit: "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e"
   git_remote: "https://github.com/viash-hub/htrnaseq"
+  git_tag: "v0.7.2-6-gf1f0c98"
   dependencies:
   - "target/nextflow/utils/listInputDir"
   - "target/nextflow/workflows/htrnaseq"
@@ -231,12 +232,28 @@ build_info:
 package_config:
   name: "htrnaseq"
   version: "update-resources"
-  description: "High-throughput pipeline [WIP]\n"
+  summary: "A workflow for high-throughput RNA-seq data analyses.\n"
+  description: "This workflow is designed to process high-throughput RNA-seq data,\
+    \ where every\nwell of a microarray plate is a sample. A fasta file provided as\
+    \ input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow\
+    \ is built in a modular fashion, where most of the base functionality\nis provided\
+    \ by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\n\
+    supplemented by custom base components and workflow components in this package.\n\
+    \nThe full workflow is split in two major subworkflows that can be run independently:\n\
+    \n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per\
+    \ well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate\
+    \ QC reports.\n\nEach of those can be started individually, or the full workflow\
+    \ can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq)\
+    \ \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner)\
+    \ where a\nnumber of choices (input/output structure and location) have been made.\n\
+    \nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other\
+    \ formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex)\
+    \ first.\n"
   info:
     test_resources:
     - path: "gs://viash-hub-resources/htrnaseq/v1"
       dest: "resources_test"
-  viash_version: "0.9.2"
+  viash_version: "0.9.4"
   source: "src"
   target: "target"
   config_mods:
@@ -248,11 +265,13 @@ package_config:
   - ".engines[.type == 'docker'].target_tag := 'update-resources'"
   keywords:
   - "bioinformatics"
-  - "sequence"
+  - "sequencing"
   - "high-throughput"
+  - "RNAseq"
   - "mapping"
   - "counting"
   - "pipeline"
+  - "workflow"
   license: "MIT"
   organization: "vsh"
   links:

target/nextflow/workflows/runner/main.nf

@@ -1,6 +1,6 @@
 // runner update-resources
 // 
-// This wrapper script is auto-generated by viash 0.9.2 and is thus a derivative
+// This wrapper script is auto-generated by viash 0.9.4 and is thus a derivative
 // work thereof. This software comes with ABSOLUTELY NO WARRANTY from Data
 // Intuitive.
 // 
@@ -82,64 +82,56 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
       foundClass = "List[${e.foundClass}]"
     }
   } else if (par.type == "string") {
-    // cast to string if need be
+    // cast to string if need be. only cast if the value is a GString
     if (value instanceof GString) {
-      value = value.toString()
+      value = value as String
     }
     expectedClass = value instanceof String ? null : "String"
   } else if (par.type == "integer") {
     // cast to integer if need be
-    if (value instanceof String) {
+    if (value !instanceof Integer) {
       try {
-        value = value.toInteger()
+        value = value as Integer
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Integer"
       }
     }
-    if (value instanceof java.math.BigInteger) {
-      value = value.intValue()
-    }
-    expectedClass = value instanceof Integer ? null : "Integer"
   } else if (par.type == "long") {
     // cast to long if need be
-    if (value instanceof String) {
+    if (value !instanceof Long) {
       try {
-        value = value.toLong()
+        value = value as Long
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Long"
       }
     }
-    if (value instanceof Integer) {
-      value = value.toLong()
-    }
-    expectedClass = value instanceof Long ? null : "Long"
   } else if (par.type == "double") {
     // cast to double if need be
-    if (value instanceof String) {
+    if (value !instanceof Double) {
       try {
-        value = value.toDouble()
+        value = value as Double
       } catch (NumberFormatException e) {
-        // do nothing
+        expectedClass = "Double"
       }
     }
-    if (value instanceof java.math.BigDecimal) {
-      value = value.doubleValue()
+  } else if (par.type == "float") {
+    // cast to float if need be
+    if (value !instanceof Float) {
+      try {
+        value = value as Float
+      } catch (NumberFormatException e) {
+        expectedClass = "Float"
+      }
     }
-    if (value instanceof Float) {
-      value = value.toDouble()
-    }
-    expectedClass = value instanceof Double ? null : "Double"
   } else if (par.type == "boolean" | par.type == "boolean_true" | par.type == "boolean_false") {
     // cast to boolean if need be
-    if (value instanceof String) {
-      def valueLower = value.toLowerCase()
-      if (valueLower == "true") {
-        value = true
-      } else if (valueLower == "false") {
-        value = false
+    if (value !instanceof Boolean) {
+      try {
+        value = value as Boolean
+      } catch (Exception e) {
+        expectedClass = "Boolean"
       }
     }
-    expectedClass = value instanceof Boolean ? null : "Boolean"
   } else if (par.type == "file" && (par.direction == "input" || stage == "output")) {
     // cast to path if need be
     if (value instanceof String) {
@@ -151,10 +143,13 @@ def _checkArgumentType(String stage, Map par, Object value, String errorIdentifi
     expectedClass = value instanceof Path ? null : "Path"
   } else if (par.type == "file" && stage == "input" && par.direction == "output") {
     // cast to string if need be
-    if (value instanceof GString) {
-      value = value.toString()
+    if (value !instanceof String) {
+      try {
+        value = value as String
+      } catch (Exception e) {
+        expectedClass = "String"
+      }
     }
-    expectedClass = value instanceof String ? null : "String"
   } else {
     // didn't find a match for par.type
     expectedClass = par.type
@@ -3050,7 +3045,7 @@ meta = [
           "create_parent" : true,
           "required" : true,
           "direction" : "input",
-          "multiple" : true,
+          "multiple" : false,
           "multiple_sep" : ";"
         },
         {
@@ -3316,14 +3311,16 @@ meta = [
     "runner" : "nextflow",
     "engine" : "native|native",
     "output" : "target/nextflow/workflows/runner",
-    "viash_version" : "0.9.2",
-    "git_commit" : "d157606b49b157cd2955acf9124f9043fbd0ca5a",
-    "git_remote" : "https://github.com/viash-hub/htrnaseq"
+    "viash_version" : "0.9.4",
+    "git_commit" : "f1f0c98bba1e84f0608978a576a49f2cc8a9aa3e",
+    "git_remote" : "https://github.com/viash-hub/htrnaseq",
+    "git_tag" : "v0.7.2-6-gf1f0c98"
   },
   "package_config" : {
     "name" : "htrnaseq",
     "version" : "update-resources",
-    "description" : "High-throughput pipeline [WIP]\n",
+    "summary" : "A workflow for high-throughput RNA-seq data analyses.\n",
+    "description" : "This workflow is designed to process high-throughput RNA-seq data, where every\nwell of a microarray plate is a sample. A fasta file provided as input\ndefines the mapping between sample barcodes and wells.\n\nThe workflow is built in a modular fashion, where most of the base functionality\nis provided by components from [`biobox`](https://www.viash-hub.com/packages/biobox/latest)\nsupplemented by custom base components and workflow components in this package.\n\nThe full workflow is split in two major subworkflows that can be run independently:\n\n* **Well-demultiplexing:** Split the input (plate/pool level) fastq files per well.\n* **Mapping, counting and QC:** Run per-well mapping, counting and generate QC reports.\n\nEach of those can be started individually, or the full workflow can be run in two ways:\n\n1. Run the [main workflow](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/htrnaseq) \ncontaining the main functionality.\n2. Run the [(opinionated) `runner`](https://www.viash-hub.com/packages/htrnaseq/v0.3.0/components/workflows/runner) where a\nnumber of choices (input/output structure and location) have been made.\n\nInput for the workflow has to be `fastq` files (zipped or not). For bcl or other formats, please consider running\n[demultiplex](https://www.viash-hub.com/packages/demultiplex) first.\n",
     "info" : {
       "test_resources" : [
         {
@@ -3332,7 +3329,7 @@ meta = [
         }
       ]
     },
-    "viash_version" : "0.9.2",
+    "viash_version" : "0.9.4",
     "source" : "src",
     "target" : "target",
     "config_mods" : [
@@ -3343,11 +3340,13 @@ meta = [
     ],
     "keywords" : [
       "bioinformatics",
-      "sequence",
+      "sequencing",
       "high-throughput",
+      "RNAseq",
       "mapping",
       "counting",
-      "pipeline"
+      "pipeline",
+      "workflow"
     ],
     "license" : "MIT",
     "organization" : "vsh",
@@ -3378,19 +3377,13 @@ workflow run_wf {
     input_ch
 
   main:
-    output_ch = input_ch
-      // Multiple runs can be provided, and the reads for these runs will
-      // be concatenated. Here, we gather the FASTQ files from each input directory first.
-      | flatMap {id, state ->
-        // Create an input event per input directory
-        def new_state = state.input.withIndex().collect{input_dir, id_index ->
-          def state_item = state + ["input": input_dir, "index": id_index, "run_id": id]
-          return ["${id}_${id_index}".toString(), state_item]
-        }
-        return new_state
-      }
+    htrnaseq_ch = input_ch
       // List the FASTQ files per input directory
       // Be careful: an event per lane is created!
+      | map {id, state ->
+        def new_state = state + ["run_id": id]
+        return [id, new_state]
+      }
       | listInputDir.run(
         fromState: [
           "input": "input",
@@ -3408,13 +3401,11 @@ workflow run_wf {
       // there might be multiple FASTQs for a single sample that correspond to the
       // lanes. So the fastq files must be gathered across lanes and input folders
       // in order to create an input lists for R1 and R2.
-      | map {id, state -> [state.sample_id, state]}
-      | groupTuple(by: 0, sort: { state1, state2 ->
-        if (state1.index == state2.index) {
-          return state1.lane <=> state2.lane
-        }
-        return state1.index <=> state2.index
-      })
+      // The ID of the event here is important! It determines the name of the output
+      // folders for the FASTQ files and these folders are published as-is later.
+      // The folder where the FASTQ files are stored in should be named after the run ID.
+      | map {id, state -> ["${state.sample_id}/${state.run_id}".toString(), state]}
+      | groupTuple(by: 0, sort: "hash")
       | map {id, states ->
         def new_r1 = states.collect{it.r1_output}
         def new_r2 = states.collect{it.r2_output}
@@ -3423,7 +3414,7 @@ workflow run_wf {
         // TODO: this can be asserted.
         def new_state = states[0] + [
           "r1": new_r1,
-          "r2": new_r2
+          "r2": new_r2,
         ]
         return [id, new_state]
       }
@@ -3432,8 +3423,7 @@ workflow run_wf {
           f_data: 'fData/$id.txt',
           p_data: 'pData/$id.txt',
           star_output: 'star_output/$id/*',
-          fastq_output_r1: 'fastq/*_R1_001.fastq',
-          fastq_output_r2: 'fastq/*_R1_001.fastq',
+          fastq_output: 'fastq/*',
           eset: 'esets/$id.rds',
           nrReadsNrGenesPerChrom: 'nrReadsNrGenesPerChrom/$id.txt',
           star_qc_metrics: 'starLogs/$id.txt',
@@ -3446,32 +3436,32 @@ workflow run_wf {
           genomeDir: "genomeDir",
           annotation: "annotation",
           umi_length: "umi_length",
+          sample_id: "sample_id",
         ],
         toState: { id, result, state -> state + result }
       )
+
       // The HT-RNAseq workflow outputs multiple events, one per 'pool' (usually a plate)
       // but for publishing the results, this is not handy because we want to use the $id
       // variable as a pointer to the target data.
       //
       // So, we should combine everything together
       //
-      // project_id / experiment_id / date_workflow
-
+      // project_id / experiment_id / "data_processed" / date_workflow
+    grouped_ch = htrnaseq_ch
       | toSortedList
-
       | map{ vs ->
           def all_fastqs
           [
             vs[0][1].run_id, // The original ID
             [
               star_output: reduce_paths(vs.collect{ it[1].star_output }.flatten()),
-              fastq_output_r1: reduce_paths(vs.collect{ it[1].fastq_output_r1 }.flatten(), 1),
-              fastq_output_r2: reduce_paths(vs.collect{ it[1].fastq_output_r2 }.flatten(), 1),
               nrReadsNrGenesPerChrom: reduce_paths(vs.collect{ it[1].nrReadsNrGenesPerChrom }),
               star_qc_metrics: reduce_paths(vs.collect{ it[1].star_qc_metrics }),
               eset: reduce_paths(vs.collect{ it[1].eset }),
               f_data: reduce_paths(vs.collect{ it[1].f_data }),
               p_data: reduce_paths(vs.collect{ it[1].p_data }),
+              fastq_output: vs.collect{ it[1].fastq_output }.flatten().unique(),
               html_report: vs.collect{ it[1].html_report }[0],  // The report is for all pools
               plain_output: vs.collect{ it[1].plain_output }[0],
               project_id: vs.collect{ it[1].project_id }[0],
@@ -3480,12 +3470,13 @@ workflow run_wf {
           ]
         }
 
+    results_publish_ch = grouped_ch
       | publish_results.run(
         fromState: { id, state ->
           def project = (state.plain_output) ? id : "${state.project_id}"
           def experiment = (state.plain_output) ? id : "${state.experiment_id}"
           def id0 = "${project}/${experiment}"
-          def id1 = (state.plain_output) ? id : "${id0}/${date}"
+          def id1 = (state.plain_output) ? id : "${id0}/data_processed/${date}"
           def id2 = (state.plain_output) ? id : "${id1}_htrnaseq_${version}"
 
           if (id == id2) {
@@ -3516,14 +3507,24 @@ workflow run_wf {
         ]
       )
 
+    fastq_publish_ch = grouped_ch
+      | flatMap{id, state ->
+        def new_states = state.fastq_output.collect{fastq_dir -> 
+          def new_id = fastq_dir.name // The folder name corresponds to the run
+          def fastq_files = fastq_dir.listFiles()
+          def new_state = [
+            "fastq_output": fastq_files
+          ]
+          return [new_id, new_state]
+        }
+        return new_states
+      }
       | publish_fastqs.run(
         fromState: { id, state ->
           def id0 = "${id}"
           def id1 = (state.plain_output) ? id : "${id0}/${date}"
           def id2 = (state.plain_output) ? id : "${id1}_htrnaseq_${version}"
 
-          println(state.plain_output)
-
           if (id == id2) {
             println("Publising fastqs to ${params.fastq_publish_dir}")
           } else {
@@ -3531,8 +3532,7 @@ workflow run_wf {
           }
 
           [
-            input_r1: state.fastq_output_r1,
-            input_r2: state.fastq_output_r2,
+            input: state.fastq_output,
             output: "${id2}",
           ]
         },
@@ -3547,7 +3547,7 @@ workflow run_wf {
       )
 
   emit:
-    output_ch
+    grouped_ch
       | map{ id, state -> [ id, [ _meta: [ join_id: state.run_id ] ] ] }
 }