vsh/demultiplex
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Build branch main with version main (f591b78)

Browse Source 
Build pipeline: viash-hub.demultiplex.main-p5t6v

Source commit: f591b7849f

Source message: Output demultiplexer log files (#41)
This commit is contained in:
CI
2025-05-14 08:38:23 +00:00
parent 9660348996
commit b720ad07f9
40 changed files with 759 additions and 105 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
CHANGELOG.md
View File

@@ -16,6 +16,8 @@
* Bump viash to 0.9.4, which adds support for nextflow versions starting major version 25.01 (PR #43 and #44).
* Output demultiplexer logs and metrics (PR #41).
# demultiplex v0.3.8
## Bug fixes

203
README.md
View File

@@ -1,6 +1,10 @@
# Demultiplex.vsh
Demultiplex.vsh is a workflow for demultiplexing of raw sequencing data. Currently data from Illumina and Element Biosciences sequencers are supported.
Demultiplex.vsh is a workflow for demultiplexing of raw sequencing data.
Currently data from Illumina and Element Biosciences sequencers are
supported.
[![ViashHub](https://img.shields.io/badge/ViashHub-demultiplex-7a4baa.svg)](https://web.viash-hub.com/packages/demultiplex)
[![GitHub](https://img.shields.io/badge/GitHub-viash--hub%2Fdemultiplex-blue.svg)](https://github.com/viash-hub/demultiplex)
@@ -11,90 +15,205 @@ Issues](https://img.shields.io/github/issues/viash-hub/demultiplex.svg)](https:/
[![Viash
version](https://img.shields.io/badge/Viash-v0.9.4-blue)](https://viash.io)
## Introcuction
This workflow is designed to demultiplex raw RNA-seq sequencing data
from Illumina and Element Biosciences sequencers.
The workflow is built in a modular fashion, where most of the base
functionality is provided by components from
[`biobox`](https://www.viash-hub.com/packages/biobox/latest)
supplemented by custom base components and workflow components in this
package. Each of these components can be used independently as
stand-alone modules with a standardized interface.
The full workflow can be run in two ways:
1. Run the [main
workflow](https://www.viash-hub.com/packages/demultiplex/v0.3.4/components/demultiplex)
containing the main functionality.
2. Run the [(opinianated)
`runner`](https://www.viash-hub.com/packages/demultiplex/v0.3.4/components/runner)
where a number of choices (input/output structure and location) have
been made.
## Workflow Overview
The workflow executes the following steps:
1. Unpacking the input data (when a TAR archive is provided)
2. Run `bclconvert` or `bases2fastq`
3. Run `falco` and convert Illumina InterOp information to csv
4. Run `multiqc` to generate a report
## Usage
## Example usage
Two variants of the same workflow are provided, depending on the flexibility in the ouput structure required:
Two variants of the same workflow are provided, depending on the
flexibility in the ouput structure required:
* The `runner` workflow provides a predifined output structure. It requires the minimal amount of parameters to be provided, at the cost of being less flexible. It is located at `target/nextflow/runner/main.nf`
* The `demultiplex` workflow (`target/nextflow/demultiplex/main.nf`) allows for more fine-grained tuning, but required more parameters to be provided.
- The `runner` workflow provides a predifined output structure. It
requires the minimal amount of parameters to be provided, at the cost
of being less flexible. It is located at
`target/nextflow/runner/main.nf`
- The `demultiplex` workflow (`target/nextflow/demultiplex/main.nf`)
allows for more fine-grained tuning, but required more parameters to
be provided.
### Test data
We have provided test data at `gs://viash-hub-test-data/demultiplex/v3/demultiplex_htrnaseq_meta/SingleCell-RNA_P3_2`, but please feel free to bring your own. The URL of the test data can be provided as-is to the workflow, or you can download everything and specify a local path.
We have provided test data at
`gs://viash-hub-resources/demultiplex/v3/demultiplex_htrnaseq_meta/SingleCell-RNA_P3_2`
(Illumina), but please feel free to bring your own. The URL of the test
data can be provided as-is to the workflow, or you can download
everything and specify a local path.
The input data should follow the structure of either Illumina or Element
Biosciences sequencers. The workflow will automatically detect which
demultiplexer to use (`bclconvert` or `bases2fastq`) based on the
presence of either `SampleSheet.csv` or `RunParameters.xml` in the input
directory. Demultiplexer can also be set explicitly using the
`--demultiplexer` parameter.
### Setup
In order to use the workflows in this package, you'll need to do the following:
* Install [nextflow](https://www.nextflow.io/docs/latest/install.html)
* Install a nextflow compatible executor. This workflow provides a profile for [docker](https://docs.docker.com/get-started/).
In order to use the workflows in this package, you’ll need to do the
following:
- Install [nextflow](https://www.nextflow.io/docs/latest/install.html)
- Install a nextflow compatible executor. This workflow provides a
profile for [docker](https://docs.docker.com/get-started/).
### Run from Viash Hub
1. Open [Viash Hub](https://www.viash-hub.com) and browse to the
[demultiplex
component](https://www.viash-hub.com/packages/demultiplex/v0.3.4/components/demultiplex).
Press the ‘Launch’ button and follow the instructions.
![](assets/demultiplex-launch-small.png)
2. We will start an example run and set profile to `docker`.
![](assets/demultiplex-launch-parameters-1.png)
3. In the next step, we provide the paramters as follows and leave the
rest as defalut:
- `input`:
`gs://viash-hub-resources/demultiplex/v3/demultiplex_htrnaseq_meta/SingleCell-RNA_P3_2`
![](assets/demultiplex-launch-parameters-2.png)
Press the ‘Launch’ button at the end to get the instructions on how to
run the workflow from the CLI.
### Run using NF-Tower / Seqera Cloud
It’s possible to run the workflow directly from [Seqera
Cloud](https://cloud.seqera.io). The necessary [Nextflow schema
file](https://nextflow-io.github.io/nf-schema/latest/nextflow_schema/nextflow_schema_specification/)
has been built and provided with the workflows in order to use the
form-based input.
1. Select the option to run the workflow using Seqera Cloud. You will
need to create an API token for your account. Once this token is
filled in in the corresponding field, we will get the option to
select a ‘Workspace’ and a ‘Compute environment’.
![](assets/demultiplex-launch-parameters-3.png)
2. Provide the parameters similar to the previous step.
3. In the next screen, pressing the ‘Launch’ button will actually start
the workflow on Seqera Cloud. A message is shown when the submit was
successful.
![](assets/demultiplex-launch-parameters-4.png)
### Setting up SCM
In order to let nextflow use the viash-hub workflows, you need to setup a [SCM](https://www.nextflow.io/docs/latest/git.html#git-configuration) file. This can be done once by creating `$HOME/.nextflow/scm` and adding the following:
```
providers {
In order to let nextflow use the viash-hub workflows, you need to setup
a [SCM](https://www.nextflow.io/docs/latest/git.html#git-configuration)
file. This can be done once by creating `$HOME/.nextflow/scm` and adding
the following:
providers {
vsh {
platform = 'gitlab'
server = "packages.viash-hub.com"
}
}
```
}
Alternatively, a custom location for the SCM file can be specified using the `NXF_SCM_FILE` environment variable.
Alternatively, a custom location for the SCM file can be specified using
the `NXF_SCM_FILE` environment variable.
You can check if everything is working by getting the `--help` for a workflow:
```bash
You can check if everything is working by getting the `--help` for a
workflow:
``` bash
nextflow run \
vsh/demultiplex \
-r v0.3.9 \
--help
```
### (Optional) Resource usage tuning
### Run from the CLI
Nextflow's labels can be used to specify the amount of resources a process can use. This workflow uses the following labels for CPU and memory:
* `verylowmem`, `lowmem`, `midmem`, `highmem`
* `verylowcpu`, `lowcpu`, `midcpu`, `highcpu`
Running from the CLI directly without using Viash hub is possible as
well. The easiest is to use the integrated help functionality, for
instance using the following:
The defaults for these labels can be found at `src/config/labels.config`. Nextflow checks that the specified resources for a process do not exceed what is available on the machine and will not start if it does. Create your own config file to tune the labels to your needs, for example:
```
// Resource labels
withLabel: verylowcpu { cpus = 2 }
withLabel: lowcpu { cpus = 8 }
withLabel: midcpu { cpus = 16 }
withLabel: highcpu { cpus = 16 }
withLabel: verylowmem { memory = 4.GB }
withLabel: lowmem { memory = 8.GB }
withLabel: midmem { memory = 8.GB }
withLabel: highmem { memory = 8.GB }
``` bash
nextflow run vsh/demultiplex \
-revision v0.3.9 \
-main-script target/nextflow/workflows/runner/main.nf \
--help
```
When starting nextflow using the CLI, you can use `-c` to provide the file to nextflow and overwrite the defaults.
Having this project available locally, you can run the following
command:
### Example
```bash
``` bash
nextflow run vsh/demultiplex \
-r v0.3.9 \
-main-script target/nextflow/runner/main.nf \
--input "gs://viash-hub-test-data/demultiplex/v3/demultiplex_htrnaseq_meta/SingleCell-RNA_P3_2" \
--input "gs://viash-hub-resources/demultiplex/v3/demultiplex_htrnaseq_meta/SingleCell-RNA_P3_2" \
--demultiplexer bclconvert \
--skip_copycomplete_check \
--publish_dir example_output/ \
-profile docker \
-c labels.config
-c src/config/labels.config
```
### (Optional) Resource usage tuning
Nextflow’s labels can be used to specify the amount of resources a
process can use. This workflow uses the following labels for CPU and
memory:
- `verylowmem`, `lowmem`, `midmem`, `highmem`
- `verylowcpu`, `lowcpu`, `midcpu`, `highcpu`
The defaults for these labels can be found at
`src/config/labels.config`. Nextflow checks that the specified resources
for a process do not exceed what is available on the machine and will
not start if it does. Create your own config file to tune the labels to
your needs, for example:
// Resource labels
withLabel: verylowcpu { cpus = 2 }
withLabel: lowcpu { cpus = 8 }
withLabel: midcpu { cpus = 16 }
withLabel: highcpu { cpus = 16 }
withLabel: verylowmem { memory = 4.GB }
withLabel: lowmem { memory = 8.GB }
withLabel: midmem { memory = 8.GB }
withLabel: highmem { memory = 8.GB }
When starting nextflow using the CLI, you can use `-c` to provide the
file to nextflow and overwrite the defaults.
## Acknowledgements
Developed in collaboration with Data Intuitive and Open Analytics.

191
README.qmd Normal file
View File

@@ -0,0 +1,191 @@
---
format: gfm
---
```{r setup, include=FALSE}
project <- yaml::read_yaml("_viash.yaml")
license <- paste0(project$links$repository, "/blob/main/LICENSE")
```
# Demultiplex.vsh
Demultiplex.vsh is a workflow for demultiplexing of raw sequencing data. Currently data from Illumina and Element Biosciences sequencers are supported.
[![ViashHub](https://img.shields.io/badge/ViashHub-demultiplex-7a4baa.svg)](https://web.viash-hub.com/packages/demultiplex)
[![GitHub](https://img.shields.io/badge/GitHub-viash--hub%2Fdemultiplex-blue.svg)](https://github.com/viash-hub/demultiplex)
[![GitHub
License](https://img.shields.io/github/license/viash-hub/demultiplex.svg)](https://github.com/viash-hub/demultiplex/blob/main/LICENSE)
[![GitHub
Issues](https://img.shields.io/github/issues/viash-hub/demultiplex.svg)](https://github.com/viash-hub/demultiplex/issues)
[![Viash
version](https://img.shields.io/badge/Viash-v0.9.4-blue)](https://viash.io)
## Introcuction
This workflow is designed to demultiplex raw RNA-seq sequencing data from Illumina and Element Biosciences sequencers.
The workflow is built in a modular fashion, where most of the base functionality is provided by components from
[`biobox`](https://www.viash-hub.com/packages/biobox/latest) supplemented by custom base components and workflow components in this package. Each of these components can be used independently as stand-alone modules with a
standardized interface.
The full workflow can be run in two ways:
1. Run the [main
workflow](https://www.viash-hub.com/packages/demultiplex/v0.3.4/components/demultiplex)
containing the main functionality.
2. Run the [(opinianated)
`runner`](https://www.viash-hub.com/packages/demultiplex/v0.3.4/components/runner)
where a number of choices (input/output structure and location) have
been made.
## Workflow Overview
The workflow executes the following steps:
1. Unpacking the input data (when a TAR archive is provided)
2. Run `bclconvert` or `bases2fastq`
3. Run `falco` and convert Illumina InterOp information to csv
4. Run `multiqc` to generate a report
## Example usage
Two variants of the same workflow are provided, depending on the flexibility in the ouput structure required:
* The `runner` workflow provides a predifined output structure. It requires the minimal amount of parameters to be provided, at the cost of being less flexible. It is located at `target/nextflow/runner/main.nf`
* The `demultiplex` workflow (`target/nextflow/demultiplex/main.nf`) allows for more fine-grained tuning, but required more parameters to be provided.
### Test data
We have provided test data at `gs://viash-hub-resources/demultiplex/v3/demultiplex_htrnaseq_meta/SingleCell-RNA_P3_2` (Illumina), but please feel free to bring your own. The URL of the test data can be provided as-is to the workflow, or you can download everything and specify a local path.
The input data should follow the structure of either Illumina or Element Biosciences sequencers. The workflow will automatically detect which demultiplexer to use (`bclconvert` or `bases2fastq`) based on the
presence of either `SampleSheet.csv` or `RunParameters.xml` in the input directory. Demultiplexer can also be set explicitly using the `--demultiplexer` parameter.
### Setup
In order to use the workflows in this package, you'll need to do the following:
* Install [nextflow](https://www.nextflow.io/docs/latest/install.html)
* Install a nextflow compatible executor. This workflow provides a profile for [docker](https://docs.docker.com/get-started/).
### Run from Viash Hub
1. Open [Viash Hub](https://www.viash-hub.com) and browse to the [demultiplex
component](https://www.viash-hub.com/packages/demultiplex/v0.3.4/components/demultiplex).
Press the ‘Launch’ button and follow the instructions.
![](assets/demultiplex-launch-small.png)
2. We will start an example run and set profile to `docker`.
![](assets/demultiplex-launch-parameters-1.png)
3. In the next step, we provide the paramters as follows and leave the rest as defalut:
- `input`:
`gs://viash-hub-resources/demultiplex/v3/demultiplex_htrnaseq_meta/SingleCell-RNA_P3_2`
![](assets/demultiplex-launch-parameters-2.png)
Press the ‘Launch’ button at the end to get the instructions on how to
run the workflow from the CLI.
### Run using NF-Tower / Seqera Cloud
It’s possible to run the workflow directly from [Seqera
Cloud](https://cloud.seqera.io). The necessary [Nextflow schema
file](https://nextflow-io.github.io/nf-schema/latest/nextflow_schema/nextflow_schema_specification/)
has been built and provided with the workflows in order to use the
form-based input.
1. Select the option to run the workflow using Seqera Cloud. You
will need to create an API token for your account. Once this token is
filled in in the corresponding field, we will get the option to select
a ‘Workspace’ and a ‘Compute environment’.
![](assets/demultiplex-launch-parameters-3.png)
2. Provide the parameters similar to the previous step.
3. In the next screen, pressing the ‘Launch’ button will actually start the
workflow on Seqera Cloud. A message is shown when the submit was
successful.
![](assets/demultiplex-launch-parameters-4.png)
### Setting up SCM
In order to let nextflow use the viash-hub workflows, you need to setup a [SCM](https://www.nextflow.io/docs/latest/git.html#git-configuration) file. This can be done once by creating `$HOME/.nextflow/scm` and adding the following:
```
providers {
vsh {
platform = 'gitlab'
server = "packages.viash-hub.com"
}
}
```
Alternatively, a custom location for the SCM file can be specified using the `NXF_SCM_FILE` environment variable.
You can check if everything is working by getting the `--help` for a workflow:
```bash
nextflow run \
vsh/demultiplex \
-r v0.3.9 \
--help
```
### Run from the CLI
Running from the CLI directly without using Viash hub is possible as well. The
easiest is to use the integrated help functionality, for instance
using the following:
``` bash
nextflow run vsh/demultiplex \
-revision v0.3.9 \
-main-script target/nextflow/workflows/runner/main.nf \
--help
```
Having this project available locally, you can run the following command:
```bash
nextflow run vsh/demultiplex \
-r v0.3.9 \
-main-script target/nextflow/runner/main.nf \
--input "gs://viash-hub-resources/demultiplex/v3/demultiplex_htrnaseq_meta/SingleCell-RNA_P3_2" \
--demultiplexer bclconvert \
--skip_copycomplete_check \
--publish_dir example_output/ \
-profile docker \
-c src/config/labels.config
```
### (Optional) Resource usage tuning
Nextflow's labels can be used to specify the amount of resources a process can use. This workflow uses the following labels for CPU and memory:
* `verylowmem`, `lowmem`, `midmem`, `highmem`
* `verylowcpu`, `lowcpu`, `midcpu`, `highcpu`
The defaults for these labels can be found at `src/config/labels.config`. Nextflow checks that the specified resources for a process do not exceed what is available on the machine and will not start if it does. Create your own config file to tune the labels to your needs, for example:
```
// Resource labels
withLabel: verylowcpu { cpus = 2 }
withLabel: lowcpu { cpus = 8 }
withLabel: midcpu { cpus = 16 }
withLabel: highcpu { cpus = 16 }
withLabel: verylowmem { memory = 4.GB }
withLabel: lowmem { memory = 8.GB }
withLabel: midmem { memory = 8.GB }
withLabel: highmem { memory = 8.GB }
```
When starting nextflow using the CLI, you can use `-c` to provide the file to nextflow and overwrite the defaults.
## Acknowledgements
Developed in collaboration with Data Intuitive and Open Analytics.

BIN
assets/demultiplex-launch-parameters-1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 59 KiB

BIN
assets/demultiplex-launch-parameters-2.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 77 KiB

BIN
assets/demultiplex-launch-parameters-3.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 110 KiB

BIN
assets/demultiplex-launch-parameters-4.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 24 KiB

BIN
assets/demultiplex-launch-small.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 54 KiB

5
src/demultiplex/config.vsh.yaml
View File

@@ -54,6 +54,11 @@ argument_groups:
direction: "output"
required: true
default: "$id/run_information.csv"
- name: "--demultiplexer_logs"
type: file
direction: output
required: true
default: "$id/demultiplexer_logs"
- name: "Other arguments"
arguments:
- name: --skip_copycomplete_check

23
src/demultiplex/main.nf
View File

@@ -124,14 +124,15 @@ workflow run_wf {
bcl_input_directory: state.input,
sample_sheet: state.run_information,
output_directory: state.output,
reports: "reports",
logs: "logs"
reports: state.demultiplexer_logs,
logs: state.demultiplexer_logs,
]
},
toState: {id, result, state ->
def toAdd = [
"output_demultiplexer" : result.output_directory,
"run_id": id,
"demultiplexer_logs": result.reports,
]
def newState = state + toAdd
return newState
@@ -141,11 +142,15 @@ workflow run_wf {
| bases2fastq.run(
runIf: {id, state -> state.demultiplexer in ["bases2fastq"]},
directives: [label: ["highmem", "midcpu"]],
fromState: [
"analysis_directory": "input",
"run_manifest": "run_information",
"output_directory": "output",
],
fromState: { id, state ->
[
"analysis_directory": state.input,
"run_manifest": state.run_information,
"output_directory": state.output,
"report": state.demultiplexer_logs + "/report.html",
"logs": state.demultiplexer_logs,
]
},
args: [
"no_projects": true, // Do not put output files in a subfolder for project
//"split_lanes": true,
@@ -156,6 +161,8 @@ workflow run_wf {
def toAdd = [
"output_demultiplexer" : result.output_directory,
"run_id": id,
"demultiplexer_logs": result.logs,
]
def newState = state + toAdd
return newState
@@ -225,6 +232,7 @@ workflow run_wf {
state + [ "output_multiqc" : result.output_report ]
}
)
| setState(
[
//"_meta": "_meta",
@@ -232,6 +240,7 @@ workflow run_wf {
"output_falco": "output_falco",
"output_multiqc": "output_multiqc",
"output_run_information": "run_information",
"demultiplexer_logs": "demultiplexer_logs"
]
)

32
src/demultiplex/test.nf
View File

@@ -23,6 +23,14 @@ workflow test_illumina {
assert output.size() == 2 : "outputs should contain two elements; [id, file]"
"Output: $output"
}
event_count_ch = output_ch
| toSortedList()
| map { state ->
assert state.size() == 1 : "Expected one event in the output channel"
}
assert_ch = output_ch
| map {id, state ->
assert state.output.isDirectory(): "Expected bclconvert output to be a directory"
state.output_falco.each{
@@ -57,6 +65,21 @@ workflow test_illumina {
|1,sampletest,PatientSample,UDP0004,ATTCCATAAG,TGCCTGGTGG
|""".stripMargin()
assert state.output_run_information.text.replaceAll("\r\n", "\n") == expected_run_information
println "ID: ${id}"
println "State: ${state}"
assert state.demultiplexer_logs.isDirectory():
"Expected BCL Convert reports to be a directory"
def logs_files = state.demultiplexer_logs.listFiles()
println "Logs files: ${logs_files}"
assert logs_files.size() > 0: "Expected BCL Convert logs dir to contain files"
assert logs_files.find { it.name == "Demultiplex_Stats.csv" }:
"Expected to find BCL Convert Demultiplex_Stats.csv"
assert logs_files.find { it.name == "Logs" }:
"Expected to find BCL Convert Logs directory"
}
}
@@ -80,5 +103,14 @@ workflow test_bases2fastq {
assert state.output.isDirectory(): "Expected bases2fastq output to be a directory"
state.output_falco.each{assert it.isDirectory(): "Expected falco output to be a directory"}
assert state.output_multiqc.isFile(): "Expected multiQC output to be a file"
def logs_files = state.demultiplexer_logs.listFiles()
println "Logs files: ${logs_files}"
assert logs_files.size() > 0: "Expected bases2fastq logs dir to contain files"
assert logs_files.find { it.name == "report.html" } != null:
"Expected to find bases2fastq report.html"
assert logs_files.find { it.name == "info" }:
"Expected to find bases2fastq info directory"
}
}

4
src/io/interop_summary_to_csv/config.vsh.yaml
View File

@@ -22,6 +22,10 @@ requirements:
resources:
- type: bash_script
path: script.sh
test_resources:
- type: bash_script
path: test.sh
- path: /testData/iseq-DI
engines:
- type: docker
image: debian:stable-slim

18
src/io/interop_summary_to_csv/test.sh Normal file
View File

@@ -0,0 +1,18 @@
#!/usr/bin/env bash
set -eo pipefail
# create tempdir
echo ">>> Creating temporary test directory."
TMPDIR=$(mktemp -d "$meta_temp_dir/$meta_functionality_name-XXXXXX")
function clean_up {
[[ -d "$TMPDIR" ]] && rm -r "$TMPDIR"
}
trap clean_up EXIT
echo ">>> Created temporary directory '$TMPDIR'."
echo ">>> Run simple execution"
./$meta_functionality_name \
--input "$meta_resources_dir/iseq-DI" \
--output_run_summary "$TMPDIR/run_summary.csv" \
--output_index_summary "$TMPDIR/index_summary.csv"

1
src/io/publish/code.sh
View File

@@ -5,6 +5,7 @@ set -eo pipefail
declare -A input_output_mapping=(["par_input"]="par_output"
["par_input_multiqc"]="par_output_multiqc"
["par_input_run_information"]="par_output_run_information"
["par_input_demultiplexer_logs"]="par_output_demultiplexer_logs"
)
for input_argument_name in "${!input_output_mapping[@]}"

7
src/io/publish/config.vsh.yaml
View File

@@ -21,6 +21,9 @@ argument_groups:
description: "Location where to write the run information to."
type: file
required: true
- name: "--input_demultiplexer_logs"
type: file
required: true
- name: Output arguments
arguments:
- name: --output
@@ -39,6 +42,10 @@ argument_groups:
type: file
direction: output
default: run_information.csv
- name: "--output_demultiplexer_logs"
type: file
direction: output
default: "demultiplexer_logs"
resources:
- type: bash_script

4
src/runner/config.vsh.yaml
View File

@@ -49,6 +49,10 @@ argument_groups:
type: file
direction: output
default: "qc/multiqc_report.html"
- name: "--demultiplexer_logs"
type: file
direction: output
default: "demultiplexer_logs"
- name: "Other arguments"
arguments:
- name: --skip_copycomplete_check

4
src/runner/main.nf
View File

@@ -28,6 +28,7 @@ workflow run_wf {
"output": "$id/fastq",
"output_falco": "$id/qc/fastqc",
"output_multiqc": "$id/qc/multiqc_report.html",
"demultiplexer_logs": "$id/demultiplexer_logs",
]
if (state.run_information) {
state_to_pass += ["output_run_information": state.run_information.getName()]
@@ -48,6 +49,7 @@ workflow run_wf {
def falco_output_1 = (id2 == "run") ? state.falco_output : "${id2}/" + state.falco_output
def multiqc_output_1 = (id2 == "run") ? state.multiqc_output : "${id2}/" + state.multiqc_output
def run_information_output_1 = (id2 == "run") ? "${state.output_run_information.getName()}" : "${id2}/${state.output_run_information.getName()}"
def demultiplexer_logs_output = (id2 == "run") ? state.demultiplexer_logs : "${id2}/${state.demultiplexer_logs.getName()}"
if (id2 == "run") {
println("Publising to ${params.publish_dir}")
@@ -60,10 +62,12 @@ workflow run_wf {
input_falco: state.output_falco,
input_multiqc: state.output_multiqc,
input_run_information: state.output_run_information,
input_demultiplexer_logs: state.demultiplexer_logs,
output: fastq_output_1,
output_falco: falco_output_1,
output_multiqc: multiqc_output_1,
output_run_information: run_information_output_1,
output_demultiplexer_logs: demultiplexer_logs_output,
]
},
toState: { id, result, state -> [:] },

10
target/executable/io/interop_summary_to_csv/.config.vsh.yaml
View File

@@ -41,6 +41,12 @@ resources:
- type: "file"
path: "nextflow_labels.config"
dest: "nextflow_labels.config"
test_resources:
- type: "bash_script"
path: "test.sh"
is_executable: true
- type: "file"
path: "iseq-DI"
info: null
status: "enabled"
scope:
@@ -151,9 +157,9 @@ build_info:
output: "target/executable/io/interop_summary_to_csv"
executable: "target/executable/io/interop_summary_to_csv/interop_summary_to_csv"
viash_version: "0.9.4"
git_commit: "7a14841ac6d2aeac5a647dc896550e9474124bb3"
git_commit: "f591b7849fe63b27602b2a759c1089d5dee1ecaa"
git_remote: "https://github.com/viash-hub/demultiplex"
git_tag: "v0.1.1-25-g7a14841"
git_tag: "v0.1.1-28-gf591b78"
package_config:
name: "demultiplex"
version: "main"

4
target/executable/io/interop_summary_to_csv/interop_summary_to_csv
View File

@@ -454,9 +454,9 @@ tar -C /tmp/ --no-same-owner --no-same-permissions -xvf /tmp/interop.tar.gz && \
mv /tmp/interop-1.3.1-Linux-GNU/bin/index-summary /tmp/interop-1.3.1-Linux-GNU/bin/summary /usr/local/bin/
LABEL org.opencontainers.image.description="Companion container for running component io interop_summary_to_csv"
LABEL org.opencontainers.image.created="2025-04-29T14:41:51Z"
LABEL org.opencontainers.image.created="2025-05-14T08:05:50Z"
LABEL org.opencontainers.image.source="https://github.com/viash-hub/demultiplex"
LABEL org.opencontainers.image.revision="7a14841ac6d2aeac5a647dc896550e9474124bb3"
LABEL org.opencontainers.image.revision="f591b7849fe63b27602b2a759c1089d5dee1ecaa"
LABEL org.opencontainers.image.version="main"
VIASHDOCKER

24
target/executable/io/publish/.config.vsh.yaml
View File

@@ -44,6 +44,15 @@ argument_groups:
direction: "input"
multiple: false
multiple_sep: ";"
- type: "file"
name: "--input_demultiplexer_logs"
info: null
must_exist: true
create_parent: true
required: true
direction: "input"
multiple: false
multiple_sep: ";"
- name: "Output arguments"
arguments:
- type: "file"
@@ -90,6 +99,17 @@ argument_groups:
direction: "output"
multiple: false
multiple_sep: ";"
- type: "file"
name: "--output_demultiplexer_logs"
info: null
default:
- "demultiplexer_logs"
must_exist: true
create_parent: true
required: false
direction: "output"
multiple: false
multiple_sep: ";"
resources:
- type: "bash_script"
path: "code.sh"
@@ -199,9 +219,9 @@ build_info:
output: "target/executable/io/publish"
executable: "target/executable/io/publish/publish"
viash_version: "0.9.4"
git_commit: "7a14841ac6d2aeac5a647dc896550e9474124bb3"
git_commit: "f591b7849fe63b27602b2a759c1089d5dee1ecaa"
git_remote: "https://github.com/viash-hub/demultiplex"
git_tag: "v0.1.1-25-g7a14841"
git_tag: "v0.1.1-28-gf591b78"
package_config:
name: "demultiplex"
version: "main"

69
target/executable/io/publish/publish
View File

@@ -450,9 +450,9 @@ RUN apt-get update && \
rm -rf /var/lib/apt/lists/*
LABEL org.opencontainers.image.description="Companion container for running component io publish"
LABEL org.opencontainers.image.created="2025-04-29T14:41:50Z"
LABEL org.opencontainers.image.created="2025-05-14T08:05:50Z"
LABEL org.opencontainers.image.source="https://github.com/viash-hub/demultiplex"
LABEL org.opencontainers.image.revision="7a14841ac6d2aeac5a647dc896550e9474124bb3"
LABEL org.opencontainers.image.revision="f591b7849fe63b27602b2a759c1089d5dee1ecaa"
LABEL org.opencontainers.image.version="main"
VIASHDOCKER
@@ -591,6 +591,9 @@ function ViashHelp {
echo " type: file, required parameter, file must exist"
echo " Location where to write the run information to."
echo ""
echo " --input_demultiplexer_logs"
echo " type: file, required parameter, file must exist"
echo ""
echo "Output arguments:"
echo " --output"
echo " type: file, output, file must exist"
@@ -608,6 +611,10 @@ function ViashHelp {
echo " type: file, output, file must exist"
echo " default: run_information.csv"
echo ""
echo " --output_demultiplexer_logs"
echo " type: file, output, file must exist"
echo " default: demultiplexer_logs"
echo ""
echo "Viash built in Computational Requirements:"
echo " ---cpus=INT"
echo " Number of CPUs to use"
@@ -708,6 +715,17 @@ while [[ $# -gt 0 ]]; do
VIASH_PAR_INPUT_RUN_INFORMATION=$(ViashRemoveFlags "$1")
shift 1
;;
--input_demultiplexer_logs)
[ -n "$VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS" ] && ViashError Bad arguments for option \'--input_demultiplexer_logs\': \'$VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS\' \& \'$2\' - you should provide exactly one argument for this option. && exit 1
VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS="$2"
[ $# -lt 2 ] && ViashError Not enough arguments passed to --input_demultiplexer_logs. Use "--help" to get more information on the parameters. && exit 1
shift 2
;;
--input_demultiplexer_logs=*)
[ -n "$VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS" ] && ViashError Bad arguments for option \'--input_demultiplexer_logs=*\': \'$VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS\' \& \'$2\' - you should provide exactly one argument for this option. && exit 1
VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS=$(ViashRemoveFlags "$1")
shift 1
;;
--output)
[ -n "$VIASH_PAR_OUTPUT" ] && ViashError Bad arguments for option \'--output\': \'$VIASH_PAR_OUTPUT\' \& \'$2\' - you should provide exactly one argument for this option. && exit 1
VIASH_PAR_OUTPUT="$2"
@@ -752,6 +770,17 @@ while [[ $# -gt 0 ]]; do
VIASH_PAR_OUTPUT_RUN_INFORMATION=$(ViashRemoveFlags "$1")
shift 1
;;
--output_demultiplexer_logs)
[ -n "$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS" ] && ViashError Bad arguments for option \'--output_demultiplexer_logs\': \'$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS\' \& \'$2\' - you should provide exactly one argument for this option. && exit 1
VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS="$2"
[ $# -lt 2 ] && ViashError Not enough arguments passed to --output_demultiplexer_logs. Use "--help" to get more information on the parameters. && exit 1
shift 2
;;
--output_demultiplexer_logs=*)
[ -n "$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS" ] && ViashError Bad arguments for option \'--output_demultiplexer_logs=*\': \'$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS\' \& \'$2\' - you should provide exactly one argument for this option. && exit 1
VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS=$(ViashRemoveFlags "$1")
shift 1
;;
---engine)
VIASH_ENGINE_ID="$2"
shift 2
@@ -940,6 +969,10 @@ if [ -z ${VIASH_PAR_INPUT_RUN_INFORMATION+x} ]; then
ViashError '--input_run_information' is a required argument. Use "--help" to get more information on the parameters.
exit 1
fi
if [ -z ${VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS+x} ]; then
ViashError '--input_demultiplexer_logs' is a required argument. Use "--help" to get more information on the parameters.
exit 1
fi
if [ -z ${VIASH_META_NAME+x} ]; then
ViashError 'name' is a required argument. Use "--help" to get more information on the parameters.
exit 1
@@ -978,6 +1011,9 @@ fi
if [ -z ${VIASH_PAR_OUTPUT_RUN_INFORMATION+x} ]; then
VIASH_PAR_OUTPUT_RUN_INFORMATION="run_information.csv"
fi
if [ -z ${VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS+x} ]; then
VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS="demultiplexer_logs"
fi
# check whether required files exist
if [ ! -z "$VIASH_PAR_INPUT" ] && [ ! -e "$VIASH_PAR_INPUT" ]; then
@@ -1004,6 +1040,10 @@ if [ ! -z "$VIASH_PAR_INPUT_RUN_INFORMATION" ] && [ ! -e "$VIASH_PAR_INPUT_RUN_I
ViashError "Input file '$VIASH_PAR_INPUT_RUN_INFORMATION' does not exist."
exit 1
fi
if [ ! -z "$VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS" ] && [ ! -e "$VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS" ]; then
ViashError "Input file '$VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS' does not exist."
exit 1
fi
# check whether parameters values are of the right type
if [[ -n "$VIASH_META_CPUS" ]]; then
@@ -1092,6 +1132,9 @@ fi
if [ ! -z "$VIASH_PAR_OUTPUT_RUN_INFORMATION" ] && [ ! -d "$(dirname "$VIASH_PAR_OUTPUT_RUN_INFORMATION")" ]; then
mkdir -p "$(dirname "$VIASH_PAR_OUTPUT_RUN_INFORMATION")"
fi
if [ ! -z "$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS" ] && [ ! -d "$(dirname "$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS")" ]; then
mkdir -p "$(dirname "$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS")"
fi
if [ "$VIASH_ENGINE_ID" == "native" ] ; then
if [ "$VIASH_MODE" == "run" ]; then
@@ -1128,6 +1171,10 @@ if [ ! -z "$VIASH_PAR_INPUT_RUN_INFORMATION" ]; then
VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$VIASH_PAR_INPUT_RUN_INFORMATION")" )
VIASH_PAR_INPUT_RUN_INFORMATION=$(ViashDockerAutodetectMount "$VIASH_PAR_INPUT_RUN_INFORMATION")
fi
if [ ! -z "$VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS" ]; then
VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS")" )
VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS=$(ViashDockerAutodetectMount "$VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS")
fi
if [ ! -z "$VIASH_PAR_OUTPUT" ]; then
VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$VIASH_PAR_OUTPUT")" )
VIASH_PAR_OUTPUT=$(ViashDockerAutodetectMount "$VIASH_PAR_OUTPUT")
@@ -1148,6 +1195,11 @@ if [ ! -z "$VIASH_PAR_OUTPUT_RUN_INFORMATION" ]; then
VIASH_PAR_OUTPUT_RUN_INFORMATION=$(ViashDockerAutodetectMount "$VIASH_PAR_OUTPUT_RUN_INFORMATION")
VIASH_CHOWN_VARS+=( "$VIASH_PAR_OUTPUT_RUN_INFORMATION" )
fi
if [ ! -z "$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS" ]; then
VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS")" )
VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS=$(ViashDockerAutodetectMount "$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS")
VIASH_CHOWN_VARS+=( "$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS" )
fi
if [ ! -z "$VIASH_META_RESOURCES_DIR" ]; then
VIASH_DIRECTORY_MOUNTS+=( "$(ViashDockerAutodetectMountArg "$VIASH_META_RESOURCES_DIR")" )
VIASH_META_RESOURCES_DIR=$(ViashDockerAutodetectMount "$VIASH_META_RESOURCES_DIR")
@@ -1221,10 +1273,12 @@ $( if [ ! -z ${VIASH_PAR_INPUT+x} ]; then echo "${VIASH_PAR_INPUT}" | sed "s#'#'
$( if [ ! -z ${VIASH_PAR_INPUT_FALCO+x} ]; then echo "${VIASH_PAR_INPUT_FALCO}" | sed "s#'#'\"'\"'#g;s#.*#par_input_falco='&'#" ; else echo "# par_input_falco="; fi )
$( if [ ! -z ${VIASH_PAR_INPUT_MULTIQC+x} ]; then echo "${VIASH_PAR_INPUT_MULTIQC}" | sed "s#'#'\"'\"'#g;s#.*#par_input_multiqc='&'#" ; else echo "# par_input_multiqc="; fi )
$( if [ ! -z ${VIASH_PAR_INPUT_RUN_INFORMATION+x} ]; then echo "${VIASH_PAR_INPUT_RUN_INFORMATION}" | sed "s#'#'\"'\"'#g;s#.*#par_input_run_information='&'#" ; else echo "# par_input_run_information="; fi )
$( if [ ! -z ${VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS+x} ]; then echo "${VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS}" | sed "s#'#'\"'\"'#g;s#.*#par_input_demultiplexer_logs='&'#" ; else echo "# par_input_demultiplexer_logs="; fi )
$( if [ ! -z ${VIASH_PAR_OUTPUT+x} ]; then echo "${VIASH_PAR_OUTPUT}" | sed "s#'#'\"'\"'#g;s#.*#par_output='&'#" ; else echo "# par_output="; fi )
$( if [ ! -z ${VIASH_PAR_OUTPUT_FALCO+x} ]; then echo "${VIASH_PAR_OUTPUT_FALCO}" | sed "s#'#'\"'\"'#g;s#.*#par_output_falco='&'#" ; else echo "# par_output_falco="; fi )
$( if [ ! -z ${VIASH_PAR_OUTPUT_MULTIQC+x} ]; then echo "${VIASH_PAR_OUTPUT_MULTIQC}" | sed "s#'#'\"'\"'#g;s#.*#par_output_multiqc='&'#" ; else echo "# par_output_multiqc="; fi )
$( if [ ! -z ${VIASH_PAR_OUTPUT_RUN_INFORMATION+x} ]; then echo "${VIASH_PAR_OUTPUT_RUN_INFORMATION}" | sed "s#'#'\"'\"'#g;s#.*#par_output_run_information='&'#" ; else echo "# par_output_run_information="; fi )
$( if [ ! -z ${VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS+x} ]; then echo "${VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS}" | sed "s#'#'\"'\"'#g;s#.*#par_output_demultiplexer_logs='&'#" ; else echo "# par_output_demultiplexer_logs="; fi )
$( if [ ! -z ${VIASH_META_NAME+x} ]; then echo "${VIASH_META_NAME}" | sed "s#'#'\"'\"'#g;s#.*#meta_name='&'#" ; else echo "# meta_name="; fi )
$( if [ ! -z ${VIASH_META_FUNCTIONALITY_NAME+x} ]; then echo "${VIASH_META_FUNCTIONALITY_NAME}" | sed "s#'#'\"'\"'#g;s#.*#meta_functionality_name='&'#" ; else echo "# meta_functionality_name="; fi )
$( if [ ! -z ${VIASH_META_RESOURCES_DIR+x} ]; then echo "${VIASH_META_RESOURCES_DIR}" | sed "s#'#'\"'\"'#g;s#.*#meta_resources_dir='&'#" ; else echo "# meta_resources_dir="; fi )
@@ -1252,6 +1306,7 @@ set -eo pipefail
declare -A input_output_mapping=(["par_input"]="par_output"
["par_input_multiqc"]="par_output_multiqc"
["par_input_run_information"]="par_output_run_information"
["par_input_demultiplexer_logs"]="par_output_demultiplexer_logs"
)
for input_argument_name in "\${!input_output_mapping[@]}"
@@ -1310,6 +1365,9 @@ if [[ "$VIASH_ENGINE_TYPE" == "docker" ]]; then
if [ ! -z "$VIASH_PAR_INPUT_RUN_INFORMATION" ]; then
VIASH_PAR_INPUT_RUN_INFORMATION=$(ViashDockerStripAutomount "$VIASH_PAR_INPUT_RUN_INFORMATION")
fi
if [ ! -z "$VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS" ]; then
VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS=$(ViashDockerStripAutomount "$VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS")
fi
if [ ! -z "$VIASH_PAR_OUTPUT" ]; then
VIASH_PAR_OUTPUT=$(ViashDockerStripAutomount "$VIASH_PAR_OUTPUT")
fi
@@ -1322,6 +1380,9 @@ if [[ "$VIASH_ENGINE_TYPE" == "docker" ]]; then
if [ ! -z "$VIASH_PAR_OUTPUT_RUN_INFORMATION" ]; then
VIASH_PAR_OUTPUT_RUN_INFORMATION=$(ViashDockerStripAutomount "$VIASH_PAR_OUTPUT_RUN_INFORMATION")
fi
if [ ! -z "$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS" ]; then
VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS=$(ViashDockerStripAutomount "$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS")
fi
if [ ! -z "$VIASH_META_RESOURCES_DIR" ]; then
VIASH_META_RESOURCES_DIR=$(ViashDockerStripAutomount "$VIASH_META_RESOURCES_DIR")
fi
@@ -1354,6 +1415,10 @@ if [ ! -z "$VIASH_PAR_OUTPUT_RUN_INFORMATION" ] && [ ! -e "$VIASH_PAR_OUTPUT_RUN
ViashError "Output file '$VIASH_PAR_OUTPUT_RUN_INFORMATION' does not exist."
exit 1
fi
if [ ! -z "$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS" ] && [ ! -e "$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS" ]; then
ViashError "Output file '$VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS' does not exist."
exit 1
fi
exit 0

4
target/executable/io/untar/.config.vsh.yaml
View File

@@ -156,9 +156,9 @@ build_info:
output: "target/executable/io/untar"
executable: "target/executable/io/untar/untar"
viash_version: "0.9.4"
git_commit: "7a14841ac6d2aeac5a647dc896550e9474124bb3"
git_commit: "f591b7849fe63b27602b2a759c1089d5dee1ecaa"
git_remote: "https://github.com/viash-hub/demultiplex"
git_tag: "v0.1.1-25-g7a14841"
git_tag: "v0.1.1-28-gf591b78"
package_config:
name: "demultiplex"
version: "main"

4
target/executable/io/untar/untar
View File

@@ -450,9 +450,9 @@ RUN apt-get update && \
rm -rf /var/lib/apt/lists/*
LABEL org.opencontainers.image.description="Companion container for running component io untar"
LABEL org.opencontainers.image.created="2025-04-29T14:41:50Z"
LABEL org.opencontainers.image.created="2025-05-14T08:05:49Z"
LABEL org.opencontainers.image.source="https://github.com/viash-hub/demultiplex"
LABEL org.opencontainers.image.revision="7a14841ac6d2aeac5a647dc896550e9474124bb3"
LABEL org.opencontainers.image.revision="f591b7849fe63b27602b2a759c1089d5dee1ecaa"
LABEL org.opencontainers.image.version="main"
VIASHDOCKER

4
target/nextflow/dataflow/combine_samples/.config.vsh.yaml
View File

@@ -165,9 +165,9 @@ build_info:
output: "target/nextflow/dataflow/combine_samples"
executable: "target/nextflow/dataflow/combine_samples/main.nf"
viash_version: "0.9.4"
git_commit: "7a14841ac6d2aeac5a647dc896550e9474124bb3"
git_commit: "f591b7849fe63b27602b2a759c1089d5dee1ecaa"
git_remote: "https://github.com/viash-hub/demultiplex"
git_tag: "v0.1.1-25-g7a14841"
git_tag: "v0.1.1-28-gf591b78"
package_config:
name: "demultiplex"
version: "main"

4
target/nextflow/dataflow/combine_samples/main.nf
View File

@@ -3230,9 +3230,9 @@ meta = [
"engine" : "native|native",
"output" : "target/nextflow/dataflow/combine_samples",
"viash_version" : "0.9.4",
"git_commit" : "7a14841ac6d2aeac5a647dc896550e9474124bb3",
"git_commit" : "f591b7849fe63b27602b2a759c1089d5dee1ecaa",
"git_remote" : "https://github.com/viash-hub/demultiplex",
"git_tag" : "v0.1.1-25-g7a14841"
"git_tag" : "v0.1.1-28-gf591b78"
},
"package_config" : {
"name" : "demultiplex",

4
target/nextflow/dataflow/gather_fastqs_and_validate/.config.vsh.yaml
View File

@@ -141,9 +141,9 @@ build_info:
output: "target/nextflow/dataflow/gather_fastqs_and_validate"
executable: "target/nextflow/dataflow/gather_fastqs_and_validate/main.nf"
viash_version: "0.9.4"
git_commit: "7a14841ac6d2aeac5a647dc896550e9474124bb3"
git_commit: "f591b7849fe63b27602b2a759c1089d5dee1ecaa"
git_remote: "https://github.com/viash-hub/demultiplex"
git_tag: "v0.1.1-25-g7a14841"
git_tag: "v0.1.1-28-gf591b78"
package_config:
name: "demultiplex"
version: "main"

4
target/nextflow/dataflow/gather_fastqs_and_validate/main.nf
View File

@@ -3203,9 +3203,9 @@ meta = [
"engine" : "native|native",
"output" : "target/nextflow/dataflow/gather_fastqs_and_validate",
"viash_version" : "0.9.4",
"git_commit" : "7a14841ac6d2aeac5a647dc896550e9474124bb3",
"git_commit" : "f591b7849fe63b27602b2a759c1089d5dee1ecaa",
"git_remote" : "https://github.com/viash-hub/demultiplex",
"git_tag" : "v0.1.1-25-g7a14841"
"git_tag" : "v0.1.1-28-gf591b78"
},
"package_config" : {
"name" : "demultiplex",

15
target/nextflow/demultiplex/.config.vsh.yaml
View File

@@ -96,6 +96,17 @@ argument_groups:
direction: "output"
multiple: false
multiple_sep: ";"
- type: "file"
name: "--demultiplexer_logs"
info: null
default:
- "$id/demultiplexer_logs"
must_exist: true
create_parent: true
required: true
direction: "output"
multiple: false
multiple_sep: ";"
- name: "Other arguments"
arguments:
- type: "boolean_true"
@@ -247,9 +258,9 @@ build_info:
output: "target/nextflow/demultiplex"
executable: "target/nextflow/demultiplex/main.nf"
viash_version: "0.9.4"
git_commit: "7a14841ac6d2aeac5a647dc896550e9474124bb3"
git_commit: "f591b7849fe63b27602b2a759c1089d5dee1ecaa"
git_remote: "https://github.com/viash-hub/demultiplex"
git_tag: "v0.1.1-25-g7a14841"
git_tag: "v0.1.1-28-gf591b78"
dependencies:
- "target/nextflow/io/untar"
- "target/nextflow/dataflow/gather_fastqs_and_validate"

40
target/nextflow/demultiplex/main.nf
View File

@@ -3139,6 +3139,19 @@ meta = [
"direction" : "output",
"multiple" : false,
"multiple_sep" : ";"
},
{
"type" : "file",
"name" : "--demultiplexer_logs",
"default" : [
"$id/demultiplexer_logs"
],
"must_exist" : true,
"create_parent" : true,
"required" : true,
"direction" : "output",
"multiple" : false,
"multiple_sep" : ";"
}
]
},
@@ -3350,9 +3363,9 @@ meta = [
"engine" : "native|native",
"output" : "target/nextflow/demultiplex",
"viash_version" : "0.9.4",
"git_commit" : "7a14841ac6d2aeac5a647dc896550e9474124bb3",
"git_commit" : "f591b7849fe63b27602b2a759c1089d5dee1ecaa",
"git_remote" : "https://github.com/viash-hub/demultiplex",
"git_tag" : "v0.1.1-25-g7a14841"
"git_tag" : "v0.1.1-28-gf591b78"
},
"package_config" : {
"name" : "demultiplex",
@@ -3530,14 +3543,15 @@ workflow run_wf {
bcl_input_directory: state.input,
sample_sheet: state.run_information,
output_directory: state.output,
reports: "reports",
logs: "logs"
reports: state.demultiplexer_logs,
logs: state.demultiplexer_logs,
]
},
toState: {id, result, state ->
def toAdd = [
"output_demultiplexer" : result.output_directory,
"run_id": id,
"demultiplexer_logs": result.reports,
]
def newState = state + toAdd
return newState
@@ -3547,11 +3561,15 @@ workflow run_wf {
| bases2fastq.run(
runIf: {id, state -> state.demultiplexer in ["bases2fastq"]},
directives: [label: ["highmem", "midcpu"]],
fromState: [
"analysis_directory": "input",
"run_manifest": "run_information",
"output_directory": "output",
],
fromState: { id, state ->
[
"analysis_directory": state.input,
"run_manifest": state.run_information,
"output_directory": state.output,
"report": state.demultiplexer_logs + "/report.html",
"logs": state.demultiplexer_logs,
]
},
args: [
"no_projects": true, // Do not put output files in a subfolder for project
//"split_lanes": true,
@@ -3562,6 +3580,8 @@ workflow run_wf {
def toAdd = [
"output_demultiplexer" : result.output_directory,
"run_id": id,
"demultiplexer_logs": result.logs,
]
def newState = state + toAdd
return newState
@@ -3631,6 +3651,7 @@ workflow run_wf {
state + [ "output_multiqc" : result.output_report ]
}
)
| setState(
[
//"_meta": "_meta",
@@ -3638,6 +3659,7 @@ workflow run_wf {
"output_falco": "output_falco",
"output_multiqc": "output_multiqc",
"output_run_information": "run_information",
"demultiplexer_logs": "demultiplexer_logs"
]
)

11
target/nextflow/demultiplex/nextflow_schema.json
View File

@@ -109,6 +109,17 @@
}
,
"demultiplexer_logs": {
"type":
"string",
"description": "Type: `file`, required, default: `$id/demultiplexer_logs`. ",
"help_text": "Type: `file`, required, default: `$id/demultiplexer_logs`. "
,
"default":"$id/demultiplexer_logs"
}
}
},

10
target/nextflow/io/interop_summary_to_csv/.config.vsh.yaml
View File

@@ -41,6 +41,12 @@ resources:
- type: "file"
path: "nextflow_labels.config"
dest: "nextflow_labels.config"
test_resources:
- type: "bash_script"
path: "test.sh"
is_executable: true
- type: "file"
path: "iseq-DI"
info: null
status: "enabled"
scope:
@@ -151,9 +157,9 @@ build_info:
output: "target/nextflow/io/interop_summary_to_csv"
executable: "target/nextflow/io/interop_summary_to_csv/main.nf"
viash_version: "0.9.4"
git_commit: "7a14841ac6d2aeac5a647dc896550e9474124bb3"
git_commit: "f591b7849fe63b27602b2a759c1089d5dee1ecaa"
git_remote: "https://github.com/viash-hub/demultiplex"
git_tag: "v0.1.1-25-g7a14841"
git_tag: "v0.1.1-28-gf591b78"
package_config:
name: "demultiplex"
version: "main"

15
target/nextflow/io/interop_summary_to_csv/main.nf
View File

@@ -3088,6 +3088,17 @@ meta = [
"dest" : "nextflow_labels.config"
}
],
"test_resources" : [
{
"type" : "bash_script",
"path" : "test.sh",
"is_executable" : true
},
{
"type" : "file",
"path" : "/testData/iseq-DI"
}
],
"status" : "enabled",
"scope" : {
"image" : "public",
@@ -3217,9 +3228,9 @@ meta = [
"engine" : "docker|native",
"output" : "target/nextflow/io/interop_summary_to_csv",
"viash_version" : "0.9.4",
"git_commit" : "7a14841ac6d2aeac5a647dc896550e9474124bb3",
"git_commit" : "f591b7849fe63b27602b2a759c1089d5dee1ecaa",
"git_remote" : "https://github.com/viash-hub/demultiplex",
"git_tag" : "v0.1.1-25-g7a14841"
"git_tag" : "v0.1.1-28-gf591b78"
},
"package_config" : {
"name" : "demultiplex",

24
target/nextflow/io/publish/.config.vsh.yaml
View File

@@ -44,6 +44,15 @@ argument_groups:
direction: "input"
multiple: false
multiple_sep: ";"
- type: "file"
name: "--input_demultiplexer_logs"
info: null
must_exist: true
create_parent: true
required: true
direction: "input"
multiple: false
multiple_sep: ";"
- name: "Output arguments"
arguments:
- type: "file"
@@ -90,6 +99,17 @@ argument_groups:
direction: "output"
multiple: false
multiple_sep: ";"
- type: "file"
name: "--output_demultiplexer_logs"
info: null
default:
- "demultiplexer_logs"
must_exist: true
create_parent: true
required: false
direction: "output"
multiple: false
multiple_sep: ";"
resources:
- type: "bash_script"
path: "code.sh"
@@ -199,9 +219,9 @@ build_info:
output: "target/nextflow/io/publish"
executable: "target/nextflow/io/publish/main.nf"
viash_version: "0.9.4"
git_commit: "7a14841ac6d2aeac5a647dc896550e9474124bb3"
git_commit: "f591b7849fe63b27602b2a759c1089d5dee1ecaa"
git_remote: "https://github.com/viash-hub/demultiplex"
git_tag: "v0.1.1-25-g7a14841"
git_tag: "v0.1.1-28-gf591b78"
package_config:
name: "demultiplex"
version: "main"

30
target/nextflow/io/publish/main.nf
View File

@@ -3080,6 +3080,16 @@ meta = [
"direction" : "input",
"multiple" : false,
"multiple_sep" : ";"
},
{
"type" : "file",
"name" : "--input_demultiplexer_logs",
"must_exist" : true,
"create_parent" : true,
"required" : true,
"direction" : "input",
"multiple" : false,
"multiple_sep" : ";"
}
]
},
@@ -3137,6 +3147,19 @@ meta = [
"direction" : "output",
"multiple" : false,
"multiple_sep" : ";"
},
{
"type" : "file",
"name" : "--output_demultiplexer_logs",
"default" : [
"demultiplexer_logs"
],
"must_exist" : true,
"create_parent" : true,
"required" : false,
"direction" : "output",
"multiple" : false,
"multiple_sep" : ";"
}
]
}
@@ -3274,9 +3297,9 @@ meta = [
"engine" : "docker|native",
"output" : "target/nextflow/io/publish",
"viash_version" : "0.9.4",
"git_commit" : "7a14841ac6d2aeac5a647dc896550e9474124bb3",
"git_commit" : "f591b7849fe63b27602b2a759c1089d5dee1ecaa",
"git_remote" : "https://github.com/viash-hub/demultiplex",
"git_tag" : "v0.1.1-25-g7a14841"
"git_tag" : "v0.1.1-28-gf591b78"
},
"package_config" : {
"name" : "demultiplex",
@@ -3330,10 +3353,12 @@ $( if [ ! -z ${VIASH_PAR_INPUT+x} ]; then echo "${VIASH_PAR_INPUT}" | sed "s#'#'
$( if [ ! -z ${VIASH_PAR_INPUT_FALCO+x} ]; then echo "${VIASH_PAR_INPUT_FALCO}" | sed "s#'#'\\"'\\"'#g;s#.*#par_input_falco='&'#" ; else echo "# par_input_falco="; fi )
$( if [ ! -z ${VIASH_PAR_INPUT_MULTIQC+x} ]; then echo "${VIASH_PAR_INPUT_MULTIQC}" | sed "s#'#'\\"'\\"'#g;s#.*#par_input_multiqc='&'#" ; else echo "# par_input_multiqc="; fi )
$( if [ ! -z ${VIASH_PAR_INPUT_RUN_INFORMATION+x} ]; then echo "${VIASH_PAR_INPUT_RUN_INFORMATION}" | sed "s#'#'\\"'\\"'#g;s#.*#par_input_run_information='&'#" ; else echo "# par_input_run_information="; fi )
$( if [ ! -z ${VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS+x} ]; then echo "${VIASH_PAR_INPUT_DEMULTIPLEXER_LOGS}" | sed "s#'#'\\"'\\"'#g;s#.*#par_input_demultiplexer_logs='&'#" ; else echo "# par_input_demultiplexer_logs="; fi )
$( if [ ! -z ${VIASH_PAR_OUTPUT+x} ]; then echo "${VIASH_PAR_OUTPUT}" | sed "s#'#'\\"'\\"'#g;s#.*#par_output='&'#" ; else echo "# par_output="; fi )
$( if [ ! -z ${VIASH_PAR_OUTPUT_FALCO+x} ]; then echo "${VIASH_PAR_OUTPUT_FALCO}" | sed "s#'#'\\"'\\"'#g;s#.*#par_output_falco='&'#" ; else echo "# par_output_falco="; fi )
$( if [ ! -z ${VIASH_PAR_OUTPUT_MULTIQC+x} ]; then echo "${VIASH_PAR_OUTPUT_MULTIQC}" | sed "s#'#'\\"'\\"'#g;s#.*#par_output_multiqc='&'#" ; else echo "# par_output_multiqc="; fi )
$( if [ ! -z ${VIASH_PAR_OUTPUT_RUN_INFORMATION+x} ]; then echo "${VIASH_PAR_OUTPUT_RUN_INFORMATION}" | sed "s#'#'\\"'\\"'#g;s#.*#par_output_run_information='&'#" ; else echo "# par_output_run_information="; fi )
$( if [ ! -z ${VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS+x} ]; then echo "${VIASH_PAR_OUTPUT_DEMULTIPLEXER_LOGS}" | sed "s#'#'\\"'\\"'#g;s#.*#par_output_demultiplexer_logs='&'#" ; else echo "# par_output_demultiplexer_logs="; fi )
$( if [ ! -z ${VIASH_META_NAME+x} ]; then echo "${VIASH_META_NAME}" | sed "s#'#'\\"'\\"'#g;s#.*#meta_name='&'#" ; else echo "# meta_name="; fi )
$( if [ ! -z ${VIASH_META_FUNCTIONALITY_NAME+x} ]; then echo "${VIASH_META_FUNCTIONALITY_NAME}" | sed "s#'#'\\"'\\"'#g;s#.*#meta_functionality_name='&'#" ; else echo "# meta_functionality_name="; fi )
$( if [ ! -z ${VIASH_META_RESOURCES_DIR+x} ]; then echo "${VIASH_META_RESOURCES_DIR}" | sed "s#'#'\\"'\\"'#g;s#.*#meta_resources_dir='&'#" ; else echo "# meta_resources_dir="; fi )
@@ -3361,6 +3386,7 @@ set -eo pipefail
declare -A input_output_mapping=(["par_input"]="par_output"
["par_input_multiqc"]="par_output_multiqc"
["par_input_run_information"]="par_output_run_information"
["par_input_demultiplexer_logs"]="par_output_demultiplexer_logs"
)
for input_argument_name in "\\${!input_output_mapping[@]}"

21
target/nextflow/io/publish/nextflow_schema.json
View File

@@ -53,6 +53,16 @@
}
,
"input_demultiplexer_logs": {
"type":
"string",
"description": "Type: `file`, required. ",
"help_text": "Type: `file`, required. "
}
}
},
@@ -107,6 +117,17 @@
}
,
"output_demultiplexer_logs": {
"type":
"string",
"description": "Type: `file`, default: `demultiplexer_logs`. ",
"help_text": "Type: `file`, default: `demultiplexer_logs`. "
,
"default":"demultiplexer_logs"
}
}
},

4
target/nextflow/io/untar/.config.vsh.yaml
View File

@@ -156,9 +156,9 @@ build_info:
output: "target/nextflow/io/untar"
executable: "target/nextflow/io/untar/main.nf"
viash_version: "0.9.4"
git_commit: "7a14841ac6d2aeac5a647dc896550e9474124bb3"
git_commit: "f591b7849fe63b27602b2a759c1089d5dee1ecaa"
git_remote: "https://github.com/viash-hub/demultiplex"
git_tag: "v0.1.1-25-g7a14841"
git_tag: "v0.1.1-28-gf591b78"
package_config:
name: "demultiplex"
version: "main"

4
target/nextflow/io/untar/main.nf
View File

@@ -3227,9 +3227,9 @@ meta = [
"engine" : "docker|native",
"output" : "target/nextflow/io/untar",
"viash_version" : "0.9.4",
"git_commit" : "7a14841ac6d2aeac5a647dc896550e9474124bb3",
"git_commit" : "f591b7849fe63b27602b2a759c1089d5dee1ecaa",
"git_remote" : "https://github.com/viash-hub/demultiplex",
"git_tag" : "v0.1.1-25-g7a14841"
"git_tag" : "v0.1.1-28-gf591b78"
},
"package_config" : {
"name" : "demultiplex",

15
target/nextflow/runner/.config.vsh.yaml
View File

@@ -84,6 +84,17 @@ argument_groups:
direction: "output"
multiple: false
multiple_sep: ";"
- type: "file"
name: "--demultiplexer_logs"
info: null
default:
- "demultiplexer_logs"
must_exist: true
create_parent: true
required: false
direction: "output"
multiple: false
multiple_sep: ";"
- name: "Other arguments"
arguments:
- type: "boolean_true"
@@ -195,9 +206,9 @@ build_info:
output: "target/nextflow/runner"
executable: "target/nextflow/runner/main.nf"
viash_version: "0.9.4"
git_commit: "7a14841ac6d2aeac5a647dc896550e9474124bb3"
git_commit: "f591b7849fe63b27602b2a759c1089d5dee1ecaa"
git_remote: "https://github.com/viash-hub/demultiplex"
git_tag: "v0.1.1-25-g7a14841"
git_tag: "v0.1.1-28-gf591b78"
dependencies:
- "target/nextflow/demultiplex"
- "target/nextflow/io/publish"

21
target/nextflow/runner/main.nf
View File

@@ -3125,6 +3125,19 @@ meta = [
"direction" : "output",
"multiple" : false,
"multiple_sep" : ";"
},
{
"type" : "file",
"name" : "--demultiplexer_logs",
"default" : [
"demultiplexer_logs"
],
"must_exist" : true,
"create_parent" : true,
"required" : false,
"direction" : "output",
"multiple" : false,
"multiple_sep" : ";"
}
]
},
@@ -3270,9 +3283,9 @@ meta = [
"engine" : "native|native",
"output" : "target/nextflow/runner",
"viash_version" : "0.9.4",
"git_commit" : "7a14841ac6d2aeac5a647dc896550e9474124bb3",
"git_commit" : "f591b7849fe63b27602b2a759c1089d5dee1ecaa",
"git_remote" : "https://github.com/viash-hub/demultiplex",
"git_tag" : "v0.1.1-25-g7a14841"
"git_tag" : "v0.1.1-28-gf591b78"
},
"package_config" : {
"name" : "demultiplex",
@@ -3348,6 +3361,7 @@ workflow run_wf {
"output": "$id/fastq",
"output_falco": "$id/qc/fastqc",
"output_multiqc": "$id/qc/multiqc_report.html",
"demultiplexer_logs": "$id/demultiplexer_logs",
]
if (state.run_information) {
state_to_pass += ["output_run_information": state.run_information.getName()]
@@ -3368,6 +3382,7 @@ workflow run_wf {
def falco_output_1 = (id2 == "run") ? state.falco_output : "${id2}/" + state.falco_output
def multiqc_output_1 = (id2 == "run") ? state.multiqc_output : "${id2}/" + state.multiqc_output
def run_information_output_1 = (id2 == "run") ? "${state.output_run_information.getName()}" : "${id2}/${state.output_run_information.getName()}"
def demultiplexer_logs_output = (id2 == "run") ? state.demultiplexer_logs : "${id2}/${state.demultiplexer_logs.getName()}"
if (id2 == "run") {
println("Publising to ${params.publish_dir}")
@@ -3380,10 +3395,12 @@ workflow run_wf {
input_falco: state.output_falco,
input_multiqc: state.output_multiqc,
input_run_information: state.output_run_information,
input_demultiplexer_logs: state.demultiplexer_logs,
output: fastq_output_1,
output_falco: falco_output_1,
output_multiqc: multiqc_output_1,
output_run_information: run_information_output_1,
output_demultiplexer_logs: demultiplexer_logs_output,
]
},
toState: { id, result, state -> [:] },

11
target/nextflow/runner/nextflow_schema.json
View File

@@ -109,6 +109,17 @@
}
,
"demultiplexer_logs": {
"type":
"string",
"description": "Type: `file`, default: `demultiplexer_logs`. ",
"help_text": "Type: `file`, default: `demultiplexer_logs`. "
,
"default":"demultiplexer_logs"
}
}
},