Build branch v0.1.x with version v0.1.1 (2c07979)

Build pipeline: viash-hub.toolbox.v0.1.x-fphr8

Source commit: 2c07979559

Source message: Merge remote-tracking branch 'origin/main' into v0.1.x
This commit is contained in:
CI
2025-05-06 20:43:01 +00:00
commit 5cb3251eaf
30 changed files with 13316 additions and 0 deletions

23
CHANGELOG.md Normal file
View File

@@ -0,0 +1,23 @@
# toolbox v0.1.1
## MINOR CHANGES
* Updated the test CI (PR #6).
* Bump viash to 0.9.0 (PR #10).
* Bump viash to 0.9.4 (PR #13).
* Update README (PR #13).
# toolbox v0.1.0
## NEW FEATURES
* `bgzip`: Add bgzip functionality to compress and decompress files (initial commit).
* `yq`: A portable YAML, JSON, XML, CSV, TOML and properties processor (PR #1).
## MINOR CHANGES
* Use newer viash-actions updated for Viash 0.9 (PR #2).

383
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,383 @@
# Contributing guidelines
We encourage contributions from the community. To contribute:
1. **Fork the Repository**: Start by forking this repository to your account.
2. **Develop Your Component**: Create your Viash component, ensuring it aligns with our best practices (detailed below).
3. **Submit a Pull Request**: After testing your component, submit a pull request for review.
## Procedure of adding a component
### Step 1: Find a component to contribute
* Find a tool to contribute to this repo.
* Check whether it is already in the [Project board](https://github.com/orgs/viash-hub/projects/1).
* Check whether there is a corresponding [Snakemake wrapper](https://github.com/snakemake/snakemake-wrappers/blob/master/bio) or [nf-core module](https://github.com/nf-core/modules/tree/master/modules/nf-core) which we can use as inspiration.
* Create an issue to show that you are working on this component.
### Step 2: Add config template
Change all occurrences of `xxx` to the name of the component.
Create a file at `src/xxx/config.vsh.yaml` with contents:
```yaml
name: xxx
description: xxx
keywords: [tag1, tag2]
links:
homepage: yyy
documentation: yyy
issue_tracker: yyy
repository: yyy
references:
doi: 12345/12345678.yz
license: MIT/Apache-2.0/GPL-3.0/...
argument_groups:
- name: Inputs
arguments: <...>
- name: Outputs
arguments: <...>
- name: Arguments
arguments: <...>
resources:
- type: bash_script
path: script.sh
test_resources:
- type: bash_script
path: test.sh
- type: file
path: test_data
engines:
- <...>
runners:
- type: executable
- type: nextflow
```
### Step 3: Fill in the metadata
Fill in the relevant metadata fields in the config. Here is an example of the metadata of an existing component.
```yaml
functionality:
name: arriba
description: Detect gene fusions from RNA-Seq data
keywords: [Gene fusion, RNA-Seq]
links:
homepage: https://arriba.readthedocs.io/en/latest/
documentation: https://arriba.readthedocs.io/en/latest/
repository: https://github.com/suhrig/arriba
issue_tracker: https://github.com/suhrig/arriba/issues
references:
doi: 10.1101/gr.257246.119
bibtex: |
@article{
... a bibtex entry in case the doi is not available ...
}
license: MIT
```
### Step 4: Find a suitable container
Google `biocontainer <name of component>` and find the container that is most suitable. Typically the link will be `https://quay.io/repository/biocontainers/xxx?tab=tags`.
If no such container is found, you can create a custom container in the next step.
### Step 5: Create help file
To help develop the component, we store the `--help` output of the tool in a file at `src/xxx/help.txt`.
````bash
cat <<EOF > src/xxx/help.txt
```sh
xxx --help
```
EOF
docker run quay.io/biocontainers/xxx:tag xxx --help >> src/xxx/help.txt
````
Notes:
* This help file has no functional purpose, but it is useful for the developer to see the help output of the tool.
* Some tools might not have a `--help` argument but instead have a `-h` argument. For example, for `arriba`, the help message is obtained by running `arriba -h`:
```bash
docker run quay.io/biocontainers/arriba:2.4.0--h0033a41_2 arriba -h
```
### Step 6: Create or fetch test data
To help develop the component, it's interesting to have some test data available. In most cases, we can use the test data from the Snakemake wrappers.
To make sure we can reproduce the test data in the future, we store the command to fetch the test data in a file at `src/xxx/test_data/script.sh`.
```bash
cat <<EOF > src/xxx/test_data/script.sh
# clone repo
if [ ! -d /tmp/snakemake-wrappers ]; then
git clone --depth 1 --single-branch --branch master https://github.com/snakemake/snakemake-wrappers /tmp/snakemake-wrappers
fi
# copy test data
cp -r /tmp/snakemake-wrappers/bio/xxx/test/* src/xxx/test_data
EOF
```
The test data should be suitable for testing this component. Ensure that the test data is small enough: ideally <1KB, preferably <10KB, if need be <100KB.
### Step 7: Add arguments for the input files
By looking at the help file, we add the input arguments to the config file. Here is an example of the input arguments of an existing component.
For instance, in the [arriba help file](src/arriba/help.txt), we see the following:
Usage: arriba [-c Chimeric.out.sam] -x Aligned.out.bam \
-g annotation.gtf -a assembly.fa [-b blacklists.tsv] [-k known_fusions.tsv] \
[-t tags.tsv] [-p protein_domains.gff3] [-d structural_variants_from_WGS.tsv] \
-o fusions.tsv [-O fusions.discarded.tsv] \
[OPTIONS]
-x FILE File in SAM/BAM/CRAM format with main alignments as generated by STAR
(Aligned.out.sam). Arriba extracts candidate reads from this file.
Based on this information, we can add the following input arguments to the config file.
```yaml
argument_groups:
- name: Inputs
arguments:
- name: --bam
alternatives: -x
type: file
description: |
File in SAM/BAM/CRAM format with main alignments as generated by STAR
(Aligned.out.sam). Arriba extracts candidate reads from this file.
required: true
example: Aligned.out.bam
```
Check the [documentation](https://viash.io/reference/config/functionality/arguments) for more information on the format of input arguments.
Several notes:
* Argument names should be formatted in `--snake_case`. This means arguments like `--foo-bar` should be formatted as `--foo_bar`, and short arguments like `-f` should receive a longer name like `--foo`.
* Input arguments can have `multiple: true` to allow the user to specify multiple files.
### Step 8: Add arguments for the output files
By looking at the help file, we now also add output arguments to the config file.
For example, in the [arriba help file](src/arriba/help.txt), we see the following:
Usage: arriba [-c Chimeric.out.sam] -x Aligned.out.bam \
-g annotation.gtf -a assembly.fa [-b blacklists.tsv] [-k known_fusions.tsv] \
[-t tags.tsv] [-p protein_domains.gff3] [-d structural_variants_from_WGS.tsv] \
-o fusions.tsv [-O fusions.discarded.tsv] \
[OPTIONS]
-o FILE Output file with fusions that have passed all filters.
-O FILE Output file with fusions that were discarded due to filtering.
Based on this information, we can add the following output arguments to the config file.
```yaml
argument_groups:
- name: Outputs
arguments:
- name: --fusions
alternatives: -o
type: file
direction: output
description: |
Output file with fusions that have passed all filters.
required: true
example: fusions.tsv
- name: --fusions_discarded
alternatives: -O
type: file
direction: output
description: |
Output file with fusions that were discarded due to filtering.
required: false
example: fusions.discarded.tsv
```
Note:
* Preferably, these outputs should not be directores but files. For example, if a tool outputs a directory `foo/` containing files `foo/bar.txt` and `foo/baz.txt`, there should be two output arguments `--bar` and `--baz` (as opposed to one output argument which outputs the whole `foo/` directory).
### Step 9: Add arguments for the other arguments
Finally, add all other arguments to the config file. There are a few exceptions:
* Arguments related to specifying CPU and memory requirements are handled separately and should not be added to the config file.
* Arguments related to printing the information such as printing the version (`-v`, `--version`) or printing the help (`-h`, `--help`) should not be added to the config file.
### Step 10: Add a Docker engine
To ensure reproducibility of components, we require that all components are run in a Docker container.
```yaml
engines:
- type: docker
image: quay.io/biocontainers/xxx:0.1.0--py_0
```
The container should have your tool installed, as well as `ps`.
If you didn't find a suitable container in the previous step, you can create a custom container. For example:
```yaml
engines:
- type: docker
image: python:3.10
setup:
- type: python
packages: numpy
```
For more information on how to do this, see the [documentation](https://viash.io/guide/component/add-dependencies.html#steps-for-creating-a-custom-docker-platform).
Here is a list of base containers we can recommend:
* Bash: [`bash`](https://hub.docker.com/_/bash), [`ubuntu`](https://hub.docker.com/_/ubuntu)
* C#: [`ghcr.io/data-intuitive/dotnet-script`](https://github.com/data-intuitive/ghcr-dotnet-script/pkgs/container/dotnet-script)
* JavaScript: [`node`](https://hub.docker.com/_/node)
* Python: [`python`](https://hub.docker.com/_/python), [`nvcr.io/nvidia/pytorch`](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch)
* R: [`eddelbuettel/r2u`](https://hub.docker.com/r/eddelbuettel/r2u), [`rocker/tidyverse`](https://hub.docker.com/r/rocker/tidyverse)
* Scala: [`sbtscala/scala-sbt`](https://hub.docker.com/r/sbtscala/scala-sbt)
### Step 11: Write a runner script
Next, we need to write a runner script that runs the tool with the input arguments. Create a Bash script named `src/xxx/script.sh` which runs the tool with the input arguments.
```bash
#!/bin/bash
## VIASH START
## VIASH END
xxx \
--input "$par_input" \
--output "$par_output" \
$([ "$par_option" = "true" ] && echo "--option")
```
When building a Viash component, Viash will automatically replace the `## VIASH START` and `## VIASH END` lines (and anything in between) with environment variables based on the arguments specified in the config.
As an example, this is what the Bash script for the `arriba` component looks like:
```bash
#!/bin/bash
## VIASH START
## VIASH END
arriba \
-x "$par_bam" \
-a "$par_genome" \
-g "$par_gene_annotation" \
-o "$par_fusions" \
${par_known_fusions:+-k "${par_known_fusions}"} \
${par_blacklist:+-b "${par_blacklist}"} \
${par_structural_variants:+-d "${par_structural_variants}"} \
$([ "$par_skip_duplicate_marking" = "true" ] && echo "-u") \
$([ "$par_extra_information" = "true" ] && echo "-X") \
$([ "$par_fill_gaps" = "true" ] && echo "-I")
```
### Step 12: Create test script
If the unit test requires test resources, these should be provided in the `test_resources` section of the component.
```yaml
functionality:
# ...
test_resources:
- type: bash_script
path: test.sh
- type: file
path: test_data
```
Create a test script at `src/xxx/test.sh` that runs the component with the test data. This script should run the component (available with `$meta_executable`) with the test data and check if the output is as expected. The script should exit with a non-zero exit code if the output is not as expected. For example:
```bash
#!/bin/bash
## VIASH START
## VIASH END
echo "> Run xxx with test data"
"$meta_executable" \
--input "$meta_resources_dir/test_data/input.txt" \
--output "output.txt" \
--option
echo ">> Checking output"
[ ! -f "output.txt" ] && echo "Output file output.txt does not exist" && exit 1
```
For example, this is what the test script for the `arriba` component looks like:
```bash
#!/bin/bash
## VIASH START
## VIASH END
echo "> Run arriba with blacklist"
"$meta_executable" \
--bam "$meta_resources_dir/test_data/A.bam" \
--genome "$meta_resources_dir/test_data/genome.fasta" \
--gene_annotation "$meta_resources_dir/test_data/annotation.gtf" \
--blacklist "$meta_resources_dir/test_data/blacklist.tsv" \
--fusions "fusions.tsv" \
--fusions_discarded "fusions_discarded.tsv" \
--interesting_contigs "1,2"
echo ">> Checking output"
[ ! -f "fusions.tsv" ] && echo "Output file fusions.tsv does not exist" && exit 1
[ ! -f "fusions_discarded.tsv" ] && echo "Output file fusions_discarded.tsv does not exist" && exit 1
echo ">> Check if output is empty"
[ ! -s "fusions.tsv" ] && echo "Output file fusions.tsv is empty" && exit 1
[ ! -s "fusions_discarded.tsv" ] && echo "Output file fusions_discarded.tsv is empty" && exit 1
```
### Step 12: Create a `/var/software_versions.txt` file
For the sake of transparency and reproducibility, we require that the versions of the software used in the component are documented.
For now, this is managed by creating a file `/var/software_versions.txt` in the `setup` section of the Docker engine.
```yaml
engines:
- type: docker
image: quay.io/biocontainers/xxx:0.1.0--py_0
setup:
- type: docker
run: |
echo "xxx: \"0.1.0\"" > /var/software_versions.txt
```

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2024 Data Intuitive
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

139
README.md Normal file
View File

@@ -0,0 +1,139 @@
# 🛠📦 toolbox
[![ViashHub](https://img.shields.io/badge/ViashHub-toolbox-7a4baa.svg)](https://www.viash-hub.com/packages/toolbox)
[![GitHub](https://img.shields.io/badge/GitHub-viash--hub%2Ftoolbox-blue.svg)](https://github.com/viash-hub/toolbox)
[![GitHub
License](https://img.shields.io/github/license/viash-hub/toolbox.svg)](https://github.com/viash-hub/toolbox/blob/main/LICENSE)
[![GitHub
Issues](https://img.shields.io/github/issues/viash-hub/toolbox.svg)](https://github.com/viash-hub/toolbox/issues)
[![Viash
version](https://img.shields.io/badge/Viash-v0.9.4-blue.svg)](https://viash.io)
A collection of curated command-line tools for general IT tasks, built
with Viash.
## Introduction
`toolbox` provides a versatile suite of IT components, following the
robust Viash (https://viash.io) framework. This package focuses on
delivering reliable, standalone tools that can be easily integrated into
larger computational workflows.
The core philosophy emphasizes **reusability**, **reproducibility**, and
adherence to **best practices** in component creation. Key features of
`toolbox` components include:
- **Standalone & Nextflow Ready:** Execute components directly from the
command line or seamlessly incorporate them into Nextflow workflows.
- **High Quality Standards:**
- Comprehensive documentation for each component and its parameters.
- Full exposure of the underlying tools arguments for maximum
flexibility.
- Containerized (Docker) to ensure consistent environments and manage
dependencies, leading to enhanced reproducibility.
- Unit tested to verify functionality and ensure reliability.
## Example Usage
Viash components in toolbox can be run in various ways:
``` mermaid lang="mermaid"
flowchart TD
A[toolbox v0.1.0] --> B(Viash Hub Launch)
A --> C(Viash CLI)
A --> D(Nextflow CLI)
A --> E(Seqera Cloud)
A --> F(As a dependency)
```
### 1. Via the Viash Hub Launch interface
You can run this component directly from the Viash Hub [Launch
interface](https://www.viash-hub.com/launch?package=toolbox&version=v0.1.0&component=yq&runner=Executable).
![](docs/viash-hub.png)
### 2. Via the Viash CLI
You can run this component directly from the command line using the
Viash CLI.
``` bash
viash run vsh://toolbox@v0.1.0/yq -- --help
viash run vsh://toolbox@v0.1.0/yq -- \
--input path/to/input.yaml \
--output output.yaml
```
This will run the component with the specified input files and output
the results to the specified output file.
### 3. Via the Nextflow CLI or Seqera Cloud
You can run this component as a Nextflow pipeline.
``` bash
nextflow run https://packages.viash-hub.com/vsh/toolbox \
-revision v0.1.0 \
-main-script target/nextflow/yq/main.nf \
-latest -resume \
-profile docker \
--input path/to/input.yaml \
--publish_dir path/to/output
```
**Note:** Make sure that the [Nextflow
SCM](https://www.nextflow.io/docs/latest/git.html#git-configuration) is
set up properly. You can do this by adding the following lines to your
`~/.nextflow/scm` file:
``` groovy
providers.vsh.platform = 'gitlab'
providers.vsh.server = 'https://packages.viash-hub.com'
```
**Tip:** This will also work with Seqera Cloud or other
Nextflow-compatible platforms.
### 4. As a dependency
In your Viash config file (`config.vsh.yaml`), you can add this
component as a dependency:
``` yaml
dependencies:
- name: yq
repository: vsh://toolbox@v0.1.0
```
**Tip:** See the [Viash
documentation](https://viash.io/guide/nextflow_vdsl3/create-a-pipeline.html#pipeline-as-a-component)
for more details on how to use Viash components as a dependency in your
own Nextflow workflows.
## Contributing
Contributions are welcome! We aim to build a comprehensive collection of
high-quality bioinformatics components. If youd like to contribute,
please follow these general steps:
1. Find a component to contribute
2. Add config template
3. Fill in the metadata
4. Find a suitable container
5. Create help file
6. Create or fetch test data
7. Add arguments for the input files
8. Add arguments for the output files
9. Add arguments for the other arguments
10. Add a Docker engine
11. Write a runner script
12. Create test script
13. Create a `/var/software_versions.txt` file
See the
[CONTRIBUTING](https://github.com/viash-hub/toolbox/blob/main/CONTRIBUTING.md)
file for more details.

119
README.qmd Normal file
View File

@@ -0,0 +1,119 @@
---
format: gfm
---
```{r setup, include=FALSE}
package <- yaml::read_yaml("_viash.yaml")
license <- paste0(package$links$repository, "/blob/main/LICENSE")
contributing <- paste0(package$links$repository, "/blob/main/CONTRIBUTING.md")
pkg <- package$name
ver <- if (!is.null(package$version)) package$version else "v0.1.0"
comp <- "yq"
```
# 🛠📦 `r pkg`
[![ViashHub](https://img.shields.io/badge/ViashHub-`r pkg`-7a4baa.svg)](https://www.viash-hub.com/packages/`r pkg`)
[![GitHub](https://img.shields.io/badge/GitHub-viash--hub%2F`r pkg`-blue.svg)](`r package$links$repository`)
[![GitHub License](https://img.shields.io/github/license/viash-hub/`r pkg`.svg)](`r license`)
[![GitHub Issues](https://img.shields.io/github/issues/viash-hub/`r pkg`.svg)](`r package$links$issue_tracker`)
[![Viash version](https://img.shields.io/badge/Viash-v`r gsub("-", "--", package$viash_version)`-blue.svg)](https://viash.io)
`r package$summary`
## Introduction
`r package$description`
## Example Usage
Viash components in `r pkg` can be run in various ways:
```{r mmd, echo=FALSE, results='asis'}
cat(
"```mermaid\n",
"flowchart TD\n",
" A[", pkg, " ", ver, "] --> B(Viash Hub Launch)\n",
" A --> C(Viash CLI)\n",
" A --> D(Nextflow CLI)\n",
" A --> E(Seqera Cloud)\n",
" A --> F(As a dependency)\n",
"```\n",
sep = ""
)
```
### 1. Via the Viash Hub Launch interface
You can run this component directly from the Viash Hub [Launch interface](https://www.viash-hub.com/launch?package=`r pkg`&version=`r ver`&component=`r comp`&runner=Executable).
![](docs/viash-hub.png)
### 2. Via the Viash CLI
You can run this component directly from the command line using the Viash CLI.
```bash
viash run vsh://`r pkg`@`r ver`/`r comp` -- --help
viash run vsh://`r pkg`@`r ver`/`r comp` -- \
--input path/to/input.yaml \
--output output.yaml
```
This will run the component with the specified input files and output the results to the specified output file.
### 3. Via the Nextflow CLI or Seqera Cloud
You can run this component as a Nextflow pipeline.
```bash
nextflow run https://packages.viash-hub.com/vsh/`r pkg` \
-revision `r ver` \
-main-script target/nextflow/`r comp`/main.nf \
-latest -resume \
-profile docker \
--input path/to/input.yaml \
--publish_dir path/to/output
```
**Note:** Make sure that the [Nextflow SCM](https://www.nextflow.io/docs/latest/git.html#git-configuration) is set up properly. You can do this by adding the following lines to your `~/.nextflow/scm` file:
```groovy
providers.vsh.platform = 'gitlab'
providers.vsh.server = 'https://packages.viash-hub.com'
```
**Tip:** This will also work with Seqera Cloud or other Nextflow-compatible platforms.
### 4. As a dependency
In your Viash config file (`config.vsh.yaml`), you can add this component as a dependency:
```yaml
dependencies:
- name: `r comp`
repository: vsh://`r pkg`@`r ver`
```
**Tip:** See the [Viash documentation](https://viash.io/guide/nextflow_vdsl3/create-a-pipeline.html#pipeline-as-a-component) for more details on how to use Viash components as a dependency in your own Nextflow workflows.
## Contributing
Contributions are welcome! We aim to build a comprehensive collection of high-quality bioinformatics components. If you'd like to contribute, please follow these general steps:
```{r echo=FALSE}
lines <- readr::read_lines("CONTRIBUTING.md")
index_start <- grep("^### Step [0-9]*:", lines)
index_end <- c(index_start[-1] - 1, length(lines))
name <- gsub("^### Step [0-9]*: *", "", lines[index_start])
knitr::asis_output(
paste(paste0(" 1. ", name, "\n"), collapse = "")
)
```
See the [CONTRIBUTING](`r contributing`) file for more details.

26
_viash.yaml Normal file
View File

@@ -0,0 +1,26 @@
name: toolbox
version: v0.1.1
summary: |
A collection of curated command-line tools for general IT tasks, built with Viash.
description: |
`toolbox` provides a versatile suite of IT components, following the robust Viash (https://viash.io) framework.
This package focuses on delivering reliable, standalone tools that can be easily integrated into larger computational workflows.
The core philosophy emphasizes **reusability**, **reproducibility**, and adherence to **best practices** in component creation. Key features of `toolbox` components include:
* **Standalone & Nextflow Ready:** Execute components directly from the command line or seamlessly incorporate them into Nextflow workflows.
* **High Quality Standards:**
* Comprehensive documentation for each component and its parameters.
* Full exposure of the underlying tool's arguments for maximum flexibility.
* Containerized (Docker) to ensure consistent environments and manage dependencies, leading to enhanced reproducibility.
* Unit tested to verify functionality and ensure reliability.
license: MIT
keywords: [toolbox, command-line, tools]
links:
issue_tracker: https://github.com/viash-hub/toolbox/issues
repository: https://github.com/viash-hub/toolbox
viash_version: 0.9.4
config_mods: |
.requirements.commands := ['ps']

BIN
docs/viash-hub.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 147 KiB

3
main.nf Normal file
View File

@@ -0,0 +1,3 @@
workflow {
print("This is a dummy placeholder for pipeline execution. Please use the corresponding nf files for running pipelines.")
}

6
nextflow.config Normal file
View File

@@ -0,0 +1,6 @@
manifest {
name = "toolbox"
version = "v0.1.1"
defaultBranch = "main"
nextflowVersion = "!>=20.12.1-edge"
}

90
src/bgzip/config.vsh.yaml Normal file
View File

@@ -0,0 +1,90 @@
name: bgzip
description: Block compression/decompression utility
links:
homepage: https://www.htslib.org/
documentation: https://www.htslib.org/doc/bgzip.html
repository: https://github.com/samtools/htslib
references:
doi: 10.1093/gigascience/giab007
license: MIT
requirements:
commands: [ bgzip ]
argument_groups:
- name: Inputs
arguments:
- name: --input
type: file
direction: input
description: file to be compressed or decompressed
required: true
- name: Outputs
arguments:
- name: --output
type: file
direction: output
description: compressed or decompressed output
required: true
- name: --index_name
alternatives: -I
type: file
direction: output
description: name of BGZF index file [file.gz.gzi]
- name: Arguments
arguments:
- name: --offset
alternatives: -b
type: integer
description: decompress at virtual file pointer (0-based uncompressed offset)
- name: --decompress
alternatives: -d
type: boolean_true
description: decompress the input file
- name: --rebgzip
alternatives: -g
type: boolean_true
description: use an index file to bgzip a file
- name: --index
alternatives: -i
type: boolean_true
description: compress and create BGZF index
- name: --compress_level
alternatives: -l
type: integer
description: compression level to use when compressing; 0 to 9, or -1 for default [-1]
min: -1
max: 9
- name: --reindex
alternatives: -r
type: boolean_true
description: (re)index the output file
- name: --size
alternatives: -s
type: integer
description: decompress INT bytes (uncompressed size)
min: 0
- name: --test
alternatives: -t
type: boolean_true
description: test integrity of compressed file
- name: --binary
type: boolean_true
description: Don't align blocks with text lines
resources:
- type: bash_script
path: script.sh
test_resources:
- type: bash_script
path: test.sh
- type: file
path: test_data
engines:
- type: docker
image: quay.io/biocontainers/htslib:1.19--h81da01d_0
setup:
- type: docker
run: |
bgzip -h | grep 'Version:' 2>&1 | sed 's/Version:\s\(.*\)/bgzip: "\1"/' > /var/software_versions.txt
runners:
- type: executable
- type: nextflow

22
src/bgzip/help.txt Normal file
View File

@@ -0,0 +1,22 @@
```bash
bgzip -h
```
Version: 1.19
Usage: bgzip [OPTIONS] [FILE] ...
Options:
-b, --offset INT decompress at virtual file pointer (0-based uncompressed offset)
-c, --stdout write on standard output, keep original files unchanged
-d, --decompress decompress
-f, --force overwrite files without asking
-g, --rebgzip use an index file to bgzip a file
-h, --help give this help
-i, --index compress and create BGZF index
-I, --index-name FILE name of BGZF index file [file.gz.gzi]
-k, --keep don't delete input files during operation
-l, --compress-level INT Compression level to use when compressing; 0 to 9, or -1 for default [-1]
-r, --reindex (re)index compressed file
-s, --size INT decompress INT bytes (uncompressed size)
-t, --test test integrity of compressed file
--binary Don't align blocks with text lines
-@, --threads INT number of compression threads to use [1]

21
src/bgzip/script.sh Executable file
View File

@@ -0,0 +1,21 @@
#!/bin/bash
[[ "$par_decompress" == "false" ]] && unset par_decompress
[[ "$par_rebgzip" == "false" ]] && unset par_rebgzip
[[ "$par_index" == "false" ]] && unset par_index
[[ "$par_reindex" == "false" ]] && unset par_reindex
[[ "$par_test" == "false" ]] && unset par_test
[[ "$par_binary" == "false" ]] && unset par_binary
bgzip -c \
${meta_cpus:+--threads "${meta_cpus}"} \
${par_offset:+-b "${par_offset}"} \
${par_decompress:+-d} \
${par_rebgzip:+-g} \
${par_index:+-i} \
${par_index_name:+-I "${par_index_name}"} \
${par_compress_level:+-l "${par_compress_level}"} \
${par_reindex:+-r} \
${par_size:+-s "${par_size}"} \
${par_test:+-t} \
${par_binary:+--binary} \
"$par_input" > "$par_output"

19
src/bgzip/test.sh Executable file
View File

@@ -0,0 +1,19 @@
set -e
"$meta_executable" --input "$meta_resources_dir/test_data/test.vcf" --output "test.vcf.gz"
echo ">> Checking output of compressing"
[ ! -f "test.vcf.gz" ] && echo "Output file test.vcf.gz does not exist" && exit 1
"$meta_executable" --input "test.vcf.gz" --output "test.vcf" --decompress
echo ">> Checking output of decompressing"
[ ! -f "test.vcf" ] && echo "Output file test.vcf does not exist" && exit 1
echo ">> Checking original and decompressed files are the same"
set +e
cmp --silent -- "$meta_resources_dir/test_data/test.vcf" "test.vcf"
[ $? -ne 0 ] && echo "files are different" && exit 1
set -e
echo "> Test successful"

View File

@@ -0,0 +1,10 @@
# bgzip test data
# Test data was obtained from https://github.com/snakemake/snakemake-wrappers/tree/master/bio/bgzip/test.
if [ ! -d /tmp/snakemake-wrappers ]; then
git clone --depth 1 --single-branch --branch master https://github.com/snakemake/snakemake-wrappers /tmp/snakemake-wrappers
fi
cp -r /tmp/snakemake-wrappers/bio/bgzip/test/* src/bgzip/test_data

View File

@@ -0,0 +1,23 @@
##fileformat=VCFv4.0
##fileDate=20090805
##source=https://www.internationalgenome.org/wiki/Analysis/vcf4.0/
##reference=1000GenomesPilot-NCBI36
##phasing=partial
##INFO=<ID=NS,Number=1,Type=Integer,Description="Number of Samples With Data">
##INFO=<ID=DP,Number=1,Type=Integer,Description="Total Depth">
##INFO=<ID=AF,Number=.,Type=Float,Description="Allele Frequency">
##INFO=<ID=AA,Number=1,Type=String,Description="Ancestral Allele">
##INFO=<ID=DB,Number=0,Type=Flag,Description="dbSNP membership, build 129">
##INFO=<ID=H2,Number=0,Type=Flag,Description="HapMap2 membership">
##FILTER=<ID=q10,Description="Quality below 10">
##FILTER=<ID=s50,Description="Less than 50% of samples have data">
##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype">
##FORMAT=<ID=GQ,Number=1,Type=Integer,Description="Genotype Quality">
##FORMAT=<ID=DP,Number=1,Type=Integer,Description="Read Depth">
##FORMAT=<ID=HQ,Number=2,Type=Integer,Description="Haplotype Quality">
#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT NA00001 NA00002 NA00003
20 14370 rs6054257 G A 29 PASS NS=3;DP=14;AF=0.5;DB;H2 GT:GQ:DP:HQ 0|0:48:1:51,51 1|0:48:8:51,51 1/1:43:5:.,.
20 17330 . T A 3 q10 NS=3;DP=11;AF=0.017 GT:GQ:DP:HQ 0|0:49:3:58,50 0|1:3:5:65,3 0/0:41:3
20 1110696 rs6040355 A G,T 67 PASS NS=2;DP=10;AF=0.333,0.667;AA=T;DB GT:GQ:DP:HQ 1|2:21:6:23,27 2|1:2:0:18,2 2/2:35:4
20 1230237 . T . 47 PASS NS=3;DP=13;AA=T GT:GQ:DP:HQ 0|0:54:7:56,60 0|0:48:4:51,51 0/0:61:2
20 1234567 microsat1 GTCT G,GTACT 50 PASS NS=3;DP=9;AA=G GT:GQ:DP 0/1:35:4 0/2:17:2 1/1:40:3

87
src/yq/config.vsh.yaml Normal file
View File

@@ -0,0 +1,87 @@
name: yq
description: A portable YAML, JSON, XML, CSV, TOML and properties processor
keywords: [ yaml, json, xml, csv, toml, properties ]
links:
homepage: https://mikefarah.gitbook.io/yq
documentation: https://mikefarah.gitbook.io/yq/
repository: https://github.com/mikefarah/yq
license: MIT
requirements:
commands: [ yq ]
argument_groups:
- name: Inputs
arguments:
- name: --input
type: file
direction: input
description: files to be processed
required: true
example: input.yaml
- name: Outputs
arguments:
- name: --output
type: file
direction: output
description: output file
required: true
example: output.yaml
- name: Arguments
arguments:
- name: --eval
type: string
description: expression to evaluate
required: true
example: '.name = "foo"'
- name: --indent
type: integer
description: sets indent level for output (default 2)
alternatives: -I
- name: --input_format
type: string
description: 'parse format for input. (default "auto")'
alternatives: -p
choices: [ auto, a, yaml, "y", json, j, props, p, csv, c, tsv, t, xml, x, base64, uri, toml, shell, s, lua, l ]
- name: --output_format
type: string
description: 'output format type. (default "auto")'
alternatives: -o
choices: [ auto, a, yaml, "y", json, j, props, p, csv, c, tsv, t, xml, x, base64, uri, toml, shell, s, lua, l ]
- name: --pretty_print
type: boolean_true
description: pretty print, shorthand for '... style = ""'
alternatives: -P
resources:
- type: bash_script
text: |
#!/bin/sh
[[ "$par_pretty_print" == "false" ]] && unset par_pretty_print
yq eval \
${par_indent:+-I "${par_indent}"} \
${par_input_format:+-p "${par_input_format}"} \
${par_output_format:+-o "${par_output_format}"} \
${par_pretty_print:+-P} \
--expression "$par_eval" \
--no-colors \
"$par_input" > "$par_output"
test_resources:
- type: bash_script
text: |
set -e
echo "name: 'bar'" > test.yaml
"$meta_executable" --input test.yaml --output output.yaml --eval '.name = "foo"'
"$meta_executable" --input output.yaml --output output2.yaml --eval '.name'
grep "^foo$" output2.yaml
engines:
- type: docker
image: alpine:latest
setup:
- type: apk
packages: [bash, yq-go]
- type: docker
run: |
/usr/bin/yq --version | sed 's/.*version\sv\(.*\)/yq: "\1"/' > /var/software_versions.txt
runners:
- type: executable
- type: nextflow

72
src/yq/help.txt Normal file
View File

@@ -0,0 +1,72 @@
yq is a portable command-line data file processor (https://github.com/mikefarah/yq/)
See https://mikefarah.gitbook.io/yq/ for detailed documentation and examples.
Usage:
yq [flags]
yq [command]
Examples:
# yq tries to auto-detect the file format based off the extension, and defaults to YAML if it's unknown (or piping through STDIN)
# Use the '-p/--input-format' flag to specify a format type.
cat file.xml | yq -p xml
# read the "stuff" node from "myfile.yml"
yq '.stuff' < myfile.yml
# update myfile.yml in place
yq -i '.stuff = "foo"' myfile.yml
# print contents of sample.json as idiomatic YAML
yq -P -oy sample.json
Available Commands:
completion Generate the autocompletion script for the specified shell
eval (default) Apply the expression to each document in each yaml file in sequence
eval-all Loads _all_ yaml documents of _all_ yaml files and runs expression once
help Help about any command
Flags:
-C, --colors force print with colors
--csv-auto-parse parse CSV YAML/JSON values (default true)
--csv-separator char CSV Separator character (default ,)
-e, --exit-status set exit status if there are no matches or null or false is returned
--expression string forcibly set the expression argument. Useful when yq argument detection thinks your expression is a file.
--from-file string Load expression from specified file.
-f, --front-matter string (extract|process) first input as yaml front-matter. Extract will pull out the yaml content, process will run the expression against the yaml content, leaving the remaining data intact
--header-preprocess Slurp any header comments and separators before processing expression. (default true)
-h, --help help for yq
-I, --indent int sets indent level for output (default 2)
-i, --inplace update the file in place of first file given.
-p, --input-format string [auto|a|yaml|y|json|j|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|lua|l] parse format for input. (default "auto")
--lua-globals output keys as top-level global variables
--lua-prefix string prefix (default "return ")
--lua-suffix string suffix (default ";\n")
--lua-unquoted output unquoted string keys (e.g. {foo="bar"})
-M, --no-colors force print with no colors
-N, --no-doc Don't print document separators (---)
-0, --nul-output Use NUL char to separate values. If unwrap scalar is also set, fail if unwrapped scalar contains NUL char.
-n, --null-input Don't read input, simply evaluate the expression given. Useful for creating docs from scratch.
-o, --output-format string [auto|a|yaml|y|json|j|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|shell|s|lua|l] output format type. (default "auto")
-P, --prettyPrint pretty print, shorthand for '... style = ""'
--properties-array-brackets use [x] in array paths (e.g. for SpringBoot)
--properties-separator string separator to use between keys and values (default " = ")
-s, --split-exp string print each result (or doc) into a file named (exp). [exp] argument must return a string. You can use $index in the expression as the result counter.
--split-exp-file string Use a file to specify the split-exp expression.
--string-interpolation Toggles strings interpolation of \(exp) (default true)
--tsv-auto-parse parse TSV YAML/JSON values (default true)
-r, --unwrapScalar unwrap scalar, print the value with no quotes, colors or comments. Defaults to true for yaml (default true)
-v, --verbose verbose mode
-V, --version Print version information and quit
--xml-attribute-prefix string prefix for xml attributes (default "+@")
--xml-content-name string name for xml content (if no attribute name is present). (default "+content")
--xml-directive-name string name for xml directives (e.g. <!DOCTYPE thing cat>) (default "+directive")
--xml-keep-namespace enables keeping namespace after parsing attributes (default true)
--xml-proc-inst-prefix string prefix for xml processing instructions (e.g. <?xml version="1"?>) (default "+p_")
--xml-raw-token enables using RawToken method instead Token. Commonly disables namespace translations. See https://pkg.go.dev/encoding/xml#Decoder.RawToken for details. (default true)
--xml-skip-directives skip over directives (e.g. <!DOCTYPE thing cat>)
--xml-skip-proc-inst skip over process instructions (e.g. <?xml version="1"?>)
--xml-strict-mode enables strict parsing of XML. See https://pkg.go.dev/encoding/xml for more details.
Use "yq [command] --help" for more information about a command.

0
target/.build.yaml Normal file
View File

View File

@@ -0,0 +1,268 @@
name: "bgzip"
version: "v0.1.1"
argument_groups:
- name: "Inputs"
arguments:
- type: "file"
name: "--input"
description: "file to be compressed or decompressed"
info: null
must_exist: true
create_parent: true
required: true
direction: "input"
multiple: false
multiple_sep: ";"
- name: "Outputs"
arguments:
- type: "file"
name: "--output"
description: "compressed or decompressed output"
info: null
must_exist: true
create_parent: true
required: true
direction: "output"
multiple: false
multiple_sep: ";"
- type: "file"
name: "--index_name"
alternatives:
- "-I"
description: "name of BGZF index file [file.gz.gzi]"
info: null
must_exist: true
create_parent: true
required: false
direction: "output"
multiple: false
multiple_sep: ";"
- name: "Arguments"
arguments:
- type: "integer"
name: "--offset"
alternatives:
- "-b"
description: "decompress at virtual file pointer (0-based uncompressed offset)"
info: null
required: false
direction: "input"
multiple: false
multiple_sep: ";"
- type: "boolean_true"
name: "--decompress"
alternatives:
- "-d"
description: "decompress the input file"
info: null
direction: "input"
- type: "boolean_true"
name: "--rebgzip"
alternatives:
- "-g"
description: "use an index file to bgzip a file"
info: null
direction: "input"
- type: "boolean_true"
name: "--index"
alternatives:
- "-i"
description: "compress and create BGZF index"
info: null
direction: "input"
- type: "integer"
name: "--compress_level"
alternatives:
- "-l"
description: "compression level to use when compressing; 0 to 9, or -1 for default\
\ [-1]"
info: null
required: false
min: -1
max: 9
direction: "input"
multiple: false
multiple_sep: ";"
- type: "boolean_true"
name: "--reindex"
alternatives:
- "-r"
description: "(re)index the output file"
info: null
direction: "input"
- type: "integer"
name: "--size"
alternatives:
- "-s"
description: "decompress INT bytes (uncompressed size)"
info: null
required: false
min: 0
direction: "input"
multiple: false
multiple_sep: ";"
- type: "boolean_true"
name: "--test"
alternatives:
- "-t"
description: "test integrity of compressed file"
info: null
direction: "input"
- type: "boolean_true"
name: "--binary"
description: "Don't align blocks with text lines"
info: null
direction: "input"
resources:
- type: "bash_script"
path: "script.sh"
is_executable: true
description: "Block compression/decompression utility"
test_resources:
- type: "bash_script"
path: "test.sh"
is_executable: true
- type: "file"
path: "test_data"
info: null
status: "enabled"
scope:
image: "public"
target: "public"
requirements:
commands:
- "ps"
license: "MIT"
references:
doi:
- "10.1093/gigascience/giab007"
links:
repository: "https://github.com/samtools/htslib"
homepage: "https://www.htslib.org/"
documentation: "https://www.htslib.org/doc/bgzip.html"
runners:
- type: "executable"
id: "executable"
docker_setup_strategy: "ifneedbepullelsecachedbuild"
- type: "nextflow"
id: "nextflow"
directives:
tag: "$id"
auto:
simplifyInput: true
simplifyOutput: false
transcript: false
publish: false
config:
labels:
mem1gb: "memory = 1000000000.B"
mem2gb: "memory = 2000000000.B"
mem5gb: "memory = 5000000000.B"
mem10gb: "memory = 10000000000.B"
mem20gb: "memory = 20000000000.B"
mem50gb: "memory = 50000000000.B"
mem100gb: "memory = 100000000000.B"
mem200gb: "memory = 200000000000.B"
mem500gb: "memory = 500000000000.B"
mem1tb: "memory = 1000000000000.B"
mem2tb: "memory = 2000000000000.B"
mem5tb: "memory = 5000000000000.B"
mem10tb: "memory = 10000000000000.B"
mem20tb: "memory = 20000000000000.B"
mem50tb: "memory = 50000000000000.B"
mem100tb: "memory = 100000000000000.B"
mem200tb: "memory = 200000000000000.B"
mem500tb: "memory = 500000000000000.B"
mem1gib: "memory = 1073741824.B"
mem2gib: "memory = 2147483648.B"
mem4gib: "memory = 4294967296.B"
mem8gib: "memory = 8589934592.B"
mem16gib: "memory = 17179869184.B"
mem32gib: "memory = 34359738368.B"
mem64gib: "memory = 68719476736.B"
mem128gib: "memory = 137438953472.B"
mem256gib: "memory = 274877906944.B"
mem512gib: "memory = 549755813888.B"
mem1tib: "memory = 1099511627776.B"
mem2tib: "memory = 2199023255552.B"
mem4tib: "memory = 4398046511104.B"
mem8tib: "memory = 8796093022208.B"
mem16tib: "memory = 17592186044416.B"
mem32tib: "memory = 35184372088832.B"
mem64tib: "memory = 70368744177664.B"
mem128tib: "memory = 140737488355328.B"
mem256tib: "memory = 281474976710656.B"
mem512tib: "memory = 562949953421312.B"
cpu1: "cpus = 1"
cpu2: "cpus = 2"
cpu5: "cpus = 5"
cpu10: "cpus = 10"
cpu20: "cpus = 20"
cpu50: "cpus = 50"
cpu100: "cpus = 100"
cpu200: "cpus = 200"
cpu500: "cpus = 500"
cpu1000: "cpus = 1000"
debug: false
container: "docker"
engines:
- type: "docker"
id: "docker"
image: "quay.io/biocontainers/htslib:1.19--h81da01d_0"
target_registry: "images.viash-hub.com"
target_tag: "v0.1.1"
namespace_separator: "/"
setup:
- type: "docker"
run:
- "bgzip -h | grep 'Version:' 2>&1 | sed 's/Version:\\s\\(.*\\)/bgzip: \"\\1\"\
/' > /var/software_versions.txt\n"
entrypoint: []
cmd: null
- type: "native"
id: "native"
build_info:
config: "src/bgzip/config.vsh.yaml"
runner: "executable"
engine: "docker|native"
output: "target/executable/bgzip"
executable: "target/executable/bgzip/bgzip"
viash_version: "0.9.4"
git_commit: "2c079795592b75ecd0db2bacaf0b748fa84e1293"
git_remote: "https://github.com/viash-hub/toolbox"
git_tag: "v0.1.0-13-g2c07979"
package_config:
name: "toolbox"
version: "v0.1.1"
summary: "A collection of curated command-line tools for general IT tasks, built\
\ with Viash.\n"
description: "`toolbox` provides a versatile suite of IT components, following the\
\ robust Viash (https://viash.io) framework.\nThis package focuses on delivering\
\ reliable, standalone tools that can be easily integrated into larger computational\
\ workflows.\n\nThe core philosophy emphasizes **reusability**, **reproducibility**,\
\ and adherence to **best practices** in component creation. Key features of `toolbox`\
\ components include:\n\n* **Standalone & Nextflow Ready:** Execute components\
\ directly from the command line or seamlessly incorporate them into Nextflow\
\ workflows.\n* **High Quality Standards:**\n * Comprehensive documentation\
\ for each component and its parameters.\n * Full exposure of the underlying\
\ tool's arguments for maximum flexibility.\n * Containerized (Docker) to ensure\
\ consistent environments and manage dependencies, leading to enhanced reproducibility.\n\
\ * Unit tested to verify functionality and ensure reliability.\n"
info: null
viash_version: "0.9.4"
source: "src"
target: "target"
config_mods:
- ".requirements.commands := ['ps']\n"
- ".engines += { type: \"native\" }"
- ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
- ".engines[.type == 'docker'].target_tag := 'v0.1.1'"
keywords:
- "toolbox"
- "command-line"
- "tools"
license: "MIT"
organization: "vsh"
links:
repository: "https://github.com/viash-hub/toolbox"
issue_tracker: "https://github.com/viash-hub/toolbox/issues"

1397
target/executable/bgzip/bgzip Executable file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,298 @@
name: "yq"
version: "v0.1.1"
argument_groups:
- name: "Inputs"
arguments:
- type: "file"
name: "--input"
description: "files to be processed"
info: null
example:
- "input.yaml"
must_exist: true
create_parent: true
required: true
direction: "input"
multiple: false
multiple_sep: ";"
- name: "Outputs"
arguments:
- type: "file"
name: "--output"
description: "output file"
info: null
example:
- "output.yaml"
must_exist: true
create_parent: true
required: true
direction: "output"
multiple: false
multiple_sep: ";"
- name: "Arguments"
arguments:
- type: "string"
name: "--eval"
description: "expression to evaluate"
info: null
example:
- ".name = \"foo\""
required: true
direction: "input"
multiple: false
multiple_sep: ";"
- type: "integer"
name: "--indent"
alternatives:
- "-I"
description: "sets indent level for output (default 2)"
info: null
required: false
direction: "input"
multiple: false
multiple_sep: ";"
- type: "string"
name: "--input_format"
alternatives:
- "-p"
description: "parse format for input. (default \"auto\")"
info: null
required: false
choices:
- "auto"
- "a"
- "yaml"
- "y"
- "json"
- "j"
- "props"
- "p"
- "csv"
- "c"
- "tsv"
- "t"
- "xml"
- "x"
- "base64"
- "uri"
- "toml"
- "shell"
- "s"
- "lua"
- "l"
direction: "input"
multiple: false
multiple_sep: ";"
- type: "string"
name: "--output_format"
alternatives:
- "-o"
description: "output format type. (default \"auto\")"
info: null
required: false
choices:
- "auto"
- "a"
- "yaml"
- "y"
- "json"
- "j"
- "props"
- "p"
- "csv"
- "c"
- "tsv"
- "t"
- "xml"
- "x"
- "base64"
- "uri"
- "toml"
- "shell"
- "s"
- "lua"
- "l"
direction: "input"
multiple: false
multiple_sep: ";"
- type: "boolean_true"
name: "--pretty_print"
alternatives:
- "-P"
description: "pretty print, shorthand for '... style = \"\"'"
info: null
direction: "input"
resources:
- type: "bash_script"
text: |
#!/bin/sh
[[ "$par_pretty_print" == "false" ]] && unset par_pretty_print
yq eval \
${par_indent:+-I "${par_indent}"} \
${par_input_format:+-p "${par_input_format}"} \
${par_output_format:+-o "${par_output_format}"} \
${par_pretty_print:+-P} \
--expression "$par_eval" \
--no-colors \
"$par_input" > "$par_output"
dest: "./script.sh"
is_executable: true
description: "A portable YAML, JSON, XML, CSV, TOML and properties processor"
test_resources:
- type: "bash_script"
text: "set -e\necho \"name: 'bar'\" > test.yaml\n\"$meta_executable\" --input test.yaml\
\ --output output.yaml --eval '.name = \"foo\"'\n\"$meta_executable\" --input\
\ output.yaml --output output2.yaml --eval '.name'\ngrep \"^foo$\" output2.yaml\n"
dest: "./script.sh"
is_executable: true
info: null
status: "enabled"
scope:
image: "public"
target: "public"
requirements:
commands:
- "ps"
keywords:
- "yaml"
- "json"
- "xml"
- "csv"
- "toml"
- "properties"
license: "MIT"
links:
repository: "https://github.com/mikefarah/yq"
homepage: "https://mikefarah.gitbook.io/yq"
documentation: "https://mikefarah.gitbook.io/yq/"
runners:
- type: "executable"
id: "executable"
docker_setup_strategy: "ifneedbepullelsecachedbuild"
- type: "nextflow"
id: "nextflow"
directives:
tag: "$id"
auto:
simplifyInput: true
simplifyOutput: false
transcript: false
publish: false
config:
labels:
mem1gb: "memory = 1000000000.B"
mem2gb: "memory = 2000000000.B"
mem5gb: "memory = 5000000000.B"
mem10gb: "memory = 10000000000.B"
mem20gb: "memory = 20000000000.B"
mem50gb: "memory = 50000000000.B"
mem100gb: "memory = 100000000000.B"
mem200gb: "memory = 200000000000.B"
mem500gb: "memory = 500000000000.B"
mem1tb: "memory = 1000000000000.B"
mem2tb: "memory = 2000000000000.B"
mem5tb: "memory = 5000000000000.B"
mem10tb: "memory = 10000000000000.B"
mem20tb: "memory = 20000000000000.B"
mem50tb: "memory = 50000000000000.B"
mem100tb: "memory = 100000000000000.B"
mem200tb: "memory = 200000000000000.B"
mem500tb: "memory = 500000000000000.B"
mem1gib: "memory = 1073741824.B"
mem2gib: "memory = 2147483648.B"
mem4gib: "memory = 4294967296.B"
mem8gib: "memory = 8589934592.B"
mem16gib: "memory = 17179869184.B"
mem32gib: "memory = 34359738368.B"
mem64gib: "memory = 68719476736.B"
mem128gib: "memory = 137438953472.B"
mem256gib: "memory = 274877906944.B"
mem512gib: "memory = 549755813888.B"
mem1tib: "memory = 1099511627776.B"
mem2tib: "memory = 2199023255552.B"
mem4tib: "memory = 4398046511104.B"
mem8tib: "memory = 8796093022208.B"
mem16tib: "memory = 17592186044416.B"
mem32tib: "memory = 35184372088832.B"
mem64tib: "memory = 70368744177664.B"
mem128tib: "memory = 140737488355328.B"
mem256tib: "memory = 281474976710656.B"
mem512tib: "memory = 562949953421312.B"
cpu1: "cpus = 1"
cpu2: "cpus = 2"
cpu5: "cpus = 5"
cpu10: "cpus = 10"
cpu20: "cpus = 20"
cpu50: "cpus = 50"
cpu100: "cpus = 100"
cpu200: "cpus = 200"
cpu500: "cpus = 500"
cpu1000: "cpus = 1000"
debug: false
container: "docker"
engines:
- type: "docker"
id: "docker"
image: "alpine:latest"
target_registry: "images.viash-hub.com"
target_tag: "v0.1.1"
namespace_separator: "/"
setup:
- type: "apk"
packages:
- "bash"
- "yq-go"
- type: "docker"
run:
- "/usr/bin/yq --version | sed 's/.*version\\sv\\(.*\\)/yq: \"\\1\"/' > /var/software_versions.txt\n"
entrypoint: []
cmd: null
- type: "native"
id: "native"
build_info:
config: "src/yq/config.vsh.yaml"
runner: "executable"
engine: "docker|native"
output: "target/executable/yq"
executable: "target/executable/yq/yq"
viash_version: "0.9.4"
git_commit: "2c079795592b75ecd0db2bacaf0b748fa84e1293"
git_remote: "https://github.com/viash-hub/toolbox"
git_tag: "v0.1.0-13-g2c07979"
package_config:
name: "toolbox"
version: "v0.1.1"
summary: "A collection of curated command-line tools for general IT tasks, built\
\ with Viash.\n"
description: "`toolbox` provides a versatile suite of IT components, following the\
\ robust Viash (https://viash.io) framework.\nThis package focuses on delivering\
\ reliable, standalone tools that can be easily integrated into larger computational\
\ workflows.\n\nThe core philosophy emphasizes **reusability**, **reproducibility**,\
\ and adherence to **best practices** in component creation. Key features of `toolbox`\
\ components include:\n\n* **Standalone & Nextflow Ready:** Execute components\
\ directly from the command line or seamlessly incorporate them into Nextflow\
\ workflows.\n* **High Quality Standards:**\n * Comprehensive documentation\
\ for each component and its parameters.\n * Full exposure of the underlying\
\ tool's arguments for maximum flexibility.\n * Containerized (Docker) to ensure\
\ consistent environments and manage dependencies, leading to enhanced reproducibility.\n\
\ * Unit tested to verify functionality and ensure reliability.\n"
info: null
viash_version: "0.9.4"
source: "src"
target: "target"
config_mods:
- ".requirements.commands := ['ps']\n"
- ".engines += { type: \"native\" }"
- ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
- ".engines[.type == 'docker'].target_tag := 'v0.1.1'"
keywords:
- "toolbox"
- "command-line"
- "tools"
license: "MIT"
organization: "vsh"
links:
repository: "https://github.com/viash-hub/toolbox"
issue_tracker: "https://github.com/viash-hub/toolbox/issues"

1260
target/executable/yq/yq Executable file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,268 @@
name: "bgzip"
version: "v0.1.1"
argument_groups:
- name: "Inputs"
arguments:
- type: "file"
name: "--input"
description: "file to be compressed or decompressed"
info: null
must_exist: true
create_parent: true
required: true
direction: "input"
multiple: false
multiple_sep: ";"
- name: "Outputs"
arguments:
- type: "file"
name: "--output"
description: "compressed or decompressed output"
info: null
must_exist: true
create_parent: true
required: true
direction: "output"
multiple: false
multiple_sep: ";"
- type: "file"
name: "--index_name"
alternatives:
- "-I"
description: "name of BGZF index file [file.gz.gzi]"
info: null
must_exist: true
create_parent: true
required: false
direction: "output"
multiple: false
multiple_sep: ";"
- name: "Arguments"
arguments:
- type: "integer"
name: "--offset"
alternatives:
- "-b"
description: "decompress at virtual file pointer (0-based uncompressed offset)"
info: null
required: false
direction: "input"
multiple: false
multiple_sep: ";"
- type: "boolean_true"
name: "--decompress"
alternatives:
- "-d"
description: "decompress the input file"
info: null
direction: "input"
- type: "boolean_true"
name: "--rebgzip"
alternatives:
- "-g"
description: "use an index file to bgzip a file"
info: null
direction: "input"
- type: "boolean_true"
name: "--index"
alternatives:
- "-i"
description: "compress and create BGZF index"
info: null
direction: "input"
- type: "integer"
name: "--compress_level"
alternatives:
- "-l"
description: "compression level to use when compressing; 0 to 9, or -1 for default\
\ [-1]"
info: null
required: false
min: -1
max: 9
direction: "input"
multiple: false
multiple_sep: ";"
- type: "boolean_true"
name: "--reindex"
alternatives:
- "-r"
description: "(re)index the output file"
info: null
direction: "input"
- type: "integer"
name: "--size"
alternatives:
- "-s"
description: "decompress INT bytes (uncompressed size)"
info: null
required: false
min: 0
direction: "input"
multiple: false
multiple_sep: ";"
- type: "boolean_true"
name: "--test"
alternatives:
- "-t"
description: "test integrity of compressed file"
info: null
direction: "input"
- type: "boolean_true"
name: "--binary"
description: "Don't align blocks with text lines"
info: null
direction: "input"
resources:
- type: "bash_script"
path: "script.sh"
is_executable: true
description: "Block compression/decompression utility"
test_resources:
- type: "bash_script"
path: "test.sh"
is_executable: true
- type: "file"
path: "test_data"
info: null
status: "enabled"
scope:
image: "public"
target: "public"
requirements:
commands:
- "ps"
license: "MIT"
references:
doi:
- "10.1093/gigascience/giab007"
links:
repository: "https://github.com/samtools/htslib"
homepage: "https://www.htslib.org/"
documentation: "https://www.htslib.org/doc/bgzip.html"
runners:
- type: "executable"
id: "executable"
docker_setup_strategy: "ifneedbepullelsecachedbuild"
- type: "nextflow"
id: "nextflow"
directives:
tag: "$id"
auto:
simplifyInput: true
simplifyOutput: false
transcript: false
publish: false
config:
labels:
mem1gb: "memory = 1000000000.B"
mem2gb: "memory = 2000000000.B"
mem5gb: "memory = 5000000000.B"
mem10gb: "memory = 10000000000.B"
mem20gb: "memory = 20000000000.B"
mem50gb: "memory = 50000000000.B"
mem100gb: "memory = 100000000000.B"
mem200gb: "memory = 200000000000.B"
mem500gb: "memory = 500000000000.B"
mem1tb: "memory = 1000000000000.B"
mem2tb: "memory = 2000000000000.B"
mem5tb: "memory = 5000000000000.B"
mem10tb: "memory = 10000000000000.B"
mem20tb: "memory = 20000000000000.B"
mem50tb: "memory = 50000000000000.B"
mem100tb: "memory = 100000000000000.B"
mem200tb: "memory = 200000000000000.B"
mem500tb: "memory = 500000000000000.B"
mem1gib: "memory = 1073741824.B"
mem2gib: "memory = 2147483648.B"
mem4gib: "memory = 4294967296.B"
mem8gib: "memory = 8589934592.B"
mem16gib: "memory = 17179869184.B"
mem32gib: "memory = 34359738368.B"
mem64gib: "memory = 68719476736.B"
mem128gib: "memory = 137438953472.B"
mem256gib: "memory = 274877906944.B"
mem512gib: "memory = 549755813888.B"
mem1tib: "memory = 1099511627776.B"
mem2tib: "memory = 2199023255552.B"
mem4tib: "memory = 4398046511104.B"
mem8tib: "memory = 8796093022208.B"
mem16tib: "memory = 17592186044416.B"
mem32tib: "memory = 35184372088832.B"
mem64tib: "memory = 70368744177664.B"
mem128tib: "memory = 140737488355328.B"
mem256tib: "memory = 281474976710656.B"
mem512tib: "memory = 562949953421312.B"
cpu1: "cpus = 1"
cpu2: "cpus = 2"
cpu5: "cpus = 5"
cpu10: "cpus = 10"
cpu20: "cpus = 20"
cpu50: "cpus = 50"
cpu100: "cpus = 100"
cpu200: "cpus = 200"
cpu500: "cpus = 500"
cpu1000: "cpus = 1000"
debug: false
container: "docker"
engines:
- type: "docker"
id: "docker"
image: "quay.io/biocontainers/htslib:1.19--h81da01d_0"
target_registry: "images.viash-hub.com"
target_tag: "v0.1.1"
namespace_separator: "/"
setup:
- type: "docker"
run:
- "bgzip -h | grep 'Version:' 2>&1 | sed 's/Version:\\s\\(.*\\)/bgzip: \"\\1\"\
/' > /var/software_versions.txt\n"
entrypoint: []
cmd: null
- type: "native"
id: "native"
build_info:
config: "src/bgzip/config.vsh.yaml"
runner: "nextflow"
engine: "docker|native"
output: "target/nextflow/bgzip"
executable: "target/nextflow/bgzip/main.nf"
viash_version: "0.9.4"
git_commit: "2c079795592b75ecd0db2bacaf0b748fa84e1293"
git_remote: "https://github.com/viash-hub/toolbox"
git_tag: "v0.1.0-13-g2c07979"
package_config:
name: "toolbox"
version: "v0.1.1"
summary: "A collection of curated command-line tools for general IT tasks, built\
\ with Viash.\n"
description: "`toolbox` provides a versatile suite of IT components, following the\
\ robust Viash (https://viash.io) framework.\nThis package focuses on delivering\
\ reliable, standalone tools that can be easily integrated into larger computational\
\ workflows.\n\nThe core philosophy emphasizes **reusability**, **reproducibility**,\
\ and adherence to **best practices** in component creation. Key features of `toolbox`\
\ components include:\n\n* **Standalone & Nextflow Ready:** Execute components\
\ directly from the command line or seamlessly incorporate them into Nextflow\
\ workflows.\n* **High Quality Standards:**\n * Comprehensive documentation\
\ for each component and its parameters.\n * Full exposure of the underlying\
\ tool's arguments for maximum flexibility.\n * Containerized (Docker) to ensure\
\ consistent environments and manage dependencies, leading to enhanced reproducibility.\n\
\ * Unit tested to verify functionality and ensure reliability.\n"
info: null
viash_version: "0.9.4"
source: "src"
target: "target"
config_mods:
- ".requirements.commands := ['ps']\n"
- ".engines += { type: \"native\" }"
- ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
- ".engines[.type == 'docker'].target_tag := 'v0.1.1'"
keywords:
- "toolbox"
- "command-line"
- "tools"
license: "MIT"
organization: "vsh"
links:
repository: "https://github.com/viash-hub/toolbox"
issue_tracker: "https://github.com/viash-hub/toolbox/issues"

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,125 @@
manifest {
name = 'bgzip'
mainScript = 'main.nf'
nextflowVersion = '!>=20.12.1-edge'
version = 'v0.1.1'
description = 'Block compression/decompression utility'
}
process.container = 'nextflow/bash:latest'
// detect tempdir
tempDir = java.nio.file.Paths.get(
System.getenv('NXF_TEMP') ?:
System.getenv('VIASH_TEMP') ?:
System.getenv('TEMPDIR') ?:
System.getenv('TMPDIR') ?:
'/tmp'
).toAbsolutePath()
profiles {
no_publish {
process {
withName: '.*' {
publishDir = [
enabled: false
]
}
}
}
mount_temp {
docker.temp = tempDir
podman.temp = tempDir
charliecloud.temp = tempDir
}
docker {
docker.enabled = true
// docker.userEmulation = true
singularity.enabled = false
podman.enabled = false
shifter.enabled = false
charliecloud.enabled = false
}
singularity {
singularity.enabled = true
singularity.autoMounts = true
docker.enabled = false
podman.enabled = false
shifter.enabled = false
charliecloud.enabled = false
}
podman {
podman.enabled = true
docker.enabled = false
singularity.enabled = false
shifter.enabled = false
charliecloud.enabled = false
}
shifter {
shifter.enabled = true
docker.enabled = false
singularity.enabled = false
podman.enabled = false
charliecloud.enabled = false
}
charliecloud {
charliecloud.enabled = true
docker.enabled = false
singularity.enabled = false
podman.enabled = false
shifter.enabled = false
}
}
process{
withLabel: mem1gb { memory = 1000000000.B }
withLabel: mem2gb { memory = 2000000000.B }
withLabel: mem5gb { memory = 5000000000.B }
withLabel: mem10gb { memory = 10000000000.B }
withLabel: mem20gb { memory = 20000000000.B }
withLabel: mem50gb { memory = 50000000000.B }
withLabel: mem100gb { memory = 100000000000.B }
withLabel: mem200gb { memory = 200000000000.B }
withLabel: mem500gb { memory = 500000000000.B }
withLabel: mem1tb { memory = 1000000000000.B }
withLabel: mem2tb { memory = 2000000000000.B }
withLabel: mem5tb { memory = 5000000000000.B }
withLabel: mem10tb { memory = 10000000000000.B }
withLabel: mem20tb { memory = 20000000000000.B }
withLabel: mem50tb { memory = 50000000000000.B }
withLabel: mem100tb { memory = 100000000000000.B }
withLabel: mem200tb { memory = 200000000000000.B }
withLabel: mem500tb { memory = 500000000000000.B }
withLabel: mem1gib { memory = 1073741824.B }
withLabel: mem2gib { memory = 2147483648.B }
withLabel: mem4gib { memory = 4294967296.B }
withLabel: mem8gib { memory = 8589934592.B }
withLabel: mem16gib { memory = 17179869184.B }
withLabel: mem32gib { memory = 34359738368.B }
withLabel: mem64gib { memory = 68719476736.B }
withLabel: mem128gib { memory = 137438953472.B }
withLabel: mem256gib { memory = 274877906944.B }
withLabel: mem512gib { memory = 549755813888.B }
withLabel: mem1tib { memory = 1099511627776.B }
withLabel: mem2tib { memory = 2199023255552.B }
withLabel: mem4tib { memory = 4398046511104.B }
withLabel: mem8tib { memory = 8796093022208.B }
withLabel: mem16tib { memory = 17592186044416.B }
withLabel: mem32tib { memory = 35184372088832.B }
withLabel: mem64tib { memory = 70368744177664.B }
withLabel: mem128tib { memory = 140737488355328.B }
withLabel: mem256tib { memory = 281474976710656.B }
withLabel: mem512tib { memory = 562949953421312.B }
withLabel: cpu1 { cpus = 1 }
withLabel: cpu2 { cpus = 2 }
withLabel: cpu5 { cpus = 5 }
withLabel: cpu10 { cpus = 10 }
withLabel: cpu20 { cpus = 20 }
withLabel: cpu50 { cpus = 50 }
withLabel: cpu100 { cpus = 100 }
withLabel: cpu200 { cpus = 200 }
withLabel: cpu500 { cpus = 500 }
withLabel: cpu1000 { cpus = 1000 }
}

View File

@@ -0,0 +1,216 @@
{
"$schema": "http://json-schema.org/draft-07/schema",
"title": "bgzip",
"description": "Block compression/decompression utility",
"type": "object",
"definitions": {
"inputs" : {
"title": "Inputs",
"type": "object",
"description": "No description",
"properties": {
"input": {
"type":
"string",
"description": "Type: `file`, required. file to be compressed or decompressed",
"help_text": "Type: `file`, required. file to be compressed or decompressed"
}
}
},
"outputs" : {
"title": "Outputs",
"type": "object",
"description": "No description",
"properties": {
"output": {
"type":
"string",
"description": "Type: `file`, required, default: `$id.$key.output`. compressed or decompressed output",
"help_text": "Type: `file`, required, default: `$id.$key.output`. compressed or decompressed output"
,
"default":"$id.$key.output"
}
,
"index_name": {
"type":
"string",
"description": "Type: `file`, default: `$id.$key.index_name`. name of BGZF index file [file",
"help_text": "Type: `file`, default: `$id.$key.index_name`. name of BGZF index file [file.gz.gzi]"
,
"default":"$id.$key.index_name"
}
}
},
"arguments" : {
"title": "Arguments",
"type": "object",
"description": "No description",
"properties": {
"offset": {
"type":
"integer",
"description": "Type: `integer`. decompress at virtual file pointer (0-based uncompressed offset)",
"help_text": "Type: `integer`. decompress at virtual file pointer (0-based uncompressed offset)"
}
,
"decompress": {
"type":
"boolean",
"description": "Type: `boolean_true`, default: `false`. decompress the input file",
"help_text": "Type: `boolean_true`, default: `false`. decompress the input file"
,
"default":false
}
,
"rebgzip": {
"type":
"boolean",
"description": "Type: `boolean_true`, default: `false`. use an index file to bgzip a file",
"help_text": "Type: `boolean_true`, default: `false`. use an index file to bgzip a file"
,
"default":false
}
,
"index": {
"type":
"boolean",
"description": "Type: `boolean_true`, default: `false`. compress and create BGZF index",
"help_text": "Type: `boolean_true`, default: `false`. compress and create BGZF index"
,
"default":false
}
,
"compress_level": {
"type":
"integer",
"description": "Type: `integer`. compression level to use when compressing; 0 to 9, or -1 for default [-1]",
"help_text": "Type: `integer`. compression level to use when compressing; 0 to 9, or -1 for default [-1]"
}
,
"reindex": {
"type":
"boolean",
"description": "Type: `boolean_true`, default: `false`. (re)index the output file",
"help_text": "Type: `boolean_true`, default: `false`. (re)index the output file"
,
"default":false
}
,
"size": {
"type":
"integer",
"description": "Type: `integer`. decompress INT bytes (uncompressed size)",
"help_text": "Type: `integer`. decompress INT bytes (uncompressed size)"
}
,
"test": {
"type":
"boolean",
"description": "Type: `boolean_true`, default: `false`. test integrity of compressed file",
"help_text": "Type: `boolean_true`, default: `false`. test integrity of compressed file"
,
"default":false
}
,
"binary": {
"type":
"boolean",
"description": "Type: `boolean_true`, default: `false`. Don\u0027t align blocks with text lines",
"help_text": "Type: `boolean_true`, default: `false`. Don\u0027t align blocks with text lines"
,
"default":false
}
}
},
"nextflow input-output arguments" : {
"title": "Nextflow input-output arguments",
"type": "object",
"description": "Input/output parameters for Nextflow itself. Please note that both publishDir and publish_dir are supported but at least one has to be configured.",
"properties": {
"publish_dir": {
"type":
"string",
"description": "Type: `string`, required, example: `output/`. Path to an output directory",
"help_text": "Type: `string`, required, example: `output/`. Path to an output directory."
}
,
"param_list": {
"type":
"string",
"description": "Type: `string`, example: `my_params.yaml`. Allows inputting multiple parameter sets to initialise a Nextflow channel",
"help_text": "Type: `string`, example: `my_params.yaml`. Allows inputting multiple parameter sets to initialise a Nextflow channel. A `param_list` can either be a list of maps, a csv file, a json file, a yaml file, or simply a yaml blob.\n\n* A list of maps (as-is) where the keys of each map corresponds to the arguments of the pipeline. Example: in a `nextflow.config` file: `param_list: [ [\u0027id\u0027: \u0027foo\u0027, \u0027input\u0027: \u0027foo.txt\u0027], [\u0027id\u0027: \u0027bar\u0027, \u0027input\u0027: \u0027bar.txt\u0027] ]`.\n* A csv file should have column names which correspond to the different arguments of this pipeline. Example: `--param_list data.csv` with columns `id,input`.\n* A json or a yaml file should be a list of maps, each of which has keys corresponding to the arguments of the pipeline. Example: `--param_list data.json` with contents `[ {\u0027id\u0027: \u0027foo\u0027, \u0027input\u0027: \u0027foo.txt\u0027}, {\u0027id\u0027: \u0027bar\u0027, \u0027input\u0027: \u0027bar.txt\u0027} ]`.\n* A yaml blob can also be passed directly as a string. Example: `--param_list \"[ {\u0027id\u0027: \u0027foo\u0027, \u0027input\u0027: \u0027foo.txt\u0027}, {\u0027id\u0027: \u0027bar\u0027, \u0027input\u0027: \u0027bar.txt\u0027} ]\"`.\n\nWhen passing a csv, json or yaml file, relative path names are relativized to the location of the parameter file. No relativation is performed when `param_list` is a list of maps (as-is) or a yaml blob.",
"hidden": true
}
}
}
},
"allOf": [
{
"$ref": "#/definitions/inputs"
},
{
"$ref": "#/definitions/outputs"
},
{
"$ref": "#/definitions/arguments"
},
{
"$ref": "#/definitions/nextflow input-output arguments"
}
]
}

View File

@@ -0,0 +1,298 @@
name: "yq"
version: "v0.1.1"
argument_groups:
- name: "Inputs"
arguments:
- type: "file"
name: "--input"
description: "files to be processed"
info: null
example:
- "input.yaml"
must_exist: true
create_parent: true
required: true
direction: "input"
multiple: false
multiple_sep: ";"
- name: "Outputs"
arguments:
- type: "file"
name: "--output"
description: "output file"
info: null
example:
- "output.yaml"
must_exist: true
create_parent: true
required: true
direction: "output"
multiple: false
multiple_sep: ";"
- name: "Arguments"
arguments:
- type: "string"
name: "--eval"
description: "expression to evaluate"
info: null
example:
- ".name = \"foo\""
required: true
direction: "input"
multiple: false
multiple_sep: ";"
- type: "integer"
name: "--indent"
alternatives:
- "-I"
description: "sets indent level for output (default 2)"
info: null
required: false
direction: "input"
multiple: false
multiple_sep: ";"
- type: "string"
name: "--input_format"
alternatives:
- "-p"
description: "parse format for input. (default \"auto\")"
info: null
required: false
choices:
- "auto"
- "a"
- "yaml"
- "y"
- "json"
- "j"
- "props"
- "p"
- "csv"
- "c"
- "tsv"
- "t"
- "xml"
- "x"
- "base64"
- "uri"
- "toml"
- "shell"
- "s"
- "lua"
- "l"
direction: "input"
multiple: false
multiple_sep: ";"
- type: "string"
name: "--output_format"
alternatives:
- "-o"
description: "output format type. (default \"auto\")"
info: null
required: false
choices:
- "auto"
- "a"
- "yaml"
- "y"
- "json"
- "j"
- "props"
- "p"
- "csv"
- "c"
- "tsv"
- "t"
- "xml"
- "x"
- "base64"
- "uri"
- "toml"
- "shell"
- "s"
- "lua"
- "l"
direction: "input"
multiple: false
multiple_sep: ";"
- type: "boolean_true"
name: "--pretty_print"
alternatives:
- "-P"
description: "pretty print, shorthand for '... style = \"\"'"
info: null
direction: "input"
resources:
- type: "bash_script"
text: |
#!/bin/sh
[[ "$par_pretty_print" == "false" ]] && unset par_pretty_print
yq eval \
${par_indent:+-I "${par_indent}"} \
${par_input_format:+-p "${par_input_format}"} \
${par_output_format:+-o "${par_output_format}"} \
${par_pretty_print:+-P} \
--expression "$par_eval" \
--no-colors \
"$par_input" > "$par_output"
dest: "./script.sh"
is_executable: true
description: "A portable YAML, JSON, XML, CSV, TOML and properties processor"
test_resources:
- type: "bash_script"
text: "set -e\necho \"name: 'bar'\" > test.yaml\n\"$meta_executable\" --input test.yaml\
\ --output output.yaml --eval '.name = \"foo\"'\n\"$meta_executable\" --input\
\ output.yaml --output output2.yaml --eval '.name'\ngrep \"^foo$\" output2.yaml\n"
dest: "./script.sh"
is_executable: true
info: null
status: "enabled"
scope:
image: "public"
target: "public"
requirements:
commands:
- "ps"
keywords:
- "yaml"
- "json"
- "xml"
- "csv"
- "toml"
- "properties"
license: "MIT"
links:
repository: "https://github.com/mikefarah/yq"
homepage: "https://mikefarah.gitbook.io/yq"
documentation: "https://mikefarah.gitbook.io/yq/"
runners:
- type: "executable"
id: "executable"
docker_setup_strategy: "ifneedbepullelsecachedbuild"
- type: "nextflow"
id: "nextflow"
directives:
tag: "$id"
auto:
simplifyInput: true
simplifyOutput: false
transcript: false
publish: false
config:
labels:
mem1gb: "memory = 1000000000.B"
mem2gb: "memory = 2000000000.B"
mem5gb: "memory = 5000000000.B"
mem10gb: "memory = 10000000000.B"
mem20gb: "memory = 20000000000.B"
mem50gb: "memory = 50000000000.B"
mem100gb: "memory = 100000000000.B"
mem200gb: "memory = 200000000000.B"
mem500gb: "memory = 500000000000.B"
mem1tb: "memory = 1000000000000.B"
mem2tb: "memory = 2000000000000.B"
mem5tb: "memory = 5000000000000.B"
mem10tb: "memory = 10000000000000.B"
mem20tb: "memory = 20000000000000.B"
mem50tb: "memory = 50000000000000.B"
mem100tb: "memory = 100000000000000.B"
mem200tb: "memory = 200000000000000.B"
mem500tb: "memory = 500000000000000.B"
mem1gib: "memory = 1073741824.B"
mem2gib: "memory = 2147483648.B"
mem4gib: "memory = 4294967296.B"
mem8gib: "memory = 8589934592.B"
mem16gib: "memory = 17179869184.B"
mem32gib: "memory = 34359738368.B"
mem64gib: "memory = 68719476736.B"
mem128gib: "memory = 137438953472.B"
mem256gib: "memory = 274877906944.B"
mem512gib: "memory = 549755813888.B"
mem1tib: "memory = 1099511627776.B"
mem2tib: "memory = 2199023255552.B"
mem4tib: "memory = 4398046511104.B"
mem8tib: "memory = 8796093022208.B"
mem16tib: "memory = 17592186044416.B"
mem32tib: "memory = 35184372088832.B"
mem64tib: "memory = 70368744177664.B"
mem128tib: "memory = 140737488355328.B"
mem256tib: "memory = 281474976710656.B"
mem512tib: "memory = 562949953421312.B"
cpu1: "cpus = 1"
cpu2: "cpus = 2"
cpu5: "cpus = 5"
cpu10: "cpus = 10"
cpu20: "cpus = 20"
cpu50: "cpus = 50"
cpu100: "cpus = 100"
cpu200: "cpus = 200"
cpu500: "cpus = 500"
cpu1000: "cpus = 1000"
debug: false
container: "docker"
engines:
- type: "docker"
id: "docker"
image: "alpine:latest"
target_registry: "images.viash-hub.com"
target_tag: "v0.1.1"
namespace_separator: "/"
setup:
- type: "apk"
packages:
- "bash"
- "yq-go"
- type: "docker"
run:
- "/usr/bin/yq --version | sed 's/.*version\\sv\\(.*\\)/yq: \"\\1\"/' > /var/software_versions.txt\n"
entrypoint: []
cmd: null
- type: "native"
id: "native"
build_info:
config: "src/yq/config.vsh.yaml"
runner: "nextflow"
engine: "docker|native"
output: "target/nextflow/yq"
executable: "target/nextflow/yq/main.nf"
viash_version: "0.9.4"
git_commit: "2c079795592b75ecd0db2bacaf0b748fa84e1293"
git_remote: "https://github.com/viash-hub/toolbox"
git_tag: "v0.1.0-13-g2c07979"
package_config:
name: "toolbox"
version: "v0.1.1"
summary: "A collection of curated command-line tools for general IT tasks, built\
\ with Viash.\n"
description: "`toolbox` provides a versatile suite of IT components, following the\
\ robust Viash (https://viash.io) framework.\nThis package focuses on delivering\
\ reliable, standalone tools that can be easily integrated into larger computational\
\ workflows.\n\nThe core philosophy emphasizes **reusability**, **reproducibility**,\
\ and adherence to **best practices** in component creation. Key features of `toolbox`\
\ components include:\n\n* **Standalone & Nextflow Ready:** Execute components\
\ directly from the command line or seamlessly incorporate them into Nextflow\
\ workflows.\n* **High Quality Standards:**\n * Comprehensive documentation\
\ for each component and its parameters.\n * Full exposure of the underlying\
\ tool's arguments for maximum flexibility.\n * Containerized (Docker) to ensure\
\ consistent environments and manage dependencies, leading to enhanced reproducibility.\n\
\ * Unit tested to verify functionality and ensure reliability.\n"
info: null
viash_version: "0.9.4"
source: "src"
target: "target"
config_mods:
- ".requirements.commands := ['ps']\n"
- ".engines += { type: \"native\" }"
- ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
- ".engines[.type == 'docker'].target_tag := 'v0.1.1'"
keywords:
- "toolbox"
- "command-line"
- "tools"
license: "MIT"
organization: "vsh"
links:
repository: "https://github.com/viash-hub/toolbox"
issue_tracker: "https://github.com/viash-hub/toolbox/issues"

3915
target/nextflow/yq/main.nf Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,125 @@
manifest {
name = 'yq'
mainScript = 'main.nf'
nextflowVersion = '!>=20.12.1-edge'
version = 'v0.1.1'
description = 'A portable YAML, JSON, XML, CSV, TOML and properties processor'
}
process.container = 'nextflow/bash:latest'
// detect tempdir
tempDir = java.nio.file.Paths.get(
System.getenv('NXF_TEMP') ?:
System.getenv('VIASH_TEMP') ?:
System.getenv('TEMPDIR') ?:
System.getenv('TMPDIR') ?:
'/tmp'
).toAbsolutePath()
profiles {
no_publish {
process {
withName: '.*' {
publishDir = [
enabled: false
]
}
}
}
mount_temp {
docker.temp = tempDir
podman.temp = tempDir
charliecloud.temp = tempDir
}
docker {
docker.enabled = true
// docker.userEmulation = true
singularity.enabled = false
podman.enabled = false
shifter.enabled = false
charliecloud.enabled = false
}
singularity {
singularity.enabled = true
singularity.autoMounts = true
docker.enabled = false
podman.enabled = false
shifter.enabled = false
charliecloud.enabled = false
}
podman {
podman.enabled = true
docker.enabled = false
singularity.enabled = false
shifter.enabled = false
charliecloud.enabled = false
}
shifter {
shifter.enabled = true
docker.enabled = false
singularity.enabled = false
podman.enabled = false
charliecloud.enabled = false
}
charliecloud {
charliecloud.enabled = true
docker.enabled = false
singularity.enabled = false
podman.enabled = false
shifter.enabled = false
}
}
process{
withLabel: mem1gb { memory = 1000000000.B }
withLabel: mem2gb { memory = 2000000000.B }
withLabel: mem5gb { memory = 5000000000.B }
withLabel: mem10gb { memory = 10000000000.B }
withLabel: mem20gb { memory = 20000000000.B }
withLabel: mem50gb { memory = 50000000000.B }
withLabel: mem100gb { memory = 100000000000.B }
withLabel: mem200gb { memory = 200000000000.B }
withLabel: mem500gb { memory = 500000000000.B }
withLabel: mem1tb { memory = 1000000000000.B }
withLabel: mem2tb { memory = 2000000000000.B }
withLabel: mem5tb { memory = 5000000000000.B }
withLabel: mem10tb { memory = 10000000000000.B }
withLabel: mem20tb { memory = 20000000000000.B }
withLabel: mem50tb { memory = 50000000000000.B }
withLabel: mem100tb { memory = 100000000000000.B }
withLabel: mem200tb { memory = 200000000000000.B }
withLabel: mem500tb { memory = 500000000000000.B }
withLabel: mem1gib { memory = 1073741824.B }
withLabel: mem2gib { memory = 2147483648.B }
withLabel: mem4gib { memory = 4294967296.B }
withLabel: mem8gib { memory = 8589934592.B }
withLabel: mem16gib { memory = 17179869184.B }
withLabel: mem32gib { memory = 34359738368.B }
withLabel: mem64gib { memory = 68719476736.B }
withLabel: mem128gib { memory = 137438953472.B }
withLabel: mem256gib { memory = 274877906944.B }
withLabel: mem512gib { memory = 549755813888.B }
withLabel: mem1tib { memory = 1099511627776.B }
withLabel: mem2tib { memory = 2199023255552.B }
withLabel: mem4tib { memory = 4398046511104.B }
withLabel: mem8tib { memory = 8796093022208.B }
withLabel: mem16tib { memory = 17592186044416.B }
withLabel: mem32tib { memory = 35184372088832.B }
withLabel: mem64tib { memory = 70368744177664.B }
withLabel: mem128tib { memory = 140737488355328.B }
withLabel: mem256tib { memory = 281474976710656.B }
withLabel: mem512tib { memory = 562949953421312.B }
withLabel: cpu1 { cpus = 1 }
withLabel: cpu2 { cpus = 2 }
withLabel: cpu5 { cpus = 5 }
withLabel: cpu10 { cpus = 10 }
withLabel: cpu20 { cpus = 20 }
withLabel: cpu50 { cpus = 50 }
withLabel: cpu100 { cpus = 100 }
withLabel: cpu200 { cpus = 200 }
withLabel: cpu500 { cpus = 500 }
withLabel: cpu1000 { cpus = 1000 }
}

View File

@@ -0,0 +1,164 @@
{
"$schema": "http://json-schema.org/draft-07/schema",
"title": "yq",
"description": "A portable YAML, JSON, XML, CSV, TOML and properties processor",
"type": "object",
"definitions": {
"inputs" : {
"title": "Inputs",
"type": "object",
"description": "No description",
"properties": {
"input": {
"type":
"string",
"description": "Type: `file`, required, example: `input.yaml`. files to be processed",
"help_text": "Type: `file`, required, example: `input.yaml`. files to be processed"
}
}
},
"outputs" : {
"title": "Outputs",
"type": "object",
"description": "No description",
"properties": {
"output": {
"type":
"string",
"description": "Type: `file`, required, default: `$id.$key.output.yaml`, example: `output.yaml`. output file",
"help_text": "Type: `file`, required, default: `$id.$key.output.yaml`, example: `output.yaml`. output file"
,
"default":"$id.$key.output.yaml"
}
}
},
"arguments" : {
"title": "Arguments",
"type": "object",
"description": "No description",
"properties": {
"eval": {
"type":
"string",
"description": "Type: `string`, required, example: `.name = \"foo\"`. expression to evaluate",
"help_text": "Type: `string`, required, example: `.name = \"foo\"`. expression to evaluate"
}
,
"indent": {
"type":
"integer",
"description": "Type: `integer`. sets indent level for output (default 2)",
"help_text": "Type: `integer`. sets indent level for output (default 2)"
}
,
"input_format": {
"type":
"string",
"description": "Type: `string`, choices: ``auto`, `a`, `yaml`, `y`, `json`, `j`, `props`, `p`, `csv`, `c`, `tsv`, `t`, `xml`, `x`, `base64`, `uri`, `toml`, `shell`, `s`, `lua`, `l``. parse format for input",
"help_text": "Type: `string`, choices: ``auto`, `a`, `yaml`, `y`, `json`, `j`, `props`, `p`, `csv`, `c`, `tsv`, `t`, `xml`, `x`, `base64`, `uri`, `toml`, `shell`, `s`, `lua`, `l``. parse format for input. (default \"auto\")",
"enum": ["auto", "a", "yaml", "y", "json", "j", "props", "p", "csv", "c", "tsv", "t", "xml", "x", "base64", "uri", "toml", "shell", "s", "lua", "l"]
}
,
"output_format": {
"type":
"string",
"description": "Type: `string`, choices: ``auto`, `a`, `yaml`, `y`, `json`, `j`, `props`, `p`, `csv`, `c`, `tsv`, `t`, `xml`, `x`, `base64`, `uri`, `toml`, `shell`, `s`, `lua`, `l``. output format type",
"help_text": "Type: `string`, choices: ``auto`, `a`, `yaml`, `y`, `json`, `j`, `props`, `p`, `csv`, `c`, `tsv`, `t`, `xml`, `x`, `base64`, `uri`, `toml`, `shell`, `s`, `lua`, `l``. output format type. (default \"auto\")",
"enum": ["auto", "a", "yaml", "y", "json", "j", "props", "p", "csv", "c", "tsv", "t", "xml", "x", "base64", "uri", "toml", "shell", "s", "lua", "l"]
}
,
"pretty_print": {
"type":
"boolean",
"description": "Type: `boolean_true`, default: `false`. pretty print, shorthand for \u0027",
"help_text": "Type: `boolean_true`, default: `false`. pretty print, shorthand for \u0027... style = \"\"\u0027"
,
"default":false
}
}
},
"nextflow input-output arguments" : {
"title": "Nextflow input-output arguments",
"type": "object",
"description": "Input/output parameters for Nextflow itself. Please note that both publishDir and publish_dir are supported but at least one has to be configured.",
"properties": {
"publish_dir": {
"type":
"string",
"description": "Type: `string`, required, example: `output/`. Path to an output directory",
"help_text": "Type: `string`, required, example: `output/`. Path to an output directory."
}
,
"param_list": {
"type":
"string",
"description": "Type: `string`, example: `my_params.yaml`. Allows inputting multiple parameter sets to initialise a Nextflow channel",
"help_text": "Type: `string`, example: `my_params.yaml`. Allows inputting multiple parameter sets to initialise a Nextflow channel. A `param_list` can either be a list of maps, a csv file, a json file, a yaml file, or simply a yaml blob.\n\n* A list of maps (as-is) where the keys of each map corresponds to the arguments of the pipeline. Example: in a `nextflow.config` file: `param_list: [ [\u0027id\u0027: \u0027foo\u0027, \u0027input\u0027: \u0027foo.txt\u0027], [\u0027id\u0027: \u0027bar\u0027, \u0027input\u0027: \u0027bar.txt\u0027] ]`.\n* A csv file should have column names which correspond to the different arguments of this pipeline. Example: `--param_list data.csv` with columns `id,input`.\n* A json or a yaml file should be a list of maps, each of which has keys corresponding to the arguments of the pipeline. Example: `--param_list data.json` with contents `[ {\u0027id\u0027: \u0027foo\u0027, \u0027input\u0027: \u0027foo.txt\u0027}, {\u0027id\u0027: \u0027bar\u0027, \u0027input\u0027: \u0027bar.txt\u0027} ]`.\n* A yaml blob can also be passed directly as a string. Example: `--param_list \"[ {\u0027id\u0027: \u0027foo\u0027, \u0027input\u0027: \u0027foo.txt\u0027}, {\u0027id\u0027: \u0027bar\u0027, \u0027input\u0027: \u0027bar.txt\u0027} ]\"`.\n\nWhen passing a csv, json or yaml file, relative path names are relativized to the location of the parameter file. No relativation is performed when `param_list` is a list of maps (as-is) or a yaml blob.",
"hidden": true
}
}
}
},
"allOf": [
{
"$ref": "#/definitions/inputs"
},
{
"$ref": "#/definitions/outputs"
},
{
"$ref": "#/definitions/arguments"
},
{
"$ref": "#/definitions/nextflow input-output arguments"
}
]
}