Build branch gunzip with version gunzip (add30ba)
Build pipeline: viash-hub.toolbox.gunzip-zjfds
Source commit: add30ba0f3
Source message: add gitignore
This commit is contained in:
18
.gitignore
vendored
Normal file
18
.gitignore
vendored
Normal file
@@ -0,0 +1,18 @@
|
||||
*.DS_Store
|
||||
*__pycache__
|
||||
|
||||
# IDE ignores
|
||||
.idea/
|
||||
.vscode/
|
||||
|
||||
# R specific ignores
|
||||
.Rhistory
|
||||
.Rproj.user
|
||||
*.Rproj
|
||||
|
||||
# viash specific ignores
|
||||
target/
|
||||
|
||||
# nextflow specific ignores
|
||||
.nextflow*
|
||||
work
|
||||
23
CHANGELOG.md
Normal file
23
CHANGELOG.md
Normal file
@@ -0,0 +1,23 @@
|
||||
# toolbox v0.1.1
|
||||
|
||||
## MINOR CHANGES
|
||||
|
||||
* Updated the test CI (PR #6).
|
||||
|
||||
* Bump viash to 0.9.0 (PR #10).
|
||||
|
||||
* Bump viash to 0.9.4 (PR #13).
|
||||
|
||||
* Update README (PR #13).
|
||||
|
||||
# toolbox v0.1.0
|
||||
|
||||
## NEW FEATURES
|
||||
|
||||
* `bgzip`: Add bgzip functionality to compress and decompress files (initial commit).
|
||||
|
||||
* `yq`: A portable YAML, JSON, XML, CSV, TOML and properties processor (PR #1).
|
||||
|
||||
## MINOR CHANGES
|
||||
|
||||
* Use newer viash-actions updated for Viash 0.9 (PR #2).
|
||||
383
CONTRIBUTING.md
Normal file
383
CONTRIBUTING.md
Normal file
@@ -0,0 +1,383 @@
|
||||
|
||||
# Contributing guidelines
|
||||
|
||||
We encourage contributions from the community. To contribute:
|
||||
|
||||
1. **Fork the Repository**: Start by forking this repository to your account.
|
||||
2. **Develop Your Component**: Create your Viash component, ensuring it aligns with our best practices (detailed below).
|
||||
3. **Submit a Pull Request**: After testing your component, submit a pull request for review.
|
||||
|
||||
## Procedure of adding a component
|
||||
|
||||
### Step 1: Find a component to contribute
|
||||
|
||||
* Find a tool to contribute to this repo.
|
||||
|
||||
* Check whether it is already in the [Project board](https://github.com/orgs/viash-hub/projects/1).
|
||||
|
||||
* Check whether there is a corresponding [Snakemake wrapper](https://github.com/snakemake/snakemake-wrappers/blob/master/bio) or [nf-core module](https://github.com/nf-core/modules/tree/master/modules/nf-core) which we can use as inspiration.
|
||||
|
||||
* Create an issue to show that you are working on this component.
|
||||
|
||||
|
||||
### Step 2: Add config template
|
||||
|
||||
Change all occurrences of `xxx` to the name of the component.
|
||||
|
||||
Create a file at `src/xxx/config.vsh.yaml` with contents:
|
||||
|
||||
```yaml
|
||||
name: xxx
|
||||
description: xxx
|
||||
keywords: [tag1, tag2]
|
||||
links:
|
||||
homepage: yyy
|
||||
documentation: yyy
|
||||
issue_tracker: yyy
|
||||
repository: yyy
|
||||
references:
|
||||
doi: 12345/12345678.yz
|
||||
license: MIT/Apache-2.0/GPL-3.0/...
|
||||
argument_groups:
|
||||
- name: Inputs
|
||||
arguments: <...>
|
||||
- name: Outputs
|
||||
arguments: <...>
|
||||
- name: Arguments
|
||||
arguments: <...>
|
||||
resources:
|
||||
- type: bash_script
|
||||
path: script.sh
|
||||
test_resources:
|
||||
- type: bash_script
|
||||
path: test.sh
|
||||
- type: file
|
||||
path: test_data
|
||||
engines:
|
||||
- <...>
|
||||
runners:
|
||||
- type: executable
|
||||
- type: nextflow
|
||||
```
|
||||
|
||||
### Step 3: Fill in the metadata
|
||||
|
||||
Fill in the relevant metadata fields in the config. Here is an example of the metadata of an existing component.
|
||||
|
||||
```yaml
|
||||
functionality:
|
||||
name: arriba
|
||||
description: Detect gene fusions from RNA-Seq data
|
||||
keywords: [Gene fusion, RNA-Seq]
|
||||
links:
|
||||
homepage: https://arriba.readthedocs.io/en/latest/
|
||||
documentation: https://arriba.readthedocs.io/en/latest/
|
||||
repository: https://github.com/suhrig/arriba
|
||||
issue_tracker: https://github.com/suhrig/arriba/issues
|
||||
references:
|
||||
doi: 10.1101/gr.257246.119
|
||||
bibtex: |
|
||||
@article{
|
||||
... a bibtex entry in case the doi is not available ...
|
||||
}
|
||||
license: MIT
|
||||
```
|
||||
|
||||
### Step 4: Find a suitable container
|
||||
|
||||
Google `biocontainer <name of component>` and find the container that is most suitable. Typically the link will be `https://quay.io/repository/biocontainers/xxx?tab=tags`.
|
||||
|
||||
If no such container is found, you can create a custom container in the next step.
|
||||
|
||||
|
||||
### Step 5: Create help file
|
||||
|
||||
To help develop the component, we store the `--help` output of the tool in a file at `src/xxx/help.txt`.
|
||||
|
||||
````bash
|
||||
cat <<EOF > src/xxx/help.txt
|
||||
```sh
|
||||
xxx --help
|
||||
```
|
||||
EOF
|
||||
|
||||
docker run quay.io/biocontainers/xxx:tag xxx --help >> src/xxx/help.txt
|
||||
````
|
||||
|
||||
Notes:
|
||||
|
||||
* This help file has no functional purpose, but it is useful for the developer to see the help output of the tool.
|
||||
|
||||
* Some tools might not have a `--help` argument but instead have a `-h` argument. For example, for `arriba`, the help message is obtained by running `arriba -h`:
|
||||
|
||||
```bash
|
||||
docker run quay.io/biocontainers/arriba:2.4.0--h0033a41_2 arriba -h
|
||||
```
|
||||
|
||||
|
||||
### Step 6: Create or fetch test data
|
||||
|
||||
To help develop the component, it's interesting to have some test data available. In most cases, we can use the test data from the Snakemake wrappers.
|
||||
|
||||
To make sure we can reproduce the test data in the future, we store the command to fetch the test data in a file at `src/xxx/test_data/script.sh`.
|
||||
|
||||
```bash
|
||||
cat <<EOF > src/xxx/test_data/script.sh
|
||||
|
||||
# clone repo
|
||||
if [ ! -d /tmp/snakemake-wrappers ]; then
|
||||
git clone --depth 1 --single-branch --branch master https://github.com/snakemake/snakemake-wrappers /tmp/snakemake-wrappers
|
||||
fi
|
||||
|
||||
# copy test data
|
||||
cp -r /tmp/snakemake-wrappers/bio/xxx/test/* src/xxx/test_data
|
||||
EOF
|
||||
```
|
||||
|
||||
The test data should be suitable for testing this component. Ensure that the test data is small enough: ideally <1KB, preferably <10KB, if need be <100KB.
|
||||
|
||||
### Step 7: Add arguments for the input files
|
||||
|
||||
By looking at the help file, we add the input arguments to the config file. Here is an example of the input arguments of an existing component.
|
||||
|
||||
For instance, in the [arriba help file](src/arriba/help.txt), we see the following:
|
||||
|
||||
Usage: arriba [-c Chimeric.out.sam] -x Aligned.out.bam \
|
||||
-g annotation.gtf -a assembly.fa [-b blacklists.tsv] [-k known_fusions.tsv] \
|
||||
[-t tags.tsv] [-p protein_domains.gff3] [-d structural_variants_from_WGS.tsv] \
|
||||
-o fusions.tsv [-O fusions.discarded.tsv] \
|
||||
[OPTIONS]
|
||||
|
||||
-x FILE File in SAM/BAM/CRAM format with main alignments as generated by STAR
|
||||
(Aligned.out.sam). Arriba extracts candidate reads from this file.
|
||||
|
||||
Based on this information, we can add the following input arguments to the config file.
|
||||
|
||||
```yaml
|
||||
argument_groups:
|
||||
- name: Inputs
|
||||
arguments:
|
||||
- name: --bam
|
||||
alternatives: -x
|
||||
type: file
|
||||
description: |
|
||||
File in SAM/BAM/CRAM format with main alignments as generated by STAR
|
||||
(Aligned.out.sam). Arriba extracts candidate reads from this file.
|
||||
required: true
|
||||
example: Aligned.out.bam
|
||||
```
|
||||
|
||||
Check the [documentation](https://viash.io/reference/config/functionality/arguments) for more information on the format of input arguments.
|
||||
|
||||
Several notes:
|
||||
|
||||
* Argument names should be formatted in `--snake_case`. This means arguments like `--foo-bar` should be formatted as `--foo_bar`, and short arguments like `-f` should receive a longer name like `--foo`.
|
||||
|
||||
* Input arguments can have `multiple: true` to allow the user to specify multiple files.
|
||||
|
||||
|
||||
|
||||
### Step 8: Add arguments for the output files
|
||||
|
||||
By looking at the help file, we now also add output arguments to the config file.
|
||||
|
||||
For example, in the [arriba help file](src/arriba/help.txt), we see the following:
|
||||
|
||||
|
||||
Usage: arriba [-c Chimeric.out.sam] -x Aligned.out.bam \
|
||||
-g annotation.gtf -a assembly.fa [-b blacklists.tsv] [-k known_fusions.tsv] \
|
||||
[-t tags.tsv] [-p protein_domains.gff3] [-d structural_variants_from_WGS.tsv] \
|
||||
-o fusions.tsv [-O fusions.discarded.tsv] \
|
||||
[OPTIONS]
|
||||
|
||||
-o FILE Output file with fusions that have passed all filters.
|
||||
|
||||
-O FILE Output file with fusions that were discarded due to filtering.
|
||||
|
||||
Based on this information, we can add the following output arguments to the config file.
|
||||
|
||||
```yaml
|
||||
argument_groups:
|
||||
- name: Outputs
|
||||
arguments:
|
||||
- name: --fusions
|
||||
alternatives: -o
|
||||
type: file
|
||||
direction: output
|
||||
description: |
|
||||
Output file with fusions that have passed all filters.
|
||||
required: true
|
||||
example: fusions.tsv
|
||||
- name: --fusions_discarded
|
||||
alternatives: -O
|
||||
type: file
|
||||
direction: output
|
||||
description: |
|
||||
Output file with fusions that were discarded due to filtering.
|
||||
required: false
|
||||
example: fusions.discarded.tsv
|
||||
```
|
||||
|
||||
Note:
|
||||
|
||||
* Preferably, these outputs should not be directores but files. For example, if a tool outputs a directory `foo/` containing files `foo/bar.txt` and `foo/baz.txt`, there should be two output arguments `--bar` and `--baz` (as opposed to one output argument which outputs the whole `foo/` directory).
|
||||
|
||||
### Step 9: Add arguments for the other arguments
|
||||
|
||||
Finally, add all other arguments to the config file. There are a few exceptions:
|
||||
|
||||
* Arguments related to specifying CPU and memory requirements are handled separately and should not be added to the config file.
|
||||
|
||||
* Arguments related to printing the information such as printing the version (`-v`, `--version`) or printing the help (`-h`, `--help`) should not be added to the config file.
|
||||
|
||||
|
||||
### Step 10: Add a Docker engine
|
||||
|
||||
To ensure reproducibility of components, we require that all components are run in a Docker container.
|
||||
|
||||
```yaml
|
||||
engines:
|
||||
- type: docker
|
||||
image: quay.io/biocontainers/xxx:0.1.0--py_0
|
||||
```
|
||||
|
||||
The container should have your tool installed, as well as `ps`.
|
||||
|
||||
If you didn't find a suitable container in the previous step, you can create a custom container. For example:
|
||||
|
||||
```yaml
|
||||
engines:
|
||||
- type: docker
|
||||
image: python:3.10
|
||||
setup:
|
||||
- type: python
|
||||
packages: numpy
|
||||
```
|
||||
|
||||
For more information on how to do this, see the [documentation](https://viash.io/guide/component/add-dependencies.html#steps-for-creating-a-custom-docker-platform).
|
||||
|
||||
Here is a list of base containers we can recommend:
|
||||
|
||||
* Bash: [`bash`](https://hub.docker.com/_/bash), [`ubuntu`](https://hub.docker.com/_/ubuntu)
|
||||
* C#: [`ghcr.io/data-intuitive/dotnet-script`](https://github.com/data-intuitive/ghcr-dotnet-script/pkgs/container/dotnet-script)
|
||||
* JavaScript: [`node`](https://hub.docker.com/_/node)
|
||||
* Python: [`python`](https://hub.docker.com/_/python), [`nvcr.io/nvidia/pytorch`](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch)
|
||||
* R: [`eddelbuettel/r2u`](https://hub.docker.com/r/eddelbuettel/r2u), [`rocker/tidyverse`](https://hub.docker.com/r/rocker/tidyverse)
|
||||
* Scala: [`sbtscala/scala-sbt`](https://hub.docker.com/r/sbtscala/scala-sbt)
|
||||
|
||||
### Step 11: Write a runner script
|
||||
|
||||
Next, we need to write a runner script that runs the tool with the input arguments. Create a Bash script named `src/xxx/script.sh` which runs the tool with the input arguments.
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
## VIASH START
|
||||
## VIASH END
|
||||
|
||||
xxx \
|
||||
--input "$par_input" \
|
||||
--output "$par_output" \
|
||||
$([ "$par_option" = "true" ] && echo "--option")
|
||||
```
|
||||
|
||||
When building a Viash component, Viash will automatically replace the `## VIASH START` and `## VIASH END` lines (and anything in between) with environment variables based on the arguments specified in the config.
|
||||
|
||||
As an example, this is what the Bash script for the `arriba` component looks like:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
## VIASH START
|
||||
## VIASH END
|
||||
|
||||
arriba \
|
||||
-x "$par_bam" \
|
||||
-a "$par_genome" \
|
||||
-g "$par_gene_annotation" \
|
||||
-o "$par_fusions" \
|
||||
${par_known_fusions:+-k "${par_known_fusions}"} \
|
||||
${par_blacklist:+-b "${par_blacklist}"} \
|
||||
${par_structural_variants:+-d "${par_structural_variants}"} \
|
||||
$([ "$par_skip_duplicate_marking" = "true" ] && echo "-u") \
|
||||
$([ "$par_extra_information" = "true" ] && echo "-X") \
|
||||
$([ "$par_fill_gaps" = "true" ] && echo "-I")
|
||||
```
|
||||
|
||||
|
||||
### Step 12: Create test script
|
||||
|
||||
|
||||
If the unit test requires test resources, these should be provided in the `test_resources` section of the component.
|
||||
|
||||
```yaml
|
||||
functionality:
|
||||
# ...
|
||||
test_resources:
|
||||
- type: bash_script
|
||||
path: test.sh
|
||||
- type: file
|
||||
path: test_data
|
||||
```
|
||||
|
||||
Create a test script at `src/xxx/test.sh` that runs the component with the test data. This script should run the component (available with `$meta_executable`) with the test data and check if the output is as expected. The script should exit with a non-zero exit code if the output is not as expected. For example:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
## VIASH START
|
||||
## VIASH END
|
||||
|
||||
echo "> Run xxx with test data"
|
||||
"$meta_executable" \
|
||||
--input "$meta_resources_dir/test_data/input.txt" \
|
||||
--output "output.txt" \
|
||||
--option
|
||||
|
||||
echo ">> Checking output"
|
||||
[ ! -f "output.txt" ] && echo "Output file output.txt does not exist" && exit 1
|
||||
```
|
||||
|
||||
|
||||
For example, this is what the test script for the `arriba` component looks like:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
|
||||
## VIASH START
|
||||
## VIASH END
|
||||
|
||||
echo "> Run arriba with blacklist"
|
||||
"$meta_executable" \
|
||||
--bam "$meta_resources_dir/test_data/A.bam" \
|
||||
--genome "$meta_resources_dir/test_data/genome.fasta" \
|
||||
--gene_annotation "$meta_resources_dir/test_data/annotation.gtf" \
|
||||
--blacklist "$meta_resources_dir/test_data/blacklist.tsv" \
|
||||
--fusions "fusions.tsv" \
|
||||
--fusions_discarded "fusions_discarded.tsv" \
|
||||
--interesting_contigs "1,2"
|
||||
|
||||
echo ">> Checking output"
|
||||
[ ! -f "fusions.tsv" ] && echo "Output file fusions.tsv does not exist" && exit 1
|
||||
[ ! -f "fusions_discarded.tsv" ] && echo "Output file fusions_discarded.tsv does not exist" && exit 1
|
||||
|
||||
echo ">> Check if output is empty"
|
||||
[ ! -s "fusions.tsv" ] && echo "Output file fusions.tsv is empty" && exit 1
|
||||
[ ! -s "fusions_discarded.tsv" ] && echo "Output file fusions_discarded.tsv is empty" && exit 1
|
||||
```
|
||||
|
||||
### Step 12: Create a `/var/software_versions.txt` file
|
||||
|
||||
For the sake of transparency and reproducibility, we require that the versions of the software used in the component are documented.
|
||||
|
||||
For now, this is managed by creating a file `/var/software_versions.txt` in the `setup` section of the Docker engine.
|
||||
|
||||
```yaml
|
||||
engines:
|
||||
- type: docker
|
||||
image: quay.io/biocontainers/xxx:0.1.0--py_0
|
||||
setup:
|
||||
- type: docker
|
||||
run: |
|
||||
echo "xxx: \"0.1.0\"" > /var/software_versions.txt
|
||||
```
|
||||
21
LICENSE
Normal file
21
LICENSE
Normal file
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2024 Data Intuitive
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
139
README.md
Normal file
139
README.md
Normal file
@@ -0,0 +1,139 @@
|
||||
|
||||
|
||||
# 🛠📦 toolbox
|
||||
|
||||
[](https://www.viash-hub.com/packages/toolbox)
|
||||
[](https://github.com/viash-hub/toolbox)
|
||||
[](https://github.com/viash-hub/toolbox/blob/main/LICENSE)
|
||||
[](https://github.com/viash-hub/toolbox/issues)
|
||||
[](https://viash.io)
|
||||
|
||||
A collection of curated command-line tools for general IT tasks, built
|
||||
with Viash.
|
||||
|
||||
## Introduction
|
||||
|
||||
`toolbox` provides a versatile suite of IT components, following the
|
||||
robust Viash (https://viash.io) framework. This package focuses on
|
||||
delivering reliable, standalone tools that can be easily integrated into
|
||||
larger computational workflows.
|
||||
|
||||
The core philosophy emphasizes **reusability**, **reproducibility**, and
|
||||
adherence to **best practices** in component creation. Key features of
|
||||
`toolbox` components include:
|
||||
|
||||
- **Standalone & Nextflow Ready:** Execute components directly from the
|
||||
command line or seamlessly incorporate them into Nextflow workflows.
|
||||
- **High Quality Standards:**
|
||||
- Comprehensive documentation for each component and its parameters.
|
||||
- Full exposure of the underlying tool’s arguments for maximum
|
||||
flexibility.
|
||||
- Containerized (Docker) to ensure consistent environments and manage
|
||||
dependencies, leading to enhanced reproducibility.
|
||||
- Unit tested to verify functionality and ensure reliability.
|
||||
|
||||
## Example Usage
|
||||
|
||||
Viash components in toolbox can be run in various ways:
|
||||
|
||||
``` mermaid lang="mermaid"
|
||||
flowchart TD
|
||||
A[toolbox v0.1.0] --> B(Viash Hub Launch)
|
||||
A --> C(Viash CLI)
|
||||
A --> D(Nextflow CLI)
|
||||
A --> E(Seqera Cloud)
|
||||
A --> F(As a dependency)
|
||||
```
|
||||
|
||||
### 1. Via the Viash Hub Launch interface
|
||||
|
||||
You can run this component directly from the Viash Hub [Launch
|
||||
interface](https://www.viash-hub.com/launch?package=toolbox&version=v0.1.0&component=yq&runner=Executable).
|
||||
|
||||

|
||||
|
||||
### 2. Via the Viash CLI
|
||||
|
||||
You can run this component directly from the command line using the
|
||||
Viash CLI.
|
||||
|
||||
``` bash
|
||||
viash run vsh://toolbox@v0.1.0/yq -- --help
|
||||
|
||||
viash run vsh://toolbox@v0.1.0/yq -- \
|
||||
--input path/to/input.yaml \
|
||||
--output output.yaml
|
||||
```
|
||||
|
||||
This will run the component with the specified input files and output
|
||||
the results to the specified output file.
|
||||
|
||||
### 3. Via the Nextflow CLI or Seqera Cloud
|
||||
|
||||
You can run this component as a Nextflow pipeline.
|
||||
|
||||
``` bash
|
||||
nextflow run https://packages.viash-hub.com/vsh/toolbox \
|
||||
-revision v0.1.0 \
|
||||
-main-script target/nextflow/yq/main.nf \
|
||||
-latest -resume \
|
||||
-profile docker \
|
||||
--input path/to/input.yaml \
|
||||
--publish_dir path/to/output
|
||||
```
|
||||
|
||||
**Note:** Make sure that the [Nextflow
|
||||
SCM](https://www.nextflow.io/docs/latest/git.html#git-configuration) is
|
||||
set up properly. You can do this by adding the following lines to your
|
||||
`~/.nextflow/scm` file:
|
||||
|
||||
``` groovy
|
||||
providers.vsh.platform = 'gitlab'
|
||||
providers.vsh.server = 'https://packages.viash-hub.com'
|
||||
```
|
||||
|
||||
**Tip:** This will also work with Seqera Cloud or other
|
||||
Nextflow-compatible platforms.
|
||||
|
||||
### 4. As a dependency
|
||||
|
||||
In your Viash config file (`config.vsh.yaml`), you can add this
|
||||
component as a dependency:
|
||||
|
||||
``` yaml
|
||||
dependencies:
|
||||
- name: yq
|
||||
repository: vsh://toolbox@v0.1.0
|
||||
```
|
||||
|
||||
**Tip:** See the [Viash
|
||||
documentation](https://viash.io/guide/nextflow_vdsl3/create-a-pipeline.html#pipeline-as-a-component)
|
||||
for more details on how to use Viash components as a dependency in your
|
||||
own Nextflow workflows.
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions are welcome! We aim to build a comprehensive collection of
|
||||
high-quality bioinformatics components. If you’d like to contribute,
|
||||
please follow these general steps:
|
||||
|
||||
1. Find a component to contribute
|
||||
2. Add config template
|
||||
3. Fill in the metadata
|
||||
4. Find a suitable container
|
||||
5. Create help file
|
||||
6. Create or fetch test data
|
||||
7. Add arguments for the input files
|
||||
8. Add arguments for the output files
|
||||
9. Add arguments for the other arguments
|
||||
10. Add a Docker engine
|
||||
11. Write a runner script
|
||||
12. Create test script
|
||||
13. Create a `/var/software_versions.txt` file
|
||||
|
||||
See the
|
||||
[CONTRIBUTING](https://github.com/viash-hub/toolbox/blob/main/CONTRIBUTING.md)
|
||||
file for more details.
|
||||
119
README.qmd
Normal file
119
README.qmd
Normal file
@@ -0,0 +1,119 @@
|
||||
---
|
||||
format: gfm
|
||||
---
|
||||
```{r setup, include=FALSE}
|
||||
package <- yaml::read_yaml("_viash.yaml")
|
||||
license <- paste0(package$links$repository, "/blob/main/LICENSE")
|
||||
contributing <- paste0(package$links$repository, "/blob/main/CONTRIBUTING.md")
|
||||
|
||||
pkg <- package$name
|
||||
ver <- if (!is.null(package$version)) package$version else "v0.1.0"
|
||||
comp <- "yq"
|
||||
```
|
||||
# 🛠📦 `r pkg`
|
||||
|
||||
[](https://www.viash-hub.com/packages/`r pkg`)
|
||||
[](`r package$links$repository`)
|
||||
[](`r license`)
|
||||
[](`r package$links$issue_tracker`)
|
||||
[`-blue.svg)](https://viash.io)
|
||||
|
||||
`r package$summary`
|
||||
|
||||
## Introduction
|
||||
|
||||
`r package$description`
|
||||
|
||||
## Example Usage
|
||||
|
||||
Viash components in `r pkg` can be run in various ways:
|
||||
|
||||
```{r mmd, echo=FALSE, results='asis'}
|
||||
cat(
|
||||
"```mermaid\n",
|
||||
"flowchart TD\n",
|
||||
" A[", pkg, " ", ver, "] --> B(Viash Hub Launch)\n",
|
||||
" A --> C(Viash CLI)\n",
|
||||
" A --> D(Nextflow CLI)\n",
|
||||
" A --> E(Seqera Cloud)\n",
|
||||
" A --> F(As a dependency)\n",
|
||||
"```\n",
|
||||
sep = ""
|
||||
)
|
||||
```
|
||||
|
||||
### 1. Via the Viash Hub Launch interface
|
||||
|
||||
You can run this component directly from the Viash Hub [Launch interface](https://www.viash-hub.com/launch?package=`r pkg`&version=`r ver`&component=`r comp`&runner=Executable).
|
||||
|
||||

|
||||
|
||||
### 2. Via the Viash CLI
|
||||
|
||||
You can run this component directly from the command line using the Viash CLI.
|
||||
|
||||
```bash
|
||||
viash run vsh://`r pkg`@`r ver`/`r comp` -- --help
|
||||
|
||||
viash run vsh://`r pkg`@`r ver`/`r comp` -- \
|
||||
--input path/to/input.yaml \
|
||||
--output output.yaml
|
||||
```
|
||||
|
||||
This will run the component with the specified input files and output the results to the specified output file.
|
||||
|
||||
### 3. Via the Nextflow CLI or Seqera Cloud
|
||||
|
||||
You can run this component as a Nextflow pipeline.
|
||||
|
||||
```bash
|
||||
nextflow run https://packages.viash-hub.com/vsh/`r pkg` \
|
||||
-revision `r ver` \
|
||||
-main-script target/nextflow/`r comp`/main.nf \
|
||||
-latest -resume \
|
||||
-profile docker \
|
||||
--input path/to/input.yaml \
|
||||
--publish_dir path/to/output
|
||||
```
|
||||
|
||||
**Note:** Make sure that the [Nextflow SCM](https://www.nextflow.io/docs/latest/git.html#git-configuration) is set up properly. You can do this by adding the following lines to your `~/.nextflow/scm` file:
|
||||
|
||||
```groovy
|
||||
providers.vsh.platform = 'gitlab'
|
||||
providers.vsh.server = 'https://packages.viash-hub.com'
|
||||
```
|
||||
|
||||
**Tip:** This will also work with Seqera Cloud or other Nextflow-compatible platforms.
|
||||
|
||||
### 4. As a dependency
|
||||
|
||||
In your Viash config file (`config.vsh.yaml`), you can add this component as a dependency:
|
||||
|
||||
```yaml
|
||||
dependencies:
|
||||
- name: `r comp`
|
||||
repository: vsh://`r pkg`@`r ver`
|
||||
```
|
||||
|
||||
**Tip:** See the [Viash documentation](https://viash.io/guide/nextflow_vdsl3/create-a-pipeline.html#pipeline-as-a-component) for more details on how to use Viash components as a dependency in your own Nextflow workflows.
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions are welcome! We aim to build a comprehensive collection of high-quality bioinformatics components. If you'd like to contribute, please follow these general steps:
|
||||
|
||||
|
||||
```{r echo=FALSE}
|
||||
lines <- readr::read_lines("CONTRIBUTING.md")
|
||||
|
||||
index_start <- grep("^### Step [0-9]*:", lines)
|
||||
|
||||
index_end <- c(index_start[-1] - 1, length(lines))
|
||||
|
||||
name <- gsub("^### Step [0-9]*: *", "", lines[index_start])
|
||||
|
||||
knitr::asis_output(
|
||||
paste(paste0(" 1. ", name, "\n"), collapse = "")
|
||||
)
|
||||
```
|
||||
|
||||
See the [CONTRIBUTING](`r contributing`) file for more details.
|
||||
25
_viash.yaml
Normal file
25
_viash.yaml
Normal file
@@ -0,0 +1,25 @@
|
||||
name: toolbox
|
||||
summary: |
|
||||
A collection of curated command-line tools for general IT tasks, built with Viash.
|
||||
description: |
|
||||
`toolbox` provides a versatile suite of IT components, following the robust Viash (https://viash.io) framework.
|
||||
This package focuses on delivering reliable, standalone tools that can be easily integrated into larger computational workflows.
|
||||
|
||||
The core philosophy emphasizes **reusability**, **reproducibility**, and adherence to **best practices** in component creation. Key features of `toolbox` components include:
|
||||
|
||||
* **Standalone & Nextflow Ready:** Execute components directly from the command line or seamlessly incorporate them into Nextflow workflows.
|
||||
* **High Quality Standards:**
|
||||
* Comprehensive documentation for each component and its parameters.
|
||||
* Full exposure of the underlying tool's arguments for maximum flexibility.
|
||||
* Containerized (Docker) to ensure consistent environments and manage dependencies, leading to enhanced reproducibility.
|
||||
* Unit tested to verify functionality and ensure reliability.
|
||||
license: MIT
|
||||
keywords: [toolbox, command-line, tools]
|
||||
links:
|
||||
issue_tracker: https://github.com/viash-hub/toolbox/issues
|
||||
repository: https://github.com/viash-hub/toolbox
|
||||
|
||||
viash_version: 0.9.4
|
||||
|
||||
config_mods: |
|
||||
.requirements.commands := ['ps']
|
||||
BIN
docs/viash-hub.png
Normal file
BIN
docs/viash-hub.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 147 KiB |
3
main.nf
Normal file
3
main.nf
Normal file
@@ -0,0 +1,3 @@
|
||||
workflow {
|
||||
print("This is a dummy placeholder for pipeline execution. Please use the corresponding nf files for running pipelines.")
|
||||
}
|
||||
6
nextflow.config
Normal file
6
nextflow.config
Normal file
@@ -0,0 +1,6 @@
|
||||
manifest {
|
||||
name = "toolbox"
|
||||
version = "gunzip"
|
||||
defaultBranch = "main"
|
||||
nextflowVersion = "!>=20.12.1-edge"
|
||||
}
|
||||
90
src/bgzip/config.vsh.yaml
Normal file
90
src/bgzip/config.vsh.yaml
Normal file
@@ -0,0 +1,90 @@
|
||||
name: bgzip
|
||||
description: Block compression/decompression utility
|
||||
links:
|
||||
homepage: https://www.htslib.org/
|
||||
documentation: https://www.htslib.org/doc/bgzip.html
|
||||
repository: https://github.com/samtools/htslib
|
||||
references:
|
||||
doi: 10.1093/gigascience/giab007
|
||||
license: MIT
|
||||
requirements:
|
||||
commands: [ bgzip ]
|
||||
argument_groups:
|
||||
- name: Inputs
|
||||
arguments:
|
||||
- name: --input
|
||||
type: file
|
||||
direction: input
|
||||
description: file to be compressed or decompressed
|
||||
required: true
|
||||
- name: Outputs
|
||||
arguments:
|
||||
- name: --output
|
||||
type: file
|
||||
direction: output
|
||||
description: compressed or decompressed output
|
||||
required: true
|
||||
- name: --index_name
|
||||
alternatives: -I
|
||||
type: file
|
||||
direction: output
|
||||
description: name of BGZF index file [file.gz.gzi]
|
||||
- name: Arguments
|
||||
arguments:
|
||||
- name: --offset
|
||||
alternatives: -b
|
||||
type: integer
|
||||
description: decompress at virtual file pointer (0-based uncompressed offset)
|
||||
- name: --decompress
|
||||
alternatives: -d
|
||||
type: boolean_true
|
||||
description: decompress the input file
|
||||
- name: --rebgzip
|
||||
alternatives: -g
|
||||
type: boolean_true
|
||||
description: use an index file to bgzip a file
|
||||
- name: --index
|
||||
alternatives: -i
|
||||
type: boolean_true
|
||||
description: compress and create BGZF index
|
||||
- name: --compress_level
|
||||
alternatives: -l
|
||||
type: integer
|
||||
description: compression level to use when compressing; 0 to 9, or -1 for default [-1]
|
||||
min: -1
|
||||
max: 9
|
||||
- name: --reindex
|
||||
alternatives: -r
|
||||
type: boolean_true
|
||||
description: (re)index the output file
|
||||
- name: --size
|
||||
alternatives: -s
|
||||
type: integer
|
||||
description: decompress INT bytes (uncompressed size)
|
||||
min: 0
|
||||
- name: --test
|
||||
alternatives: -t
|
||||
type: boolean_true
|
||||
description: test integrity of compressed file
|
||||
- name: --binary
|
||||
type: boolean_true
|
||||
description: Don't align blocks with text lines
|
||||
resources:
|
||||
- type: bash_script
|
||||
path: script.sh
|
||||
test_resources:
|
||||
- type: bash_script
|
||||
path: test.sh
|
||||
- type: file
|
||||
path: test_data
|
||||
|
||||
engines:
|
||||
- type: docker
|
||||
image: quay.io/biocontainers/htslib:1.19--h81da01d_0
|
||||
setup:
|
||||
- type: docker
|
||||
run: |
|
||||
bgzip -h | grep 'Version:' 2>&1 | sed 's/Version:\s\(.*\)/bgzip: "\1"/' > /var/software_versions.txt
|
||||
runners:
|
||||
- type: executable
|
||||
- type: nextflow
|
||||
22
src/bgzip/help.txt
Normal file
22
src/bgzip/help.txt
Normal file
@@ -0,0 +1,22 @@
|
||||
```bash
|
||||
bgzip -h
|
||||
```
|
||||
|
||||
Version: 1.19
|
||||
Usage: bgzip [OPTIONS] [FILE] ...
|
||||
Options:
|
||||
-b, --offset INT decompress at virtual file pointer (0-based uncompressed offset)
|
||||
-c, --stdout write on standard output, keep original files unchanged
|
||||
-d, --decompress decompress
|
||||
-f, --force overwrite files without asking
|
||||
-g, --rebgzip use an index file to bgzip a file
|
||||
-h, --help give this help
|
||||
-i, --index compress and create BGZF index
|
||||
-I, --index-name FILE name of BGZF index file [file.gz.gzi]
|
||||
-k, --keep don't delete input files during operation
|
||||
-l, --compress-level INT Compression level to use when compressing; 0 to 9, or -1 for default [-1]
|
||||
-r, --reindex (re)index compressed file
|
||||
-s, --size INT decompress INT bytes (uncompressed size)
|
||||
-t, --test test integrity of compressed file
|
||||
--binary Don't align blocks with text lines
|
||||
-@, --threads INT number of compression threads to use [1]
|
||||
21
src/bgzip/script.sh
Executable file
21
src/bgzip/script.sh
Executable file
@@ -0,0 +1,21 @@
|
||||
#!/bin/bash
|
||||
|
||||
[[ "$par_decompress" == "false" ]] && unset par_decompress
|
||||
[[ "$par_rebgzip" == "false" ]] && unset par_rebgzip
|
||||
[[ "$par_index" == "false" ]] && unset par_index
|
||||
[[ "$par_reindex" == "false" ]] && unset par_reindex
|
||||
[[ "$par_test" == "false" ]] && unset par_test
|
||||
[[ "$par_binary" == "false" ]] && unset par_binary
|
||||
bgzip -c \
|
||||
${meta_cpus:+--threads "${meta_cpus}"} \
|
||||
${par_offset:+-b "${par_offset}"} \
|
||||
${par_decompress:+-d} \
|
||||
${par_rebgzip:+-g} \
|
||||
${par_index:+-i} \
|
||||
${par_index_name:+-I "${par_index_name}"} \
|
||||
${par_compress_level:+-l "${par_compress_level}"} \
|
||||
${par_reindex:+-r} \
|
||||
${par_size:+-s "${par_size}"} \
|
||||
${par_test:+-t} \
|
||||
${par_binary:+--binary} \
|
||||
"$par_input" > "$par_output"
|
||||
19
src/bgzip/test.sh
Executable file
19
src/bgzip/test.sh
Executable file
@@ -0,0 +1,19 @@
|
||||
set -e
|
||||
|
||||
"$meta_executable" --input "$meta_resources_dir/test_data/test.vcf" --output "test.vcf.gz"
|
||||
|
||||
echo ">> Checking output of compressing"
|
||||
[ ! -f "test.vcf.gz" ] && echo "Output file test.vcf.gz does not exist" && exit 1
|
||||
|
||||
"$meta_executable" --input "test.vcf.gz" --output "test.vcf" --decompress
|
||||
|
||||
echo ">> Checking output of decompressing"
|
||||
[ ! -f "test.vcf" ] && echo "Output file test.vcf does not exist" && exit 1
|
||||
|
||||
echo ">> Checking original and decompressed files are the same"
|
||||
set +e
|
||||
cmp --silent -- "$meta_resources_dir/test_data/test.vcf" "test.vcf"
|
||||
[ $? -ne 0 ] && echo "files are different" && exit 1
|
||||
set -e
|
||||
|
||||
echo "> Test successful"
|
||||
10
src/bgzip/test_data/script.sh
Normal file
10
src/bgzip/test_data/script.sh
Normal file
@@ -0,0 +1,10 @@
|
||||
# bgzip test data
|
||||
|
||||
# Test data was obtained from https://github.com/snakemake/snakemake-wrappers/tree/master/bio/bgzip/test.
|
||||
|
||||
if [ ! -d /tmp/snakemake-wrappers ]; then
|
||||
git clone --depth 1 --single-branch --branch master https://github.com/snakemake/snakemake-wrappers /tmp/snakemake-wrappers
|
||||
fi
|
||||
|
||||
cp -r /tmp/snakemake-wrappers/bio/bgzip/test/* src/bgzip/test_data
|
||||
|
||||
23
src/bgzip/test_data/test.vcf
Normal file
23
src/bgzip/test_data/test.vcf
Normal file
@@ -0,0 +1,23 @@
|
||||
##fileformat=VCFv4.0
|
||||
##fileDate=20090805
|
||||
##source=https://www.internationalgenome.org/wiki/Analysis/vcf4.0/
|
||||
##reference=1000GenomesPilot-NCBI36
|
||||
##phasing=partial
|
||||
##INFO=<ID=NS,Number=1,Type=Integer,Description="Number of Samples With Data">
|
||||
##INFO=<ID=DP,Number=1,Type=Integer,Description="Total Depth">
|
||||
##INFO=<ID=AF,Number=.,Type=Float,Description="Allele Frequency">
|
||||
##INFO=<ID=AA,Number=1,Type=String,Description="Ancestral Allele">
|
||||
##INFO=<ID=DB,Number=0,Type=Flag,Description="dbSNP membership, build 129">
|
||||
##INFO=<ID=H2,Number=0,Type=Flag,Description="HapMap2 membership">
|
||||
##FILTER=<ID=q10,Description="Quality below 10">
|
||||
##FILTER=<ID=s50,Description="Less than 50% of samples have data">
|
||||
##FORMAT=<ID=GT,Number=1,Type=String,Description="Genotype">
|
||||
##FORMAT=<ID=GQ,Number=1,Type=Integer,Description="Genotype Quality">
|
||||
##FORMAT=<ID=DP,Number=1,Type=Integer,Description="Read Depth">
|
||||
##FORMAT=<ID=HQ,Number=2,Type=Integer,Description="Haplotype Quality">
|
||||
#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT NA00001 NA00002 NA00003
|
||||
20 14370 rs6054257 G A 29 PASS NS=3;DP=14;AF=0.5;DB;H2 GT:GQ:DP:HQ 0|0:48:1:51,51 1|0:48:8:51,51 1/1:43:5:.,.
|
||||
20 17330 . T A 3 q10 NS=3;DP=11;AF=0.017 GT:GQ:DP:HQ 0|0:49:3:58,50 0|1:3:5:65,3 0/0:41:3
|
||||
20 1110696 rs6040355 A G,T 67 PASS NS=2;DP=10;AF=0.333,0.667;AA=T;DB GT:GQ:DP:HQ 1|2:21:6:23,27 2|1:2:0:18,2 2/2:35:4
|
||||
20 1230237 . T . 47 PASS NS=3;DP=13;AA=T GT:GQ:DP:HQ 0|0:54:7:56,60 0|0:48:4:51,51 0/0:61:2
|
||||
20 1234567 microsat1 GTCT G,GTACT 50 PASS NS=3;DP=9;AA=G GT:GQ:DP 0/1:35:4 0/2:17:2 1/1:40:3
|
||||
87
src/yq/config.vsh.yaml
Normal file
87
src/yq/config.vsh.yaml
Normal file
@@ -0,0 +1,87 @@
|
||||
name: yq
|
||||
description: A portable YAML, JSON, XML, CSV, TOML and properties processor
|
||||
keywords: [ yaml, json, xml, csv, toml, properties ]
|
||||
links:
|
||||
homepage: https://mikefarah.gitbook.io/yq
|
||||
documentation: https://mikefarah.gitbook.io/yq/
|
||||
repository: https://github.com/mikefarah/yq
|
||||
license: MIT
|
||||
requirements:
|
||||
commands: [ yq ]
|
||||
argument_groups:
|
||||
- name: Inputs
|
||||
arguments:
|
||||
- name: --input
|
||||
type: file
|
||||
direction: input
|
||||
description: files to be processed
|
||||
required: true
|
||||
example: input.yaml
|
||||
- name: Outputs
|
||||
arguments:
|
||||
- name: --output
|
||||
type: file
|
||||
direction: output
|
||||
description: output file
|
||||
required: true
|
||||
example: output.yaml
|
||||
- name: Arguments
|
||||
arguments:
|
||||
- name: --eval
|
||||
type: string
|
||||
description: expression to evaluate
|
||||
required: true
|
||||
example: '.name = "foo"'
|
||||
- name: --indent
|
||||
type: integer
|
||||
description: sets indent level for output (default 2)
|
||||
alternatives: -I
|
||||
- name: --input_format
|
||||
type: string
|
||||
description: 'parse format for input. (default "auto")'
|
||||
alternatives: -p
|
||||
choices: [ auto, a, yaml, "y", json, j, props, p, csv, c, tsv, t, xml, x, base64, uri, toml, shell, s, lua, l ]
|
||||
- name: --output_format
|
||||
type: string
|
||||
description: 'output format type. (default "auto")'
|
||||
alternatives: -o
|
||||
choices: [ auto, a, yaml, "y", json, j, props, p, csv, c, tsv, t, xml, x, base64, uri, toml, shell, s, lua, l ]
|
||||
- name: --pretty_print
|
||||
type: boolean_true
|
||||
description: pretty print, shorthand for '... style = ""'
|
||||
alternatives: -P
|
||||
|
||||
resources:
|
||||
- type: bash_script
|
||||
text: |
|
||||
#!/bin/sh
|
||||
[[ "$par_pretty_print" == "false" ]] && unset par_pretty_print
|
||||
yq eval \
|
||||
${par_indent:+-I "${par_indent}"} \
|
||||
${par_input_format:+-p "${par_input_format}"} \
|
||||
${par_output_format:+-o "${par_output_format}"} \
|
||||
${par_pretty_print:+-P} \
|
||||
--expression "$par_eval" \
|
||||
--no-colors \
|
||||
"$par_input" > "$par_output"
|
||||
test_resources:
|
||||
- type: bash_script
|
||||
text: |
|
||||
set -e
|
||||
echo "name: 'bar'" > test.yaml
|
||||
"$meta_executable" --input test.yaml --output output.yaml --eval '.name = "foo"'
|
||||
"$meta_executable" --input output.yaml --output output2.yaml --eval '.name'
|
||||
grep "^foo$" output2.yaml
|
||||
|
||||
engines:
|
||||
- type: docker
|
||||
image: alpine:latest
|
||||
setup:
|
||||
- type: apk
|
||||
packages: [bash, yq-go]
|
||||
- type: docker
|
||||
run: |
|
||||
/usr/bin/yq --version | sed 's/.*version\sv\(.*\)/yq: "\1"/' > /var/software_versions.txt
|
||||
runners:
|
||||
- type: executable
|
||||
- type: nextflow
|
||||
72
src/yq/help.txt
Normal file
72
src/yq/help.txt
Normal file
@@ -0,0 +1,72 @@
|
||||
yq is a portable command-line data file processor (https://github.com/mikefarah/yq/)
|
||||
See https://mikefarah.gitbook.io/yq/ for detailed documentation and examples.
|
||||
|
||||
Usage:
|
||||
yq [flags]
|
||||
yq [command]
|
||||
|
||||
Examples:
|
||||
|
||||
# yq tries to auto-detect the file format based off the extension, and defaults to YAML if it's unknown (or piping through STDIN)
|
||||
# Use the '-p/--input-format' flag to specify a format type.
|
||||
cat file.xml | yq -p xml
|
||||
|
||||
# read the "stuff" node from "myfile.yml"
|
||||
yq '.stuff' < myfile.yml
|
||||
|
||||
# update myfile.yml in place
|
||||
yq -i '.stuff = "foo"' myfile.yml
|
||||
|
||||
# print contents of sample.json as idiomatic YAML
|
||||
yq -P -oy sample.json
|
||||
|
||||
|
||||
Available Commands:
|
||||
completion Generate the autocompletion script for the specified shell
|
||||
eval (default) Apply the expression to each document in each yaml file in sequence
|
||||
eval-all Loads _all_ yaml documents of _all_ yaml files and runs expression once
|
||||
help Help about any command
|
||||
|
||||
Flags:
|
||||
-C, --colors force print with colors
|
||||
--csv-auto-parse parse CSV YAML/JSON values (default true)
|
||||
--csv-separator char CSV Separator character (default ,)
|
||||
-e, --exit-status set exit status if there are no matches or null or false is returned
|
||||
--expression string forcibly set the expression argument. Useful when yq argument detection thinks your expression is a file.
|
||||
--from-file string Load expression from specified file.
|
||||
-f, --front-matter string (extract|process) first input as yaml front-matter. Extract will pull out the yaml content, process will run the expression against the yaml content, leaving the remaining data intact
|
||||
--header-preprocess Slurp any header comments and separators before processing expression. (default true)
|
||||
-h, --help help for yq
|
||||
-I, --indent int sets indent level for output (default 2)
|
||||
-i, --inplace update the file in place of first file given.
|
||||
-p, --input-format string [auto|a|yaml|y|json|j|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|lua|l] parse format for input. (default "auto")
|
||||
--lua-globals output keys as top-level global variables
|
||||
--lua-prefix string prefix (default "return ")
|
||||
--lua-suffix string suffix (default ";\n")
|
||||
--lua-unquoted output unquoted string keys (e.g. {foo="bar"})
|
||||
-M, --no-colors force print with no colors
|
||||
-N, --no-doc Don't print document separators (---)
|
||||
-0, --nul-output Use NUL char to separate values. If unwrap scalar is also set, fail if unwrapped scalar contains NUL char.
|
||||
-n, --null-input Don't read input, simply evaluate the expression given. Useful for creating docs from scratch.
|
||||
-o, --output-format string [auto|a|yaml|y|json|j|props|p|csv|c|tsv|t|xml|x|base64|uri|toml|shell|s|lua|l] output format type. (default "auto")
|
||||
-P, --prettyPrint pretty print, shorthand for '... style = ""'
|
||||
--properties-array-brackets use [x] in array paths (e.g. for SpringBoot)
|
||||
--properties-separator string separator to use between keys and values (default " = ")
|
||||
-s, --split-exp string print each result (or doc) into a file named (exp). [exp] argument must return a string. You can use $index in the expression as the result counter.
|
||||
--split-exp-file string Use a file to specify the split-exp expression.
|
||||
--string-interpolation Toggles strings interpolation of \(exp) (default true)
|
||||
--tsv-auto-parse parse TSV YAML/JSON values (default true)
|
||||
-r, --unwrapScalar unwrap scalar, print the value with no quotes, colors or comments. Defaults to true for yaml (default true)
|
||||
-v, --verbose verbose mode
|
||||
-V, --version Print version information and quit
|
||||
--xml-attribute-prefix string prefix for xml attributes (default "+@")
|
||||
--xml-content-name string name for xml content (if no attribute name is present). (default "+content")
|
||||
--xml-directive-name string name for xml directives (e.g. <!DOCTYPE thing cat>) (default "+directive")
|
||||
--xml-keep-namespace enables keeping namespace after parsing attributes (default true)
|
||||
--xml-proc-inst-prefix string prefix for xml processing instructions (e.g. <?xml version="1"?>) (default "+p_")
|
||||
--xml-raw-token enables using RawToken method instead Token. Commonly disables namespace translations. See https://pkg.go.dev/encoding/xml#Decoder.RawToken for details. (default true)
|
||||
--xml-skip-directives skip over directives (e.g. <!DOCTYPE thing cat>)
|
||||
--xml-skip-proc-inst skip over process instructions (e.g. <?xml version="1"?>)
|
||||
--xml-strict-mode enables strict parsing of XML. See https://pkg.go.dev/encoding/xml for more details.
|
||||
|
||||
Use "yq [command] --help" for more information about a command.
|
||||
0
target/.build.yaml
Normal file
0
target/.build.yaml
Normal file
267
target/executable/bgzip/.config.vsh.yaml
Normal file
267
target/executable/bgzip/.config.vsh.yaml
Normal file
@@ -0,0 +1,267 @@
|
||||
name: "bgzip"
|
||||
version: "gunzip"
|
||||
argument_groups:
|
||||
- name: "Inputs"
|
||||
arguments:
|
||||
- type: "file"
|
||||
name: "--input"
|
||||
description: "file to be compressed or decompressed"
|
||||
info: null
|
||||
must_exist: true
|
||||
create_parent: true
|
||||
required: true
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- name: "Outputs"
|
||||
arguments:
|
||||
- type: "file"
|
||||
name: "--output"
|
||||
description: "compressed or decompressed output"
|
||||
info: null
|
||||
must_exist: true
|
||||
create_parent: true
|
||||
required: true
|
||||
direction: "output"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "file"
|
||||
name: "--index_name"
|
||||
alternatives:
|
||||
- "-I"
|
||||
description: "name of BGZF index file [file.gz.gzi]"
|
||||
info: null
|
||||
must_exist: true
|
||||
create_parent: true
|
||||
required: false
|
||||
direction: "output"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- name: "Arguments"
|
||||
arguments:
|
||||
- type: "integer"
|
||||
name: "--offset"
|
||||
alternatives:
|
||||
- "-b"
|
||||
description: "decompress at virtual file pointer (0-based uncompressed offset)"
|
||||
info: null
|
||||
required: false
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "boolean_true"
|
||||
name: "--decompress"
|
||||
alternatives:
|
||||
- "-d"
|
||||
description: "decompress the input file"
|
||||
info: null
|
||||
direction: "input"
|
||||
- type: "boolean_true"
|
||||
name: "--rebgzip"
|
||||
alternatives:
|
||||
- "-g"
|
||||
description: "use an index file to bgzip a file"
|
||||
info: null
|
||||
direction: "input"
|
||||
- type: "boolean_true"
|
||||
name: "--index"
|
||||
alternatives:
|
||||
- "-i"
|
||||
description: "compress and create BGZF index"
|
||||
info: null
|
||||
direction: "input"
|
||||
- type: "integer"
|
||||
name: "--compress_level"
|
||||
alternatives:
|
||||
- "-l"
|
||||
description: "compression level to use when compressing; 0 to 9, or -1 for default\
|
||||
\ [-1]"
|
||||
info: null
|
||||
required: false
|
||||
min: -1
|
||||
max: 9
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "boolean_true"
|
||||
name: "--reindex"
|
||||
alternatives:
|
||||
- "-r"
|
||||
description: "(re)index the output file"
|
||||
info: null
|
||||
direction: "input"
|
||||
- type: "integer"
|
||||
name: "--size"
|
||||
alternatives:
|
||||
- "-s"
|
||||
description: "decompress INT bytes (uncompressed size)"
|
||||
info: null
|
||||
required: false
|
||||
min: 0
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "boolean_true"
|
||||
name: "--test"
|
||||
alternatives:
|
||||
- "-t"
|
||||
description: "test integrity of compressed file"
|
||||
info: null
|
||||
direction: "input"
|
||||
- type: "boolean_true"
|
||||
name: "--binary"
|
||||
description: "Don't align blocks with text lines"
|
||||
info: null
|
||||
direction: "input"
|
||||
resources:
|
||||
- type: "bash_script"
|
||||
path: "script.sh"
|
||||
is_executable: true
|
||||
description: "Block compression/decompression utility"
|
||||
test_resources:
|
||||
- type: "bash_script"
|
||||
path: "test.sh"
|
||||
is_executable: true
|
||||
- type: "file"
|
||||
path: "test_data"
|
||||
info: null
|
||||
status: "enabled"
|
||||
scope:
|
||||
image: "public"
|
||||
target: "public"
|
||||
requirements:
|
||||
commands:
|
||||
- "ps"
|
||||
license: "MIT"
|
||||
references:
|
||||
doi:
|
||||
- "10.1093/gigascience/giab007"
|
||||
links:
|
||||
repository: "https://github.com/samtools/htslib"
|
||||
homepage: "https://www.htslib.org/"
|
||||
documentation: "https://www.htslib.org/doc/bgzip.html"
|
||||
runners:
|
||||
- type: "executable"
|
||||
id: "executable"
|
||||
docker_setup_strategy: "ifneedbepullelsecachedbuild"
|
||||
- type: "nextflow"
|
||||
id: "nextflow"
|
||||
directives:
|
||||
tag: "$id"
|
||||
auto:
|
||||
simplifyInput: true
|
||||
simplifyOutput: false
|
||||
transcript: false
|
||||
publish: false
|
||||
config:
|
||||
labels:
|
||||
mem1gb: "memory = 1000000000.B"
|
||||
mem2gb: "memory = 2000000000.B"
|
||||
mem5gb: "memory = 5000000000.B"
|
||||
mem10gb: "memory = 10000000000.B"
|
||||
mem20gb: "memory = 20000000000.B"
|
||||
mem50gb: "memory = 50000000000.B"
|
||||
mem100gb: "memory = 100000000000.B"
|
||||
mem200gb: "memory = 200000000000.B"
|
||||
mem500gb: "memory = 500000000000.B"
|
||||
mem1tb: "memory = 1000000000000.B"
|
||||
mem2tb: "memory = 2000000000000.B"
|
||||
mem5tb: "memory = 5000000000000.B"
|
||||
mem10tb: "memory = 10000000000000.B"
|
||||
mem20tb: "memory = 20000000000000.B"
|
||||
mem50tb: "memory = 50000000000000.B"
|
||||
mem100tb: "memory = 100000000000000.B"
|
||||
mem200tb: "memory = 200000000000000.B"
|
||||
mem500tb: "memory = 500000000000000.B"
|
||||
mem1gib: "memory = 1073741824.B"
|
||||
mem2gib: "memory = 2147483648.B"
|
||||
mem4gib: "memory = 4294967296.B"
|
||||
mem8gib: "memory = 8589934592.B"
|
||||
mem16gib: "memory = 17179869184.B"
|
||||
mem32gib: "memory = 34359738368.B"
|
||||
mem64gib: "memory = 68719476736.B"
|
||||
mem128gib: "memory = 137438953472.B"
|
||||
mem256gib: "memory = 274877906944.B"
|
||||
mem512gib: "memory = 549755813888.B"
|
||||
mem1tib: "memory = 1099511627776.B"
|
||||
mem2tib: "memory = 2199023255552.B"
|
||||
mem4tib: "memory = 4398046511104.B"
|
||||
mem8tib: "memory = 8796093022208.B"
|
||||
mem16tib: "memory = 17592186044416.B"
|
||||
mem32tib: "memory = 35184372088832.B"
|
||||
mem64tib: "memory = 70368744177664.B"
|
||||
mem128tib: "memory = 140737488355328.B"
|
||||
mem256tib: "memory = 281474976710656.B"
|
||||
mem512tib: "memory = 562949953421312.B"
|
||||
cpu1: "cpus = 1"
|
||||
cpu2: "cpus = 2"
|
||||
cpu5: "cpus = 5"
|
||||
cpu10: "cpus = 10"
|
||||
cpu20: "cpus = 20"
|
||||
cpu50: "cpus = 50"
|
||||
cpu100: "cpus = 100"
|
||||
cpu200: "cpus = 200"
|
||||
cpu500: "cpus = 500"
|
||||
cpu1000: "cpus = 1000"
|
||||
debug: false
|
||||
container: "docker"
|
||||
engines:
|
||||
- type: "docker"
|
||||
id: "docker"
|
||||
image: "quay.io/biocontainers/htslib:1.19--h81da01d_0"
|
||||
target_registry: "images.viash-hub.com"
|
||||
target_tag: "gunzip"
|
||||
namespace_separator: "/"
|
||||
setup:
|
||||
- type: "docker"
|
||||
run:
|
||||
- "bgzip -h | grep 'Version:' 2>&1 | sed 's/Version:\\s\\(.*\\)/bgzip: \"\\1\"\
|
||||
/' > /var/software_versions.txt\n"
|
||||
entrypoint: []
|
||||
cmd: null
|
||||
- type: "native"
|
||||
id: "native"
|
||||
build_info:
|
||||
config: "src/bgzip/config.vsh.yaml"
|
||||
runner: "executable"
|
||||
engine: "docker|native"
|
||||
output: "target/executable/bgzip"
|
||||
executable: "target/executable/bgzip/bgzip"
|
||||
viash_version: "0.9.4"
|
||||
git_commit: "add30ba0f36bd8a8b07f0ba640707016d04e11b2"
|
||||
git_remote: "https://github.com/viash-hub/toolbox"
|
||||
package_config:
|
||||
name: "toolbox"
|
||||
version: "gunzip"
|
||||
summary: "A collection of curated command-line tools for general IT tasks, built\
|
||||
\ with Viash.\n"
|
||||
description: "`toolbox` provides a versatile suite of IT components, following the\
|
||||
\ robust Viash (https://viash.io) framework.\nThis package focuses on delivering\
|
||||
\ reliable, standalone tools that can be easily integrated into larger computational\
|
||||
\ workflows.\n\nThe core philosophy emphasizes **reusability**, **reproducibility**,\
|
||||
\ and adherence to **best practices** in component creation. Key features of `toolbox`\
|
||||
\ components include:\n\n* **Standalone & Nextflow Ready:** Execute components\
|
||||
\ directly from the command line or seamlessly incorporate them into Nextflow\
|
||||
\ workflows.\n* **High Quality Standards:**\n * Comprehensive documentation\
|
||||
\ for each component and its parameters.\n * Full exposure of the underlying\
|
||||
\ tool's arguments for maximum flexibility.\n * Containerized (Docker) to ensure\
|
||||
\ consistent environments and manage dependencies, leading to enhanced reproducibility.\n\
|
||||
\ * Unit tested to verify functionality and ensure reliability.\n"
|
||||
info: null
|
||||
viash_version: "0.9.4"
|
||||
source: "src"
|
||||
target: "target"
|
||||
config_mods:
|
||||
- ".requirements.commands := ['ps']\n"
|
||||
- ".engines += { type: \"native\" }"
|
||||
- ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
|
||||
- ".engines[.type == 'docker'].target_tag := 'gunzip'"
|
||||
keywords:
|
||||
- "toolbox"
|
||||
- "command-line"
|
||||
- "tools"
|
||||
license: "MIT"
|
||||
organization: "vsh"
|
||||
links:
|
||||
repository: "https://github.com/viash-hub/toolbox"
|
||||
issue_tracker: "https://github.com/viash-hub/toolbox/issues"
|
||||
1397
target/executable/bgzip/bgzip
Executable file
1397
target/executable/bgzip/bgzip
Executable file
File diff suppressed because it is too large
Load Diff
297
target/executable/yq/.config.vsh.yaml
Normal file
297
target/executable/yq/.config.vsh.yaml
Normal file
@@ -0,0 +1,297 @@
|
||||
name: "yq"
|
||||
version: "gunzip"
|
||||
argument_groups:
|
||||
- name: "Inputs"
|
||||
arguments:
|
||||
- type: "file"
|
||||
name: "--input"
|
||||
description: "files to be processed"
|
||||
info: null
|
||||
example:
|
||||
- "input.yaml"
|
||||
must_exist: true
|
||||
create_parent: true
|
||||
required: true
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- name: "Outputs"
|
||||
arguments:
|
||||
- type: "file"
|
||||
name: "--output"
|
||||
description: "output file"
|
||||
info: null
|
||||
example:
|
||||
- "output.yaml"
|
||||
must_exist: true
|
||||
create_parent: true
|
||||
required: true
|
||||
direction: "output"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- name: "Arguments"
|
||||
arguments:
|
||||
- type: "string"
|
||||
name: "--eval"
|
||||
description: "expression to evaluate"
|
||||
info: null
|
||||
example:
|
||||
- ".name = \"foo\""
|
||||
required: true
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "integer"
|
||||
name: "--indent"
|
||||
alternatives:
|
||||
- "-I"
|
||||
description: "sets indent level for output (default 2)"
|
||||
info: null
|
||||
required: false
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "string"
|
||||
name: "--input_format"
|
||||
alternatives:
|
||||
- "-p"
|
||||
description: "parse format for input. (default \"auto\")"
|
||||
info: null
|
||||
required: false
|
||||
choices:
|
||||
- "auto"
|
||||
- "a"
|
||||
- "yaml"
|
||||
- "y"
|
||||
- "json"
|
||||
- "j"
|
||||
- "props"
|
||||
- "p"
|
||||
- "csv"
|
||||
- "c"
|
||||
- "tsv"
|
||||
- "t"
|
||||
- "xml"
|
||||
- "x"
|
||||
- "base64"
|
||||
- "uri"
|
||||
- "toml"
|
||||
- "shell"
|
||||
- "s"
|
||||
- "lua"
|
||||
- "l"
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "string"
|
||||
name: "--output_format"
|
||||
alternatives:
|
||||
- "-o"
|
||||
description: "output format type. (default \"auto\")"
|
||||
info: null
|
||||
required: false
|
||||
choices:
|
||||
- "auto"
|
||||
- "a"
|
||||
- "yaml"
|
||||
- "y"
|
||||
- "json"
|
||||
- "j"
|
||||
- "props"
|
||||
- "p"
|
||||
- "csv"
|
||||
- "c"
|
||||
- "tsv"
|
||||
- "t"
|
||||
- "xml"
|
||||
- "x"
|
||||
- "base64"
|
||||
- "uri"
|
||||
- "toml"
|
||||
- "shell"
|
||||
- "s"
|
||||
- "lua"
|
||||
- "l"
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "boolean_true"
|
||||
name: "--pretty_print"
|
||||
alternatives:
|
||||
- "-P"
|
||||
description: "pretty print, shorthand for '... style = \"\"'"
|
||||
info: null
|
||||
direction: "input"
|
||||
resources:
|
||||
- type: "bash_script"
|
||||
text: |
|
||||
#!/bin/sh
|
||||
[[ "$par_pretty_print" == "false" ]] && unset par_pretty_print
|
||||
yq eval \
|
||||
${par_indent:+-I "${par_indent}"} \
|
||||
${par_input_format:+-p "${par_input_format}"} \
|
||||
${par_output_format:+-o "${par_output_format}"} \
|
||||
${par_pretty_print:+-P} \
|
||||
--expression "$par_eval" \
|
||||
--no-colors \
|
||||
"$par_input" > "$par_output"
|
||||
|
||||
|
||||
dest: "./script.sh"
|
||||
is_executable: true
|
||||
description: "A portable YAML, JSON, XML, CSV, TOML and properties processor"
|
||||
test_resources:
|
||||
- type: "bash_script"
|
||||
text: "set -e\necho \"name: 'bar'\" > test.yaml\n\"$meta_executable\" --input test.yaml\
|
||||
\ --output output.yaml --eval '.name = \"foo\"'\n\"$meta_executable\" --input\
|
||||
\ output.yaml --output output2.yaml --eval '.name'\ngrep \"^foo$\" output2.yaml\n"
|
||||
dest: "./script.sh"
|
||||
is_executable: true
|
||||
info: null
|
||||
status: "enabled"
|
||||
scope:
|
||||
image: "public"
|
||||
target: "public"
|
||||
requirements:
|
||||
commands:
|
||||
- "ps"
|
||||
keywords:
|
||||
- "yaml"
|
||||
- "json"
|
||||
- "xml"
|
||||
- "csv"
|
||||
- "toml"
|
||||
- "properties"
|
||||
license: "MIT"
|
||||
links:
|
||||
repository: "https://github.com/mikefarah/yq"
|
||||
homepage: "https://mikefarah.gitbook.io/yq"
|
||||
documentation: "https://mikefarah.gitbook.io/yq/"
|
||||
runners:
|
||||
- type: "executable"
|
||||
id: "executable"
|
||||
docker_setup_strategy: "ifneedbepullelsecachedbuild"
|
||||
- type: "nextflow"
|
||||
id: "nextflow"
|
||||
directives:
|
||||
tag: "$id"
|
||||
auto:
|
||||
simplifyInput: true
|
||||
simplifyOutput: false
|
||||
transcript: false
|
||||
publish: false
|
||||
config:
|
||||
labels:
|
||||
mem1gb: "memory = 1000000000.B"
|
||||
mem2gb: "memory = 2000000000.B"
|
||||
mem5gb: "memory = 5000000000.B"
|
||||
mem10gb: "memory = 10000000000.B"
|
||||
mem20gb: "memory = 20000000000.B"
|
||||
mem50gb: "memory = 50000000000.B"
|
||||
mem100gb: "memory = 100000000000.B"
|
||||
mem200gb: "memory = 200000000000.B"
|
||||
mem500gb: "memory = 500000000000.B"
|
||||
mem1tb: "memory = 1000000000000.B"
|
||||
mem2tb: "memory = 2000000000000.B"
|
||||
mem5tb: "memory = 5000000000000.B"
|
||||
mem10tb: "memory = 10000000000000.B"
|
||||
mem20tb: "memory = 20000000000000.B"
|
||||
mem50tb: "memory = 50000000000000.B"
|
||||
mem100tb: "memory = 100000000000000.B"
|
||||
mem200tb: "memory = 200000000000000.B"
|
||||
mem500tb: "memory = 500000000000000.B"
|
||||
mem1gib: "memory = 1073741824.B"
|
||||
mem2gib: "memory = 2147483648.B"
|
||||
mem4gib: "memory = 4294967296.B"
|
||||
mem8gib: "memory = 8589934592.B"
|
||||
mem16gib: "memory = 17179869184.B"
|
||||
mem32gib: "memory = 34359738368.B"
|
||||
mem64gib: "memory = 68719476736.B"
|
||||
mem128gib: "memory = 137438953472.B"
|
||||
mem256gib: "memory = 274877906944.B"
|
||||
mem512gib: "memory = 549755813888.B"
|
||||
mem1tib: "memory = 1099511627776.B"
|
||||
mem2tib: "memory = 2199023255552.B"
|
||||
mem4tib: "memory = 4398046511104.B"
|
||||
mem8tib: "memory = 8796093022208.B"
|
||||
mem16tib: "memory = 17592186044416.B"
|
||||
mem32tib: "memory = 35184372088832.B"
|
||||
mem64tib: "memory = 70368744177664.B"
|
||||
mem128tib: "memory = 140737488355328.B"
|
||||
mem256tib: "memory = 281474976710656.B"
|
||||
mem512tib: "memory = 562949953421312.B"
|
||||
cpu1: "cpus = 1"
|
||||
cpu2: "cpus = 2"
|
||||
cpu5: "cpus = 5"
|
||||
cpu10: "cpus = 10"
|
||||
cpu20: "cpus = 20"
|
||||
cpu50: "cpus = 50"
|
||||
cpu100: "cpus = 100"
|
||||
cpu200: "cpus = 200"
|
||||
cpu500: "cpus = 500"
|
||||
cpu1000: "cpus = 1000"
|
||||
debug: false
|
||||
container: "docker"
|
||||
engines:
|
||||
- type: "docker"
|
||||
id: "docker"
|
||||
image: "alpine:latest"
|
||||
target_registry: "images.viash-hub.com"
|
||||
target_tag: "gunzip"
|
||||
namespace_separator: "/"
|
||||
setup:
|
||||
- type: "apk"
|
||||
packages:
|
||||
- "bash"
|
||||
- "yq-go"
|
||||
- type: "docker"
|
||||
run:
|
||||
- "/usr/bin/yq --version | sed 's/.*version\\sv\\(.*\\)/yq: \"\\1\"/' > /var/software_versions.txt\n"
|
||||
entrypoint: []
|
||||
cmd: null
|
||||
- type: "native"
|
||||
id: "native"
|
||||
build_info:
|
||||
config: "src/yq/config.vsh.yaml"
|
||||
runner: "executable"
|
||||
engine: "docker|native"
|
||||
output: "target/executable/yq"
|
||||
executable: "target/executable/yq/yq"
|
||||
viash_version: "0.9.4"
|
||||
git_commit: "add30ba0f36bd8a8b07f0ba640707016d04e11b2"
|
||||
git_remote: "https://github.com/viash-hub/toolbox"
|
||||
package_config:
|
||||
name: "toolbox"
|
||||
version: "gunzip"
|
||||
summary: "A collection of curated command-line tools for general IT tasks, built\
|
||||
\ with Viash.\n"
|
||||
description: "`toolbox` provides a versatile suite of IT components, following the\
|
||||
\ robust Viash (https://viash.io) framework.\nThis package focuses on delivering\
|
||||
\ reliable, standalone tools that can be easily integrated into larger computational\
|
||||
\ workflows.\n\nThe core philosophy emphasizes **reusability**, **reproducibility**,\
|
||||
\ and adherence to **best practices** in component creation. Key features of `toolbox`\
|
||||
\ components include:\n\n* **Standalone & Nextflow Ready:** Execute components\
|
||||
\ directly from the command line or seamlessly incorporate them into Nextflow\
|
||||
\ workflows.\n* **High Quality Standards:**\n * Comprehensive documentation\
|
||||
\ for each component and its parameters.\n * Full exposure of the underlying\
|
||||
\ tool's arguments for maximum flexibility.\n * Containerized (Docker) to ensure\
|
||||
\ consistent environments and manage dependencies, leading to enhanced reproducibility.\n\
|
||||
\ * Unit tested to verify functionality and ensure reliability.\n"
|
||||
info: null
|
||||
viash_version: "0.9.4"
|
||||
source: "src"
|
||||
target: "target"
|
||||
config_mods:
|
||||
- ".requirements.commands := ['ps']\n"
|
||||
- ".engines += { type: \"native\" }"
|
||||
- ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
|
||||
- ".engines[.type == 'docker'].target_tag := 'gunzip'"
|
||||
keywords:
|
||||
- "toolbox"
|
||||
- "command-line"
|
||||
- "tools"
|
||||
license: "MIT"
|
||||
organization: "vsh"
|
||||
links:
|
||||
repository: "https://github.com/viash-hub/toolbox"
|
||||
issue_tracker: "https://github.com/viash-hub/toolbox/issues"
|
||||
1260
target/executable/yq/yq
Executable file
1260
target/executable/yq/yq
Executable file
File diff suppressed because it is too large
Load Diff
267
target/nextflow/bgzip/.config.vsh.yaml
Normal file
267
target/nextflow/bgzip/.config.vsh.yaml
Normal file
@@ -0,0 +1,267 @@
|
||||
name: "bgzip"
|
||||
version: "gunzip"
|
||||
argument_groups:
|
||||
- name: "Inputs"
|
||||
arguments:
|
||||
- type: "file"
|
||||
name: "--input"
|
||||
description: "file to be compressed or decompressed"
|
||||
info: null
|
||||
must_exist: true
|
||||
create_parent: true
|
||||
required: true
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- name: "Outputs"
|
||||
arguments:
|
||||
- type: "file"
|
||||
name: "--output"
|
||||
description: "compressed or decompressed output"
|
||||
info: null
|
||||
must_exist: true
|
||||
create_parent: true
|
||||
required: true
|
||||
direction: "output"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "file"
|
||||
name: "--index_name"
|
||||
alternatives:
|
||||
- "-I"
|
||||
description: "name of BGZF index file [file.gz.gzi]"
|
||||
info: null
|
||||
must_exist: true
|
||||
create_parent: true
|
||||
required: false
|
||||
direction: "output"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- name: "Arguments"
|
||||
arguments:
|
||||
- type: "integer"
|
||||
name: "--offset"
|
||||
alternatives:
|
||||
- "-b"
|
||||
description: "decompress at virtual file pointer (0-based uncompressed offset)"
|
||||
info: null
|
||||
required: false
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "boolean_true"
|
||||
name: "--decompress"
|
||||
alternatives:
|
||||
- "-d"
|
||||
description: "decompress the input file"
|
||||
info: null
|
||||
direction: "input"
|
||||
- type: "boolean_true"
|
||||
name: "--rebgzip"
|
||||
alternatives:
|
||||
- "-g"
|
||||
description: "use an index file to bgzip a file"
|
||||
info: null
|
||||
direction: "input"
|
||||
- type: "boolean_true"
|
||||
name: "--index"
|
||||
alternatives:
|
||||
- "-i"
|
||||
description: "compress and create BGZF index"
|
||||
info: null
|
||||
direction: "input"
|
||||
- type: "integer"
|
||||
name: "--compress_level"
|
||||
alternatives:
|
||||
- "-l"
|
||||
description: "compression level to use when compressing; 0 to 9, or -1 for default\
|
||||
\ [-1]"
|
||||
info: null
|
||||
required: false
|
||||
min: -1
|
||||
max: 9
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "boolean_true"
|
||||
name: "--reindex"
|
||||
alternatives:
|
||||
- "-r"
|
||||
description: "(re)index the output file"
|
||||
info: null
|
||||
direction: "input"
|
||||
- type: "integer"
|
||||
name: "--size"
|
||||
alternatives:
|
||||
- "-s"
|
||||
description: "decompress INT bytes (uncompressed size)"
|
||||
info: null
|
||||
required: false
|
||||
min: 0
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "boolean_true"
|
||||
name: "--test"
|
||||
alternatives:
|
||||
- "-t"
|
||||
description: "test integrity of compressed file"
|
||||
info: null
|
||||
direction: "input"
|
||||
- type: "boolean_true"
|
||||
name: "--binary"
|
||||
description: "Don't align blocks with text lines"
|
||||
info: null
|
||||
direction: "input"
|
||||
resources:
|
||||
- type: "bash_script"
|
||||
path: "script.sh"
|
||||
is_executable: true
|
||||
description: "Block compression/decompression utility"
|
||||
test_resources:
|
||||
- type: "bash_script"
|
||||
path: "test.sh"
|
||||
is_executable: true
|
||||
- type: "file"
|
||||
path: "test_data"
|
||||
info: null
|
||||
status: "enabled"
|
||||
scope:
|
||||
image: "public"
|
||||
target: "public"
|
||||
requirements:
|
||||
commands:
|
||||
- "ps"
|
||||
license: "MIT"
|
||||
references:
|
||||
doi:
|
||||
- "10.1093/gigascience/giab007"
|
||||
links:
|
||||
repository: "https://github.com/samtools/htslib"
|
||||
homepage: "https://www.htslib.org/"
|
||||
documentation: "https://www.htslib.org/doc/bgzip.html"
|
||||
runners:
|
||||
- type: "executable"
|
||||
id: "executable"
|
||||
docker_setup_strategy: "ifneedbepullelsecachedbuild"
|
||||
- type: "nextflow"
|
||||
id: "nextflow"
|
||||
directives:
|
||||
tag: "$id"
|
||||
auto:
|
||||
simplifyInput: true
|
||||
simplifyOutput: false
|
||||
transcript: false
|
||||
publish: false
|
||||
config:
|
||||
labels:
|
||||
mem1gb: "memory = 1000000000.B"
|
||||
mem2gb: "memory = 2000000000.B"
|
||||
mem5gb: "memory = 5000000000.B"
|
||||
mem10gb: "memory = 10000000000.B"
|
||||
mem20gb: "memory = 20000000000.B"
|
||||
mem50gb: "memory = 50000000000.B"
|
||||
mem100gb: "memory = 100000000000.B"
|
||||
mem200gb: "memory = 200000000000.B"
|
||||
mem500gb: "memory = 500000000000.B"
|
||||
mem1tb: "memory = 1000000000000.B"
|
||||
mem2tb: "memory = 2000000000000.B"
|
||||
mem5tb: "memory = 5000000000000.B"
|
||||
mem10tb: "memory = 10000000000000.B"
|
||||
mem20tb: "memory = 20000000000000.B"
|
||||
mem50tb: "memory = 50000000000000.B"
|
||||
mem100tb: "memory = 100000000000000.B"
|
||||
mem200tb: "memory = 200000000000000.B"
|
||||
mem500tb: "memory = 500000000000000.B"
|
||||
mem1gib: "memory = 1073741824.B"
|
||||
mem2gib: "memory = 2147483648.B"
|
||||
mem4gib: "memory = 4294967296.B"
|
||||
mem8gib: "memory = 8589934592.B"
|
||||
mem16gib: "memory = 17179869184.B"
|
||||
mem32gib: "memory = 34359738368.B"
|
||||
mem64gib: "memory = 68719476736.B"
|
||||
mem128gib: "memory = 137438953472.B"
|
||||
mem256gib: "memory = 274877906944.B"
|
||||
mem512gib: "memory = 549755813888.B"
|
||||
mem1tib: "memory = 1099511627776.B"
|
||||
mem2tib: "memory = 2199023255552.B"
|
||||
mem4tib: "memory = 4398046511104.B"
|
||||
mem8tib: "memory = 8796093022208.B"
|
||||
mem16tib: "memory = 17592186044416.B"
|
||||
mem32tib: "memory = 35184372088832.B"
|
||||
mem64tib: "memory = 70368744177664.B"
|
||||
mem128tib: "memory = 140737488355328.B"
|
||||
mem256tib: "memory = 281474976710656.B"
|
||||
mem512tib: "memory = 562949953421312.B"
|
||||
cpu1: "cpus = 1"
|
||||
cpu2: "cpus = 2"
|
||||
cpu5: "cpus = 5"
|
||||
cpu10: "cpus = 10"
|
||||
cpu20: "cpus = 20"
|
||||
cpu50: "cpus = 50"
|
||||
cpu100: "cpus = 100"
|
||||
cpu200: "cpus = 200"
|
||||
cpu500: "cpus = 500"
|
||||
cpu1000: "cpus = 1000"
|
||||
debug: false
|
||||
container: "docker"
|
||||
engines:
|
||||
- type: "docker"
|
||||
id: "docker"
|
||||
image: "quay.io/biocontainers/htslib:1.19--h81da01d_0"
|
||||
target_registry: "images.viash-hub.com"
|
||||
target_tag: "gunzip"
|
||||
namespace_separator: "/"
|
||||
setup:
|
||||
- type: "docker"
|
||||
run:
|
||||
- "bgzip -h | grep 'Version:' 2>&1 | sed 's/Version:\\s\\(.*\\)/bgzip: \"\\1\"\
|
||||
/' > /var/software_versions.txt\n"
|
||||
entrypoint: []
|
||||
cmd: null
|
||||
- type: "native"
|
||||
id: "native"
|
||||
build_info:
|
||||
config: "src/bgzip/config.vsh.yaml"
|
||||
runner: "nextflow"
|
||||
engine: "docker|native"
|
||||
output: "target/nextflow/bgzip"
|
||||
executable: "target/nextflow/bgzip/main.nf"
|
||||
viash_version: "0.9.4"
|
||||
git_commit: "add30ba0f36bd8a8b07f0ba640707016d04e11b2"
|
||||
git_remote: "https://github.com/viash-hub/toolbox"
|
||||
package_config:
|
||||
name: "toolbox"
|
||||
version: "gunzip"
|
||||
summary: "A collection of curated command-line tools for general IT tasks, built\
|
||||
\ with Viash.\n"
|
||||
description: "`toolbox` provides a versatile suite of IT components, following the\
|
||||
\ robust Viash (https://viash.io) framework.\nThis package focuses on delivering\
|
||||
\ reliable, standalone tools that can be easily integrated into larger computational\
|
||||
\ workflows.\n\nThe core philosophy emphasizes **reusability**, **reproducibility**,\
|
||||
\ and adherence to **best practices** in component creation. Key features of `toolbox`\
|
||||
\ components include:\n\n* **Standalone & Nextflow Ready:** Execute components\
|
||||
\ directly from the command line or seamlessly incorporate them into Nextflow\
|
||||
\ workflows.\n* **High Quality Standards:**\n * Comprehensive documentation\
|
||||
\ for each component and its parameters.\n * Full exposure of the underlying\
|
||||
\ tool's arguments for maximum flexibility.\n * Containerized (Docker) to ensure\
|
||||
\ consistent environments and manage dependencies, leading to enhanced reproducibility.\n\
|
||||
\ * Unit tested to verify functionality and ensure reliability.\n"
|
||||
info: null
|
||||
viash_version: "0.9.4"
|
||||
source: "src"
|
||||
target: "target"
|
||||
config_mods:
|
||||
- ".requirements.commands := ['ps']\n"
|
||||
- ".engines += { type: \"native\" }"
|
||||
- ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
|
||||
- ".engines[.type == 'docker'].target_tag := 'gunzip'"
|
||||
keywords:
|
||||
- "toolbox"
|
||||
- "command-line"
|
||||
- "tools"
|
||||
license: "MIT"
|
||||
organization: "vsh"
|
||||
links:
|
||||
repository: "https://github.com/viash-hub/toolbox"
|
||||
issue_tracker: "https://github.com/viash-hub/toolbox/issues"
|
||||
3917
target/nextflow/bgzip/main.nf
Normal file
3917
target/nextflow/bgzip/main.nf
Normal file
File diff suppressed because it is too large
Load Diff
125
target/nextflow/bgzip/nextflow.config
Normal file
125
target/nextflow/bgzip/nextflow.config
Normal file
@@ -0,0 +1,125 @@
|
||||
manifest {
|
||||
name = 'bgzip'
|
||||
mainScript = 'main.nf'
|
||||
nextflowVersion = '!>=20.12.1-edge'
|
||||
version = 'gunzip'
|
||||
description = 'Block compression/decompression utility'
|
||||
}
|
||||
|
||||
process.container = 'nextflow/bash:latest'
|
||||
|
||||
// detect tempdir
|
||||
tempDir = java.nio.file.Paths.get(
|
||||
System.getenv('NXF_TEMP') ?:
|
||||
System.getenv('VIASH_TEMP') ?:
|
||||
System.getenv('TEMPDIR') ?:
|
||||
System.getenv('TMPDIR') ?:
|
||||
'/tmp'
|
||||
).toAbsolutePath()
|
||||
|
||||
profiles {
|
||||
no_publish {
|
||||
process {
|
||||
withName: '.*' {
|
||||
publishDir = [
|
||||
enabled: false
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
mount_temp {
|
||||
docker.temp = tempDir
|
||||
podman.temp = tempDir
|
||||
charliecloud.temp = tempDir
|
||||
}
|
||||
docker {
|
||||
docker.enabled = true
|
||||
// docker.userEmulation = true
|
||||
singularity.enabled = false
|
||||
podman.enabled = false
|
||||
shifter.enabled = false
|
||||
charliecloud.enabled = false
|
||||
}
|
||||
singularity {
|
||||
singularity.enabled = true
|
||||
singularity.autoMounts = true
|
||||
docker.enabled = false
|
||||
podman.enabled = false
|
||||
shifter.enabled = false
|
||||
charliecloud.enabled = false
|
||||
}
|
||||
podman {
|
||||
podman.enabled = true
|
||||
docker.enabled = false
|
||||
singularity.enabled = false
|
||||
shifter.enabled = false
|
||||
charliecloud.enabled = false
|
||||
}
|
||||
shifter {
|
||||
shifter.enabled = true
|
||||
docker.enabled = false
|
||||
singularity.enabled = false
|
||||
podman.enabled = false
|
||||
charliecloud.enabled = false
|
||||
}
|
||||
charliecloud {
|
||||
charliecloud.enabled = true
|
||||
docker.enabled = false
|
||||
singularity.enabled = false
|
||||
podman.enabled = false
|
||||
shifter.enabled = false
|
||||
}
|
||||
}
|
||||
|
||||
process{
|
||||
withLabel: mem1gb { memory = 1000000000.B }
|
||||
withLabel: mem2gb { memory = 2000000000.B }
|
||||
withLabel: mem5gb { memory = 5000000000.B }
|
||||
withLabel: mem10gb { memory = 10000000000.B }
|
||||
withLabel: mem20gb { memory = 20000000000.B }
|
||||
withLabel: mem50gb { memory = 50000000000.B }
|
||||
withLabel: mem100gb { memory = 100000000000.B }
|
||||
withLabel: mem200gb { memory = 200000000000.B }
|
||||
withLabel: mem500gb { memory = 500000000000.B }
|
||||
withLabel: mem1tb { memory = 1000000000000.B }
|
||||
withLabel: mem2tb { memory = 2000000000000.B }
|
||||
withLabel: mem5tb { memory = 5000000000000.B }
|
||||
withLabel: mem10tb { memory = 10000000000000.B }
|
||||
withLabel: mem20tb { memory = 20000000000000.B }
|
||||
withLabel: mem50tb { memory = 50000000000000.B }
|
||||
withLabel: mem100tb { memory = 100000000000000.B }
|
||||
withLabel: mem200tb { memory = 200000000000000.B }
|
||||
withLabel: mem500tb { memory = 500000000000000.B }
|
||||
withLabel: mem1gib { memory = 1073741824.B }
|
||||
withLabel: mem2gib { memory = 2147483648.B }
|
||||
withLabel: mem4gib { memory = 4294967296.B }
|
||||
withLabel: mem8gib { memory = 8589934592.B }
|
||||
withLabel: mem16gib { memory = 17179869184.B }
|
||||
withLabel: mem32gib { memory = 34359738368.B }
|
||||
withLabel: mem64gib { memory = 68719476736.B }
|
||||
withLabel: mem128gib { memory = 137438953472.B }
|
||||
withLabel: mem256gib { memory = 274877906944.B }
|
||||
withLabel: mem512gib { memory = 549755813888.B }
|
||||
withLabel: mem1tib { memory = 1099511627776.B }
|
||||
withLabel: mem2tib { memory = 2199023255552.B }
|
||||
withLabel: mem4tib { memory = 4398046511104.B }
|
||||
withLabel: mem8tib { memory = 8796093022208.B }
|
||||
withLabel: mem16tib { memory = 17592186044416.B }
|
||||
withLabel: mem32tib { memory = 35184372088832.B }
|
||||
withLabel: mem64tib { memory = 70368744177664.B }
|
||||
withLabel: mem128tib { memory = 140737488355328.B }
|
||||
withLabel: mem256tib { memory = 281474976710656.B }
|
||||
withLabel: mem512tib { memory = 562949953421312.B }
|
||||
withLabel: cpu1 { cpus = 1 }
|
||||
withLabel: cpu2 { cpus = 2 }
|
||||
withLabel: cpu5 { cpus = 5 }
|
||||
withLabel: cpu10 { cpus = 10 }
|
||||
withLabel: cpu20 { cpus = 20 }
|
||||
withLabel: cpu50 { cpus = 50 }
|
||||
withLabel: cpu100 { cpus = 100 }
|
||||
withLabel: cpu200 { cpus = 200 }
|
||||
withLabel: cpu500 { cpus = 500 }
|
||||
withLabel: cpu1000 { cpus = 1000 }
|
||||
}
|
||||
|
||||
|
||||
127
target/nextflow/bgzip/nextflow_schema.json
Normal file
127
target/nextflow/bgzip/nextflow_schema.json
Normal file
@@ -0,0 +1,127 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"title": "bgzip",
|
||||
"description": "Block compression/decompression utility",
|
||||
"type": "object",
|
||||
"$defs": {
|
||||
"inputs": {
|
||||
"title": "Inputs",
|
||||
"type": "object",
|
||||
"description": "No description",
|
||||
"properties": {
|
||||
"input": {
|
||||
"type": "string",
|
||||
"format": "path",
|
||||
"exists": true,
|
||||
"description": "file to be compressed or decompressed",
|
||||
"help_text": "Type: `file`, multiple: `False`, required, direction: `input`. "
|
||||
}
|
||||
}
|
||||
},
|
||||
"outputs": {
|
||||
"title": "Outputs",
|
||||
"type": "object",
|
||||
"description": "No description",
|
||||
"properties": {
|
||||
"output": {
|
||||
"type": "string",
|
||||
"format": "path",
|
||||
"description": "compressed or decompressed output",
|
||||
"help_text": "Type: `file`, multiple: `False`, required, default: `\"$id.$key.output\"`, direction: `output`. ",
|
||||
"default": "$id.$key.output"
|
||||
},
|
||||
"index_name": {
|
||||
"type": "string",
|
||||
"format": "path",
|
||||
"description": "name of BGZF index file [file.gz.gzi]",
|
||||
"help_text": "Type: `file`, multiple: `False`, default: `\"$id.$key.index_name\"`, direction: `output`. ",
|
||||
"default": "$id.$key.index_name"
|
||||
}
|
||||
}
|
||||
},
|
||||
"arguments": {
|
||||
"title": "Arguments",
|
||||
"type": "object",
|
||||
"description": "No description",
|
||||
"properties": {
|
||||
"offset": {
|
||||
"type": "integer",
|
||||
"description": "decompress at virtual file pointer (0-based uncompressed offset)",
|
||||
"help_text": "Type: `integer`, multiple: `False`. "
|
||||
},
|
||||
"decompress": {
|
||||
"type": "boolean",
|
||||
"description": "decompress the input file",
|
||||
"help_text": "Type: `boolean_true`, multiple: `False`, default: `false`. ",
|
||||
"default": false
|
||||
},
|
||||
"rebgzip": {
|
||||
"type": "boolean",
|
||||
"description": "use an index file to bgzip a file",
|
||||
"help_text": "Type: `boolean_true`, multiple: `False`, default: `false`. ",
|
||||
"default": false
|
||||
},
|
||||
"index": {
|
||||
"type": "boolean",
|
||||
"description": "compress and create BGZF index",
|
||||
"help_text": "Type: `boolean_true`, multiple: `False`, default: `false`. ",
|
||||
"default": false
|
||||
},
|
||||
"compress_level": {
|
||||
"type": "integer",
|
||||
"description": "compression level to use when compressing; 0 to 9, or -1 for default [-1]",
|
||||
"help_text": "Type: `integer`, multiple: `False`. "
|
||||
},
|
||||
"reindex": {
|
||||
"type": "boolean",
|
||||
"description": "(re)index the output file",
|
||||
"help_text": "Type: `boolean_true`, multiple: `False`, default: `false`. ",
|
||||
"default": false
|
||||
},
|
||||
"size": {
|
||||
"type": "integer",
|
||||
"description": "decompress INT bytes (uncompressed size)",
|
||||
"help_text": "Type: `integer`, multiple: `False`. "
|
||||
},
|
||||
"test": {
|
||||
"type": "boolean",
|
||||
"description": "test integrity of compressed file",
|
||||
"help_text": "Type: `boolean_true`, multiple: `False`, default: `false`. ",
|
||||
"default": false
|
||||
},
|
||||
"binary": {
|
||||
"type": "boolean",
|
||||
"description": "Don't align blocks with text lines",
|
||||
"help_text": "Type: `boolean_true`, multiple: `False`, default: `false`. ",
|
||||
"default": false
|
||||
}
|
||||
}
|
||||
},
|
||||
"nextflow input-output arguments": {
|
||||
"title": "Nextflow input-output arguments",
|
||||
"type": "object",
|
||||
"description": "Input/output parameters for Nextflow itself. Please note that both publishDir and publish_dir are supported but at least one has to be configured.",
|
||||
"properties": {
|
||||
"publish_dir": {
|
||||
"type": "string",
|
||||
"description": "Path to an output directory.",
|
||||
"help_text": "Type: `string`, multiple: `False`, required, example: `\"output/\"`. "
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"allOf": [
|
||||
{
|
||||
"$ref": "#/$defs/inputs"
|
||||
},
|
||||
{
|
||||
"$ref": "#/$defs/outputs"
|
||||
},
|
||||
{
|
||||
"$ref": "#/$defs/arguments"
|
||||
},
|
||||
{
|
||||
"$ref": "#/$defs/nextflow input-output arguments"
|
||||
}
|
||||
]
|
||||
}
|
||||
297
target/nextflow/yq/.config.vsh.yaml
Normal file
297
target/nextflow/yq/.config.vsh.yaml
Normal file
@@ -0,0 +1,297 @@
|
||||
name: "yq"
|
||||
version: "gunzip"
|
||||
argument_groups:
|
||||
- name: "Inputs"
|
||||
arguments:
|
||||
- type: "file"
|
||||
name: "--input"
|
||||
description: "files to be processed"
|
||||
info: null
|
||||
example:
|
||||
- "input.yaml"
|
||||
must_exist: true
|
||||
create_parent: true
|
||||
required: true
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- name: "Outputs"
|
||||
arguments:
|
||||
- type: "file"
|
||||
name: "--output"
|
||||
description: "output file"
|
||||
info: null
|
||||
example:
|
||||
- "output.yaml"
|
||||
must_exist: true
|
||||
create_parent: true
|
||||
required: true
|
||||
direction: "output"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- name: "Arguments"
|
||||
arguments:
|
||||
- type: "string"
|
||||
name: "--eval"
|
||||
description: "expression to evaluate"
|
||||
info: null
|
||||
example:
|
||||
- ".name = \"foo\""
|
||||
required: true
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "integer"
|
||||
name: "--indent"
|
||||
alternatives:
|
||||
- "-I"
|
||||
description: "sets indent level for output (default 2)"
|
||||
info: null
|
||||
required: false
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "string"
|
||||
name: "--input_format"
|
||||
alternatives:
|
||||
- "-p"
|
||||
description: "parse format for input. (default \"auto\")"
|
||||
info: null
|
||||
required: false
|
||||
choices:
|
||||
- "auto"
|
||||
- "a"
|
||||
- "yaml"
|
||||
- "y"
|
||||
- "json"
|
||||
- "j"
|
||||
- "props"
|
||||
- "p"
|
||||
- "csv"
|
||||
- "c"
|
||||
- "tsv"
|
||||
- "t"
|
||||
- "xml"
|
||||
- "x"
|
||||
- "base64"
|
||||
- "uri"
|
||||
- "toml"
|
||||
- "shell"
|
||||
- "s"
|
||||
- "lua"
|
||||
- "l"
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "string"
|
||||
name: "--output_format"
|
||||
alternatives:
|
||||
- "-o"
|
||||
description: "output format type. (default \"auto\")"
|
||||
info: null
|
||||
required: false
|
||||
choices:
|
||||
- "auto"
|
||||
- "a"
|
||||
- "yaml"
|
||||
- "y"
|
||||
- "json"
|
||||
- "j"
|
||||
- "props"
|
||||
- "p"
|
||||
- "csv"
|
||||
- "c"
|
||||
- "tsv"
|
||||
- "t"
|
||||
- "xml"
|
||||
- "x"
|
||||
- "base64"
|
||||
- "uri"
|
||||
- "toml"
|
||||
- "shell"
|
||||
- "s"
|
||||
- "lua"
|
||||
- "l"
|
||||
direction: "input"
|
||||
multiple: false
|
||||
multiple_sep: ";"
|
||||
- type: "boolean_true"
|
||||
name: "--pretty_print"
|
||||
alternatives:
|
||||
- "-P"
|
||||
description: "pretty print, shorthand for '... style = \"\"'"
|
||||
info: null
|
||||
direction: "input"
|
||||
resources:
|
||||
- type: "bash_script"
|
||||
text: |
|
||||
#!/bin/sh
|
||||
[[ "$par_pretty_print" == "false" ]] && unset par_pretty_print
|
||||
yq eval \
|
||||
${par_indent:+-I "${par_indent}"} \
|
||||
${par_input_format:+-p "${par_input_format}"} \
|
||||
${par_output_format:+-o "${par_output_format}"} \
|
||||
${par_pretty_print:+-P} \
|
||||
--expression "$par_eval" \
|
||||
--no-colors \
|
||||
"$par_input" > "$par_output"
|
||||
|
||||
|
||||
dest: "./script.sh"
|
||||
is_executable: true
|
||||
description: "A portable YAML, JSON, XML, CSV, TOML and properties processor"
|
||||
test_resources:
|
||||
- type: "bash_script"
|
||||
text: "set -e\necho \"name: 'bar'\" > test.yaml\n\"$meta_executable\" --input test.yaml\
|
||||
\ --output output.yaml --eval '.name = \"foo\"'\n\"$meta_executable\" --input\
|
||||
\ output.yaml --output output2.yaml --eval '.name'\ngrep \"^foo$\" output2.yaml\n"
|
||||
dest: "./script.sh"
|
||||
is_executable: true
|
||||
info: null
|
||||
status: "enabled"
|
||||
scope:
|
||||
image: "public"
|
||||
target: "public"
|
||||
requirements:
|
||||
commands:
|
||||
- "ps"
|
||||
keywords:
|
||||
- "yaml"
|
||||
- "json"
|
||||
- "xml"
|
||||
- "csv"
|
||||
- "toml"
|
||||
- "properties"
|
||||
license: "MIT"
|
||||
links:
|
||||
repository: "https://github.com/mikefarah/yq"
|
||||
homepage: "https://mikefarah.gitbook.io/yq"
|
||||
documentation: "https://mikefarah.gitbook.io/yq/"
|
||||
runners:
|
||||
- type: "executable"
|
||||
id: "executable"
|
||||
docker_setup_strategy: "ifneedbepullelsecachedbuild"
|
||||
- type: "nextflow"
|
||||
id: "nextflow"
|
||||
directives:
|
||||
tag: "$id"
|
||||
auto:
|
||||
simplifyInput: true
|
||||
simplifyOutput: false
|
||||
transcript: false
|
||||
publish: false
|
||||
config:
|
||||
labels:
|
||||
mem1gb: "memory = 1000000000.B"
|
||||
mem2gb: "memory = 2000000000.B"
|
||||
mem5gb: "memory = 5000000000.B"
|
||||
mem10gb: "memory = 10000000000.B"
|
||||
mem20gb: "memory = 20000000000.B"
|
||||
mem50gb: "memory = 50000000000.B"
|
||||
mem100gb: "memory = 100000000000.B"
|
||||
mem200gb: "memory = 200000000000.B"
|
||||
mem500gb: "memory = 500000000000.B"
|
||||
mem1tb: "memory = 1000000000000.B"
|
||||
mem2tb: "memory = 2000000000000.B"
|
||||
mem5tb: "memory = 5000000000000.B"
|
||||
mem10tb: "memory = 10000000000000.B"
|
||||
mem20tb: "memory = 20000000000000.B"
|
||||
mem50tb: "memory = 50000000000000.B"
|
||||
mem100tb: "memory = 100000000000000.B"
|
||||
mem200tb: "memory = 200000000000000.B"
|
||||
mem500tb: "memory = 500000000000000.B"
|
||||
mem1gib: "memory = 1073741824.B"
|
||||
mem2gib: "memory = 2147483648.B"
|
||||
mem4gib: "memory = 4294967296.B"
|
||||
mem8gib: "memory = 8589934592.B"
|
||||
mem16gib: "memory = 17179869184.B"
|
||||
mem32gib: "memory = 34359738368.B"
|
||||
mem64gib: "memory = 68719476736.B"
|
||||
mem128gib: "memory = 137438953472.B"
|
||||
mem256gib: "memory = 274877906944.B"
|
||||
mem512gib: "memory = 549755813888.B"
|
||||
mem1tib: "memory = 1099511627776.B"
|
||||
mem2tib: "memory = 2199023255552.B"
|
||||
mem4tib: "memory = 4398046511104.B"
|
||||
mem8tib: "memory = 8796093022208.B"
|
||||
mem16tib: "memory = 17592186044416.B"
|
||||
mem32tib: "memory = 35184372088832.B"
|
||||
mem64tib: "memory = 70368744177664.B"
|
||||
mem128tib: "memory = 140737488355328.B"
|
||||
mem256tib: "memory = 281474976710656.B"
|
||||
mem512tib: "memory = 562949953421312.B"
|
||||
cpu1: "cpus = 1"
|
||||
cpu2: "cpus = 2"
|
||||
cpu5: "cpus = 5"
|
||||
cpu10: "cpus = 10"
|
||||
cpu20: "cpus = 20"
|
||||
cpu50: "cpus = 50"
|
||||
cpu100: "cpus = 100"
|
||||
cpu200: "cpus = 200"
|
||||
cpu500: "cpus = 500"
|
||||
cpu1000: "cpus = 1000"
|
||||
debug: false
|
||||
container: "docker"
|
||||
engines:
|
||||
- type: "docker"
|
||||
id: "docker"
|
||||
image: "alpine:latest"
|
||||
target_registry: "images.viash-hub.com"
|
||||
target_tag: "gunzip"
|
||||
namespace_separator: "/"
|
||||
setup:
|
||||
- type: "apk"
|
||||
packages:
|
||||
- "bash"
|
||||
- "yq-go"
|
||||
- type: "docker"
|
||||
run:
|
||||
- "/usr/bin/yq --version | sed 's/.*version\\sv\\(.*\\)/yq: \"\\1\"/' > /var/software_versions.txt\n"
|
||||
entrypoint: []
|
||||
cmd: null
|
||||
- type: "native"
|
||||
id: "native"
|
||||
build_info:
|
||||
config: "src/yq/config.vsh.yaml"
|
||||
runner: "nextflow"
|
||||
engine: "docker|native"
|
||||
output: "target/nextflow/yq"
|
||||
executable: "target/nextflow/yq/main.nf"
|
||||
viash_version: "0.9.4"
|
||||
git_commit: "add30ba0f36bd8a8b07f0ba640707016d04e11b2"
|
||||
git_remote: "https://github.com/viash-hub/toolbox"
|
||||
package_config:
|
||||
name: "toolbox"
|
||||
version: "gunzip"
|
||||
summary: "A collection of curated command-line tools for general IT tasks, built\
|
||||
\ with Viash.\n"
|
||||
description: "`toolbox` provides a versatile suite of IT components, following the\
|
||||
\ robust Viash (https://viash.io) framework.\nThis package focuses on delivering\
|
||||
\ reliable, standalone tools that can be easily integrated into larger computational\
|
||||
\ workflows.\n\nThe core philosophy emphasizes **reusability**, **reproducibility**,\
|
||||
\ and adherence to **best practices** in component creation. Key features of `toolbox`\
|
||||
\ components include:\n\n* **Standalone & Nextflow Ready:** Execute components\
|
||||
\ directly from the command line or seamlessly incorporate them into Nextflow\
|
||||
\ workflows.\n* **High Quality Standards:**\n * Comprehensive documentation\
|
||||
\ for each component and its parameters.\n * Full exposure of the underlying\
|
||||
\ tool's arguments for maximum flexibility.\n * Containerized (Docker) to ensure\
|
||||
\ consistent environments and manage dependencies, leading to enhanced reproducibility.\n\
|
||||
\ * Unit tested to verify functionality and ensure reliability.\n"
|
||||
info: null
|
||||
viash_version: "0.9.4"
|
||||
source: "src"
|
||||
target: "target"
|
||||
config_mods:
|
||||
- ".requirements.commands := ['ps']\n"
|
||||
- ".engines += { type: \"native\" }"
|
||||
- ".engines[.type == 'docker'].target_registry := 'images.viash-hub.com'"
|
||||
- ".engines[.type == 'docker'].target_tag := 'gunzip'"
|
||||
keywords:
|
||||
- "toolbox"
|
||||
- "command-line"
|
||||
- "tools"
|
||||
license: "MIT"
|
||||
organization: "vsh"
|
||||
links:
|
||||
repository: "https://github.com/viash-hub/toolbox"
|
||||
issue_tracker: "https://github.com/viash-hub/toolbox/issues"
|
||||
3914
target/nextflow/yq/main.nf
Normal file
3914
target/nextflow/yq/main.nf
Normal file
File diff suppressed because it is too large
Load Diff
125
target/nextflow/yq/nextflow.config
Normal file
125
target/nextflow/yq/nextflow.config
Normal file
@@ -0,0 +1,125 @@
|
||||
manifest {
|
||||
name = 'yq'
|
||||
mainScript = 'main.nf'
|
||||
nextflowVersion = '!>=20.12.1-edge'
|
||||
version = 'gunzip'
|
||||
description = 'A portable YAML, JSON, XML, CSV, TOML and properties processor'
|
||||
}
|
||||
|
||||
process.container = 'nextflow/bash:latest'
|
||||
|
||||
// detect tempdir
|
||||
tempDir = java.nio.file.Paths.get(
|
||||
System.getenv('NXF_TEMP') ?:
|
||||
System.getenv('VIASH_TEMP') ?:
|
||||
System.getenv('TEMPDIR') ?:
|
||||
System.getenv('TMPDIR') ?:
|
||||
'/tmp'
|
||||
).toAbsolutePath()
|
||||
|
||||
profiles {
|
||||
no_publish {
|
||||
process {
|
||||
withName: '.*' {
|
||||
publishDir = [
|
||||
enabled: false
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
mount_temp {
|
||||
docker.temp = tempDir
|
||||
podman.temp = tempDir
|
||||
charliecloud.temp = tempDir
|
||||
}
|
||||
docker {
|
||||
docker.enabled = true
|
||||
// docker.userEmulation = true
|
||||
singularity.enabled = false
|
||||
podman.enabled = false
|
||||
shifter.enabled = false
|
||||
charliecloud.enabled = false
|
||||
}
|
||||
singularity {
|
||||
singularity.enabled = true
|
||||
singularity.autoMounts = true
|
||||
docker.enabled = false
|
||||
podman.enabled = false
|
||||
shifter.enabled = false
|
||||
charliecloud.enabled = false
|
||||
}
|
||||
podman {
|
||||
podman.enabled = true
|
||||
docker.enabled = false
|
||||
singularity.enabled = false
|
||||
shifter.enabled = false
|
||||
charliecloud.enabled = false
|
||||
}
|
||||
shifter {
|
||||
shifter.enabled = true
|
||||
docker.enabled = false
|
||||
singularity.enabled = false
|
||||
podman.enabled = false
|
||||
charliecloud.enabled = false
|
||||
}
|
||||
charliecloud {
|
||||
charliecloud.enabled = true
|
||||
docker.enabled = false
|
||||
singularity.enabled = false
|
||||
podman.enabled = false
|
||||
shifter.enabled = false
|
||||
}
|
||||
}
|
||||
|
||||
process{
|
||||
withLabel: mem1gb { memory = 1000000000.B }
|
||||
withLabel: mem2gb { memory = 2000000000.B }
|
||||
withLabel: mem5gb { memory = 5000000000.B }
|
||||
withLabel: mem10gb { memory = 10000000000.B }
|
||||
withLabel: mem20gb { memory = 20000000000.B }
|
||||
withLabel: mem50gb { memory = 50000000000.B }
|
||||
withLabel: mem100gb { memory = 100000000000.B }
|
||||
withLabel: mem200gb { memory = 200000000000.B }
|
||||
withLabel: mem500gb { memory = 500000000000.B }
|
||||
withLabel: mem1tb { memory = 1000000000000.B }
|
||||
withLabel: mem2tb { memory = 2000000000000.B }
|
||||
withLabel: mem5tb { memory = 5000000000000.B }
|
||||
withLabel: mem10tb { memory = 10000000000000.B }
|
||||
withLabel: mem20tb { memory = 20000000000000.B }
|
||||
withLabel: mem50tb { memory = 50000000000000.B }
|
||||
withLabel: mem100tb { memory = 100000000000000.B }
|
||||
withLabel: mem200tb { memory = 200000000000000.B }
|
||||
withLabel: mem500tb { memory = 500000000000000.B }
|
||||
withLabel: mem1gib { memory = 1073741824.B }
|
||||
withLabel: mem2gib { memory = 2147483648.B }
|
||||
withLabel: mem4gib { memory = 4294967296.B }
|
||||
withLabel: mem8gib { memory = 8589934592.B }
|
||||
withLabel: mem16gib { memory = 17179869184.B }
|
||||
withLabel: mem32gib { memory = 34359738368.B }
|
||||
withLabel: mem64gib { memory = 68719476736.B }
|
||||
withLabel: mem128gib { memory = 137438953472.B }
|
||||
withLabel: mem256gib { memory = 274877906944.B }
|
||||
withLabel: mem512gib { memory = 549755813888.B }
|
||||
withLabel: mem1tib { memory = 1099511627776.B }
|
||||
withLabel: mem2tib { memory = 2199023255552.B }
|
||||
withLabel: mem4tib { memory = 4398046511104.B }
|
||||
withLabel: mem8tib { memory = 8796093022208.B }
|
||||
withLabel: mem16tib { memory = 17592186044416.B }
|
||||
withLabel: mem32tib { memory = 35184372088832.B }
|
||||
withLabel: mem64tib { memory = 70368744177664.B }
|
||||
withLabel: mem128tib { memory = 140737488355328.B }
|
||||
withLabel: mem256tib { memory = 281474976710656.B }
|
||||
withLabel: mem512tib { memory = 562949953421312.B }
|
||||
withLabel: cpu1 { cpus = 1 }
|
||||
withLabel: cpu2 { cpus = 2 }
|
||||
withLabel: cpu5 { cpus = 5 }
|
||||
withLabel: cpu10 { cpus = 10 }
|
||||
withLabel: cpu20 { cpus = 20 }
|
||||
withLabel: cpu50 { cpus = 50 }
|
||||
withLabel: cpu100 { cpus = 100 }
|
||||
withLabel: cpu200 { cpus = 200 }
|
||||
withLabel: cpu500 { cpus = 500 }
|
||||
withLabel: cpu1000 { cpus = 1000 }
|
||||
}
|
||||
|
||||
|
||||
141
target/nextflow/yq/nextflow_schema.json
Normal file
141
target/nextflow/yq/nextflow_schema.json
Normal file
@@ -0,0 +1,141 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"title": "yq",
|
||||
"description": "A portable YAML, JSON, XML, CSV, TOML and properties processor",
|
||||
"type": "object",
|
||||
"$defs": {
|
||||
"inputs": {
|
||||
"title": "Inputs",
|
||||
"type": "object",
|
||||
"description": "No description",
|
||||
"properties": {
|
||||
"input": {
|
||||
"type": "string",
|
||||
"format": "path",
|
||||
"exists": true,
|
||||
"description": "files to be processed",
|
||||
"help_text": "Type: `file`, multiple: `False`, required, direction: `input`, example: `\"input.yaml\"`. "
|
||||
}
|
||||
}
|
||||
},
|
||||
"outputs": {
|
||||
"title": "Outputs",
|
||||
"type": "object",
|
||||
"description": "No description",
|
||||
"properties": {
|
||||
"output": {
|
||||
"type": "string",
|
||||
"format": "path",
|
||||
"description": "output file",
|
||||
"help_text": "Type: `file`, multiple: `False`, required, default: `\"$id.$key.output.yaml\"`, direction: `output`, example: `\"output.yaml\"`. ",
|
||||
"default": "$id.$key.output.yaml"
|
||||
}
|
||||
}
|
||||
},
|
||||
"arguments": {
|
||||
"title": "Arguments",
|
||||
"type": "object",
|
||||
"description": "No description",
|
||||
"properties": {
|
||||
"eval": {
|
||||
"type": "string",
|
||||
"description": "expression to evaluate",
|
||||
"help_text": "Type: `string`, multiple: `False`, required, example: `\".name = \"foo\"\"`. "
|
||||
},
|
||||
"indent": {
|
||||
"type": "integer",
|
||||
"description": "sets indent level for output (default 2)",
|
||||
"help_text": "Type: `integer`, multiple: `False`. "
|
||||
},
|
||||
"input_format": {
|
||||
"type": "string",
|
||||
"description": "parse format for input",
|
||||
"help_text": "Type: `string`, multiple: `False`, choices: ``auto`, `a`, `yaml`, `y`, `json`, `j`, `props`, `p`, `csv`, `c`, `tsv`, `t`, `xml`, `x`, `base64`, `uri`, `toml`, `shell`, `s`, `lua`, `l``. ",
|
||||
"enum": [
|
||||
"auto",
|
||||
"a",
|
||||
"yaml",
|
||||
"y",
|
||||
"json",
|
||||
"j",
|
||||
"props",
|
||||
"p",
|
||||
"csv",
|
||||
"c",
|
||||
"tsv",
|
||||
"t",
|
||||
"xml",
|
||||
"x",
|
||||
"base64",
|
||||
"uri",
|
||||
"toml",
|
||||
"shell",
|
||||
"s",
|
||||
"lua",
|
||||
"l"
|
||||
]
|
||||
},
|
||||
"output_format": {
|
||||
"type": "string",
|
||||
"description": "output format type",
|
||||
"help_text": "Type: `string`, multiple: `False`, choices: ``auto`, `a`, `yaml`, `y`, `json`, `j`, `props`, `p`, `csv`, `c`, `tsv`, `t`, `xml`, `x`, `base64`, `uri`, `toml`, `shell`, `s`, `lua`, `l``. ",
|
||||
"enum": [
|
||||
"auto",
|
||||
"a",
|
||||
"yaml",
|
||||
"y",
|
||||
"json",
|
||||
"j",
|
||||
"props",
|
||||
"p",
|
||||
"csv",
|
||||
"c",
|
||||
"tsv",
|
||||
"t",
|
||||
"xml",
|
||||
"x",
|
||||
"base64",
|
||||
"uri",
|
||||
"toml",
|
||||
"shell",
|
||||
"s",
|
||||
"lua",
|
||||
"l"
|
||||
]
|
||||
},
|
||||
"pretty_print": {
|
||||
"type": "boolean",
|
||||
"description": "pretty print, shorthand for '..",
|
||||
"help_text": "Type: `boolean_true`, multiple: `False`, default: `false`. ",
|
||||
"default": false
|
||||
}
|
||||
}
|
||||
},
|
||||
"nextflow input-output arguments": {
|
||||
"title": "Nextflow input-output arguments",
|
||||
"type": "object",
|
||||
"description": "Input/output parameters for Nextflow itself. Please note that both publishDir and publish_dir are supported but at least one has to be configured.",
|
||||
"properties": {
|
||||
"publish_dir": {
|
||||
"type": "string",
|
||||
"description": "Path to an output directory.",
|
||||
"help_text": "Type: `string`, multiple: `False`, required, example: `\"output/\"`. "
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"allOf": [
|
||||
{
|
||||
"$ref": "#/$defs/inputs"
|
||||
},
|
||||
{
|
||||
"$ref": "#/$defs/outputs"
|
||||
},
|
||||
{
|
||||
"$ref": "#/$defs/arguments"
|
||||
},
|
||||
{
|
||||
"$ref": "#/$defs/nextflow input-output arguments"
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user