From 340f3b20ca2ffb5370efbe6567e88b32bb3a86e1 Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Wed, 8 Jul 2026 14:05:20 +0100 Subject: [PATCH 01/18] polish contributions.md --- docs/CONTRIBUTING.md | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md index 67a4370..94bbd9f 100644 --- a/docs/CONTRIBUTING.md +++ b/docs/CONTRIBUTING.md @@ -204,13 +204,11 @@ When adding new functionality (e.g. a new submission mode such as `isolate_genom All inputs in seqsubmit are defined via strict samplesheet schemas. -- Any new feature must: - - Extend or introduce a corresponding schema (assets/schema*input*\*.json) - - Clearly define required and optional fields - Maintain: - Column order consistency (critical for pipeline execution) - Backward compatibility where possible - Avoid duplicating metadata logic—reuse existing parsing and validation patterns. +- New feature should extend or introduce a corresponding schema (`assets/schema_\*.json`) if pipeline inputs are affected. ⸻ @@ -220,8 +218,6 @@ Submissions to [ENA](https://www.ebi.ac.uk/ena/browser/home) are governed by a w Contributors should refer directly to the [official ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/general-guide.html) to understand latest submission requirements and ensure that pipeline behaviour remains aligned with ENA standards. - Follow ENA-defined submission workflows and required steps for each data type - - Ensure that: - New pipeline workflows follow ENA-defined submission workflows and required steps for each data type - Metadata fields align with ENA requirements @@ -237,8 +233,8 @@ Contributors should refer directly to the [official ENA documentation](https://e All submissions are performed via [Webin-CLI](https://github.com/enasequence/webin-cli). Contributors should rely on official Webin-CLI documentation to understand expected inputs, parameters, and modes of operation. Make sure to always use the latest version of the tool. - Contributions affecting submission logic must: - - Respect --webincli_mode (validate vs submit) - - Support --test_upload for safe testing against ENA test server + - Respect `--webincli_mode` (validate vs submit) + - Support `--test_upload` for safe testing against ENA test server - Never introduce changes that: - Risk accidental submission to production without explicit user intent - Ensure compatibility with: @@ -276,7 +272,7 @@ Some data processing steps may be resource-intensive (e.g. database preparation 7. Documentation and Usability -Every contribution should improve usability: +Every contribution should aim to improve usability: - Update: - README.md From 9692ab3827dea6abb994b3361ecd125892a037f9 Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Wed, 8 Jul 2026 14:38:08 +0100 Subject: [PATCH 02/18] add limitations block --- README.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/README.md b/README.md index 40f2f6e..7133ca1 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,26 @@ Currently, the pipeline supports four submission modes, each routed to a dedicat ![seqsubmit workflow diagram](assets/seqsubmit_schema.png) +## Limitations + +nf-core/seqsubmit does not yet support the following data types and scenarios: + +- **Co-assemblies**: Metagenomic co-assemblies, and MAGs/bins generated from co-assemblies, are not supported. The pipeline currently expects each assembly/MAG/bin to be derived from a single run or an assembly generated from a single run. + +- **Eukaryotic and viral bins/MAGs**: Submitting eukaryotic or viral genomes that require additional metadata generation is not supported. + - For **eukaryotic bins/MAGs**, RNA prediction and quality assessment (completeness/contamination estimation) are not implemented. + - For **viral bins/MAGs**, RNA prediction, quality assessment (completeness/contamination estimation) and taxonomy assignment are not implemented. + +- **Reads in non-FASTQ formats**: Only reads provided as FASTQ (`.fastq.gz`) can be submitted with `--mode reads`. Other formats (e.g. BAM) are not supported as input. + +- **Submission of MAG/bin without submitting the assembly or raw reads**: If a MAG/bin cannot be traced back to an assembly accession or a reads accession — i.e. only a sample accession is available — it cannot currently be submitted, since the pipeline relies on that lineage to build the required metadata. + +- **Submission of assembly without submitting the raw reads**: Similarly, assemblies that have no associated reads accession (`run_accession` column in the sample sheet) are not supported for the same reason. + +- **Single-contig assemblies/MAGs/bins**: Assemblies or MAGs/bins that consist of only one contig cannot be uploaded through this pipeline. ENA classifies these as "chromosomal assemblies", which follow a different submission procedure with different metadata requirements. If you have single-contig assembly/MAG/bin to submit, please contact [ENA support](https://www.ebi.ac.uk/ena/browser/support) for guidance. + +- **Coverage calculation from long reads**: Coverage calculation via `coverm` has only been tested and validated with short reads (e.g. Illumina). It has not been tested with long reads (PacBio, Nanopore). Exercise caution when processing data generated from long reads, as issues may occur. + ## Requirements - [Nextflow](https://www.nextflow.io/) `>=25.04.0` From 4452cb8975d5c91f8df5ac1b20e28ce5ed8cd586 Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Wed, 8 Jul 2026 14:50:19 +0100 Subject: [PATCH 03/18] refactor limitations block --- README.md | 50 ++++++++++++++++++++++++++++++-------------------- 1 file changed, 30 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index 7133ca1..1e17529 100644 --- a/README.md +++ b/README.md @@ -31,26 +31,6 @@ Currently, the pipeline supports four submission modes, each routed to a dedicat ![seqsubmit workflow diagram](assets/seqsubmit_schema.png) -## Limitations - -nf-core/seqsubmit does not yet support the following data types and scenarios: - -- **Co-assemblies**: Metagenomic co-assemblies, and MAGs/bins generated from co-assemblies, are not supported. The pipeline currently expects each assembly/MAG/bin to be derived from a single run or an assembly generated from a single run. - -- **Eukaryotic and viral bins/MAGs**: Submitting eukaryotic or viral genomes that require additional metadata generation is not supported. - - For **eukaryotic bins/MAGs**, RNA prediction and quality assessment (completeness/contamination estimation) are not implemented. - - For **viral bins/MAGs**, RNA prediction, quality assessment (completeness/contamination estimation) and taxonomy assignment are not implemented. - -- **Reads in non-FASTQ formats**: Only reads provided as FASTQ (`.fastq.gz`) can be submitted with `--mode reads`. Other formats (e.g. BAM) are not supported as input. - -- **Submission of MAG/bin without submitting the assembly or raw reads**: If a MAG/bin cannot be traced back to an assembly accession or a reads accession — i.e. only a sample accession is available — it cannot currently be submitted, since the pipeline relies on that lineage to build the required metadata. - -- **Submission of assembly without submitting the raw reads**: Similarly, assemblies that have no associated reads accession (`run_accession` column in the sample sheet) are not supported for the same reason. - -- **Single-contig assemblies/MAGs/bins**: Assemblies or MAGs/bins that consist of only one contig cannot be uploaded through this pipeline. ENA classifies these as "chromosomal assemblies", which follow a different submission procedure with different metadata requirements. If you have single-contig assembly/MAG/bin to submit, please contact [ENA support](https://www.ebi.ac.uk/ena/browser/support) for guidance. - -- **Coverage calculation from long reads**: Coverage calculation via `coverm` has only been tested and validated with short reads (e.g. Illumina). It has not been tested with long reads (PacBio, Nanopore). Exercise caution when processing data generated from long reads, as issues may occur. - ## Requirements - [Nextflow](https://www.nextflow.io/) `>=25.04.0` @@ -297,6 +277,36 @@ Key output locations in `--outdir`: For full details, see the [output documentation](https://nf-co.re/seqsubmit/output). +## Limitations + +nf-core/seqsubmit does not yet support the following data types and scenarios, grouped by the mode they affect: + +#### `mags` and `bins` modes (`GENOMESUBMIT`) + +- **Co-assemblies**: MAGs/bins generated from co-assemblies are not supported. The pipeline currently expects each MAG/bin to be derived from an assembly generated from a single run. + +- **Eukaryotic and viral bins/MAGs**: Submitting eukaryotic or viral genomes that require additional metadata generation is not supported. For such genomes tRNA and rRNA prediction, quality assessment (completeness/contamination estimation) and taxonomy assignment are not implemented. If all the required metadata (`completeness`, `contamination`, `stats_generation_software`, `RNA_presence` and `NCBI_lineage`) is already provided in the samplesheet, eukaryotic and viral bins/MAGs can still be submitted normally. + +- **Submission of a MAG/bin without submitting the assembly or raw reads**: If a MAG/bin cannot be traced back to an assembly accession or a reads accession — i.e. only a sample accession is available — it cannot currently be submitted, since the pipeline relies on that lineage to build the required metadata. + +- **Single-contig MAGs/bins**: MAGs/bins that consist of only one contig cannot be uploaded through this pipeline. ENA classifies these as "chromosomal assemblies", which follow a different submission procedure with different metadata requirements. If you have a single-contig MAG/bin to submit, please contact [ENA support](https://www.ebi.ac.uk/ena/browser/support) for guidance. + +- **Coverage calculation from long reads**: `genome_coverage` calculation via `coverm` has only been tested and validated with short reads (e.g. Illumina). It has not been tested with long reads (PacBio, Nanopore). Exercise caution when processing data generated from long reads, as issues may occur. + +#### `metagenomic_assemblies` mode (`ASSEMBLYSUBMIT`) + +- **Co-assemblies**: Metagenomic co-assemblies are not supported. The pipeline currently expects each assembly to be derived from a single run. + +- **Submission of an assembly without submitting the raw reads**: Assemblies that have no associated reads accession (`run_accession` column in the samplesheet) are not supported, since the pipeline relies on that lineage to build the required metadata. + +- **Single-contig assemblies**: Assemblies that consist of only one contig cannot be uploaded through this pipeline. ENA classifies these as "chromosomal assemblies", which follow a different submission procedure with different metadata requirements. If you have a single-contig assembly to submit, please contact [ENA support](https://www.ebi.ac.uk/ena/browser/support) for guidance. + +- **Coverage calculation from long reads**: `coverage` calculation via `coverm` has only been tested and validated with short reads (e.g. Illumina). It has not been tested with long reads (PacBio, Nanopore). Exercise caution when processing data generated from long reads, as issues may occur. + +#### `reads` mode (`READSUBMIT`) + +- **Reads in non-FASTQ formats**: Only reads provided as FASTQ (`.fastq.gz`) can be submitted. Other formats (e.g. BAM) are not supported as input. + ## Credits nf-core/seqsubmit was originally written by [Martin Beracochea](https://github.com/mberacochea), [Ekaterina Sakharova](https://github.com/KateSakharova), [Sofia Ochkalova](https://github.com/ochkalova), [Evangelos Karatzas](https://github.com/vagkaratzas) and [Tim Rozday](https://github.com/timrozday-mgnify). From 6bda9c774448e48ff84b17af3cf00a8154589ff2 Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Wed, 8 Jul 2026 15:00:15 +0100 Subject: [PATCH 04/18] add scalability section to README.md --- README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/README.md b/README.md index 1e17529..e699631 100644 --- a/README.md +++ b/README.md @@ -277,6 +277,14 @@ Key output locations in `--outdir`: For full details, see the [output documentation](https://nf-co.re/seqsubmit/output). +## Scalability + +Some processes in this pipeline are resource intensive — in particular coverage calculation (`coverm`), quality assessment (`CheckM2`), and taxonomic classification of MAGs/bins (`BAT`). For real life data these steps should be run on an HPC cluster or another compute environment with sufficient CPU, memory, and job scheduling capacity, rather than on a single local machine. + +Depending on the resources available on your machine or cluster, you will likely need to adapt the Nextflow configuration to limit the maximum number of concurrently running jobs and other resource settings (e.g. `max_cpus`, `max_memory`, `max_time`, or `executor.queueSize` in a custom config), so the pipeline doesn't try to claim more resources than are actually available. See the [nf-core configuration documentation](https://nf-co.re/docs/usage/getting_started/configuration) for guidance on writing a custom config for your infrastructure. + +When submitting a large number of records, be aware that ENA recommends limiting submissions to 5000 per day to avoid overfilling their processing queue. If you need to submit more than this, consider splitting your samplesheet and spreading the submission across several days. + ## Limitations nf-core/seqsubmit does not yet support the following data types and scenarios, grouped by the mode they affect: From 9074e16fc2ba4495a48360ade7d84565f82f32f1 Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Wed, 8 Jul 2026 15:22:35 +0100 Subject: [PATCH 05/18] refactor contributing guidelines --- docs/CONTRIBUTING.md | 66 +++++--------------------------------------- 1 file changed, 7 insertions(+), 59 deletions(-) diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md index 94bbd9f..e48f6a7 100644 --- a/docs/CONTRIBUTING.md +++ b/docs/CONTRIBUTING.md @@ -184,35 +184,7 @@ If you update images or graphics, follow the nf-core [style guidelines](https:// To ensure consistency and maintainability of the nf-core/seqsubmit pipeline, contributors should follow these guidelines when implementing new features or modifying existing functionality. -1. Understand Workflow Structure - -The pipeline consists of independent workflows, each corresponding to a certain data type: - -- GENOMESUBMIT (mags, bins) -- ASSEMBLYSUBMIT (metagenomic_assemblies) -- READSUBMIT (raw reads) - -When adding new functionality (e.g. a new submission mode such as `isolate_genomes`), ensure that: - -- It integrates cleanly into the appropriate workflow or introduces a well-structured new one. -- Existing workflows remain unaffected unless changes are explicitly required. -- Shared logic is abstracted into reusable modules or subworkflows where possible. - -⸻ - -2. Follow Samplesheet and Schema-Driven Design - -All inputs in seqsubmit are defined via strict samplesheet schemas. - -- Maintain: - - Column order consistency (critical for pipeline execution) - - Backward compatibility where possible -- Avoid duplicating metadata logic—reuse existing parsing and validation patterns. -- New feature should extend or introduce a corresponding schema (`assets/schema_\*.json`) if pipeline inputs are affected. - -⸻ - -3. ENA Submission Requirements +### ENA Submission Requirements Submissions to [ENA](https://www.ebi.ac.uk/ena/browser/home) are governed by a well-defined data model and submission process. These include relationships between entities (e.g. studies, samples, experiments, runs, analyses), required submission steps, and accepted file types and formats. @@ -228,7 +200,7 @@ Contributors should refer directly to the [official ENA documentation](https://e ⸻ -4. Integration with Webin-CLI +### Integration with Webin-CLI All submissions are performed via [Webin-CLI](https://github.com/enasequence/webin-cli). Contributors should rely on official Webin-CLI documentation to understand expected inputs, parameters, and modes of operation. Make sure to always use the latest version of the tool. @@ -242,7 +214,7 @@ All submissions are performed via [Webin-CLI](https://github.com/enasequence/web ⸻ -5. Testing Requirements +### Testing Requirements All new functionality must be covered by tests. @@ -264,36 +236,12 @@ Where applicable: ⸻ -6. Performance and Resource Awareness - -Some data processing steps may be resource-intensive (e.g. database preparation for CAT_pack). When introducing new workflows or features avoid unnecessary recomputation. Add option to reuse cached results or provide prebuilt databases when possible. - -⸻ - -7. Documentation and Usability - -Every contribution should aim to improve usability: - -- Update: - - README.md - - docs/usage.md - - Schema descriptions -- Provide: - - Clear examples (e.g. samplesheets) - - Meaningful parameter descriptions - -The goal is to ensure that users can successfully submit data without needing deep knowledge of ENA submission requirements. - -⸻ - -8. Design Principles +### Performance and Resource Awareness -When contributing, prioritise: +Some data processing steps may be resource-intensive (e.g. database preparation for CAT_pack). -- Reproducibility (consistent results across environments) -- Automation (minimise manual user intervention) -- Clarity (explicit inputs and outputs) -- Extensibility (easy to add new submission types) +- When introducing new workflows or features avoid unnecessary recomputation. +- Add option to reuse cached results or provide prebuilt databases when possible. ⸻ From 44c38d9fb33c78ccdaf0964f0fb27f51d0efd013 Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Thu, 9 Jul 2026 14:50:50 +0100 Subject: [PATCH 06/18] update CONTRIBUTING.md to clarify testing requirements for ENA submission --- docs/CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md index e48f6a7..7552e65 100644 --- a/docs/CONTRIBUTING.md +++ b/docs/CONTRIBUTING.md @@ -216,7 +216,7 @@ All submissions are performed via [Webin-CLI](https://github.com/enasequence/web ### Testing Requirements -All new functionality must be covered by tests. +All new functionality must be covered by tests. Running nf-tests that perform ENA submission locally requires ENA webin credentials. Contributors may ask for testing credentials on the nf-core Slack [#seqsubmit](https://nfcore.slack.com/channels/seqsubmit) channel. At minimum, include: From b0343f445ce70c0933beb76eb6ba72d69b5b4734 Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Thu, 9 Jul 2026 15:07:00 +0100 Subject: [PATCH 07/18] update README.md to reference Methods documentation for metadata statistics and coverage calculation --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index e699631..07b3f72 100644 --- a/README.md +++ b/README.md @@ -80,7 +80,7 @@ Additional supported columns: - `RNA_presence` - `NCBI_lineage` -If `genome_coverage`, `stats_generation_software`, `completeness`, `contamination`, `RNA_presence`, or `NCBI_lineage` are missing, the workflow can calculate or infer them when the required inputs are available. +If `genome_coverage`, `stats_generation_software`, `completeness`, `contamination`, `RNA_presence`, or `NCBI_lineage` are missing, the workflow can calculate or infer them when the required inputs are available. See the [Methods documentation](docs/methods.md) for more information on how metadata statistics are obtained. Those fields are metadata required for the [genome_uploader](https://github.com/EBI-Metagenomics/genome_uploader) package. @@ -111,7 +111,7 @@ At least one of the following must be provided per row: - reads (`fastq_1`, optional `fastq_2` for paired-end) - `coverage` -If `coverage` is missing and reads are provided, the workflow calculates average coverage with `coverm`. +If `coverage` is missing and reads are provided, the workflow calculates average coverage with `coverm`. See the [Methods documentation](docs/methods.md) for more information on how coverage is calculated. Example `samplesheet_assembly.csv`: From f7f99b32df13817b972d0a73b708dae848504b2b Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Thu, 9 Jul 2026 15:07:23 +0100 Subject: [PATCH 08/18] refactor methods documentation to clarify RNA presence detection and update metadata schema --- docs/methods.md | 50 ++++++++----------- .../local/create_genome_metadata_tsv/meta.yml | 2 +- nextflow_schema.json | 8 +-- 3 files changed, 25 insertions(+), 35 deletions(-) diff --git a/docs/methods.md b/docs/methods.md index 0f15a33..acbf12f 100644 --- a/docs/methods.md +++ b/docs/methods.md @@ -1,15 +1,8 @@ # nf-core/seqsubmit: Methods -## Overview +This page provides a detailed overview of the methods and procedures that are executed throughout the nf-core/seqsubmit pipeline. -`nf-core/seqsubmit` currently contains two workflow implementations: - -- `GENOMESUBMIT` for `--mode mags` and `--mode bins` -- `ASSEMBLYSUBMIT` for `--mode metagenomic_assemblies` - -This page documents the methods that are currently implemented in the pipeline. - -## `GENOMESUBMIT` methods +## `GENOMESUBMIT` ### Overview @@ -17,22 +10,24 @@ The `GENOMESUBMIT` workflow: 1. Reads the samplesheet and associated genome FASTA files. 2. Validates genome FASTA files. -3. Reuses provided or calculates missing values for RNA genes presence, coverage, taxonomy, and genome quality metrics. +3. Reuses provided or calculates missing values for tRNA and rRNA genes presence, coverage, taxonomy, and genome quality metrics. 4. Collects genome metadata into the tabular format required by `genome_uploader`. 5. Generates submission manifests for ENA. 6. Performs submission to ENA. ### Genome FASTA validation -Genome FASTA files are validated with the `FASTAVALIDATOR` module before downstream processing. Each file is checked for FASTA format validity and contig count. A genome must contain at least two contigs to pass validation, which is an ENA requirement for contig-level submissions. +Genome FASTA files are validated with the `fa-lint` before downstream processing. Each file is checked for FASTA format validity and contig count. A genome must contain at least two contigs to pass validation, which is an ENA requirement for contig-level submissions. Only FASTA files that pass validation are retained for downstream processing and submission. -### RNA presence detection +### tRNA and rRNA genes detection -The workflow only runs internal RNA detection for entries where the `RNA_presence` column is empty. If a value is already supplied in the samplesheet, that value is passed through unchanged. +The workflow only runs tRNA and rRNA genes detection for entries where the `RNA_presence` column is empty. If a value is already supplied in the samplesheet, that value is passed through unchanged. -RNA detection is implemented through the `RNA_DETECTION` subworkflow and combines: +The resulting `RNA_presence` value is a decisive factor affecting MISAG/MIMAG assembly quality classification of a genome. + +tRNA and rRNA genes detection is implemented through the `RNA_DETECTION` subworkflow and combines: - `barrnap` for rRNA prediction - `tRNAscan-SE` for tRNA prediction @@ -42,11 +37,7 @@ RNA detection is implemented through the `RNA_DETECTION` subworkflow and combine `barrnap` is run in bacterial mode (`"bac"`). -The custom parser then scans the GFF output and keeps only the following ribosomal RNA subunits: - -- `16S_rRNA` -- `23S_rRNA` -- `5S_rRNA` +The custom parser then scans the GFF output and keeps only the following rRNA subunits: `16S_rRNA`, `23S_rRNA`, `5S_rRNA`. For each detected feature, the recovered length is calculated as: @@ -66,7 +57,7 @@ $$ If multiple hits are found for the same subunit, the workflow keeps the best recovered percentage for that subunit. -A subunit is considered present when its best recovered percentage is greater than or equal to `--rrna_limit`. The current default is `80`. +A subunit is considered present when its best recovered percentage is greater than or equal to `--rrna_limit`. The current default is `80`. This threshold is arbitrary, as no strict consensus length is defined by ENA or MISAG/MIMAG for considering an rRNA gene present. #### tRNA detection @@ -79,16 +70,16 @@ The parser sums the counts assigned to the 20 standard amino-acid isotypes: - `Leu`, `Lys`, `Met`, `Phe`, `Pro` - `Ser`, `Thr`, `Trp`, `Tyr`, `Val` -The total number of predicted tRNAs is compared against `params.trna_limit`. The current default is `18`. +The total number of predicted tRNAs is compared against `params.trna_limit`. The current default is `18`. This value is defined by the MISAG/MIMAG standard and must not be modified. -#### Final RNA presence decision +#### Final `RNA_presence` decision The final decision stored in `RNA_presence` is: - `Yes` when at least 18 tRNAs are detected and all three required rRNA subunits pass the recovery threshold - `No` otherwise -The result is written as a two-column TSV file containing the genome identifier and the final `Yes`/`No` decision, and this value is then merged back into the submission metadata. +The final `Yes`/`No` decision is then merged back into the submission metadata. ### Genome coverage calculation @@ -98,7 +89,7 @@ Genome coverage values from `coverm genome` output TSV are parsed and merged int ### Taxonomy assignment -If `NCBI_lineage` is already present in the input samlesheet, the value is retained. If it is missing, the workflow runs taxonomy classification using the `CAT_pack` tool. +If `NCBI_lineage` is already present in the input samplesheet, the value is retained. If it is missing, the workflow runs taxonomy classification using the `CAT_pack` tool. Before classification, input FASTA files are normalized to a `.fasta` suffix by `RENAME_FASTA_FOR_CATPACK`. Classification is then run in bin mode (`CAT_pack bins`, followed by `CAT_pack add_names`). @@ -112,7 +103,7 @@ The workflow checks three samplesheet fields: `completeness`, `contamination`, a For records that run `CheckM2`, completeness and contamination are extracted from the generated quality report (`quality_report.tsv`) and used `CheckM2` version is recorded as `stats_generation_software`. -## `ASSEMBLYSUBMIT` methods +## `ASSEMBLYSUBMIT` ### Overview @@ -120,14 +111,13 @@ The `ASSEMBLYSUBMIT` workflow: 1. Reads the samplesheet and associated assembly FASTA files. 2. Validates assembly FASTA files. -3. Reuses `coverage` values supplied in the samplesheet when they are already present. -4. Calculates `coverage` internally for entries where this field is missing and reads are available. -5. Builds the metadata table used for manifest generation. -6. Generates assembly manifests and submits the assemblies to ENA. +3. Reuses provided or calculates missing `coverage` values. +4. Builds the metadata table used for manifest generation. +5. Generates assembly manifests and submits the assemblies to ENA. ### Assembly FASTA validation -Assembly FASTA files are validated with `FASTAVALIDATOR` before downstream processing. Each file is checked for FASTA format validity and contig count. A genome must contain at least two contigs to pass validation, which is an ENA requirement for contig-level submissions. +Assembly FASTA files are validated with `fa-lint` before downstream processing. Each file is checked for FASTA format validity and contig count. A genome must contain at least two contigs to pass validation, which is an ENA requirement for contig-level submissions. Only assemblies with successful validation are forwarded to coverage estimation, metadata/manifest generation and submission. diff --git a/modules/local/create_genome_metadata_tsv/meta.yml b/modules/local/create_genome_metadata_tsv/meta.yml index 74752ad..870b889 100644 --- a/modules/local/create_genome_metadata_tsv/meta.yml +++ b/modules/local/create_genome_metadata_tsv/meta.yml @@ -35,7 +35,7 @@ input: - broad_environment: Broad environmental context - local_environment: Local environmental context - environmental_medium: Environmental medium description - - RNA_presence: RNA presence status ("Yes" or other) + - RNA_presence: status of tRNA and rRNA genes presence ("Yes" or other) - NCBI_lineage: NCBI taxonomic lineage - fasta: type: file diff --git a/nextflow_schema.json b/nextflow_schema.json index 59c6dc0..b7d7da5 100644 --- a/nextflow_schema.json +++ b/nextflow_schema.json @@ -43,11 +43,11 @@ } }, "rna_detection_options": { - "title": "RNA detection options", + "title": "rRNA and tRNA detection options", "type": "object", "fa_icon": "fas fa-university", - "description": "Parameters used to make a decision about presence/absence of RNA in data", - "help_text": "ENA requires a boolean value (True/False) defining RNA. Decision of presence/absence of RNA is based on number of rRNA and tRNA. If both values are bigger than their corresponding limits - RNA is present (True), otherwise - not (False).", + "description": "Parameters affecting a decision about presence/absence of tRNA and rRNA genes (RNA_presence) in a genome", + "help_text": "The value of RNA_presence is based on the number of rRNA and tRNA. If at least 18 tRNAs are detected and 16S, 23S, and 5S rRNA subunits pass the recovery threshold - RNA_presence is True, otherwise - False. The resulting value is a decisive factor affecting MISAG/MIMAG assembly quality classification of a genome.", "properties": { "rrna_limit": { "type": "number", @@ -57,7 +57,7 @@ }, "trna_limit": { "type": "number", - "description": "Minimum number of tRNA.", + "description": "Minimum number of tRNA genes detected to count tRNA as present. This value is defined by the MISAG/MIMAG standard and must not be modified.", "default": 18, "fa_icon": "fas fa-users-cog" } From 36567f412f06f80241e09396939c27729ed09fe0 Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Thu, 9 Jul 2026 15:09:19 +0100 Subject: [PATCH 09/18] add SVG pipeline metromap --- assets/seqsubmit_schema.svg | 114 ++++++++++++++++++++++++++++++++++++ 1 file changed, 114 insertions(+) create mode 100644 assets/seqsubmit_schema.svg diff --git a/assets/seqsubmit_schema.svg b/assets/seqsubmit_schema.svg new file mode 100644 index 0000000..2338fce --- /dev/null +++ b/assets/seqsubmit_schema.svg @@ -0,0 +1,114 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + From c57e86589021709490079d438a7a2e6c03c8872e Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Fri, 10 Jul 2026 11:19:09 +0100 Subject: [PATCH 10/18] update README.md Limitations to include issue tracking --- README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 07b3f72..db00849 100644 --- a/README.md +++ b/README.md @@ -291,11 +291,11 @@ nf-core/seqsubmit does not yet support the following data types and scenarios, g #### `mags` and `bins` modes (`GENOMESUBMIT`) -- **Co-assemblies**: MAGs/bins generated from co-assemblies are not supported. The pipeline currently expects each MAG/bin to be derived from an assembly generated from a single run. +- **Co-assemblies**: MAGs/bins generated from co-assemblies are not supported. The pipeline currently expects each MAG/bin to be derived from an assembly generated from a single run. Tracked in [issue #61](https://github.com/nf-core/seqsubmit/issues/61). -- **Eukaryotic and viral bins/MAGs**: Submitting eukaryotic or viral genomes that require additional metadata generation is not supported. For such genomes tRNA and rRNA prediction, quality assessment (completeness/contamination estimation) and taxonomy assignment are not implemented. If all the required metadata (`completeness`, `contamination`, `stats_generation_software`, `RNA_presence` and `NCBI_lineage`) is already provided in the samplesheet, eukaryotic and viral bins/MAGs can still be submitted normally. +- **Eukaryotic and viral bins/MAGs**: Submitting eukaryotic or viral genomes that require additional metadata generation is not supported. For such genomes tRNA and rRNA prediction, quality assessment (completeness/contamination estimation) and taxonomy assignment are not implemented. If all the required metadata (`completeness`, `contamination`, `stats_generation_software`, `RNA_presence` and `NCBI_lineage`) is already provided in the samplesheet, eukaryotic and viral bins/MAGs can still be submitted normally. Tracked in [issue #40](https://github.com/nf-core/seqsubmit/issues/40) (eukaryotic) and [issue #63](https://github.com/nf-core/seqsubmit/issues/63) (viral). -- **Submission of a MAG/bin without submitting the assembly or raw reads**: If a MAG/bin cannot be traced back to an assembly accession or a reads accession — i.e. only a sample accession is available — it cannot currently be submitted, since the pipeline relies on that lineage to build the required metadata. +- **Submission of a MAG/bin without submitting the assembly or raw reads**: If a MAG/bin cannot be traced back to an assembly accession or a reads accession — i.e. only a sample accession is available — it cannot currently be submitted, since the pipeline relies on that lineage to build the required metadata. Tracked in [issue #87](https://github.com/nf-core/seqsubmit/issues/87). - **Single-contig MAGs/bins**: MAGs/bins that consist of only one contig cannot be uploaded through this pipeline. ENA classifies these as "chromosomal assemblies", which follow a different submission procedure with different metadata requirements. If you have a single-contig MAG/bin to submit, please contact [ENA support](https://www.ebi.ac.uk/ena/browser/support) for guidance. @@ -303,9 +303,9 @@ nf-core/seqsubmit does not yet support the following data types and scenarios, g #### `metagenomic_assemblies` mode (`ASSEMBLYSUBMIT`) -- **Co-assemblies**: Metagenomic co-assemblies are not supported. The pipeline currently expects each assembly to be derived from a single run. +- **Co-assemblies**: Metagenomic co-assemblies are not supported. The pipeline currently expects each assembly to be derived from a single run. Tracked in [issue #61](https://github.com/nf-core/seqsubmit/issues/61), addressed in [PR #66](https://github.com/nf-core/seqsubmit/pull/66). -- **Submission of an assembly without submitting the raw reads**: Assemblies that have no associated reads accession (`run_accession` column in the samplesheet) are not supported, since the pipeline relies on that lineage to build the required metadata. +- **Submission of an assembly without submitting the raw reads**: Assemblies that have no associated reads accession (`run_accession` column in the samplesheet) are not supported, since the pipeline relies on that lineage to build the required metadata. Tracked in [issue #29](https://github.com/nf-core/seqsubmit/issues/29). - **Single-contig assemblies**: Assemblies that consist of only one contig cannot be uploaded through this pipeline. ENA classifies these as "chromosomal assemblies", which follow a different submission procedure with different metadata requirements. If you have a single-contig assembly to submit, please contact [ENA support](https://www.ebi.ac.uk/ena/browser/support) for guidance. From f759e9c2f76fa86d860be68f70867502ef004b66 Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Fri, 10 Jul 2026 11:36:48 +0100 Subject: [PATCH 11/18] update README.md to add a Table of Contents --- README.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index db00849..9fb879a 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,24 @@ [![Get help on Slack](http://img.shields.io/badge/slack-nf--core%20%23seqsubmit-4A154B?labelColor=000000&logo=slack)](https://nfcore.slack.com/channels/seqsubmit)[![Follow on Bluesky](https://img.shields.io/badge/bluesky-%40nf__core-1185fe?labelColor=000000&logo=bluesky)](https://bsky.app/profile/nf-co.re)[![Follow on Mastodon](https://img.shields.io/badge/mastodon-nf__core-6364ff?labelColor=FFFFFF&logo=mastodon)](https://mstdn.science/@nf_core)[![Watch on YouTube](http://img.shields.io/badge/youtube-nf--core-FF0000?labelColor=000000&logo=youtube)](https://www.youtube.com/c/nf-core) +## Table of Contents + +- [Introduction](#introduction) +- [Requirements](#requirements) +- [Input samplesheets](#input-samplesheets) +- [Usage](#usage) + - [Submission study](#submission-study) + - [Data privacy](#data-privacy) + - [Database setup (`CheckM2` and `CAT_pack`)](#database-setup-checkm2-and-cat_pack) + - [Required parameters:](#required-parameters) + - [Optional parameters:](#optional-parameters) +- [Pipeline output](#pipeline-output) +- [Scalability](#scalability) +- [Limitations](#limitations) +- [Credits](#credits) +- [Contributions and Support](#contributions-and-support) +- [Citations](#citations) + ## Introduction **nf-core/seqsubmit** is a Nextflow pipeline for submitting sequence data to [ENA](https://www.ebi.ac.uk/ena/browser/home). @@ -205,7 +223,7 @@ The `mags`/`bins` workflow requires databases for completeness/contamination est | `--upload_tpa` | Flag to control the type of assembly study (third party assembly or not). Default: false | | `--test_upload` | Upload to TEST ENA server instead of LIVE. Default: true | | `--webincli_mode` | Choose Webin-CLI mode: `submit` or `validate`. Default: `submit` | -| `--private` | Use that flag if you are referring to private data in ENA | +| `--is_private` | Use that flag if you are referring private data accessions in your submission | | `--release_date` | Use that flag if you want to keep your data private after submission until particular date | General command template: From a7f6dd63fd0e884ccbd89a8bc7cc0e00bb49d372 Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Fri, 10 Jul 2026 11:59:12 +0100 Subject: [PATCH 12/18] Move Data privacy section to usage.md --- README.md | 15 +-------------- docs/usage.md | 17 +++++++++++++++++ 2 files changed, 18 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 9fb879a..1a8fd84 100644 --- a/README.md +++ b/README.md @@ -187,20 +187,7 @@ See the [usage documentation](docs/usage.md#submission-study) for more details. ### Data privacy -You can reference private ENA data if you have access to it via your Webin account (it was submitted previously under your credentials). To use private data accessions in your submission metadata, specify the `--is_private` flag. The pipeline will then use your provided Webin credentials to fetch the required metadata. - -You can also control the privacy of your submitted data: - -- If you provide an existing study via `--submission_study`, your submission will inherit the same privacy status as that study. -- If no study is provided, the pipeline will register a new one for you. To keep this new study private, you must specify a release date using `--release_date YYYY-MM-DD` (up to 2 years from the current date). - -| Example | Source Data | Submission Visibility | Required Arguments | Result | -| --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | --------------------- | ------------------------------------------------------ | ----------------------------------------------------- | -| Assemblies generated from public reads. Assemblies should be public immediately after submission | public | public | – | SUCCESS | -| Assemblies generated from public reads. Assemblies should remain private for 1 year after submission | public | private | `--release_date YYYY-MM-DD` (+1 year) | SUCCESS | -| Bins generated from private reads and assemblies. Bins should be public after submission. User **has access** to the reads and assemblies via Webin account | private | public | `--is_private` | SUCCESS | -| Bins generated from private reads and assemblies. Bins should remain private for 2 years after submission. User **has access** via Webin account | private | private | `--is_private`, `--release_date YYYY-MM-DD` (+2 years) | SUCCESS | -| MAGs generated from private reads and assemblies. MAGs should be public after submission. User **does not have access** to the reads and assemblies via Webin account | private | public | – | ERROR (submission can only reference accessible data) | +The pipeline supports private submissions. See the [usage documentation](docs/usage.md#data-privacy) for more details. ### Database setup (`CheckM2` and `CAT_pack`) diff --git a/docs/usage.md b/docs/usage.md index 7d3b5ef..54177c5 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -198,6 +198,23 @@ study-soil-2026 Soil microbiome study Survey of soil microbiota | `existing_study_type` | No | ENA study type (e.g. `Metagenomics`, `Other`). | | `new_study_type` | No | Custom study type. Only used when `existing_study_type` is set to `Other`. | +## Data privacy + +You can reference private ENA data if you have access to it via your Webin account (it was submitted previously under your credentials). To use private data accessions in your submission metadata, specify the `--is_private` flag. The pipeline will then use your provided Webin credentials to fetch the required metadata. + +You can also control the privacy of your submitted data: + +- If you provide an existing study via `--submission_study`, your submission will inherit the same privacy status as that study. +- If no study is provided, the pipeline will register a new one for you. To keep this new study private, you must specify a release date using `--release_date YYYY-MM-DD` (up to 2 years from the current date). + +| Example | Source Data | Submission Visibility | Required Arguments | Result | +| --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | --------------------- | ------------------------------------------------------ | ----------------------------------------------------- | +| Assemblies generated from public reads. Assemblies should be public immediately after submission | public | public | – | SUCCESS | +| Assemblies generated from public reads. Assemblies should remain private for 1 year after submission | public | private | `--release_date YYYY-MM-DD` (+1 year) | SUCCESS | +| Bins generated from private reads and assemblies. Bins should be public after submission. User **has access** to the reads and assemblies via Webin account | private | public | `--is_private` | SUCCESS | +| Bins generated from private reads and assemblies. Bins should remain private for 2 years after submission. User **has access** via Webin account | private | private | `--is_private`, `--release_date YYYY-MM-DD` (+2 years) | SUCCESS | +| MAGs generated from private reads and assemblies. MAGs should be public after submission. User **does not have access** to the reads and assemblies via Webin account | private | public | – | ERROR (submission can only reference accessible data) | + ## Database preparation (`mags` / `bins`) The `GENOMESUBMIT` workflow uses `CheckM2` and `CAT_pack` that require specialized databases for completeness/contamination assessment and taxonomy assignment. From 81e64bdb908cdcad2bb2f98add04ce0db1aaa8c1 Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Fri, 10 Jul 2026 12:11:00 +0100 Subject: [PATCH 13/18] use one format for tables with column descriptions in usage.md --- docs/usage.md | 90 +++++++++++++++++++++++++-------------------------- 1 file changed, 45 insertions(+), 45 deletions(-) diff --git a/docs/usage.md b/docs/usage.md index 54177c5..bc30044 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -51,27 +51,27 @@ mag_001,data/mag_001.fasta.gz,SRR24458089,,,SPAdes 3.15.5,MetaBAT2 2.15,default, > [!IMPORTANT] > **Samplesheet column requirements**: All columns shown in the example above must be present in your samplesheet, even if some values are empty. Columns must be in exactly the same order as shown. -| Column | Description | -| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `sample` | A unique identifier for this data entry. Must be globally unique within the input dataset (used as `meta.id` throughout the pipeline). | -| `fasta` | Path to MAG/bin contigs in FASTA format compressed with `gzip`. All names of the FASTA files must be unique to prevent pipeline errors. | -| `accession` | ENA accession of the run or metagenomic assembly used to generate the MAG/bin. | -| `fastq_1` | Path to the read file in FASTQ format used to generate the source metagenomic assembly. Required if `genome_coverage` is not provided. | -| `fastq_2` | Path to the second read file in FASTQ format for paired-end data used to generate the source metagenomic assembly. Leave empty for single-end reads. | -| `assembly_software` | Tool name and version that were used to generate the source metagenomic assembly. | -| `binning_software` | Binning tool, including version, that was used to generate the bins. | -| `binning_parameters` | Arguments that were used during binning. | -| `stats_generation_software` | Tool, including version, that was used to calculate completeness and contamination. | -| `completeness` | Genome completeness value. | -| `contamination` | Genome contamination value. | -| `genome_coverage` | Estimated average sequencing depth across the genome. If the value is missing, it is computed automatically during pipeline execution when reads are provided. | -| `metagenome` | Registered metagenome taxonomic identifier or name that matches an existing ENA taxonomy entry. For more details see https://ena-docs.readthedocs.io/en/latest/faq/taxonomy.html | -| `co-assembly` | Whether a co-assembly strategy was used for the initial metagenomic assembly generation. Options: Yes or No. | -| `broad_environment` | Broad ecological context of the sample, for example 'marine biome', 'desert biome'. It is recommended to use subclasses of EnvO 'biome' class (http://purl.obolibrary.org/obo/ENVO_00000428) | -| `local_environment` | Local environmental context of the sample, for example 'tropical dry broadleaf forest biome', 'marine abyssal zone biome'. It is recommended to use EnvO terms which are of smaller spatial grain than your entry for "broad-scale environmental context". | -| `environmental_medium` | Material displaced by the sample, or the material in which the sample was embedded before sampling, for example 'mucus', 'lake water'. It is recommended to use subclasses of EnvO 'environmental material' class (http://purl.obolibrary.org/obo/ENVO_00010483). | -| `RNA_presence` | Presence or absence of the 23S, 16S, and 5S rRNA genes and at least 18 tRNAs. This is used for MISAG/MIMAG assembly quality classification. Options: Yes or No. | -| `NCBI_lineage` | NCBI taxonomy lineage of the genome. Can be composted of either numerical IDs or taxon names separated by ";". | +| Column | Required | Description | +| --------------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `sample` | Yes | A unique identifier for this data entry. Must be globally unique within the input dataset (used as `meta.id` throughout the pipeline). | +| `fasta` | Yes | Path to MAG/bin contigs in FASTA format compressed with `gzip`. All names of the FASTA files must be unique to prevent pipeline errors. | +| `accession` | Yes | ENA accession of the run or metagenomic assembly used to generate the MAG/bin. | +| `fastq_1` | Conditional | Path to the read file in FASTQ format used to generate the source metagenomic assembly. Required if `genome_coverage` is not provided. | +| `fastq_2` | No | Path to the second read file in FASTQ format for paired-end data used to generate the source metagenomic assembly. Leave empty for single-end reads. | +| `assembly_software` | Yes | Tool name and version that were used to generate the source metagenomic assembly. | +| `binning_software` | Yes | Binning tool, including version, that was used to generate the bins. | +| `binning_parameters` | Yes | Arguments that were used during binning. | +| `stats_generation_software` | No | Tool, including version, that was used to calculate completeness and contamination. | +| `completeness` | No | Genome completeness value. | +| `contamination` | No | Genome contamination value. | +| `genome_coverage` | Conditional | Estimated average sequencing depth across the genome. If the value is missing, it is computed automatically during pipeline execution when reads are provided. | +| `metagenome` | Yes | Registered metagenome taxonomic identifier or name that matches an existing ENA taxonomy entry. For more details see https://ena-docs.readthedocs.io/en/latest/faq/taxonomy.html | +| `co-assembly` | Yes | Whether a co-assembly strategy was used for the initial metagenomic assembly generation. Options: Yes or No. | +| `broad_environment` | Yes | Broad ecological context of the sample, for example 'marine biome', 'desert biome'. It is recommended to use subclasses of EnvO 'biome' class (http://purl.obolibrary.org/obo/ENVO_00000428) | +| `local_environment` | Yes | Local environmental context of the sample, for example 'tropical dry broadleaf forest biome', 'marine abyssal zone biome'. It is recommended to use EnvO terms which are of smaller spatial grain than your entry for "broad-scale environmental context". | +| `environmental_medium` | Yes | Material displaced by the sample, or the material in which the sample was embedded before sampling, for example 'mucus', 'lake water'. It is recommended to use subclasses of EnvO 'environmental material' class (http://purl.obolibrary.org/obo/ENVO_00010483). | +| `RNA_presence` | No | Presence or absence of the 23S, 16S, and 5S rRNA genes and at least 18 tRNAs. This is used for MISAG/MIMAG assembly quality classification. Options: Yes or No. | +| `NCBI_lineage` | No | NCBI taxonomy lineage of the genome. Can be composted of either numerical IDs or taxon names separated by ";". | > [!NOTE] > More information about envioronment tags can be found at checklists [ERC000050](https://www.ebi.ac.uk/ena/browser/view/ERC000050) for bins and [ERC000047](https://www.ebi.ac.uk/ena/browser/view/ERC000047) for MAGs under the field names "broad-scale environmental context", "local environmental context", and "environmental medium". @@ -93,16 +93,16 @@ assembly_002,data/assembly_002.fasta.gz,,,42.7,ERR011323,MEGAHIT,1.2.9 > [!IMPORTANT] > **Samplesheet column requirements**: All columns shown in the example above must be present in your samplesheet, even if some values are empty. Columns must be in exactly the same order as shown. -| Column | Description | -| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | -| `sample` | A unique identifier for this data entry. Must be globally unique within the input dataset (used as `meta.id` throughout the pipeline). | -| `fasta` | Path to assembly contigs in FASTA format compressed with `gzip`. | -| `fastq_1` | Path to the read file in FASTQ format used to generate the metagenomic assembly. Required if `coverage` is not provided. | -| `fastq_2` | Path to the second read file in FASTQ format for paired-end data used to generate the source metagenomic assembly. Leave empty for single-end reads. | -| `coverage` | Estimated sequencing depth of the assembly. If this value is missing, it is computed automatically during pipeline execution when reads are provided. | -| `run_accession` | ENA run accession for the reads used to generate the metagenomic assembly. Reads must already be submitted to ENA. | -| `assembler` | Name of the assembler software used to generate the assembly. | -| `assembler_version` | Version of the assembler software used to generate the assembly. | +| Column | Required | Description | +| ------------------- | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | +| `sample` | Yes | A unique identifier for this data entry. Must be globally unique within the input dataset (used as `meta.id` throughout the pipeline). | +| `fasta` | Yes | Path to assembly contigs in FASTA format compressed with `gzip`. | +| `fastq_1` | Conditional | Path to the read file in FASTQ format used to generate the metagenomic assembly. Required if `coverage` is not provided. | +| `fastq_2` | No | Path to the second read file in FASTQ format for paired-end data used to generate the source metagenomic assembly. Leave empty for single-end reads. | +| `coverage` | Conditional | Estimated sequencing depth of the assembly. If this value is missing, it is computed automatically during pipeline execution when reads are provided. | +| `run_accession` | Yes | ENA run accession for the reads used to generate the metagenomic assembly. Reads must already be submitted to ENA. | +| `assembler` | Yes | Name of the assembler software used to generate the assembly. | +| `assembler_version` | Yes | Version of the assembler software used to generate the assembly. | An example file is available at [assets/samplesheet_assembly.csv](../assets/samplesheet_assembly.csv). @@ -121,20 +121,20 @@ pacbio_run_001,SAMEA7654321,data/pacbio_reads.fastq.gz,,PACBIO_SMRT,PacBio Seque > [!IMPORTANT] > **Samplesheet column requirements**: All columns shown in the example above must be present in your samplesheet, even if some values are empty. Columns must be in exactly the same order as shown. -| Column | Type | Required | Description | -| ------------------- | --------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `sample` | str | Yes | Unique identifier of this particular data entry. Used as an experiment name. | -| `sample_accession` | str | Yes | ENA sample accession (starting with SAMEA) of the sample used to generate raw reads. | -| `fastq_1` | file path | Yes | Path to forward reads in FASTQ format (optionally gzipped). | -| `fastq_2` | file path | No | Path to reverse reads for paired-end data. Leave empty for single-end reads. | -| `platform` | str | Yes | Sequencing platform. Supported values: `ILLUMINA`, `PACBIO_SMRT`, `OXFORD_NANOPORE`, `ION_TORRENT`, `CAPILLARY`, `DNBSEQ`, `ELEMENT`, `GENAPSYS`, `GENEMIND`, `HELICOS`, `LS454`, `BGISEQ`, `ULTIMA`, `VELA_DIAGNOSTICS`. See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#metadata-validation) for complete list. | -| `instrument` | str | Yes | Sequencer model, e.g. "Illumina HiSeq 2000", "PacBio Sequel", "MinION". | -| `library_source` | str | Yes | Library source type. Options: `GENOMIC`, `METAGENOMIC`, `TRANSCRIPTOMIC`, `METAGENOMIC SINGLE CELL`, `TRANSCRIPTOMIC SINGLE CELL`, `SYNTHETIC`, `VIRAL RNA`, `OTHER`. | -| `library_selection` | str | Yes | Library selection method. Options: `RANDOM`, `PCR`, `RANDOM PCR`, `RT-PCR`, `MF`, `cDNA`, `cDNA_randomPriming`, `cDNA_oligo_dT`, `PolyA`, `Inverse rRNA`, `ChIP`, `MNase`, `DNase`, `Hybrid Selection`, etc. See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#metadata-validation) for complete list. | -| `library_strategy` | str | Yes | Library strategy. Options: `WGS`, `WGA`, `WXS`, `RNA-Seq`, `miRNA-Seq`, `ncRNA-Seq`, `EST`, `Hi-C`, `ATAC-seq`, `WCS`, `RAD-Seq`, `CLONE`, `AMPLICON`, `POOLCLONE`, `etc`. See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#metadata-validation) for complete list. | -| `insert_size` | number | No | Fragment/insert size for paired-end reads (e.g., 500 for 500 bp inserts). Leave empty if not applicable. | -| `library_name` | str | No | Descriptive library name (optional). | -| `description` | str | No | Free-text description of the experiment (optional). | +| Column | Required | Description | +| ------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `sample` | Yes | Unique identifier of this particular data entry. Used as an experiment name. | +| `sample_accession` | Yes | ENA sample accession (starting with SAMEA) of the sample used to generate raw reads. | +| `fastq_1` | Yes | Path to forward reads in FASTQ format (optionally gzipped). | +| `fastq_2` | No | Path to reverse reads for paired-end data. Leave empty for single-end reads. | +| `platform` | Yes | Sequencing platform. Supported values: `ILLUMINA`, `PACBIO_SMRT`, `OXFORD_NANOPORE`, `ION_TORRENT`, `CAPILLARY`, `DNBSEQ`, `ELEMENT`, `GENAPSYS`, `GENEMIND`, `HELICOS`, `LS454`, `BGISEQ`, `ULTIMA`, `VELA_DIAGNOSTICS`. See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#metadata-validation) for complete list. | +| `instrument` | Yes | Sequencer model, e.g. "Illumina HiSeq 2000", "PacBio Sequel", "MinION". | +| `library_source` | Yes | Library source type. Options: `GENOMIC`, `METAGENOMIC`, `TRANSCRIPTOMIC`, `METAGENOMIC SINGLE CELL`, `TRANSCRIPTOMIC SINGLE CELL`, `SYNTHETIC`, `VIRAL RNA`, `OTHER`. | +| `library_selection` | Yes | Library selection method. Options: `RANDOM`, `PCR`, `RANDOM PCR`, `RT-PCR`, `MF`, `cDNA`, `cDNA_randomPriming`, `cDNA_oligo_dT`, `PolyA`, `Inverse rRNA`, `ChIP`, `MNase`, `DNase`, `Hybrid Selection`, etc. See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#metadata-validation) for complete list. | +| `library_strategy` | Yes | Library strategy. Options: `WGS`, `WGA`, `WXS`, `RNA-Seq`, `miRNA-Seq`, `ncRNA-Seq`, `EST`, `Hi-C`, `ATAC-seq`, `WCS`, `RAD-Seq`, `CLONE`, `AMPLICON`, `POOLCLONE`, `etc`. See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#metadata-validation) for complete list. | +| `insert_size` | No | Fragment/insert size for paired-end reads (e.g., 500 for 500 bp inserts). Leave empty if not applicable. | +| `library_name` | No | Descriptive library name (optional). | +| `description` | No | Free-text description of the experiment (optional). | ## Submission study From 6aea0f85bc13d2b711b0e760c6ab220b7e318fd7 Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Fri, 10 Jul 2026 13:07:11 +0100 Subject: [PATCH 14/18] slim Samplesheet descriptions in README to minimum --- README.md | 93 +++++---------------------------------------------- docs/usage.md | 6 ++-- 2 files changed, 11 insertions(+), 88 deletions(-) diff --git a/README.md b/README.md index 1a8fd84..f13b535 100644 --- a/README.md +++ b/README.md @@ -65,42 +65,9 @@ Make sure you update commands above with your authorised credentials. ## Input samplesheets -For detailed descriptions of all samplesheet columns, see the [usage documentation](docs/usage.md#samplesheet-input). +### `mags` and `bins` modes -### `mags` and `bins` modes (`GENOMESUBMIT`) - -The input must follow `assets/schema_input_genome.json`. - -Required columns: - -- `sample` -- `fasta` (must end with `.fa.gz`, `.fasta.gz`, or `.fna.gz`) -- `accession` -- `assembly_software` -- `binning_software` -- `binning_parameters` -- `metagenome` -- `environmental_medium` -- `broad_environment` -- `local_environment` -- `co-assembly` - -At least one of the following must be provided per row: - -- reads (`fastq_1`, optional `fastq_2` for paired-end) -- `genome_coverage` - -Additional supported columns: - -- `stats_generation_software` -- `completeness` -- `contamination` -- `RNA_presence` -- `NCBI_lineage` - -If `genome_coverage`, `stats_generation_software`, `completeness`, `contamination`, `RNA_presence`, or `NCBI_lineage` are missing, the workflow can calculate or infer them when the required inputs are available. See the [Methods documentation](docs/methods.md) for more information on how metadata statistics are obtained. - -Those fields are metadata required for the [genome_uploader](https://github.com/EBI-Metagenomics/genome_uploader) package. +The input must follow [assets/schema_input_genome.json](assets/schema_input_genome.json). See the [corresponding samplesheet section](docs/usage.md#mags-and-bins-modes-genomesubmit) of the usage documentation for the full list of required and optional columns. Example `samplesheet_genomes.csv`: @@ -109,27 +76,9 @@ sample,fasta,accession,fastq_1,fastq_2,assembly_software,binning_software,binnin lachnospira_eligens,data/bin_lachnospira_eligens.fa.gz,SRR24458089,,,spades_v3.15.5,metabat2_v2.6,default,CheckM2_v1.0.1,61.0,0.21,32.07,sediment metagenome,No,marine,cable_bacteria,marine_sediment,No,d__Bacteria;p__Proteobacteria;s__unclassified_Proteobacteria ``` -> [!IMPORTANT] -> **Samplesheet column requirements**: All columns shown in the example above must be present in your samplesheet, even if some values are empty. Columns must be in exactly the same order as shown. - -### `metagenomic_assemblies` mode (`ASSEMBLYSUBMIT`) - -The input must follow `assets/schema_input_assembly.json`. +### `metagenomic_assemblies` mode -Required columns: - -- `sample` -- `fasta` (must end with `.fa.gz`, `.fasta.gz`, or `.fna.gz`) -- `run_accession` -- `assembler` -- `assembler_version` - -At least one of the following must be provided per row: - -- reads (`fastq_1`, optional `fastq_2` for paired-end) -- `coverage` - -If `coverage` is missing and reads are provided, the workflow calculates average coverage with `coverm`. See the [Methods documentation](docs/methods.md) for more information on how coverage is calculated. +The input must follow [assets/schema_input_assembly.json](assets/schema_input_assembly.json). See the [corresponding samplesheet section](docs/usage.md#metagenomic_assemblies-mode-assemblysubmit) of the usage documentation for more details. Example `samplesheet_assembly.csv`: @@ -139,30 +88,9 @@ assembly_1,data/contigs_1.fasta.gz,data/reads_1.fastq.gz,data/reads_2.fastq.gz,, assembly_2,data/contigs_2.fasta.gz,,,42.7,ERR011323,MEGAHIT,1.2.9 ``` -> [!IMPORTANT] -> **Samplesheet column requirements**: All columns shown in the example above must be present in your samplesheet, even if some values are empty. Columns must be in exactly the same order as shown. - -### `reads` mode (`READSUBMIT`) - -The input must follow `assets/schema_input_reads.json`. +### `reads` mode -Required columns: - -- `sample` -- `sample_accession` -- `fastq_1` -- `fastq_2` -- `platform` -- `instrument` -- `library_source` -- `library_selection` -- `library_strategy` - -Optional columns: - -- `insert_size` -- `library_name` -- `description` +The input must follow [assets/schema_input_reads.json](assets/schema_input_reads.json). See the [corresponding samplesheet section](docs/usage.md#reads-mode-readsubmit) of the usage documentation for more details. Example `samplesheet_reads.csv`: @@ -171,9 +99,6 @@ sample,sample_accession,fastq_1,fastq_2,platform,instrument,library_source,libra illumina_run_001,SAMEA1234567,data/reads_R1.fastq.gz,data/reads_R2.fastq.gz,ILLUMINA,Illumina HiSeq 2000,GENOMIC,RANDOM,WGS,500,HiSeq_library_001,Illumina sequencing of sample XYZ ``` -> [!IMPORTANT] -> **Samplesheet column requirements**: All columns shown in the example above must be present in your samplesheet, even if some values are empty. Columns must be in exactly the same order as shown. - ## Usage > [!NOTE] @@ -181,9 +106,7 @@ illumina_run_001,SAMEA1234567,data/reads_R1.fastq.gz,data/reads_R2.fastq.gz,ILLU ### Submission study -All data submitted through this pipeline must be associated with an ENA study (project). You can either pass an accession of your existing study via `--submission_study`or provide a metadata file via `--study_metadata` and the pipeline will register the study with ENA before submitting your data. - -See the [usage documentation](docs/usage.md#submission-study) for more details. +All data submitted through this pipeline must be associated with an ENA study (project). You can either pass an accession of your existing study via `--submission_study`or provide a metadata file via `--study_metadata` and the pipeline will register the study with ENA before submitting your data. See the [usage documentation](docs/usage.md#submission-study) for more details. ### Data privacy @@ -191,7 +114,7 @@ The pipeline supports private submissions. See the [usage documentation](docs/us ### Database setup (`CheckM2` and `CAT_pack`) -The `mags`/`bins` workflow requires databases for completeness/contamination estimation and taxonomy assignment. See [Usage documentation](usage.md) for details. +The `mags`/`bins` workflow requires databases for completeness/contamination estimation and taxonomy assignment. See [Usage documentation](usage.md) for more details. ### Required parameters: diff --git a/docs/usage.md b/docs/usage.md index bc30044..d666aad 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -76,12 +76,12 @@ mag_001,data/mag_001.fasta.gz,SRR24458089,,,SPAdes 3.15.5,MetaBAT2 2.15,default, > [!NOTE] > More information about envioronment tags can be found at checklists [ERC000050](https://www.ebi.ac.uk/ena/browser/view/ERC000050) for bins and [ERC000047](https://www.ebi.ac.uk/ena/browser/view/ERC000047) for MAGs under the field names "broad-scale environmental context", "local environmental context", and "environmental medium". +If `genome_coverage`, `stats_generation_software`, `completeness`, `contamination`, `RNA_presence`, or `NCBI_lineage` are missing, the workflow can calculate or infer them when the required inputs are available. See the [Methods documentation](docs/methods.md) for more information on how metadata statistics are obtained. + ### `metagenomic_assemblies` mode (`ASSEMBLYSUBMIT`) Use this samplesheet structure for metagenomic assembly submission. The input format follows [assets/schema_input_assembly.json](../assets/schema_input_assembly.json). -Provide either read files (`fastq_1`, optionally `fastq_2`) or a `coverage` value for each row. If `coverage` is missing and reads are provided, the workflow calculates average coverage automatically. - Example: ```csv title="samplesheet_assembly.csv" @@ -104,7 +104,7 @@ assembly_002,data/assembly_002.fasta.gz,,,42.7,ERR011323,MEGAHIT,1.2.9 | `assembler` | Yes | Name of the assembler software used to generate the assembly. | | `assembler_version` | Yes | Version of the assembler software used to generate the assembly. | -An example file is available at [assets/samplesheet_assembly.csv](../assets/samplesheet_assembly.csv). +Provide either read files (`fastq_1`, optionally `fastq_2`) or a `coverage` value for each row. If `coverage` is missing and reads are provided, the workflow calculates average coverage with `coverm`. See the [Methods documentation](docs/methods.md) for more information on how coverage is calculated. ### `reads` mode (`READSUBMIT`) From 93940733a3fea8ff3757c12158e985bfc0ec9af5 Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Fri, 10 Jul 2026 13:49:34 +0100 Subject: [PATCH 15/18] Add Table of Contents to usage.md --- docs/usage.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/docs/usage.md b/docs/usage.md index d666aad..995ec2a 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -4,6 +4,23 @@ > _Documentation of pipeline parameters is generated automatically from the pipeline schema and can no longer be found in markdown files._ +## Table of Contents + +- [Introduction](#introduction) +- [Before you start](#before-you-start) +- [Samplesheet input](#samplesheet-input) + - [`mags` and `bins` modes](#mags-and-bins-modes-genomesubmit) + - [`metagenomic_assemblies` mode](#metagenomic_assemblies-mode-assemblysubmit) + - [`reads` mode](#reads-mode-readsubmit) +- [Submission study](#submission-study) +- [Data privacy](#data-privacy) +- [Database preparation (`mags` / `bins` modes)](#database-preparation-mags--bins) +- [Running the pipeline](#running-the-pipeline) +- [Core Nextflow arguments](#core-nextflow-arguments) +- [Custom configuration](#custom-configuration) +- [Running in the background](#running-in-the-background) +- [Nextflow memory requirements](#nextflow-memory-requirements) + ## Introduction `nf-core/seqsubmit` is a Nextflow pipeline for submitting metagenomic assemblies, MAGs, bins, and raw reads to ENA. From cd8d99a23b7b76ce70fde5c03533de786d3ad7e0 Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Fri, 10 Jul 2026 13:54:33 +0100 Subject: [PATCH 16/18] move Submission Study section up in usage.md --- docs/usage.md | 126 +++++++++++++++++++++++++------------------------- 1 file changed, 63 insertions(+), 63 deletions(-) diff --git a/docs/usage.md b/docs/usage.md index 995ec2a..eaef351 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -8,11 +8,11 @@ - [Introduction](#introduction) - [Before you start](#before-you-start) +- [Submission study](#submission-study) - [Samplesheet input](#samplesheet-input) - [`mags` and `bins` modes](#mags-and-bins-modes-genomesubmit) - [`metagenomic_assemblies` mode](#metagenomic_assemblies-mode-assemblysubmit) - [`reads` mode](#reads-mode-readsubmit) -- [Submission study](#submission-study) - [Data privacy](#data-privacy) - [Database preparation (`mags` / `bins` modes)](#database-preparation-mags--bins) - [Running the pipeline](#running-the-pipeline) @@ -46,6 +46,68 @@ nextflow secrets set ENA_WEBIN "Webin-XXX" nextflow secrets set ENA_WEBIN_PASSWORD "XXX" ``` +## Submission study + +All data submitted through this pipeline must be associated with an ENA study (project). You have two options: + +### Option 1 — Use an existing study + +If you already have an ENA study, pass its accession (starting with `PRJ` or `ERP`) via `--submission_study`: + +```bash +--submission_study PRJEB12345 +``` + +You can create a study manually via the [Webin Portal](https://www.ebi.ac.uk/ena/submit/webin/login) and then use the assigned accession here. + +### Option 2 — Register a new study automatically + +Provide a study metadata file via `--study_metadata` and the pipeline will register the study with ENA before submitting your data: + +```bash +--study_metadata study_metadata.json +``` + +The pipeline accepts JSON, CSV, and TSV formats. + +#### JSON formats + +Single study as a flat object: + +```json +{ + "alias": "study-gut-2026", + "study_title": "Gut microbiome study", + "study_abstract": "Characterisation of gut microbial communities" +} +``` + +#### CSV format + +```csv +alias,study_title,study_abstract +study-gut-2026,Gut microbiome study,Characterisation of gut microbial communities +``` + +#### TSV format + +```tsv +alias study_title study_abstract +study-soil-2026 Soil microbiome study Survey of soil microbiota +``` + +#### Study metadata fields + +| Field | Required | Description | +| --------------------- | -------- | ---------------------------------------------------------------------------- | +| `study_title` | Yes | Descriptive title of the study. | +| `alias` | Yes | Unique project alias within your Webin account. Max length is 50 characters. | +| `study_abstract` | No | Free-text abstract describing the study. | +| `study_description` | No | Alternative to `study_abstract`. | +| `project_name` | No | Project name. Defaults to `study_title`. | +| `existing_study_type` | No | ENA study type (e.g. `Metagenomics`, `Other`). | +| `new_study_type` | No | Custom study type. Only used when `existing_study_type` is set to `Other`. | + ## Samplesheet input You will need to create a samplesheet with information about the data entries you would like to process before running the pipeline. Use `--input` parameter to specify its location. It has to be a comma-separated file with the structure defined by the execution `--mode`. @@ -153,68 +215,6 @@ pacbio_run_001,SAMEA7654321,data/pacbio_reads.fastq.gz,,PACBIO_SMRT,PacBio Seque | `library_name` | No | Descriptive library name (optional). | | `description` | No | Free-text description of the experiment (optional). | -## Submission study - -All data submitted through this pipeline must be associated with an ENA study (project). You have two options: - -### Option 1 — Use an existing study - -If you already have an ENA study, pass its accession (starting with `PRJ` or `ERP`) via `--submission_study`: - -```bash ---submission_study PRJEB12345 -``` - -You can create a study manually via the [Webin Portal](https://www.ebi.ac.uk/ena/submit/webin/login) and then use the assigned accession here. - -### Option 2 — Register a new study automatically - -Provide a study metadata file via `--study_metadata` and the pipeline will register the study with ENA before submitting your data: - -```bash ---study_metadata study_metadata.json -``` - -The pipeline accepts JSON, CSV, and TSV formats. - -#### JSON formats - -Single study as a flat object: - -```json -{ - "alias": "study-gut-2026", - "study_title": "Gut microbiome study", - "study_abstract": "Characterisation of gut microbial communities" -} -``` - -#### CSV format - -```csv -alias,study_title,study_abstract -study-gut-2026,Gut microbiome study,Characterisation of gut microbial communities -``` - -#### TSV format - -```tsv -alias study_title study_abstract -study-soil-2026 Soil microbiome study Survey of soil microbiota -``` - -#### Study metadata fields - -| Field | Required | Description | -| --------------------- | -------- | ---------------------------------------------------------------------------- | -| `study_title` | Yes | Descriptive title of the study. | -| `alias` | Yes | Unique project alias within your Webin account. Max length is 50 characters. | -| `study_abstract` | No | Free-text abstract describing the study. | -| `study_description` | No | Alternative to `study_abstract`. | -| `project_name` | No | Project name. Defaults to `study_title`. | -| `existing_study_type` | No | ENA study type (e.g. `Metagenomics`, `Other`). | -| `new_study_type` | No | Custom study type. Only used when `existing_study_type` is set to `Other`. | - ## Data privacy You can reference private ENA data if you have access to it via your Webin account (it was submitted previously under your credentials). To use private data accessions in your submission metadata, specify the `--is_private` flag. The pipeline will then use your provided Webin credentials to fetch the required metadata. From ae6b4627c82977799cfe88f844bfd5a251a02ecc Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Fri, 10 Jul 2026 14:14:50 +0100 Subject: [PATCH 17/18] Expand Requirements to include required pre-submissions --- README.md | 6 ++++-- docs/usage.md | 15 +++++++++------ 2 files changed, 13 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index f13b535..e66d1de 100644 --- a/README.md +++ b/README.md @@ -53,9 +53,11 @@ Currently, the pipeline supports four submission modes, each routed to a dedicat - [Nextflow](https://www.nextflow.io/) `>=25.04.0` - Webin account registered at https://www.ebi.ac.uk/ena/submit/webin/login -- Raw reads used to assemble contigs submitted to [INSDC](https://www.insdc.org/) and associated accessions available +- For metagenomic assemblies submission the raw reads used to generate the assemblies have to be submitted to INSDC/ENA to obtain the corresponding run accessions. You can use the pipeline in mode `reads` to perform this. +- For MAGs or bins submission the raw reads only or raw reads and metagenomic assemblies used to generate the MAGs/bins have to be submitted to INSDC/ENA to obtain the corresponding run or assembly accessions. You can use the pipeline in mode `reads` or `metagenomic_assemblies` to perform this. +- For raw reads submission it is required to register source BioSamples and obtain the sample accessions. -Setup your environment secrets before running the pipeline: +Setup Webin credentials as Nextflow secrets before running the pipeline: `nextflow secrets set ENA_WEBIN "Webin-XXX"` diff --git a/docs/usage.md b/docs/usage.md index eaef351..949e1b4 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -37,14 +37,17 @@ Before running the pipeline, make sure that: - Nextflow `>=25.04.0` is available. - You have a Webin account registered at . -- The raw reads used to generate the submitted assemblies have already been submitted to INSDC/ENA and the relevant accessions are available. +- For metagenomic assemblies submission the raw reads used to generate the assemblies have to be submitted to INSDC/ENA to obtain the corresponding run accessions. You can use the pipeline in mode `reads` to perform this. +- For MAGs or bins submission the raw reads only or raw reads and metagenomic assemblies used to generate the MAGs/bins have to be submitted to INSDC/ENA to obtain the corresponding run or assembly accessions. You can use the pipeline in mode `reads` or `metagenomic_assemblies` to perform this. +- For raw reads submission it is required to register source BioSamples and obtain the sample accessions. -Set your Webin credentials as Nextflow secrets: +Setup Webin credentials as Nextflow secrets before running the pipeline: -```bash -nextflow secrets set ENA_WEBIN "Webin-XXX" -nextflow secrets set ENA_WEBIN_PASSWORD "XXX" -``` +`nextflow secrets set ENA_WEBIN "Webin-XXX"` + +`nextflow secrets set ENA_WEBIN_PASSWORD "XXX"` + +Make sure you update commands above with your authorised credentials. ## Submission study From bea141a3c6b922941b68e823ff2ac93af6d5514e Mon Sep 17 00:00:00 2001 From: Sofia Ochkalova Date: Fri, 10 Jul 2026 16:26:42 +0100 Subject: [PATCH 18/18] Apply James' suggesstions in usage.md --- docs/usage.md | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/docs/usage.md b/docs/usage.md index 949e1b4..ad69cdb 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -153,10 +153,10 @@ mag_001,data/mag_001.fasta.gz,SRR24458089,,,SPAdes 3.15.5,MetaBAT2 2.15,default, | `local_environment` | Yes | Local environmental context of the sample, for example 'tropical dry broadleaf forest biome', 'marine abyssal zone biome'. It is recommended to use EnvO terms which are of smaller spatial grain than your entry for "broad-scale environmental context". | | `environmental_medium` | Yes | Material displaced by the sample, or the material in which the sample was embedded before sampling, for example 'mucus', 'lake water'. It is recommended to use subclasses of EnvO 'environmental material' class (http://purl.obolibrary.org/obo/ENVO_00010483). | | `RNA_presence` | No | Presence or absence of the 23S, 16S, and 5S rRNA genes and at least 18 tRNAs. This is used for MISAG/MIMAG assembly quality classification. Options: Yes or No. | -| `NCBI_lineage` | No | NCBI taxonomy lineage of the genome. Can be composted of either numerical IDs or taxon names separated by ";". | +| `NCBI_lineage` | No | NCBI taxonomy lineage of the genome. Can be composted of either numerical IDs or official NCBI taxon names separated by ";". | > [!NOTE] -> More information about envioronment tags can be found at checklists [ERC000050](https://www.ebi.ac.uk/ena/browser/view/ERC000050) for bins and [ERC000047](https://www.ebi.ac.uk/ena/browser/view/ERC000047) for MAGs under the field names "broad-scale environmental context", "local environmental context", and "environmental medium". +> More information about environment tags can be found at checklists [ERC000050](https://www.ebi.ac.uk/ena/browser/view/ERC000050) for bins and [ERC000047](https://www.ebi.ac.uk/ena/browser/view/ERC000047) for MAGs under the field names "broad-scale environmental context", "local environmental context", and "environmental medium". If `genome_coverage`, `stats_generation_software`, `completeness`, `contamination`, `RNA_presence`, or `NCBI_lineage` are missing, the workflow can calculate or infer them when the required inputs are available. See the [Methods documentation](docs/methods.md) for more information on how metadata statistics are obtained. @@ -210,7 +210,7 @@ pacbio_run_001,SAMEA7654321,data/pacbio_reads.fastq.gz,,PACBIO_SMRT,PacBio Seque | `fastq_1` | Yes | Path to forward reads in FASTQ format (optionally gzipped). | | `fastq_2` | No | Path to reverse reads for paired-end data. Leave empty for single-end reads. | | `platform` | Yes | Sequencing platform. Supported values: `ILLUMINA`, `PACBIO_SMRT`, `OXFORD_NANOPORE`, `ION_TORRENT`, `CAPILLARY`, `DNBSEQ`, `ELEMENT`, `GENAPSYS`, `GENEMIND`, `HELICOS`, `LS454`, `BGISEQ`, `ULTIMA`, `VELA_DIAGNOSTICS`. See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#metadata-validation) for complete list. | -| `instrument` | Yes | Sequencer model, e.g. "Illumina HiSeq 2000", "PacBio Sequel", "MinION". | +| `instrument` | Yes | Sequencer model, e.g. "Illumina HiSeq 2000", "PacBio Sequel", "MinION". See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#permitted-values-for-instrument) | | `library_source` | Yes | Library source type. Options: `GENOMIC`, `METAGENOMIC`, `TRANSCRIPTOMIC`, `METAGENOMIC SINGLE CELL`, `TRANSCRIPTOMIC SINGLE CELL`, `SYNTHETIC`, `VIRAL RNA`, `OTHER`. | | `library_selection` | Yes | Library selection method. Options: `RANDOM`, `PCR`, `RANDOM PCR`, `RT-PCR`, `MF`, `cDNA`, `cDNA_randomPriming`, `cDNA_oligo_dT`, `PolyA`, `Inverse rRNA`, `ChIP`, `MNase`, `DNase`, `Hybrid Selection`, etc. See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#metadata-validation) for complete list. | | `library_strategy` | Yes | Library strategy. Options: `WGS`, `WGA`, `WXS`, `RNA-Seq`, `miRNA-Seq`, `ncRNA-Seq`, `EST`, `Hi-C`, `ATAC-seq`, `WCS`, `RAD-Seq`, `CLONE`, `AMPLICON`, `POOLCLONE`, `etc`. See [ENA documentation](https://ena-docs.readthedocs.io/en/latest/submit/reads/webin-cli.html#metadata-validation) for complete list. | @@ -241,21 +241,19 @@ The `GENOMESUBMIT` workflow uses `CheckM2` and `CAT_pack` that require specializ You can either provide pre-existing databases or let the pipeline prepare them during execution. +As databases preparation can take significant time, we strongly recommend downloading them locally and storing them in a local cache folder for reuse across runs. +In particular the `CAT_pack` database is extremely large and may take a long time to download and compute. + - `CheckM2`: - - provide the path to local database with `--checkm2_db`, otherwise the pipeline downloads version specified with `--checkm2_db_download_id` (by default `14897628`). + - provide the path to local database with `--checkm2_db`, otherwise the pipeline downloads version specified with `--checkm2_db_download_id` (by default Zenodo accession `14897628`). - `CAT_pack`: - provide the path to local database (containing `tax/` and `db/` folders or `tar.gz` archive) with `--cat_db`, otherwise the pipeline constructs version specified with `--cat_db_download_id` (by default `nr`). See [CAT_pack documentation](https://github.com/MGXlab/CAT_pack) and [CheckM2 documentation](https://github.com/chklovski/CheckM2) for more details on usage and creation of databases. -> [!IMPORTANT] -> `CAT_pack` database creation can take significant time. -> -> Reusing an existing database is strongly recommended for repeated runs. -> -> Databases created/downloaded by the pipeline are published under: -> `${params.outdir}/databases/` +Databases created/downloaded by the pipeline are published under: +`/databases/`. In the subsequent pipeline runs it is strongly recommended to reuse them using `--cat_db /databases/cat_pack/` and `--cat_db /databases/checkm2/`. ## Running the pipeline