-
Notifications
You must be signed in to change notification settings - Fork 8
Update and restructure docs for v1.0 release #83
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: 1.0.0
Are you sure you want to change the base?
Changes from 9 commits
340f3b2
9692ab3
4452cb8
6bda9c7
9074e16
44c38d9
b0343f4
f7f99b3
36567f4
c57e865
f759e9c
a7f6dd6
81e64bd
6aea0f8
9394073
cd8d99a
ae6b462
bea141a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -80,7 +80,7 @@ Additional supported columns: | |
| - `RNA_presence` | ||
| - `NCBI_lineage` | ||
|
|
||
| If `genome_coverage`, `stats_generation_software`, `completeness`, `contamination`, `RNA_presence`, or `NCBI_lineage` are missing, the workflow can calculate or infer them when the required inputs are available. | ||
| If `genome_coverage`, `stats_generation_software`, `completeness`, `contamination`, `RNA_presence`, or `NCBI_lineage` are missing, the workflow can calculate or infer them when the required inputs are available. See the [Methods documentation](docs/methods.md) for more information on how metadata statistics are obtained. | ||
|
|
||
| Those fields are metadata required for the [genome_uploader](https://github.com/EBI-Metagenomics/genome_uploader) package. | ||
|
|
||
|
|
@@ -111,7 +111,7 @@ At least one of the following must be provided per row: | |
| - reads (`fastq_1`, optional `fastq_2` for paired-end) | ||
| - `coverage` | ||
|
|
||
| If `coverage` is missing and reads are provided, the workflow calculates average coverage with `coverm`. | ||
| If `coverage` is missing and reads are provided, the workflow calculates average coverage with `coverm`. See the [Methods documentation](docs/methods.md) for more information on how coverage is calculated. | ||
|
|
||
| Example `samplesheet_assembly.csv`: | ||
|
|
||
|
|
@@ -277,6 +277,44 @@ Key output locations in `--outdir`: | |
|
|
||
| For full details, see the [output documentation](https://nf-co.re/seqsubmit/output). | ||
|
|
||
| ## Scalability | ||
|
|
||
| Some processes in this pipeline are resource intensive — in particular coverage calculation (`coverm`), quality assessment (`CheckM2`), and taxonomic classification of MAGs/bins (`BAT`). For real life data these steps should be run on an HPC cluster or another compute environment with sufficient CPU, memory, and job scheduling capacity, rather than on a single local machine. | ||
|
|
||
| Depending on the resources available on your machine or cluster, you will likely need to adapt the Nextflow configuration to limit the maximum number of concurrently running jobs and other resource settings (e.g. `max_cpus`, `max_memory`, `max_time`, or `executor.queueSize` in a custom config), so the pipeline doesn't try to claim more resources than are actually available. See the [nf-core configuration documentation](https://nf-co.re/docs/usage/getting_started/configuration) for guidance on writing a custom config for your infrastructure. | ||
|
|
||
| When submitting a large number of records, be aware that ENA recommends limiting submissions to 5000 per day to avoid overfilling their processing queue. If you need to submit more than this, consider splitting your samplesheet and spreading the submission across several days. | ||
|
|
||
| ## Limitations | ||
|
|
||
| nf-core/seqsubmit does not yet support the following data types and scenarios, grouped by the mode they affect: | ||
|
|
||
| #### `mags` and `bins` modes (`GENOMESUBMIT`) | ||
|
|
||
| - **Co-assemblies**: MAGs/bins generated from co-assemblies are not supported. The pipeline currently expects each MAG/bin to be derived from an assembly generated from a single run. | ||
|
|
||
| - **Eukaryotic and viral bins/MAGs**: Submitting eukaryotic or viral genomes that require additional metadata generation is not supported. For such genomes tRNA and rRNA prediction, quality assessment (completeness/contamination estimation) and taxonomy assignment are not implemented. If all the required metadata (`completeness`, `contamination`, `stats_generation_software`, `RNA_presence` and `NCBI_lineage`) is already provided in the samplesheet, eukaryotic and viral bins/MAGs can still be submitted normally. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe we should link issues here? We kind of target to fix those |
||
|
|
||
| - **Submission of a MAG/bin without submitting the assembly or raw reads**: If a MAG/bin cannot be traced back to an assembly accession or a reads accession — i.e. only a sample accession is available — it cannot currently be submitted, since the pipeline relies on that lineage to build the required metadata. | ||
|
|
||
| - **Single-contig MAGs/bins**: MAGs/bins that consist of only one contig cannot be uploaded through this pipeline. ENA classifies these as "chromosomal assemblies", which follow a different submission procedure with different metadata requirements. If you have a single-contig MAG/bin to submit, please contact [ENA support](https://www.ebi.ac.uk/ena/browser/support) for guidance. | ||
|
|
||
| - **Coverage calculation from long reads**: `genome_coverage` calculation via `coverm` has only been tested and validated with short reads (e.g. Illumina). It has not been tested with long reads (PacBio, Nanopore). Exercise caution when processing data generated from long reads, as issues may occur. | ||
|
|
||
| #### `metagenomic_assemblies` mode (`ASSEMBLYSUBMIT`) | ||
|
|
||
| - **Co-assemblies**: Metagenomic co-assemblies are not supported. The pipeline currently expects each assembly to be derived from a single run. | ||
|
|
||
| - **Submission of an assembly without submitting the raw reads**: Assemblies that have no associated reads accession (`run_accession` column in the samplesheet) are not supported, since the pipeline relies on that lineage to build the required metadata. | ||
|
|
||
| - **Single-contig assemblies**: Assemblies that consist of only one contig cannot be uploaded through this pipeline. ENA classifies these as "chromosomal assemblies", which follow a different submission procedure with different metadata requirements. If you have a single-contig assembly to submit, please contact [ENA support](https://www.ebi.ac.uk/ena/browser/support) for guidance. | ||
|
|
||
| - **Coverage calculation from long reads**: `coverage` calculation via `coverm` has only been tested and validated with short reads (e.g. Illumina). It has not been tested with long reads (PacBio, Nanopore). Exercise caution when processing data generated from long reads, as issues may occur. | ||
|
|
||
| #### `reads` mode (`READSUBMIT`) | ||
|
|
||
| - **Reads in non-FASTQ formats**: Only reads provided as FASTQ (`.fastq.gz`) can be submitted. Other formats (e.g. BAM) are not supported as input. | ||
|
|
||
| ## Credits | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we have a quite big readme and need a table of contents in the beggining of file (it was also one of James's suggestions) |
||
|
|
||
| nf-core/seqsubmit was originally written by [Martin Beracochea](https://github.com/mberacochea), [Ekaterina Sakharova](https://github.com/KateSakharova), [Sofia Ochkalova](https://github.com/ochkalova), [Evangelos Karatzas](https://github.com/vagkaratzas) and [Tim Rozday](https://github.com/timrozday-mgnify). | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would point to your PR and say it is in development