fix(rose): preserve treatment metadata in aggregate outputs

.gitignore

@@ -21,6 +21,7 @@ tmp
 x.sh
 WIP
 dryrun.log
+reset_rose_test_outputs.sh
 POOL_CONTROLS_FEATURE.md
 .snakemake
 .vscode

CHANGELOG.md

@@ -1,3 +1,25 @@
+## development version
+
+### New Features
+
+- **Treatment-level merged peak workflow + cross-treatment HOMER/ROSE aggregation**: Added consensus peak merging across treatment replicates (no peak re-calling) and parallel treatment-level annotation workflows. New outputs include `treatment_merged` peak sets for each caller/mode, treatment-level HOMER/AME outputs (`annotation/homer_treatment`), treatment-level ROSE outputs (`annotation/rose_treatment`), and global aggregate files across all treatments for HOMER annotation summaries and ROSE enhancer-gene/gene-enhancer mappings. Existing replicate-level outputs remain unchanged.
+- **ROSE gene mapping outputs for super-enhancers**: Integrated Younglab `ROSE_geneMapper.py` into the ROSE rule so super-enhancer runs now emit `rose_input_SuperEnhancers_ENHANCER_TO_GENE.txt` and `rose_input_SuperEnhancers_GENE_TO_ENHANCER.txt` in each ROSE output folder. Added output-contract coverage for low-peak fallback by writing placeholders when ROSE is skipped. (#248)
+- **Slurm run-state markers and CLI run commentary**: `runmode=run` now manages explicit workdir marker files (`pipeline.running`, `pipeline.completed`, `pipeline.failed`, `pipeline.canceled`) plus `pipeline.status.json` sidecar metadata (state, timestamp, job id, reason). Starting a new slurm run in the same workdir rotates old markers and writes a fresh `pipeline.running`. Submission/runtime transitions are handled directly in launcher/submit script logic, and CLI output now includes a concise run summary + monitor hints. (#250)
+- **Enriched `pipeline.status.json` metadata**: Extended the run-state sidecar with audit and timing fields: `user`, `submission_timestamp_utc`, `start_timestamp_utc`, `duration_seconds`, `exit_code`, `tasks_done`, `tasks_total`, and `snakemake_log`. Duration and task counts are populated by the SLURM job itself; submission timestamp is persisted via a `pipeline.submit_epoch` helper file so it survives the headnode-to-compute-node handoff. Stale sbatch scripts from earlier releases are automatically detected and regenerated. (#250, @kopardev)
+- **Live progress summary in `pipeline.running` during `run`**: The generated SLURM submit script now runs a background monitor that periodically parses Snakemake progress lines and writes a human-readable status block to `pipeline.running` (completed steps, percent complete, remaining steps, timestamp). It includes delayed-log handling when `snakemake.log` is not immediately available after submission, plus clean monitor shutdown when Snakemake exits. (#254, @kopardev)
+- **Advanced scripts sync mode in wrapper**: Added `-m=syncscripts` as an advanced wrapper mode to re-sync `workflow/scripts` into an initialized work directory without running `init` or `reset`. The legacy `rescript` mode remains available as a backward-compatible alias, and script sync now excludes `__pycache__/` directories. (#251, @kopardev)
+
+### Documentation Improvements
+
+- **User guide: treatment-level merged outputs documented**: Added `treatment_merged` peak outputs and new treatment-level annotation directories (`homer_treatment`, `rose_treatment`) to `output.md`, including global aggregate output paths under `peaks/annotation/`.
+- **User guide: `pipeline.status.json` field reference**: Added a table documenting all 18 fields in `pipeline.status.json`, their availability (headnode vs SLURM job), and what each represents. (#250, @kopardev)
+
+### Bug Fixes
+
+- **HOMER annotation: removed unsupported `annotatePeaks.pl -preparsedDir` option**: `annotatePeaks.pl` does not accept `-preparsedDir`, which caused HOMER annotation failures in replicate-level, DEG-level, and treatment-level motif rules. The option is now removed from `annotatePeaks.pl` calls while retained on `findMotifsGenome.pl`, where writable scratch-backed preparsed caching is still supported. (#255, @kopardev)
+- **Treatment-level motif Slurm resources**: Added explicit `homer_motif_treatment` and `ame_motif_enrichment_treatment` entries to the Biowulf cluster configuration so treatment-level HOMER/AME jobs use the intended 120GB, 32-thread, 2-day resource requests instead of falling back to the 40GB, 2-hour defaults. (#252)
+- **ROSE geneMapper schema robustness**: Some ROSE runs emitted `rose_input_SuperEnhancers.table.txt` without trailing `enhancerRank`/`isSuper` columns, causing `ROSE_geneMapper.py` to fail when parsing rank values. The `rose` rule now detects this schema variant and normalizes the table before gene mapping so jobs do not hard-fail on this format difference. (#248)
+
 ## CARLISLE 2.8.0
 
 ### New Features

VERSION

@@ -1 +1 @@
-2.8.0
+2.8.0-dev