Conversation
Normalize ROSE super-enhancer tables when enhancerRank/isSuper columns are missing to avoid geneMapper crashes on schema variants.\n\nIncludes updated output expectations, rule wiring, docs, and changelog notes for issue #248.\n\n⚡ Generated using AI ⚡
- Fix ROSE super-enhancer normalization to use chr(10)/chr(9) instead of escape sequences, ensuring Python 2 compatibility in ROSE container - Add best-effort jobby generation on sbatch submission failure to ensure artifact exists even when Snakemake callbacks don't run - Fix submit script regeneration to detect missing marker logic (not just scheduler flags) and force rewrite for consistency - Set /data/CCBR_Pipeliner/SIFs as default singularity cache when writable, falling back to /data/$USER/.singularity - Improve launcher cache detection logic to handle permission constraints - Update documentation and changelog for state markers and jobby availability Fixes job failures when ROSE normalization script encounters Python 2 interpreter and ensures reliable pipeline state tracking across all submission modes.
Add consensus peak merging across treatment replicates without re-calling peaks, then run HOMER and ROSE on the treatment-level merged peak sets in parallel with the existing replicate-level workflow. Also add global aggregate outputs for treatment-level HOMER and ROSE results and document the new output layout. ⚡ Generated using AI ⚡
Add a documented advanced run mode to sync workflow/scripts into an existing workdir.\nKeep rescript as a backward-compatible alias and exclude __pycache__/ during sync.\n\nRefs #251\n\n⚡ Generated using AI ⚡
- Add log_info/step/ok/warn/error/next helpers for consistent CLI output - Standardize all runtime output blocks (init, dryrun, run, syncscripts, synccluster, runlocal, runslurm) to use INFO/STEP/OK/WARN/NEXT prefixes - Rewrite usage() block: version in header, RUNMODE/WORKDIR sections, ADVANCED ARGUMENTS and OTHER ARGUMENTS, workdir-first arg order - Add syncscripts advanced mode (rsync workflow/scripts, exclude __pycache__/) - Add synccluster advanced mode (cp cluster config into workdir/config/) - Make cluster config file configurable via --cluster-config or CARLISLE_CLUSTER_CONFIG env var (default: cluster_biowulf.yaml) - Switch sbatch to --parsable for reliable job ID capture; replace awk pattern-match extraction with cut -d';' -f1 so SLURM job ID is propagated correctly into pipeline.status.json - Print SLURM job ID in submission confirmation log line - Add -preparsedDir to all annotatePeaks.pl and findMotifsGenome.pl calls (homer_motif, homer_motif_deg, homer_motif_treatment) using lscratch/TMPDIR to avoid read-only HOMER system path error - Replace treatment-merged BED column 4 (support_count) with unique peak ID (merged_peak_NNNNNN); support_count derivable from col 5 - Update peakcalls.smk rule docstring to reflect new column schema Refs #251, #253 ⚡ Generated using AI ⚡
Add a background monitor in the generated submit script to parse Snakemake\nprogress lines and write a readable status summary into pipeline.running.\n\nIncludes delayed snakemake.log handling and clean monitor shutdown.\n\nRefs #254\n\n⚡ Generated using AI ⚡
Remove the unsupported -preparsedDir option from annotatePeaks.pl\ncalls in replicate-level, DEG-level, and treatment-level HOMER rules,\nwhile retaining writable preparsedDir usage for findMotifsGenome.pl.\n\nRefs #255\n\n⚡ Generated using AI ⚡
Add user, submission_timestamp_utc, start_timestamp_utc, duration_seconds, exit_code, tasks_done, tasks_total, and snakemake_log fields to pipeline.status.json. - Outer write_pipeline_state_marker (headnode): persists submission epoch to pipeline.submit_epoch; reads it back for submission_timestamp_utc on subsequent writes; adds user and snakemake_log; duration_seconds/exit_code/tasks_* are null headnode-side. - Inner _write_pipeline_state_marker (sbatch heredoc): reads pipeline.submit_epoch for submission_timestamp_utc; uses _START_EPOCH/_START_TS (set at job start) for start_timestamp_utc and duration_seconds; parses snakemake.log for tasks_done/tasks_total; accepts exit_code as arg 4. - _START_EPOCH/_START_TS are set in the sbatch body before snakemake. - Traps (SIGTERM/SIGINT) now pass job_id and exit_code=130. - All call sites updated to pass exit_code. - REWRITE_SUBMIT_SCRIPT guard added for submission_timestamp_utc to force regeneration of stale sbatch scripts. - docs/user-guide/run.md: updated with full field-reference table. Closes #250 ⚡ Generated using AI ⚡
Move post-2.8.0 entries (treatment-level merged peaks, ROSE gene mapping, Slurm state markers, pipeline.status.json enrichment, live progress, syncscripts, HOMER preparsedDir fix, treatment motif resources) out of CARLISLE 2.8.0 into a new 'development version' section at the top. CARLISLE 2.8.0 now reflects only what was present at the v2.8.0 release commit. Adds changelog entry for pipeline.status.json enrichment (#250). ⚡ Generated using AI ⚡
Update treatment-level ROSE aggregation to pass structured labels (peak caller, control mode, treatment sample, dup status, stitch distance) from Snakemake rule parameters into the aggregation script, and emit these as explicit columns in aggregated outputs. Also move the ROSE geneMapper schema robustness entry (#248) into the development section of CHANGELOG so it is not listed under 2.8.0. ⚡ Generated using AI ⚡
Adds a new 'Understanding Log Output' section to the run guide explaining: - STEP / INFO / OK / WARN / ERROR / NEXT prefix meanings with a table - Annotated example terminal output for carlisle --runmode=run - Description of live progress updates written to pipeline.running every 60s _commit message is ai-generated
…ogress updates Remove false claim that SLURM email notifications are automatically sent for job start, error, and completion — no --mail-type directives exist in the generated submit_script.sbatch. Document the pipeline.running live progress feature: updated every 60 s from logs/snakemake.log, showing steps done/total/pct and a timestamp. Include example output and the cat command for checking it. ⚡ Generated using AI ⚡
Render a Home-only last-updated line and populate it automatically from the page modification timestamp. Refs #258 ⚡ Generated using AI ⚡
Handle unparseable browser lastModified values with fallbacks so docs Home never shows 'date unavailable'. Refs #258 ⚡ Generated using AI ⚡
Use MkDocs build_date_utc in the Home template so the landing page date does not depend on browser JavaScript execution. Refs #258 ⚡ Generated using AI ⚡
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
peak_caller|control_mode|treatment_sample|dup_status|stitch_distance) into the aggregation script.workflow/scripts/_aggregate_rose_treatment.py.peak_callertreatment_samplecontrol_modedup_statusstitch_distanceROSE geneMapper schema robustnesschangelog item (#248) intodevelopment versionso it is not listed under2.8.0.Issues
PR Checklist
(
Strikethroughany points that are not applicable.)Update docs if there are any API changes.CHANGELOG.mdwith a short description of any user-facing changes and reference the PR number. Guidelines: https://keepachangelog.com/en/1.1.0/Generated using AI