Tracking waveform analysis & clustering

[mgignac]

I am updating ldmx-sw, here are the details:

• Added ESA DAQ map — electronics (feb, hybrid) translated to Acts surface/geometry
• SiStripWaveformFitProcessor — real-data pulse fitter, address via DAQ map, per-channel pedestal noise
  - recoil.gdml geometry fix — corrected stereo/axial z-order + stereo tilt sign
• Gutted SiStripWaveform and haromized with simulation workflow in FittedSiStripHit. Added noise field.
• cluster de-hardcoding — added real per-channel noise & per-sensor strip counts, MC path unaffected
• Added/updated a number of example configs:, raw_to_measurements (with ability to stop at a given stage), plot_*)
• Loosened the waveform-builder gates from 5σ/4 high, 3σ/5 low and clustering 4/3/4 to loose-A 4σ/3, 2.5σ/3 and clustering 3/2/3 — recovering ~3× the hits and track candidates

What are the issues that this addresses?

Check List

• I successfully compiled ldmx-sw with my developments.
• I read, understood and follow the coding rules.

• I ran my developments and the following shows that they are successful.

[tvami]

For the geom update we really should have
#2115
in, otherwise the CI doesnt run so your change is not being tested

[mgignac]

I still have a bit more work to do before this should go in, so that shouldn't be a problem. I'll wait until #2115 is merged in before removing the draft PR from here.

[mgignac]

/run-validation

[github-actions]

The validation workflow is running here: https://github.com/LDMX-Software/ldmx-sw/actions/runs/33128766145.

[github-actions]

Validation Results

Some validation samples failed! ❌

Sample	Status
cascade_history	✅ PASS
deep_ecal_gun	✅ PASS
eat_signal	✅ PASS
ecal_pn	✅ PASS
hcal	✅ PASS
inclusive	✅ PASS
it_pileup	✅ PASS
kaon_enhanced	✅ PASS
signal	✅ PASS
signal_target_al	✅ PASS
target_genie	✅ PASS
target_pn_lyso	✅ PASS
target_ti_en	✅ PASS
wab_lhe	✅ PASS
reduced_ldmx	❌ FAIL (95 histograms failed KS test) (artifact)

reduced_ldmx:

• 95 plots failed the KS test against gold.
• Text Differences Between Logs (271568 lines differ)
• Log character count differs by 9%: gold=33305648, new=36540055
• Timing for reduced_ldmx: gold=146s, new=143s
• Timing within 2% of gold (tolerance 10%)

cascade_history:

• Text Differences Between Logs (2 lines differ)
• Timing for cascade_history: gold=6224s, new=6216s
• Timing within 0% of gold (tolerance 10%)

deep_ecal_gun:

• Text Differences Between Logs (18 lines differ)
• Timing for deep_ecal_gun: gold=4076s, new=3988s
• Timing within 2% of gold (tolerance 10%)

eat_signal:

• Timing regression for eat_signal: new=1569s vs gold=942s (+66%, tolerance 10%)
• Timing for eat_signal: gold=942s, new=1569s

ecal_pn:

• Text Differences Between Logs (56 lines differ)
• Log character count differs by 0% (gold=28889426, new=28889425); within tolerance
• Timing anomaly for ecal_pn: new=4403s vs gold=6747s (-34%, tolerance 10%)
• Timing for ecal_pn: gold=6747s, new=4403s

hcal:

• Text Differences Between Logs (2 lines differ)
• Timing for hcal: gold=810s, new=819s
• Timing within 1% of gold (tolerance 10%)

inclusive:

• Text Differences Between Logs (34 lines differ)
• Timing for inclusive: gold=6595s, new=6660s
• Timing within 0% of gold (tolerance 10%)

it_pileup:

• Text Differences Between Logs (32 lines differ)
• Timing for it_pileup: gold=562s, new=561s
• Timing within 0% of gold (tolerance 10%)

kaon_enhanced:

• Text Differences Between Logs (40 lines differ)
• Timing for kaon_enhanced: gold=2405s, new=2406s
• Timing within 0% of gold (tolerance 10%)

signal:

• Text Differences Between Logs (38 lines differ)
• Timing for signal: gold=1208s, new=1262s
• Timing within 4% of gold (tolerance 10%)

signal_target_al:

• Text Differences Between Logs (32 lines differ)
• Timing for signal_target_al: gold=923s, new=930s
• Timing within 0% of gold (tolerance 10%)

target_genie:

• Text Differences Between Logs (232 lines differ)
• Timing for target_genie: gold=9818s, new=9412s
• Timing within 4% of gold (tolerance 10%)

target_pn_lyso:

• Text Differences Between Logs (38 lines differ)
• Timing for target_pn_lyso: gold=6137s, new=6347s
• Timing within 3% of gold (tolerance 10%)

target_ti_en:

• Text Differences Between Logs (40 lines differ)
• Timing anomaly for target_ti_en: new=2131s vs gold=2431s (-12%, tolerance 10%)
• Timing for target_ti_en: gold=2431s, new=2131s

wab_lhe:

• Text Differences Between Logs (36 lines differ)
• Timing anomaly for wab_lhe: new=5322s vs gold=6930s (-23%, tolerance 10%)
• Timing for wab_lhe: gold=6930s, new=5322s

[github-actions]

Validation Results

Some validation samples failed! ❌

Sample	Status
cascade_history	✅ PASS
deep_ecal_gun	✅ PASS
eat_signal	✅ PASS
ecal_pn	✅ PASS
hcal	✅ PASS
inclusive	✅ PASS
it_pileup	✅ PASS
kaon_enhanced	✅ PASS
signal	✅ PASS
signal_target_al	✅ PASS
target_genie	✅ PASS
target_pn_lyso	✅ PASS
target_ti_en	✅ PASS
wab_lhe	✅ PASS
reduced_ldmx	❌ FAIL (95 histograms failed KS test) (artifact)

reduced_ldmx:

• 95 plots failed the KS test against gold.
• Text Differences Between Logs (271568 lines differ)
• Log character count differs by 9%: gold=33305648, new=36540055
• Timing regression for reduced_ldmx: new=177s vs gold=146s (+21%, tolerance 10%)
• Timing for reduced_ldmx: gold=146s, new=177s

cascade_history:

• Text Differences Between Logs (2 lines differ)
• Timing for cascade_history: gold=6224s, new=6261s
• Timing within 0% of gold (tolerance 10%)

deep_ecal_gun:

• Text Differences Between Logs (22 lines differ)
• Timing for deep_ecal_gun: gold=4076s, new=4047s
• Timing within 0% of gold (tolerance 10%)

eat_signal:

• Timing regression for eat_signal: new=1520s vs gold=942s (+61%, tolerance 10%)
• Timing for eat_signal: gold=942s, new=1520s

ecal_pn:

• Text Differences Between Logs (24 lines differ)
• Timing for ecal_pn: gold=6747s, new=6721s
• Timing within 0% of gold (tolerance 10%)

hcal:

• Text Differences Between Logs (2 lines differ)
• Timing anomaly for hcal: new=690s vs gold=810s (-14%, tolerance 10%)
• Timing for hcal: gold=810s, new=690s

inclusive:

• Text Differences Between Logs (44 lines differ)
• Timing for inclusive: gold=6595s, new=6131s
• Timing within 7% of gold (tolerance 10%)

it_pileup:

• Text Differences Between Logs (36 lines differ)
• Timing for it_pileup: gold=562s, new=561s
• Timing within 0% of gold (tolerance 10%)

kaon_enhanced:

• Text Differences Between Logs (42 lines differ)
• Timing anomaly for kaon_enhanced: new=2072s vs gold=2405s (-13%, tolerance 10%)
• Timing for kaon_enhanced: gold=2405s, new=2072s

signal:

• Text Differences Between Logs (18 lines differ)
• Timing for signal: gold=1208s, new=1217s
• Timing within 0% of gold (tolerance 10%)

signal_target_al:

• Text Differences Between Logs (22 lines differ)
• Timing for signal_target_al: gold=923s, new=919s
• Timing within 0% of gold (tolerance 10%)

target_genie:

• Text Differences Between Logs (232 lines differ)
• Timing for target_genie: gold=9818s, new=9689s
• Timing within 1% of gold (tolerance 10%)

target_pn_lyso:

• Text Differences Between Logs (32 lines differ)
• Timing for target_pn_lyso: gold=6137s, new=6197s
• Timing within 0% of gold (tolerance 10%)

target_ti_en:

• Text Differences Between Logs (36 lines differ)
• Timing for target_ti_en: gold=2431s, new=2285s
• Timing within 6% of gold (tolerance 10%)

wab_lhe:

• Text Differences Between Logs (22 lines differ)
• Timing for wab_lhe: gold=6930s, new=6805s
• Timing within 1% of gold (tolerance 10%)

[mgignac]

The geometry changes are a bit more involved than I thought, due to how sensors are mapped in TrackingUtils.cxx -- so I'm removing those from this PR and will follow up with one dedicated to that. Core of the issue is that we are recycling the ldmx recoil geometry in the ESA (and future cosmic ray) test stand, but the hardware setup was fundamentally different.

[mgignac]

/run-validation

[github-actions]

The validation workflow is running here: https://github.com/LDMX-Software/ldmx-sw/actions/runs/33181060040.

[github-actions]

Validation Results

All validation samples passed! ✅

Sample	Status
cascade_history	✅ PASS
deep_ecal_gun	✅ PASS
eat_signal	✅ PASS
ecal_pn	✅ PASS
hcal	✅ PASS
inclusive	✅ PASS
it_pileup	✅ PASS
kaon_enhanced	✅ PASS
reduced_ldmx	✅ PASS
signal	✅ PASS
signal_target_al	✅ PASS
target_genie	✅ PASS
target_pn_lyso	✅ PASS
target_ti_en	✅ PASS
wab_lhe	✅ PASS

cascade_history:

• Text Differences Between Logs (2 lines differ)
• Timing for cascade_history: gold=6224s, new=5985s
• Timing within 3% of gold (tolerance 10%)

deep_ecal_gun:

• Text Differences Between Logs (18 lines differ)
• Timing anomaly for deep_ecal_gun: new=3512s vs gold=4076s (-13%, tolerance 10%)
• Timing for deep_ecal_gun: gold=4076s, new=3512s

eat_signal:

• Timing regression for eat_signal: new=1561s vs gold=942s (+65%, tolerance 10%)
• Timing for eat_signal: gold=942s, new=1561s

ecal_pn:

• Text Differences Between Logs (38 lines differ)
• Timing for ecal_pn: gold=6747s, new=6619s
• Timing within 1% of gold (tolerance 10%)

hcal:

• Text Differences Between Logs (2 lines differ)
• Timing for hcal: gold=810s, new=846s
• Timing within 4% of gold (tolerance 10%)

inclusive:

• Text Differences Between Logs (26 lines differ)
• Timing for inclusive: gold=6595s, new=6675s
• Timing within 1% of gold (tolerance 10%)

it_pileup:

• Text Differences Between Logs (38 lines differ)
• Timing for it_pileup: gold=562s, new=570s
• Timing within 1% of gold (tolerance 10%)

kaon_enhanced:

• Text Differences Between Logs (52 lines differ)
• Timing regression for kaon_enhanced: new=3006s vs gold=2405s (+24%, tolerance 10%)
• Timing for kaon_enhanced: gold=2405s, new=3006s

reduced_ldmx:

• Text Differences Between Logs (20 lines differ)
• Timing for reduced_ldmx: gold=146s, new=149s
• Timing within 2% of gold (tolerance 10%)

signal:

• Text Differences Between Logs (34 lines differ)
• Timing for signal: gold=1208s, new=1194s
• Timing within 1% of gold (tolerance 10%)

signal_target_al:

• Text Differences Between Logs (36 lines differ)
• Timing for signal_target_al: gold=923s, new=947s
• Timing within 2% of gold (tolerance 10%)

target_genie:

• Text Differences Between Logs (232 lines differ)
• Timing for target_genie: gold=9818s, new=8824s
• Timing within 10% of gold (tolerance 10%)

target_pn_lyso:

• Text Differences Between Logs (50 lines differ)
• Timing anomaly for target_pn_lyso: new=4862s vs gold=6137s (-20%, tolerance 10%)
• Timing for target_pn_lyso: gold=6137s, new=4862s

target_ti_en:

• Text Differences Between Logs (38 lines differ)
• Timing regression for target_ti_en: new=3169s vs gold=2431s (+30%, tolerance 10%)
• Timing for target_ti_en: gold=2431s, new=3169s

wab_lhe:

• Text Differences Between Logs (22 lines differ)
• Timing for wab_lhe: gold=6930s, new=6806s
• Timing within 1% of gold (tolerance 10%)