Rename `Phase2Timing_resources` to `resources` by Parsifal-2045 · Pull Request #2828 · cms-sw/cms-bot

Parsifal-2045 · 2026-08-27T08:44:41Z

Title says it all, some forward-looking changes in preparation for the reworked benchmark scripts cms-patatrack/patatrack-scripts#61

cmsbuild · 2026-08-27T08:45:04Z

A new Pull Request was created by @Parsifal-2045 for branch master.

@akritkbehera, @cmsbuild, @iarspider, @raoatifshad, @smuzaffar can you please review it and eventually sign? Thanks.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.
cms-bot commands are listed here

cmsbuild · 2026-08-27T08:45:05Z

cms-bot internal usage

mmusich · 2026-08-27T08:55:38Z

test parameters:

enable = hlt_p2_timing
pull_request = Rename Phase2Timing_resources to resources cmssw#51766

mmusich · 2026-08-27T08:57:05Z

@cmsbuild, please test

cmsbuild · 2026-08-27T09:36:15Z

Pull request #2828 was updated.

mmusich · 2026-08-27T09:42:33Z

@cmsbuild, please test

mmusich · 2026-08-27T09:42:43Z

@cmsbuild, please test for CMSSW_20_0_X

cmsbuild · 2026-08-27T09:46:26Z

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-78d331/55561/summary.html
COMMIT: 932fe0b
CMSSW: CMSSW_20_0_X_2026-08-26-2300/el8_amd64_gcc13
Additional Tests: HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cms-bot/2828/55561/install.sh to create a dev area with all the needed externals and cmssw changes.

This pull request cannot be automatically merged, could you please rebase it?
You can see the log for git cms-merge-topic here: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-78d331/55561/git-merge-result

mmusich · 2026-08-27T09:47:58Z

@cmsbuild, please test

cmsbuild · 2026-08-27T11:40:22Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-78d331/55563/summary.html
COMMIT: 932fe0b
CMSSW: CMSSW_20_1_X_2026-08-26-2300/el9_amd64_gcc14
Additional Tests: HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cms-bot/2828/55563/install.sh to create a dev area with all the needed externals and cmssw changes.

HLT P2 Timing: chart

Comparison Summary

Summary:

You potentially added 2 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 45
DQMHistoTests: Total histograms compared: 3592269
DQMHistoTests: Total failures: 11
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3592240
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
Checked 195 log files, 163 edm output root files, 45 DQM output files
TriggerResults: no differences found

AMD_MI300X Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 78 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 173739
DQMHistoTests: Total failures: 6986
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 166753
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

AMD_W7900 Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 50 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 173739
DQMHistoTests: Total failures: 8058
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 165681
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

NVIDIA_H100 Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 47 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 173739
DQMHistoTests: Total failures: 6915
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 166824
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

NVIDIA_L4 Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 96 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 173739
DQMHistoTests: Total failures: 8225
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 165514
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

NVIDIA_L40S Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 100 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 173739
DQMHistoTests: Total failures: 9266
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 164473
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

NVIDIA_T4 Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 106 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 173739
DQMHistoTests: Total failures: 9487
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 164252
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

mmusich · 2026-08-27T14:08:45Z

test parameters:

enable = hlt_p2_timing

cmsbuild · 2026-09-02T10:44:02Z

Pull request #2828 was updated.

Parsifal-2045 · 2026-09-02T10:44:06Z

I managed to get in a quick fix, it should follow the better suggestion, thanks!

mmusich · 2026-09-02T10:49:43Z

@cmsbuild, please test

mmusich · 2026-09-02T10:50:08Z

@cmsbuild, please test for CMSSW_20_0_X

smuzaffar · 2026-09-02T11:37:27Z

please test with cms-sw/cmssw#51766

As current 20.1.X tests are running with cmssw change, so I see that https://cmssdt.cern.ch/jenkins/job/ib-run-pr-hlt_p2_timing/528/console is using

++ python3 -c 'from HLTrigger.Configuration.HLT_75e33.services.FastTimerService_cfi import FastTimerService; print(FastTimerService.jsonFileName.value())'
++ sed 's|[.]json$||'
+ RES_PREFIX=Phase2Timing_resources

let rerun 20.1.X tests with cmssw PR

cmsbuild · 2026-09-02T12:38:56Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-78d331/55652/summary.html
COMMIT: 665dd02
CMSSW: CMSSW_20_0_X_2026-09-02-1100/el8_amd64_gcc13
Additional Tests: HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cms-bot/2828/55652/install.sh to create a dev area with all the needed externals and cmssw changes.

HLT P2 Timing: chart

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3248666
DQMHistoTests: Total failures: 12
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3248636
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 191 log files, 159 edm output root files, 44 DQM output files
TriggerResults: no differences found

cmsbuild · 2026-09-02T13:36:36Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-78d331/55653/summary.html
COMMIT: 665dd02
CMSSW: CMSSW_20_1_X_2026-09-01-2300/el9_amd64_gcc14
Additional Tests: HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cms-bot/2828/55653/install.sh to create a dev area with all the needed externals and cmssw changes.

HLT P2 Timing: chart

Comparison Summary

Summary:

You potentially added 1 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 45
DQMHistoTests: Total histograms compared: 3592269
DQMHistoTests: Total failures: 19
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3592232
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
Checked 195 log files, 163 edm output root files, 45 DQM output files
TriggerResults: no differences found

AMD_MI300X Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 75 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 173739
DQMHistoTests: Total failures: 7839
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 165900
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

AMD_W7900 Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 49 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 173739
DQMHistoTests: Total failures: 7543
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 166196
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

NVIDIA_H100 Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 88 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 173739
DQMHistoTests: Total failures: 8612
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 165127
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

NVIDIA_L4 Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 95 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 173739
DQMHistoTests: Total failures: 9780
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 163959
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

NVIDIA_L40S Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 60 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 173739
DQMHistoTests: Total failures: 7673
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 166066
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

NVIDIA_T4 Comparison Summary

Summary:

You potentially removed 1 lines from the logs
Reco comparison results: 67 differences found in the comparisons
DQMHistoTests: Total files compared: 7
DQMHistoTests: Total histograms compared: 173739
DQMHistoTests: Total failures: 8605
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 165134
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
Checked 25 log files, 20 edm output root files, 7 DQM output files
TriggerResults: no differences found

Max Memory Comparisons exceeding threshold NVIDIA_H100

@cms-sw/core-l2 , I found 1 workflow step(s) with memory usage exceeding the error threshold:

Expand to see workflows ...

Error: Workflow 37634.7503_TTbar_14TeV+Run4D127_HLTHeterogeneousValid step2 max memory diff -202.4 exceeds +/- 30.0 MiB

Max Memory Comparisons exceeding threshold NVIDIA_L40S

@cms-sw/core-l2 , I found 1 workflow step(s) with memory usage exceeding the error threshold:

Expand to see workflows ...

Error: Workflow 37634.7503_TTbar_14TeV+Run4D127_HLTHeterogeneousValid step2 max memory diff 231.0 exceeds +/- 30.0 MiB

smuzaffar · 2026-09-02T13:43:12Z

@mmusich , can you please checkif hlt p2 timing tests have generated the correct results/output?

mmusich · 2026-09-02T14:57:05Z

@smuzaffar it looks like things did run correctly for 20_1_X.
For 20_0_X though, despite the chart, logs and resource usage summary are available, the bot flagged an issue in the summary page.

I am not sure what the problem might be.

smuzaffar · 2026-09-02T18:30:36Z

@cmsbuild, please test for CMSSW_20_0_X

@mmusich , I also do not understand the issue with 20.0.X results. Looks like

cms-bot/pr_testing/run-pr-hlt-p2-timing.sh

Lines 43 to 44 in adf8838

    
           echo "ERROR: Failed to copy Generated CSV files" >> ${WORKSPACE}/hlt-p2-timing.log 
        
           echo "ERROR" > $JENKINS_UPLOAD_DIR/hlt-p2-timing/status

code was activated and resultsed in ERROR: Failed to copy Generated CSV files message in log file but in jenkins job I do not see any failure in copying the csv files

14:23:45 + upload_gpu_csvs
14:23:45 + mkdir -p /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/upload/hlt-p2-timing
14:23:45 + cp /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/rundir/logs.Phase2_L1P2GT_HLT/cpu_memory.csv /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/upload/hlt-p2-timing/cpu_memory_ph2_hlt.csv
14:23:45 + cp /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/rundir/logs.Phase2_L1P2GT_HLT/gpu_memory.csv /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/upload/hlt-p2-timing/gpu_memory_ph2_hlt.csv
14:23:45 + cp /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/rundir/logs.Phase2_L1P2GT_HLT/gpu_usage.csv /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/upload/hlt-p2-timing/gpu_usage_ph2_hlt.csv
14:23:45 + cp /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/rundir/logs.Phase2_L1P2GT_HLT_OnCPU/cpu_memory.csv /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/upload/hlt-p2-timing/cpu_memory_ph2_hlt_onCPU.csv
14:23:45 + cp /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/rundir/logs.NGTScouting_L1P2GT_HLT/cpu_memory.csv /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/upload/hlt-p2-timing/cpu_memory_ph2_ngt.csv
14:23:45 + cp /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/rundir/logs.NGTScouting_L1P2GT_HLT/gpu_memory.csv /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/upload/hlt-p2-timing/gpu_memory_ph2_ngt.csv
14:23:45 + cp /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/rundir/logs.NGTScouting_L1P2GT_HLT/gpu_usage.csv /data/cmsbuild/workspace/ib-run-pr-hlt_p2_timing/upload/hlt-p2-timing/gpu_usage_ph2_ngt.csv
14:23:45 + set +o pipefail

let me re-run the tests.

cmsbuild · 2026-09-02T19:58:04Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-78d331/55662/summary.html
COMMIT: 665dd02
CMSSW: CMSSW_20_0_X_2026-09-02-1100/el8_amd64_gcc13
Additional Tests: HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cms-bot/2828/55662/install.sh to create a dev area with all the needed externals and cmssw changes.

HLT P2 Timing: chart

Comparison Summary

Summary:

You potentially added 1 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3248666
DQMHistoTests: Total failures: 12
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3248636
DQMHistoTests: Total skipped: 18
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 191 log files, 159 edm output root files, 44 DQM output files
TriggerResults: no differences found

smuzaffar · 2026-09-02T20:01:42Z

+externals

looks good to go

cmsbuild · 2026-09-02T20:02:06Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @mandrenguyen, @sextonkennedy, @ftenchini (and backports should be raised in the release meeting by the corresponding L2)

Rename Phase2Timing_resources to resources

6e1d2c3

cmsbuild added orp-pending pending-signatures externals-pending tests-pending labels Aug 27, 2026

Parsifal-2045 mentioned this pull request Aug 27, 2026

Rename Phase2Timing_resources to resources cms-sw/cmssw#51766

Merged

cmsbuild added the requires-external label Aug 27, 2026

cmsbuild added tests-started tests-pending and removed tests-pending tests-started labels Aug 27, 2026

cmsbuild added tests-started and removed tests-pending labels Aug 27, 2026

cmsbuild added tests-rejected and removed tests-started labels Aug 27, 2026

cmsbuild added tests-started and removed tests-rejected labels Aug 27, 2026

cmsbuild added tests-approved and removed tests-started labels Aug 27, 2026

cmsbuild added the tests-pending label Sep 2, 2026

cmsbuild added tests-started and removed tests-pending labels Sep 2, 2026

cmsbuild added the requires-external label Sep 2, 2026

cmsbuild added tests-approved and removed tests-started labels Sep 2, 2026

cmsbuild added tests-started and removed requires-external tests-approved labels Sep 2, 2026

cmsbuild added tests-approved and removed tests-started labels Sep 2, 2026

smuzaffar merged commit f2e548a into cms-sw:master Sep 2, 2026
37 checks passed

cmsbuild added fully-signed externals-approved and removed pending-signatures externals-pending labels Sep 2, 2026

Parsifal-2045 deleted the renamePhase2TimingResources branch September 4, 2026 16:55

Uh oh!

Conversation

Parsifal-2045 commented Aug 27, 2026

Uh oh!

cmsbuild commented Aug 27, 2026

Uh oh!

cmsbuild commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mmusich commented Aug 27, 2026

Uh oh!

mmusich commented Aug 27, 2026

Uh oh!

cmsbuild commented Aug 27, 2026

Uh oh!

mmusich commented Aug 27, 2026

Uh oh!

mmusich commented Aug 27, 2026

Uh oh!

cmsbuild commented Aug 27, 2026

Uh oh!

mmusich commented Aug 27, 2026

Uh oh!

cmsbuild commented Aug 27, 2026

Comparison Summary

AMD_MI300X Comparison Summary

AMD_W7900 Comparison Summary

NVIDIA_H100 Comparison Summary

NVIDIA_L4 Comparison Summary

NVIDIA_L40S Comparison Summary

NVIDIA_T4 Comparison Summary

Uh oh!

mmusich commented Aug 27, 2026

Uh oh!

cmsbuild commented Sep 2, 2026

Uh oh!

Parsifal-2045 commented Sep 2, 2026

Uh oh!

mmusich commented Sep 2, 2026

Uh oh!

mmusich commented Sep 2, 2026

Uh oh!

smuzaffar commented Sep 2, 2026

Uh oh!

cmsbuild commented Sep 2, 2026

Comparison Summary

Uh oh!

cmsbuild commented Sep 2, 2026

Comparison Summary

AMD_MI300X Comparison Summary

AMD_W7900 Comparison Summary

NVIDIA_H100 Comparison Summary

NVIDIA_L4 Comparison Summary

NVIDIA_L40S Comparison Summary

NVIDIA_T4 Comparison Summary

Max Memory Comparisons exceeding threshold NVIDIA_H100

Max Memory Comparisons exceeding threshold NVIDIA_L40S

Uh oh!

smuzaffar commented Sep 2, 2026

Uh oh!

mmusich commented Sep 2, 2026

Uh oh!

smuzaffar commented Sep 2, 2026

Uh oh!

cmsbuild commented Sep 2, 2026

Comparison Summary

Uh oh!

smuzaffar commented Sep 2, 2026

Uh oh!

Uh oh!

cmsbuild commented Sep 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cmsbuild commented Aug 27, 2026 •

edited

Loading