Massive update of Python Packages

[smuzaffar]

No description provided.

[smuzaffar]

please test

[cmsbuild]

A new Pull Request was created by @smuzaffar for branch IB/CMSSW_20_1_X/master.

@akritkbehera, @iarspider, @raoatifshad, @smuzaffar can you please review it and eventually sign? Thanks.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.
cms-bot commands are listed here

[cmsbuild]

cms-bot internal usage

[cmsbuild]

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-449e09/55655/summary.html
COMMIT: a8c7f47
CMSSW: CMSSW_20_1_X_2026-09-01-2300/el9_amd64_gcc14
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10821/55655/install.sh to create a dev area with all the needed externals and cmssw changes.

Failed External Build

I found compilation error when building:

++ CMS_EIGEN_CXX_FLAGS='-DEIGEN_DONT_PARALLELIZE -DEIGEN_MAX_ALIGN_BYTES=64'
+++ uname -m
++ '[' x86_64 = aarch64 ']'
+ rm -f /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/config.sub /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/config.guess
+ curl -L -k -s -o /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/config.guess http://cmsrep.cern.ch/cmssw/download/config/config.guess
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.JLs0MY (%build)

RPM build warnings:
Macro expanded in comment on line 569: %{pkginstroot}/${PYTHON3_LIB_SITE_PACKAGES}

Macro expanded in comment on line 570: %{realversion}-src.tgz


[cmsbuild]

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-449e09/55668/summary.html
COMMIT: a8c7f47
CMSSW: CMSSW_20_1_X_2026-09-02-2300/el9_amd64_gcc14
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10821/55668/install.sh to create a dev area with all the needed externals and cmssw changes.

[cmsbuild]

Pull request #10821 was updated.

[smuzaffar]

please test using full cmssw

[cmsbuild]

-1

Failed Tests: UnitTests RelVals AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-449e09/55676/summary.html
COMMIT: f942d27
CMSSW: CMSSW_20_1_X_2026-09-02-2300/el9_amd64_gcc14
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10821/55676/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @cms-sw Update Boost version to 1.92.0 in spec file #10819
• @Kirutheeka7 Updating Phase-2 FastSim Tracker material parameters and workflow cmssw#51604
• @felicepantaleo Truth Graph: Introducing Levels cmssw#51716
• @tvami Drop clamped pixels, bounds-check cluster ids in SiPixelClusterizer cmssw#51753
• @Parsifal-2045 Rename Phase2Timing_resources to resources cmssw#51766
• @bsunanda Phase2-hgx368T Remove cyclic dependency between Geometry/HGCalGeometry and Geometry/CaloTopology cmssw#51800
• @smuzaffar [fmt 12.2] use fmt/format.h instead of heavy fmt/core.h cmssw#51801

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-449e09/55676/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-449e09/55676/git-merge-result

Failed Unit Tests

I found 12 errors in the following unit tests:

---> test test_MC_22_crosscheck had ERRORS
---> test test_MC_23_crosscheck had ERRORS
---> test test_GRunFullOutput had ERRORS
and more ...

Failed RelVals

• 135.4

----- Begin Fatal Exception 03-Sep-2026 18:38:49 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=PATMuonProducer label='patMuons'
Exception Message:
A std::exception was thrown.
Failed to load XGBoost model
----- End Fatal Exception -------------------------------------------------

• 4.53

----- Begin Fatal Exception 03-Sep-2026 18:39:11 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=PATMuonProducer label='patMuons'
Exception Message:
A std::exception was thrown.
Failed to load XGBoost model
----- End Fatal Exception -------------------------------------------------

• 1306.0

----- Begin Fatal Exception 03-Sep-2026 18:39:17 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=PATMuonProducer label='patMuons'
Exception Message:
A std::exception was thrown.
Failed to load XGBoost model
----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

• 136.731
• 1330.0
• 139.001
• 2022.0010001
• 136.793
• 136.874
• 2025.0000002
• 2025.0010001
• 9.0
• 2023.0020001
• 2024.0000001
• 25.0
• 2024.0010001
• 2024.0020001
• 2024.0040001
• 2024.0030001
• 2024.0060001
• 2024.0050001
• 2024.0070001
• 13234.0
• 14034.0
• 12846.0
• 12434.0
• 11634.0
• 12834.0
• 18434.0
• 16834.0
• 37696.0
• 37700.0
• 17034.96
• 37634.0
• 37634.911
• 37834.0

Failed AddOn Tests

• hlt_mc_GRun

----- Begin Fatal Exception 03-Sep-2026 18:37:49 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=PhotonXGBoostProducer label='hltPhotonXGBoostProducer'
Exception Message:
A std::exception was thrown.
Failed to load XGBoost model
----- End Fatal Exception -------------------------------------------------

• hlt_mc_GRun

----- Begin Fatal Exception 03-Sep-2026 18:38:29 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=PhotonXGBoostProducer label='hltPhotonXGBoostProducer'
Exception Message:
A std::exception was thrown.
Failed to load XGBoost model
----- End Fatal Exception -------------------------------------------------

• hlt_data_GRun

----- Begin Fatal Exception 03-Sep-2026 18:36:53 CEST-----------------------
An exception of category 'StdException' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=PhotonXGBoostProducer label='hltPhotonXGBoostProducer'
Exception Message:
A std::exception was thrown.
Failed to load XGBoost model
----- End Fatal Exception -------------------------------------------------

Expand to see more addon errors ...

• hlt_data_GRun

[cmsbuild]

Pull request #10821 was updated.

[smuzaffar]

please test using full cmssw

[smuzaffar]

enable gpu

[cmsbuild]

Pull request #10821 was updated.

[smuzaffar]

please test using full cmssw

[smuzaffar]

please test using full cmssw

[cmsbuild]

Pull request #10821 was updated.

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-449e09/55727/summary.html
COMMIT: 94fb867
CMSSW: CMSSW_20_1_X_2026-09-04-1100/el9_amd64_gcc14
Additional Tests: GPU,AMD_MI300X,AMD_W7900,NVIDIA_H100,NVIDIA_L40S,NVIDIA_T4
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/10821/55727/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @cms-sw Add get_source_type, and use it to check for pLHE workflow #10830
• @sbaldu Update CLUEstering to version 2.12.0 #10831

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-449e09/55727/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-449e09/55727/git-merge-result

Comparison Summary

Summary:

• You potentially removed 172 lines from the logs
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 45
• DQMHistoTests: Total histograms compared: 3592269
• DQMHistoTests: Total failures: 22
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3592229
• DQMHistoTests: Total skipped: 18
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 44 files compared)
• Checked 195 log files, 163 edm output root files, 45 DQM output files
• TriggerResults: no differences found

AMD_MI300X Comparison Summary

Summary:

• You potentially removed 25 lines from the logs
• Reco comparison results: 83 differences found in the comparisons
• DQMHistoTests: Total files compared: 7
• DQMHistoTests: Total histograms compared: 173739
• DQMHistoTests: Total failures: 8751
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 164988
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
• Checked 25 log files, 20 edm output root files, 7 DQM output files
• TriggerResults: no differences found

AMD_W7900 Comparison Summary

Summary:

• You potentially removed 25 lines from the logs
• Reco comparison results: 65 differences found in the comparisons
• DQMHistoTests: Total files compared: 7
• DQMHistoTests: Total histograms compared: 173739
• DQMHistoTests: Total failures: 6360
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 167379
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
• Checked 25 log files, 20 edm output root files, 7 DQM output files
• TriggerResults: no differences found

NVIDIA_H100 Comparison Summary

Summary:

• You potentially removed 25 lines from the logs
• Reco comparison results: 52 differences found in the comparisons
• DQMHistoTests: Total files compared: 7
• DQMHistoTests: Total histograms compared: 173739
• DQMHistoTests: Total failures: 8024
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 165715
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
• Checked 25 log files, 20 edm output root files, 7 DQM output files
• TriggerResults: no differences found

NVIDIA_L40S Comparison Summary

Summary:

• You potentially removed 25 lines from the logs
• Reco comparison results: 84 differences found in the comparisons
• DQMHistoTests: Total files compared: 7
• DQMHistoTests: Total histograms compared: 173739
• DQMHistoTests: Total failures: 10059
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 163680
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
• Checked 25 log files, 20 edm output root files, 7 DQM output files
• TriggerResults: no differences found

NVIDIA_T4 Comparison Summary

Summary:

• You potentially removed 25 lines from the logs
• Reco comparison results: 73 differences found in the comparisons
• DQMHistoTests: Total files compared: 7
• DQMHistoTests: Total histograms compared: 173739
• DQMHistoTests: Total failures: 8591
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 165148
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 6 files compared)
• Checked 25 log files, 20 edm output root files, 7 DQM output files
• TriggerResults: no differences found

[smuzaffar]

+externals

good to go in 20.1.X. some of newer versions of python packages also contain security fixes

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_20_1_X/master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @mandrenguyen, @sextonkennedy, @ftenchini (and backports should be raised in the release meeting by the corresponding L2)