Skip to content

OCPBUGS-115309: Re-enable sandboxed-containers extension for 5.0 - #1965

Merged
sdodson merged 1 commit into
openshift:release-5.0from
aaradhak:kata-cont-ar
Sep 15, 2026
Merged

sdodson merged 1 commit into
openshift:release-5.0from
aaradhak:kata-cont-ar

Conversation

@aaradhak

@aaradhak aaradhak commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

KATA-4726 made kata-containers available again for RHCOS 9.8 and 10.2. Restore the sandboxed-containers extension in those definitions so 5.0 builds can ship it.

KATA-4726 made kata-containers available again for RHCOS 9.8 and 10.2.
Restore the sandboxed-containers extension in those definitions so 5.0
builds can ship it.
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci-robot openshift-ci-robot added jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Sep 2, 2026
@openshift-ci-robot

openshift-ci-robot commented Sep 2, 2026

Copy link
Copy Markdown

@aaradhak: This pull request references Jira Issue OCPBUGS-115309, which is invalid:

  • expected the bug to target either version "5.1.0." or "openshift-5.1.0.", but it targets "5.0.0" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

KATA-4726 made kata-containers available again for RHCOS 9.8 and 10.2. Restore the sandboxed-containers extension in those definitions so 5.0 builds can ship it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 2, 2026
@tlbueno

tlbueno commented Sep 2, 2026

Copy link
Copy Markdown
Member

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Sep 2, 2026
@tlbueno

tlbueno commented Sep 2, 2026

Copy link
Copy Markdown
Member

/jira refresh

@openshift-ci-robot

Copy link
Copy Markdown

@tlbueno: This pull request references Jira Issue OCPBUGS-115309, which is invalid:

  • expected the bug to target either version "5.1.0." or "openshift-5.1.0.", but it targets "5.0.0" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@yasminvalim yasminvalim left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/LGTM

@tlbueno

tlbueno commented Sep 3, 2026

Copy link
Copy Markdown
Member

/approve

@tlbueno

tlbueno commented Sep 3, 2026

Copy link
Copy Markdown
Member

/test images

@aaradhak

aaradhak commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

prow/images failure seems to happening in all PRs , it doesnt seem to be related to the changes done here -

  * could not run steps: step rhel-coreos-extensions failed: error occurred handling build rhel-coreos-extensions-amd64: the build rhel-coreos-extensions-amd64 failed after 13m18s with reason DockerBuildFailed: Dockerfile build strategy has failed. 
INFO[2026-09-03T15:45:00Z] Reporting job state 'failed' with reason 'executing_graph:step_failed:building_project_image' 

@ngopalak-redhat

Copy link
Copy Markdown

@aaradhak Is the build issue already being tracked?
This PR as you know is a blocker for 5.0 and important for kata containers
I hit the same failure trying to build a cluster with this PR: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/release-openshift-origin-installer-launch-azure-modern/2095864325007740928
We'd like to test kata with 5.0 and would be great to have this merged soon.
cc: @gkurz

@tlbueno

tlbueno commented Sep 4, 2026

Copy link
Copy Markdown
Member

/test images

@tlbueno

tlbueno commented Sep 4, 2026

Copy link
Copy Markdown
Member

looks like the issue is in the repos. kernel package not found:

dnf --repo=rhel-9.8-baseos,rhel-9.8-appstream,rhel-9.8-early-kernel,rhel-9.8-fast-datapath,rhel-9.8-server-ose-5.0,rhel-9.8-highavailability,rhel-9.8-nfv install --assumeyes --nobest --downloadonly --setopt=skip_if_unavailable=True --destdir=/usr/share/rpm-ostree/extensions/ libreswan NetworkManager-libreswan openvswitch3.5-ipsec krb5-workstation libkadm5 kernel kernel-core kernel-modules kernel-modules-core kernel-modules-extra kernel-devel kernel-headers kernel-rt-core kernel-rt-modules kernel-rt-modules-core kernel-rt-modules-extra kernel-rt-devel kata-containers sysstat pacemaker pcs fence-agents-all usbguard
Updating Subscription Management repositories.
Unable to read consumer identity
This system is not registered with an entitlement server. You can use subscription-manager to register.
rhel-9.8-baseos                                 309 MB/s | 143 MB     00:00    
rhel-9.8-appstream                              237 MB/s | 109 MB     00:00    
rhel-9.8-nfv                                    221 MB/s | 123 MB     00:00    
rhel-9.8-highavailability                        22 MB/s | 3.4 MB     00:00    
rhel-9.8-fast-datapath                          4.1 MB/s | 700 kB     00:00    
rhel-9.8-server-ose-5.0                          17 MB/s | 6.4 MB     00:00    
rhel-9.8-early-kernel                           190 MB/s | 6.4 MB     00:00    
Package kernel-5.14.0-687.45.1.el9_8.x86_64 is already installed.
Package kernel-core-5.14.0-687.45.1.el9_8.x86_64 is already installed.
Package kernel-modules-5.14.0-687.45.1.el9_8.x86_64 is already installed.
Package kernel-modules-core-5.14.0-687.45.1.el9_8.x86_64 is already installed.
Package kernel-modules-extra-5.14.0-687.45.1.el9_8.x86_64 is already installed.
All matches were filtered out by exclude filtering for argument: kernel-devel
All matches were filtered out by exclude filtering for argument: kernel-headers
All matches were filtered out by exclude filtering for argument: kernel-rt-core
All matches were filtered out by exclude filtering for argument: kernel-rt-devel
Error: Unable to find a match: kernel-devel kernel-headers kernel-rt-core kernel-rt-devel

@sdodson

sdodson commented Sep 10, 2026

Copy link
Copy Markdown
Member

/test images

@sdodson

sdodson commented Sep 10, 2026

Copy link
Copy Markdown
Member

/jira refresh

@openshift-ci-robot

Copy link
Copy Markdown

@sdodson: This pull request references Jira Issue OCPBUGS-115309, which is invalid:

  • expected the bug to be in one of the following states: NEW, ASSIGNED, POST, but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@sdodson

sdodson commented Sep 10, 2026

Copy link
Copy Markdown
Member

@tlbueno New error

Error: Unknown repo: 'rhel-10.2-server-ose-5.1'
error: build error: building at STEP "RUN --mount=type=bind,target=/run/src --mount=type=secret,id=yumrepos,target=/run/src/secret.repo /run/src/build-node-image.sh": while running runtime: exit status 1

Same for rhel-9.8-server-ose-5.1.

@aaradhak

Copy link
Copy Markdown
Contributor Author

ci/prow/images is failing in the shared rhel-coreos-extensions image build, not because of the changes in this PR. Recent failures show both missing kernel-devel packages due to exclude filtering and most recently - Unknown repo: rhel-10.2-server-ose-5.1 / rhel-9.8-server-ose-5.1. This appears to be a shared yum repository/image-build configuration issue affecting multiple PRs.

@aaradhak

Copy link
Copy Markdown
Contributor Author

@sdodson can we override the CI failure if the error is not related to the changes made here?

@aaradhak

Copy link
Copy Markdown
Contributor Author

/jira refresh

@openshift-ci-robot

Copy link
Copy Markdown

@aaradhak: This pull request references Jira Issue OCPBUGS-115309, which is invalid:

  • expected the bug to be in one of the following states: NEW, ASSIGNED, POST, but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label Sep 10, 2026
@aaradhak

aaradhak commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

I updated CI to use the rhel-10.2-server-ose-5.1 and rhel-9.8-server-ose-5.1 repos in openshift/release#84660.

I'm trying to land #1964 so master tracks OCP 5.1 now, but I'm seeing a CI failure there that was seen in this PR as well.

If #1964 lands, then master becomes 5.1, so I might have to target this PR to release-5.0 branch

Yes, the CI error is affecting all the PRs in this repo

@tlbueno

tlbueno commented Sep 11, 2026

Copy link
Copy Markdown
Member

I think there are 2 issues now.

One is due the 5.1 branching and the 2nd commit of this PR which I am not sure if that is correct to fetch 5.1 as optional url inside 5.0 branch (the the branching still not completed as far as I know).
@marmijo could have more info about that.

The original issue was relate to kernel-rt packages. Looks like the kernel version pinned is 5.14.0-687.47.1.el9_8 based on the base image but on this version seams to do not exist in rhel-9.8-nfv repo (I can't access the CDN but looking on https://access.redhat.com/downloads/content/kernel-rt-core/6.12.0-264.2.1.el10_3/x86_64/fd431d51/package it seams to be not released yet). So, I suppose that should be then available in the rhel-9.8-early-kernel but I can't access it directly (https://openshift-mirror-list.ci-systems.workers.dev/enterprise/reposync/5.0/rhel-9-server-ose-rpms/) but looking into the plashet (https://ocp-artifacts.engineering.redhat.com/pub/RHOCP/plashets/5.0/stream/el9/latest/x86_64/os/Packages/) it has the kernel version but do not have the kernel-rt. So, I think that is the missing part. the kernel package version 5.14.0-687.47.1.el9_8 on brew has the early-kernel-candidate as well as rhaos-5.0-rhel-9-candidate.
@sdodson @joepvd, wdyt?

@aaradhak

Copy link
Copy Markdown
Contributor Author

I have removed the second commit as I was trying to see what would happen if I fetch 5.1 as optional url inside 5.0 branch

I think there are 2 issues now.

One is due the 5.1 branching and the 2nd commit of this PR which I am not sure if that is correct to fetch 5.1 as optional url inside 5.0 branch (the the branching still not completed as far as I know). @marmijo could have more info about that.

The original issue was relate to kernel-rt packages. Looks like the kernel version pinned is 5.14.0-687.47.1.el9_8 based on the base image but on this version seams to do not exist in rhel-9.8-nfv repo (I can't access the CDN but looking on https://access.redhat.com/downloads/content/kernel-rt-core/6.12.0-264.2.1.el10_3/x86_64/fd431d51/package it seams to be not released yet). So, I suppose that should be then available in the rhel-9.8-early-kernel but I can't access it directly (https://openshift-mirror-list.ci-systems.workers.dev/enterprise/reposync/5.0/rhel-9-server-ose-rpms/) but looking into the plashet (https://ocp-artifacts.engineering.redhat.com/pub/RHOCP/plashets/5.0/stream/el9/latest/x86_64/os/Packages/) it has the kernel version but do not have the kernel-rt. So, I think that is the missing part. the kernel package version 5.14.0-687.47.1.el9_8 on brew has the early-kernel-candidate as well as rhaos-5.0-rhel-9-candidate. @sdodson @joepvd, wdyt?

@sdodson

sdodson commented Sep 11, 2026

Copy link
Copy Markdown
Member

@tlbueno Those are directories based on the build-arch tuple I guess. Inside them are the actual RPMs, ie: https://ocp-artifacts.engineering.redhat.com/pub/RHOCP/plashets/5.0/stream/el9/latest/x86_64/os/Packages/kernel-5.14.0-687.47.1.el9_8__x86_64__release4%2Crelease2%2Cima/kernel-rt-5.14.0-687.47.1.el9_8.x86_64.rpm

And there's a 5.1 repo with the same content too. Are those enabled for these builds?

@tlbueno

tlbueno commented Sep 11, 2026

Copy link
Copy Markdown
Member

@tlbueno Those are directories based on the build-arch tuple I guess. Inside them are the actual RPMs, ie: https://ocp-artifacts.engineering.redhat.com/pub/RHOCP/plashets/5.0/stream/el9/latest/x86_64/os/Packages/kernel-5.14.0-687.47.1.el9_8__x86_64__release4%2Crelease2%2Cima/kernel-rt-5.14.0-687.47.1.el9_8.x86_64.rpm

And there's a 5.1 repo with the same content too. Are those enabled for these builds?

Oh, I didn't realize they are directories. :-/
I see it as enabled. Need to dig a bit more.

@tlbueno

tlbueno commented Sep 11, 2026

Copy link
Copy Markdown
Member

/test images

@tlbueno

tlbueno commented Sep 11, 2026

Copy link
Copy Markdown
Member
dnf repoquery --repofrompath=temp-repo,https://ocp-artifacts.engineering.redhat.com/pub/RHOCP/plashets/5.0/stream/el9/latest/x86_64/os --repo=temp-repo kernel-rt
Updating and loading repositories:
Repositories loaded.
kernel-rt-0:5.14.0-687.46.1.el9_8.x86_64
kernel-rt-0:5.14.0-687.47.1.el9_8.x86_64

@aaradhak

aaradhak commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Meanwhile, I have reached out to the ART team to check if they can enable the matching kernel-RT packages

@aaradhak

aaradhak commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

/retest

2 similar comments
@aaradhak

Copy link
Copy Markdown
Contributor Author

/retest

@aaradhak

Copy link
Copy Markdown
Contributor Author

/retest

@aaradhak
aaradhak changed the base branch from master to release-5.0 September 11, 2026 17:47
@openshift-ci-robot

Copy link
Copy Markdown

@aaradhak: This pull request references Jira Issue OCPBUGS-115309, which is invalid:

  • expected Jira Issue OCPBUGS-115309 to depend on a bug targeting a version in 5.1.0 and in one of the following states: MODIFIED, ON_QA, VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but no dependents were found

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

KATA-4726 made kata-containers available again for RHCOS 9.8 and 10.2. Restore the sandboxed-containers extension in those definitions so 5.0 builds can ship it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@aaradhak

Copy link
Copy Markdown
Contributor Author

/test images

@openshift-ci

openshift-ci Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

@aaradhak: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/okd-scos-images e17b57a link true /test okd-scos-images

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@sdodson

sdodson commented Sep 15, 2026

Copy link
Copy Markdown
Member

/test images

@tlbueno

tlbueno commented Sep 15, 2026

Copy link
Copy Markdown
Member

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Sep 15, 2026
@openshift-ci

openshift-ci Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aaradhak, tlbueno, yasminvalim

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [aaradhak,tlbueno,yasminvalim]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@aaradhak

Copy link
Copy Markdown
Contributor Author

@sdodson Can I get the backport-risk assessed and jira/valid-bug labels added here

@sdodson

sdodson commented Sep 15, 2026

Copy link
Copy Markdown
Member

@aaradhak Does this change also affect 5.1 because we haven't forked yet or what's the strategy to get this change into 5.1 as well?

@aaradhak

Copy link
Copy Markdown
Contributor Author

@sdodson Yes, this change should apply to 5.1 as well. I am keeping this PR targeted to release-5.0 as it is the 5.0 fix and its CI is now passing.
Once #1964 lands and master transitions to 5.1, I will apply the same changes to master in a follow-up PR or by cherry-picking the commit.

@sdodson sdodson added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Sep 15, 2026
@sdodson

sdodson commented Sep 15, 2026

Copy link
Copy Markdown
Member

/tide refresh

@sdodson

sdodson commented Sep 15, 2026

Copy link
Copy Markdown
Member

The bots seem to be on lunch break.

@sdodson
sdodson merged commit 7324ccd into openshift:release-5.0 Sep 15, 2026
2 of 3 checks passed
@openshift-ci-robot

Copy link
Copy Markdown

@aaradhak: Jira Issue OCPBUGS-115309 is in an unrecognized state (ON_QA) and will not be moved to the MODIFIED state.

Details

In response to this:

KATA-4726 made kata-containers available again for RHCOS 9.8 and 10.2. Restore the sandboxed-containers extension in those definitions so 5.0 builds can ship it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants