Skip to content

CMP-4475: Install OpenShift Virtualization operator for OCP-Virt e2e#78

Open
Vincent056 wants to merge 5 commits into
ComplianceAsCode:mainfrom
Vincent056:cmp-4475-ocpvirt-e2e
Open

CMP-4475: Install OpenShift Virtualization operator for OCP-Virt e2e#78
Vincent056 wants to merge 5 commits into
ComplianceAsCode:mainfrom
Vincent056:cmp-4475-ocpvirt-e2e

Conversation

@Vincent056

@Vincent056 Vincent056 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

What

Adds the setup and plumbing to run the CIS OpenShift Virtualization (cis-vm-extension) CEL profiles e2e on a cluster that doesn't ship CNV.

  • CNV installer (helpers/virtualization.go, gated on -install-virt, default false): creates the openshift-cnv namespace, an OperatorGroup, a Subscription for kubevirt-hyperconverged (channel stable, redhat-operators), waits for the HyperConverged API, creates the HyperConverged CR (retrying to tolerate the operator webhook), and waits until it is Available. Idempotent; leaves existing CNV in place. Called from helpers/Setup.
  • CEL ProfileBundle support: bumps the vendored ComplianceAsCode/compliance-operator API to v1.9.0 (adds ProfileBundle.spec.celContentFile) and sets pb.Spec.CELContentFile for the ocp4 bundle when -cel-content-file is provided, so CEL profiles are parsed. The k8s stack is aligned to match v1.9.0 (k8s.io/* v0.30 -> v0.35.3, controller-runtime v0.20.4 -> v0.23.3) — no harness code changes were needed for the upgrade.
  • Flags + Makefile wiring: -install-virt / -cel-content-file in config; INSTALL_VIRT and CEL_CONTENT_FILE make vars are passed to make e2e-profile and make e2e-profile-remediations.
  • Opt-in live test TestInstallVirtLive (RUN_VIRT_LIVE=1) exercising the installer.

Usage, the proposed CI lane, and full validation details are in a PR comment.

Validation (OCP 4.21 + AWS + CNV)

  • TestInstallVirtLive passed — CNV installs and HyperConverged reaches Available in ~4 min (no metal nodes needed for the operator/HCO config the CEL rules read).
  • With a CEL content image, a ProfileBundle using celContentFile: ocp4-cel-content.yaml parsed VALID; the operator created the cis-vm-extension CEL profile and its scannerType: CEL rules; a ScanSettingBinding ran the scan to completion and produced per-rule ComplianceCheckResults. (Validated with a manually-created ProfileBundle; the typed CELContentFile wiring constructs the same object.)
  • go build / go vet / test-compile pass; gofmt clean. The dep bump is not yet runtime-tested end-to-end (cluster expired) — worth a smoke run on the next cluster before merge.

Follow-ups (not in this PR)

  • Assertion generation for the CEL + mixed rule set (CEL platform + manual who-can + OpenSCAP node).
  • CI lane in openshift/release (e2e-aws-ocp4-cis-vm-extension) — see the PR comment.
  • Apply the scanner RBAC from CMP-4425 in the flow.

Jira: https://issues.redhat.com/browse/CMP-4475

🤖 Generated with Claude Code

Add an optional setup step that installs the OpenShift Virtualization (CNV)
operator and creates the HyperConverged CR before the tests run, so the CIS
OpenShift Virtualization profiles can be scanned on a cluster that does not
already have CNV. Gated by the new -install-virt flag (default false) and
idempotent if CNV is already present.

Also adds an -install-virt and -cel-content-file config plumbing; wiring the
CEL ProfileBundle (spec.celContentFile) is marked as a follow-up TODO pending
a bump of the vendored compliance-operator API to >= v1.9.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@openshift-ci

openshift-ci Bot commented Jul 2, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Vincent056 and others added 3 commits July 7, 2026 02:20
Retry HyperConverged CR creation to tolerate the operator webhook not being
ready right after the CRD appears. Add an opt-in TestInstallVirtLive
(RUN_VIRT_LIVE=1) that exercises the installer against a real cluster.

Validated on an OCP 4.21 cluster: CNV installs and HyperConverged reaches
Available in ~4 minutes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…docs

Expose INSTALL_VIRT and CEL_CONTENT_FILE make variables and pass the
corresponding -install-virt / -cel-content-file flags from the e2e-profile
and e2e-profile-remediations targets, so a CI lane can run the OCP-Virt CEL
profiles (which need CNV installed and a CEL content file).

Add docs/ocp-virt-e2e.md documenting the flags, local run, the CNV install
step, a proposed CI lane, and the on-cluster validation results.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Keep the Makefile wiring; the OCP-Virt e2e usage notes live in the PR
discussion instead of a checked-in doc.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Vincent056

Copy link
Copy Markdown
Collaborator Author

OpenShift Virtualization (CIS VM Extension) e2e — usage & validation

The OCP-Virt profiles (e.g. cis-vm-extension) are CEL profiles that check OpenShift Virtualization resources. Running them e2e differs from the SCAP profiles in two ways, both handled by this PR:

  1. The cluster needs OpenShift Virtualization (CNV) installed. CI clusters (e.g. ipi-aws) do not ship CNV, so the suite installs it during setup.
  2. The profile is delivered via a CEL content file, not the XCCDF datastream, so the ProfileBundle needs spec.celContentFile set.

Flags

Flag Make var Default Purpose
-install-virt INSTALL_VIRT false Install the CNV operator + HyperConverged CR during setup. Idempotent; no-ops if CNV already present.
-cel-content-file CEL_CONTENT_FILE "" Path (inside the content image) to the CEL content YAML, e.g. ocp4-cel-content.yaml. When set, a CEL ProfileBundle is created so CEL profiles are parsed.

Running locally

# 1. Build the content image with the CEL content bundle (in the content checkout):
./build_product ocp4 --datastream --cel-content=ocp4
#    -> build/ssg-ocp4-ds.xml and build/ocp4-cel-content.yaml
#    Package both into an image and push to a registry the cluster can pull.

# 2. Run the OCP-Virt profile e2e (installs CNV itself):
export PROFILE=cis-vm-extension PRODUCT=ocp4
export INSTALL_VIRT=true CEL_CONTENT_FILE=ocp4-cel-content.yaml
make e2e-profile   # go test also needs -content-image=<your image>

What the CNV install step does

installVirtualizationOperator (helpers/virtualization.go), gated on -install-virt:

  1. Creates the openshift-cnv namespace, an OperatorGroup, and a Subscription for kubevirt-hyperconverged (channel stable, source redhat-operators).
  2. Waits for the HyperConverged API to be served.
  3. Creates the HyperConverged CR (retrying to tolerate the operator webhook not being ready immediately) and waits until it reports Available.

Opt-in live test for just this path:

RUN_VIRT_LIVE=1 go test ./helpers -run TestInstallVirtLive -timeout 40m -v

Proposed CI lane (openshift/release, tracked separately)

- as: e2e-aws-ocp4-cis-vm-extension
  steps:
    cluster_profile: quay-aws
    test:
    - as: test
      commands: |
        export PROFILE=cis-vm-extension PRODUCT=ocp4
        export INSTALL_VIRT=true CEL_CONTENT_FILE=ocp4-cel-content.yaml
        make e2e-profile
    workflow: ipi-aws

Validation (OCP 4.21 + AWS)

  • TestInstallVirtLive installs CNV and HyperConverged reaches Available in ~4 min (no metal nodes required for the operator/HCO config the CEL rules read).
  • With a CEL content image, a ProfileBundle using celContentFile: ocp4-cel-content.yaml parses VALID; the operator creates the cis-vm-extension CEL profile and its scannerType: CEL rules; a ScanSettingBinding runs the scan to completion and produces per-rule ComplianceCheckResults.

@Vincent056 Vincent056 marked this pull request as ready for review July 7, 2026 09:29
…tFile

Bump the vendored ComplianceAsCode/compliance-operator API from v1.7.0 to
v1.9.0 (the release that adds ProfileBundle.spec.celContentFile), aligning the
k8s stack to match: k8s.io/* v0.30 -> v0.35.3, controller-runtime v0.20.4 ->
v0.23.3, plus the openshift/api and openshift/client-go replaces from v1.9.0.

With the typed field available, set pb.Spec.CELContentFile directly in
ensureTestProfileBundles for the ocp4 bundle (removing the earlier TODO /
unstructured workaround), so the harness creates a CEL ProfileBundle for the
OCP-Virt profiles.

go build/vet/test-compile pass; the k8s/controller-runtime upgrade required no
code changes in the harness.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@openshift-ci

openshift-ci Bot commented Jul 8, 2026

Copy link
Copy Markdown

@Vincent056: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ocp4-cis c4dbe5e link true /test e2e-aws-ocp4-cis
ci/prow/e2e-aws-openshift-platform-compliance c4dbe5e link true /test e2e-aws-openshift-platform-compliance
ci/prow/e2e-aws-openshift-node-compliance c4dbe5e link true /test e2e-aws-openshift-node-compliance
ci/prow/e2e-aws-rhcos4-moderate c4dbe5e link true /test e2e-aws-rhcos4-moderate
ci/prow/e2e-aws-ocp4-stig c4dbe5e link true /test e2e-aws-ocp4-stig

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant