Add SSH-based BMC reset fallback in BMCReconciler - #713
Merged
Conversation
stefanhipfel
changed the title
stefanhipfel
marked this pull request as ready for review
February 27, 2026 12:51
Contributor
📝 WalkthroughWalkthroughThe BMC reconciler now falls back to timed SSH resets after eligible Redfish failures. It adds reset annotations, error classification, timeout configuration, reset status updates, and tests for success, failure, and duplicate prevention. ChangesBMC reset fallback
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@internal/bmcutils/bmcutils.go`:
- Around line 226-229: The SSH client config currently uses
ssh.InsecureIgnoreHostKey() which disables host key verification; update the SSH
setup in internal/bmcutils (the code that builds the ssh.ClientConfig with User,
Auth, HostKeyCallback, Timeout) to follow the secure fallback pattern used in
cmd/metalctl/app/console.go: attempt to create a knownhosts callback via
knownhosts.New(knownHostsPath) and, if that fails, fall back to
ssh.InsecureIgnoreHostKey(); set HostKeyCallback to the resulting callback and
handle the error path cleanly (import golang.org/x/crypto/ssh/knownhosts and
ensure any returned error is logged/handled as in the reference).
In `@internal/controller/bmc_controller.go`:
- Around line 503-504: The SSH fallback goroutine is started with
context.Background() (see resetBMCViaSSH usage) which detaches it from the
controller lifecycle; change it to use the reconciliation/controller-scoped
context (the ctx passed into the reconcile loop or a derived context) instead of
context.Background(), and ensure the goroutine has a bounded lifetime (e.g.,
context.WithTimeout or use the controller stop context) and checks ctx.Done()
inside resetBMCViaSSH so it is cancellable; also make the operation idempotent
by guarding re-entrant runs (e.g., set/read a BMC status/annotation or in-flight
marker before launching) so repeated reconciles (lines around resetBMCViaSSH and
513-552) won’t trigger uncontrolled concurrent resets.
- Around line 495-511: The resetBMC block can lose the original triggering error
(bmcClient may be nil and err nil), and it marks the BMC as pending even when no
reset was started; fix resetBMC so it preserves and uses the original error
value for classification and logging (don’t rely on a possibly-nil local err),
only call r.resetBMCViaSSH and set metalv1alpha1.BMCStatePending when a reset
was actually initiated, and return the original error (or a wrapped version)
when no reset occurs. Concretely: in resetBMC capture the upstream error (e.g.
connectionErr/origErr) and use that instead of err for the httpErr type
assertion (schemas.Error) and status-code checks, guard the goroutine +
updateBMCState call so BMCStatePending is set only on successful start of
r.resetBMCViaSSH, and ensure returned errors.Join or fmt.Errorf wraps the
preserved original error so callers see the real cause.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
Makefilecmd/bmctools/cmds/cmds.gocmd/bmctools/cmds/reset.gocmd/bmctools/main.gointernal/bmcutils/bmcutils.gointernal/controller/bmc_controller.go
💤 Files with no reviewable changes (4)
- Makefile
- cmd/bmctools/cmds/reset.go
- cmd/bmctools/main.go
- cmd/bmctools/cmds/cmds.go
Contributor
There was a problem hiding this comment.
♻️ Duplicate comments (2)
internal/controller/bmc_controller.go (2)
506-510:⚠️ Potential issue | 🟠 MajorTie async SSH reset to controller lifecycle context, not plain background context.
The goroutine is timeout-bounded, but
context.Background()still detaches it from controller shutdown/cancellation semantics.As per coding guidelines
internal/controller/**/*_controller.go: "Implement idempotent reconciliation logic - safe to run the same reconciliation multiple times without side effects".🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal/controller/bmc_controller.go` around lines 506 - 510, The goroutine uses context.Background() which detaches the SSH reset from the controller's lifecycle; change the timeout parent to the controller/reconciliation context (e.g., use the existing ctx or r.ctx passed into the reconcile function) by creating resetCtx with context.WithTimeout(ctx, 5*time.Minute) so resetBMCViaSSH(resetCtx, bmcObj) is cancelled if the controller/reconcile is stopped; keep the defer cancel() and the async invocation of resetBMCViaSSH as-is to preserve timeout behavior.
137-140:⚠️ Potential issue | 🔴 CriticalPreserve the triggering error and mark reset state only after reset initiation.
resetBMCcurrently setsReset=Trueup front, then for nil client / 4xx / unknown paths it can return without any reset actually starting. That leaves state inconsistent and can block future auto-resets becauseshouldResetBMCsees reset already in progress. It also drops the original caller error context from Line 137-140.Proposed fix
-func (r *BMCReconciler) resetBMC(ctx context.Context, bmcObj *metalv1alpha1.BMC, bmcClient bmc.BMC, reason, message string) error { +func (r *BMCReconciler) resetBMC(ctx context.Context, bmcObj *metalv1alpha1.BMC, bmcClient bmc.BMC, triggerErr error, reason, message string) error { log := ctrl.LoggerFrom(ctx) - if err := r.updateConditions(ctx, bmcObj, true, bmcResetConditionType, corev1.ConditionTrue, reason, message); err != nil { - return fmt.Errorf("failed to set BMC resetting condition: %w", err) - } - var err error + err := triggerErr if bmcClient != nil { if err = bmcClient.ResetManager(ctx, bmcObj.Spec.BMCUUID, schemas.GracefulRestartResetType); err == nil { + if err := r.updateConditions(ctx, bmcObj, true, bmcResetConditionType, corev1.ConditionTrue, reason, message); err != nil { + return fmt.Errorf("failed to set BMC resetting condition: %w", err) + } log.Info("Successfully reset BMC via Redfish", "BMC", bmcObj.Name) return r.updateBMCStateToPending(ctx, bmcObj) } log.Error(err, "Could not reset BMC via Redfish", "BMC", bmcObj.Name) } - ... - return errors.Join(r.updateBMCStateToPending(ctx, bmcObj), fmt.Errorf("cannot reset bmc: client unavailable")) + var httpErr *schemas.Error + if errors.As(err, &httpErr) && httpErr.HTTPReturnedStatusCode >= 500 && httpErr.HTTPReturnedStatusCode < 600 { + if err := r.updateConditions(ctx, bmcObj, true, bmcResetConditionType, corev1.ConditionTrue, reason, message); err != nil { + return fmt.Errorf("failed to set BMC resetting condition: %w", err) + } + // launch SSH fallback... + return r.updateBMCStateToPending(ctx, bmcObj) + } + _ = r.updateConditions(ctx, bmcObj, false, bmcResetConditionType, corev1.ConditionFalse, bmcUnknownErrorReason, "BMC reset did not start") + return fmt.Errorf("cannot reset bmc: %w", err) }As per coding guidelines
internal/controller/**/*_controller.go: "Implement idempotent reconciliation logic - safe to run the same reconciliation multiple times without side effects".Also applies to: 483-521
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal/controller/bmc_controller.go` around lines 137 - 140, The code currently sets the BMC Reset condition inside resetBMC before actually starting a reset, which can leave state inconsistent and lose the original triggering error; change resetBMC so it does not modify the BMC status up-front but instead returns any error (preserving/wrapping the original error) to the caller, and move the status update (setting Reset=True with bmcAutoResetReason/bmcAutoResetMessage) into the caller only after reset initiation succeeds; update shouldResetBMC usage accordingly and apply the same pattern to the other reset invocation sites in this file so reset state is only marked after a successful reset start and errors are preserved.
🧹 Nitpick comments (1)
internal/controller/bmc_controller.go (1)
471-479: Re-fetch the BMC object before patching Pending state.
updateBMCStateToPendingpatches status from the in-memory object without a fresh read. This increases conflict risk during concurrent status updates.Proposed refactor
func (r *BMCReconciler) updateBMCStateToPending(ctx context.Context, bmcObj *metalv1alpha1.BMC) error { - if bmcObj.Status.State == metalv1alpha1.BMCStatePending { + current := &metalv1alpha1.BMC{} + if err := r.Get(ctx, client.ObjectKeyFromObject(bmcObj), current); err != nil { + return fmt.Errorf("failed to fetch BMC before patching state: %w", err) + } + if current.Status.State == metalv1alpha1.BMCStatePending { return nil } - bmcBase := bmcObj.DeepCopy() - bmcObj.Status.State = metalv1alpha1.BMCStatePending - if err := r.Status().Patch(ctx, bmcObj, client.MergeFrom(bmcBase)); err != nil { + bmcBase := current.DeepCopy() + current.Status.State = metalv1alpha1.BMCStatePending + if err := r.Status().Patch(ctx, current, client.MergeFrom(bmcBase)); err != nil { return fmt.Errorf("failed to patch BMC state to Pending: %w", err) } + bmcObj.Status.State = current.Status.State return nil }As per coding guidelines
internal/controller/**/*_controller.go: "Always re-fetch objects before updating them in reconciliation to avoid conflicts - callr.Get(ctx, req.NamespacedName, obj)beforer.Update".🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal/controller/bmc_controller.go` around lines 471 - 479, The updateBMCStateToPending function is patching status from the existing in-memory bmcObj which can cause conflicts; before creating bmcBase and calling r.Status().Patch, re-fetch the latest BMC from the API (e.g., newBMC := &metalv1alpha1.BMC{} and r.Get(ctx, client.ObjectKeyFromObject(bmcObj), newBMC)), then deep-copy that fresh object (use newBMC.DeepCopy() as the MergeFrom base), set newBMC.Status.State = metalv1alpha1.BMCStatePending and call r.Status().Patch(ctx, newBMC, client.MergeFrom(baseCopy)) to ensure you patch against the latest resource and reduce conflicts in updateBMCStateToPending.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@internal/controller/bmc_controller.go`:
- Around line 506-510: The goroutine uses context.Background() which detaches
the SSH reset from the controller's lifecycle; change the timeout parent to the
controller/reconciliation context (e.g., use the existing ctx or r.ctx passed
into the reconcile function) by creating resetCtx with context.WithTimeout(ctx,
5*time.Minute) so resetBMCViaSSH(resetCtx, bmcObj) is cancelled if the
controller/reconcile is stopped; keep the defer cancel() and the async
invocation of resetBMCViaSSH as-is to preserve timeout behavior.
- Around line 137-140: The code currently sets the BMC Reset condition inside
resetBMC before actually starting a reset, which can leave state inconsistent
and lose the original triggering error; change resetBMC so it does not modify
the BMC status up-front but instead returns any error (preserving/wrapping the
original error) to the caller, and move the status update (setting Reset=True
with bmcAutoResetReason/bmcAutoResetMessage) into the caller only after reset
initiation succeeds; update shouldResetBMC usage accordingly and apply the same
pattern to the other reset invocation sites in this file so reset state is only
marked after a successful reset start and errors are preserved.
---
Nitpick comments:
In `@internal/controller/bmc_controller.go`:
- Around line 471-479: The updateBMCStateToPending function is patching status
from the existing in-memory bmcObj which can cause conflicts; before creating
bmcBase and calling r.Status().Patch, re-fetch the latest BMC from the API
(e.g., newBMC := &metalv1alpha1.BMC{} and r.Get(ctx,
client.ObjectKeyFromObject(bmcObj), newBMC)), then deep-copy that fresh object
(use newBMC.DeepCopy() as the MergeFrom base), set newBMC.Status.State =
metalv1alpha1.BMCStatePending and call r.Status().Patch(ctx, newBMC,
client.MergeFrom(baseCopy)) to ensure you patch against the latest resource and
reduce conflicts in updateBMCStateToPending.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
internal/bmcutils/bmcutils.gointernal/controller/bmc_controller.go
🚧 Files skipped from review as they are similar to previous changes (1)
- internal/bmcutils/bmcutils.go
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@internal/controller/bmc_controller.go`:
- Around line 579-591: In updateLastResetTime, the r.Get call builds
client.ObjectKey only with Name which will fail for namespaced BMCs; update the
Get key to include the Namespace from bmcObj (use client.ObjectKey{Name:
bmcObj.Name, Namespace: bmcObj.Namespace}) so currentBMC is fetched from the
correct namespace before updating Status.LastResetTime and patching with
client.MergeFrom(bmcBase).
- Around line 238-244: The code reads bmcManager.LastResetTime before verifying
bmcManager is non-nil, causing a potential nil pointer panic; move the nil-check
(bmcManager != nil) so it executes before any access to bmcManager fields (e.g.,
LastResetTime) and only call time.Parse and set lastResetTime when bmcManager is
confirmed non-nil, preserving the existing behavior when LastResetTime is empty
or unparsable.
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (4)
internal/controller/bmc_controller.go (3)
584-585:⚠️ Potential issue | 🟠 MajorInclude namespace when re-fetching BMC in
updateLastResetTime.Line 584 uses only
Nameinclient.ObjectKey. For namespaced BMCs this can fetch the wrong object or fail.Proposed fix
- if err := r.Get(ctx, client.ObjectKey{Name: bmcObj.Name}, currentBMC); err != nil { + if err := r.Get(ctx, client.ObjectKey{Name: bmcObj.Name, Namespace: bmcObj.Namespace}, currentBMC); err != nil { return fmt.Errorf("failed to fetch BMC: %w", err) }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal/controller/bmc_controller.go` around lines 584 - 585, The re-fetch in updateLastResetTime uses client.ObjectKey with only bmcObj.Name which can target the wrong namespaced resource; update the r.Get call in updateLastResetTime to include both Name and Namespace (e.g., client.ObjectKey{Name: bmcObj.Name, Namespace: bmcObj.Namespace}) when fetching currentBMC so the correct namespaced BMC is retrieved.
240-248:⚠️ Potential issue | 🔴 CriticalGuard
bmcManagerbefore dereferencingLastResetTime.Line 242 dereferences
bmcManagerbefore the nil check at Line 248. IfGetManagerreturns(nil, nil), this panics.Proposed fix
bmcManager, err := bmcClient.GetManager(bmcObj.Spec.BMCUUID) if err != nil { return fmt.Errorf("failed to get manager details for BMC %s: %w", bmcObj.Name, err) } + if bmcManager == nil { + log.V(1).Info("Manager details not available for BMC", "BMC", bmcObj.Name) + return nil + } // parse time to metav1.Time: ISO 8601 format lastResetTime := &metav1.Time{} if bmcManager.LastResetTime != "" {🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal/controller/bmc_controller.go` around lines 240 - 248, The code dereferences bmcManager.LastResetTime before checking if bmcManager is nil, which can panic; change the logic in the function (where bmcManager is set) to guard bmcManager != nil first (e.g., check bmcManager == nil and skip parsing) and only then parse LastResetTime into lastResetTime (the metav1.Time creation and time.Parse call). In short: move the nil check for bmcManager above the block that reads bmcManager.LastResetTime (or add an explicit if bmcManager != nil around the time.Parse/assignment) so the code using LastResetTime only runs when bmcManager is non-nil.
499-534:⚠️ Potential issue | 🟠 MajorSet Reset condition to
Trueonly after reset is actually initiated.
resetBMCmarks Reset=True before Redfish success / SSH enqueue. On non-5xx or unknown failures, no reset starts but Reset can stay True, which can suppress future auto-reset attempts.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal/controller/bmc_controller.go` around lines 499 - 534, The reset condition (bmcResetConditionType) is being set True before a reset is actually initiated; move the call to r.updateConditions so it runs only after a successful Redfish reset (after bmcClient.ResetManager returns nil) or after successfully enqueueing an SSH reset (inside the select case), and do not set the condition for 4xx or unknown failures; update the branches that currently return on errors to leave conditions unchanged (or explicitly set False/clear) and keep the idempotency guard comments referencing waitForBMCReset and handlePreviousBMCResetAnnotations intact so future retries aren’t suppressed.internal/bmcutils/bmcutils.go (1)
236-239:⚠️ Potential issue | 🟠 MajorRe-enable SSH host key verification for reset connections.
Line 238 uses
ssh.InsecureIgnoreHostKey(), which disables MITM protection for privileged reset operations. Please switch to a verified host-key callback (e.g., known_hosts-based) and keep insecure mode only as an explicit fallback/opt-in.Suggested direction
- HostKeyCallback: ssh.InsecureIgnoreHostKey(), // `#nosec` G106 - See security note above + HostKeyCallback: hostKeyCallback, // resolved via knownhosts.New(...) with explicit fallback policy🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal/bmcutils/bmcutils.go` around lines 236 - 239, The SSH client config currently sets HostKeyCallback to ssh.InsecureIgnoreHostKey() which disables host-key verification; change it to use a known-hosts based callback by loading/using golang.org/x/crypto/ssh/knownhosts (e.g., call knownhosts.New with the default known_hosts path and set HostKeyCallback to that). Update the code in internal/bmcutils/bmcutils.go where User/Auth/HostKeyCallback/Timeout are assembled (the SSH client config creation) to construct and use the knownhosts callback, handle any error from knownhosts.New, and only fall back to ssh.InsecureIgnoreHostKey when an explicit opt-in flag or environment override (e.g., allowInsecureReset) is set and documented; ensure the fallback is gated and logged so insecure mode is not the default.
🧹 Nitpick comments (1)
cmd/main.go (1)
120-123: Validate SSH timeout flag bounds before constructing the reconciler.Please reject non-positive durations and
ssh-reset-worker-timeout < ssh-reset-timeout; otherwise worker contexts can cancel SSH resets earlier than configured.Suggested patch
flag.Parse() ctrl.SetLogger(zap.New(zap.UseFlagOptions(&opts))) + + if sshResetTimeout <= 0 || sshResetWorkerTimeout <= 0 { + setupLog.Error(nil, "ssh reset timeouts must be > 0") + os.Exit(1) + } + if sshResetWorkerTimeout < sshResetTimeout { + setupLog.Error(nil, "ssh-reset-worker-timeout must be greater than or equal to ssh-reset-timeout") + os.Exit(1) + }Also applies to: 356-357
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@cmd/main.go` around lines 120 - 123, Validate sshResetTimeout and sshResetWorkerTimeout immediately after flag parsing and before creating the reconciler: check that both durations are > 0 and that sshResetWorkerTimeout >= sshResetTimeout; if any check fails, log a clear error and exit (e.g., using log.Fatalf or similar). Update the validation code path that runs prior to constructing the reconciler (referencing the sshResetTimeout and sshResetWorkerTimeout variables) and apply the same checks where those flags might be parsed/used (the other occurrence around the second flag parsing block).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@internal/controller/bmc_controller_test.go`:
- Around line 749-760: The test uses shared vars updated inside the
bmcutils.SSHResetBMCFunc callback (sshResetCalled, capturedIP,
capturedManufacturer, capturedUsername, capturedPassword, capturedTimeout) and
other callbacks and reads them from the main test goroutine without
synchronization; replace raw bools/counters (sshResetCalled, callCount,
panicOccurred, sshCalled) with sync/atomic operations and protect captured
parameter fields (capturedIP, capturedManufacturer, capturedUsername,
capturedPassword, capturedTimeout) with a sync.Mutex (or a small struct guarded
by a mutex) so the callback locks, writes fields, and the test locks to read
them before asserting; apply the same pattern to the other callback sites noted
(the other similar ssh/call/panic variables) to eliminate data races when
running with the race detector.
---
Duplicate comments:
In `@internal/bmcutils/bmcutils.go`:
- Around line 236-239: The SSH client config currently sets HostKeyCallback to
ssh.InsecureIgnoreHostKey() which disables host-key verification; change it to
use a known-hosts based callback by loading/using
golang.org/x/crypto/ssh/knownhosts (e.g., call knownhosts.New with the default
known_hosts path and set HostKeyCallback to that). Update the code in
internal/bmcutils/bmcutils.go where User/Auth/HostKeyCallback/Timeout are
assembled (the SSH client config creation) to construct and use the knownhosts
callback, handle any error from knownhosts.New, and only fall back to
ssh.InsecureIgnoreHostKey when an explicit opt-in flag or environment override
(e.g., allowInsecureReset) is set and documented; ensure the fallback is gated
and logged so insecure mode is not the default.
In `@internal/controller/bmc_controller.go`:
- Around line 584-585: The re-fetch in updateLastResetTime uses client.ObjectKey
with only bmcObj.Name which can target the wrong namespaced resource; update the
r.Get call in updateLastResetTime to include both Name and Namespace (e.g.,
client.ObjectKey{Name: bmcObj.Name, Namespace: bmcObj.Namespace}) when fetching
currentBMC so the correct namespaced BMC is retrieved.
- Around line 240-248: The code dereferences bmcManager.LastResetTime before
checking if bmcManager is nil, which can panic; change the logic in the function
(where bmcManager is set) to guard bmcManager != nil first (e.g., check
bmcManager == nil and skip parsing) and only then parse LastResetTime into
lastResetTime (the metav1.Time creation and time.Parse call). In short: move the
nil check for bmcManager above the block that reads bmcManager.LastResetTime (or
add an explicit if bmcManager != nil around the time.Parse/assignment) so the
code using LastResetTime only runs when bmcManager is non-nil.
- Around line 499-534: The reset condition (bmcResetConditionType) is being set
True before a reset is actually initiated; move the call to r.updateConditions
so it runs only after a successful Redfish reset (after bmcClient.ResetManager
returns nil) or after successfully enqueueing an SSH reset (inside the select
case), and do not set the condition for 4xx or unknown failures; update the
branches that currently return on errors to leave conditions unchanged (or
explicitly set False/clear) and keep the idempotency guard comments referencing
waitForBMCReset and handlePreviousBMCResetAnnotations intact so future retries
aren’t suppressed.
---
Nitpick comments:
In `@cmd/main.go`:
- Around line 120-123: Validate sshResetTimeout and sshResetWorkerTimeout
immediately after flag parsing and before creating the reconciler: check that
both durations are > 0 and that sshResetWorkerTimeout >= sshResetTimeout; if any
check fails, log a clear error and exit (e.g., using log.Fatalf or similar).
Update the validation code path that runs prior to constructing the reconciler
(referencing the sshResetTimeout and sshResetWorkerTimeout variables) and apply
the same checks where those flags might be parsed/used (the other occurrence
around the second flag parsing block).
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
cmd/main.gointernal/bmcutils/bmcutils.gointernal/controller/bmc_controller.gointernal/controller/bmc_controller_test.gointernal/controller/suite_test.go
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (1)
internal/controller/bmc_controller.go (1)
259-265:⚠️ Potential issue | 🔴 CriticalGuard
bmcManagerbefore dereferencing fields.
bmcManager.LastResetTimeis accessed on Line 259 before the nil check on Line 265. IfGetManager()returns(nil, nil), this can panic reconciliation.🐛 Proposed fix
bmcManager, err := bmcClient.GetManager(bmcObj.Spec.BMCUUID) if err != nil { return fmt.Errorf("failed to get manager details for BMC %s: %w", bmcObj.Name, err) } + if bmcManager == nil { + log.V(1).Info("Manager details not available for BMC", "BMC", bmcObj.Name) + return nil + } // parse time to metav1.Time: ISO 8601 format lastResetTime := &metav1.Time{} if bmcManager.LastResetTime != "" { t, err := time.Parse(time.RFC3339, bmcManager.LastResetTime) if err == nil { lastResetTime = &metav1.Time{Time: t} } } - if bmcManager != nil { - bmcBase := bmcObj.DeepCopy() + bmcBase := bmcObj.DeepCopy() bmcObj.Status.Manufacturer = bmcManager.Manufacturer ... if err := r.Status().Patch(ctx, bmcObj, client.MergeFrom(bmcBase)); err != nil { return fmt.Errorf("failed to patch manager details for BMC %s: %w", bmcObj.Name, err) } - } else { - log.V(1).Info("Manager details not available for BMC", "BMC", bmcObj.Name) - }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal/controller/bmc_controller.go` around lines 259 - 265, The code dereferences bmcManager.LastResetTime before confirming bmcManager is non-nil which can panic; update the logic to guard bmcManager first (check bmcManager != nil immediately after GetManager()/where bmcManager is assigned) and only then parse bmcManager.LastResetTime and set lastResetTime; effectively move or add the nil check for bmcManager before the block that references LastResetTime (referencing the bmcManager variable and the LastResetTime handling).
🧹 Nitpick comments (1)
internal/controller/bmc_controller_test.go (1)
658-659: Replace microsecond polling with millisecond interval to avoid hot loop.The
1 * time.Microsecondpolling interval creates a hot loop; use50 * time.Millisecondinstead for more stable, lower-CPU assertions.Similar patterns exist in
internal/controller/biossettings_controller_test.goat lines 1721, 1725, and 1761.♻️ Proposed change
-Eventually(Object(bmc)).WithPolling(1 * time.Microsecond).MustPassRepeatedly(1).Should( +Eventually(Object(bmc)).WithPolling(50 * time.Millisecond).Should(Run
make lint-fixandmake testafter editing.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal/controller/bmc_controller_test.go` around lines 658 - 659, Replace the hot-loop polling intervals by changing calls like Eventually(Object(bmc)).WithPolling(1 * time.Microsecond) to use a millisecond interval (e.g., 50 * time.Millisecond) so assertions run at a stable, lower-CPU cadence; apply the same change for the identical patterns found in biossettings_controller_test.go (the occurrences using WithPolling(1 * time.Microsecond)). After edits run make lint-fix and make test to validate.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@internal/controller/bmc_controller_test.go`:
- Around line 1035-1073: The test "Should handle BMC deletion during SSH
processing" never triggers the SSH reset path because it doesn't set the
required operation annotation; update the test to set the OperationAnnotation to
ForceResetBMC on the BMC created by createBMCForSSHTest (or call the same helper
to create a BMC with that annotation) before deleting so
bmcutils.SSHResetBMCFunc is actually invoked; specifically, patch or update the
BMC object's annotations (OperationAnnotation -> ForceResetBMC) after
manufacturer is populated and before deleting the BMC/secret, then proceed with
the deletion and the existing Eventually(IsNotFound) check to ensure the worker
handled the in-flight SSH operation without panicking.
In `@internal/controller/bmc_controller.go`:
- Around line 550-559: The current logic always calls updateBMCStateToPending
even when the sshResetQueue send falls through to the default (queue full),
which can set Reset=True without scheduling any work; change the control flow so
that updateBMCStateToPending(ctx, bmcObj) is invoked only when the non-blocking
send to r.sshResetQueue (creating &sshResetRequest{ bmcName: bmcObj.Name,
bmcNamespace: bmcObj.Namespace }) actually succeeds (the case branch), and in
the default branch simply log the queue-full message (log.V(1).Info("SSH reset
queue full, will retry on next reconciliation", "BMC", bmcObj.Name)) and return
without calling updateBMCStateToPending; i.e., move the return/update into the
successful-case block to ensure we don't mark reset pending when no reset was
enqueued.
- Around line 543-564: The code is using a direct type assertion on err
(err.(*schemas.Error)); replace it with errors.As to match existing patterns and
handle wrapped errors—use a local variable (e.g., var httpErr *schemas.Error)
and call errors.As(err, &httpErr) before inspecting
httpErr.HTTPReturnedStatusCode; update the conditional block that enqueues
sshResetRequest and the subsequent returns to use the httpErr from errors.As,
keeping the same behavior in the functions updateBMCStateToPending,
waitForBMCReset, handlePreviousBMCResetAnnotations, and the sshResetQueue
enqueue logic.
---
Duplicate comments:
In `@internal/controller/bmc_controller.go`:
- Around line 259-265: The code dereferences bmcManager.LastResetTime before
confirming bmcManager is non-nil which can panic; update the logic to guard
bmcManager first (check bmcManager != nil immediately after GetManager()/where
bmcManager is assigned) and only then parse bmcManager.LastResetTime and set
lastResetTime; effectively move or add the nil check for bmcManager before the
block that references LastResetTime (referencing the bmcManager variable and the
LastResetTime handling).
---
Nitpick comments:
In `@internal/controller/bmc_controller_test.go`:
- Around line 658-659: Replace the hot-loop polling intervals by changing calls
like Eventually(Object(bmc)).WithPolling(1 * time.Microsecond) to use a
millisecond interval (e.g., 50 * time.Millisecond) so assertions run at a
stable, lower-CPU cadence; apply the same change for the identical patterns
found in biossettings_controller_test.go (the occurrences using WithPolling(1 *
time.Microsecond)). After edits run make lint-fix and make test to validate.
ℹ️ Review info
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (9)
api/v1alpha1/constants.gobmc/mock/server/data/Managers/BMC/index.jsoninternal/controller/biossettings_controller.gointernal/controller/biosversion_controller.gointernal/controller/bmc_controller.gointernal/controller/bmc_controller_test.gointernal/controller/bmcsettings_controller.gointernal/controller/bmcversion_controller.gointernal/controller/helper.go
afritzler
changed the title
afritzler
requested changes
Mar 5, 2026
stefanhipfel
added a commit
that referenced
this pull request
Apr 9, 2026
This commit addresses all feedback from coderabbitai and afritzler: 🔴 Critical Fixes: 1. Fix resetBMC error handling bug (internal/controller/bmc_controller.go) - Add clientErr parameter to preserve triggering error context - Initialize err with clientErr instead of nil to prevent loss of error information - Clear reset condition to False when reset doesn't actually start - Add nil error guard to prevent silent failures - Use bmcAuthenticationFailedReason for 4xx client errors 2. Fix nil pointer dereference (internal/controller/bmc_controller.go) - Move bmcManager nil check BEFORE accessing LastResetTime field - Return early if manager is nil to prevent panic - Fixes potential crash when GetManager returns (nil, nil) 🟢 Test Improvements: 3. Replace test comments with By() statements (internal/controller/bmc_controller_test.go) - Convert ~20+ inline comments to By() for better test output - Improves test readability and reporting with -v flag - Consistent with existing test patterns in codebase ✅ Already Fixed: 4. SSH host key verification - Already using ssh.InsecureIgnoreHostKey() (internal/bmcutils/bmcutils.go was already updated) All tests passing (106/106 controller specs). Fixes: #713
Contributor
Author
|
CI: Branch updated with lint fixes and test improvements (commit 1abf5a6) |
stefanhipfel
added a commit
that referenced
this pull request
Apr 9, 2026
This commit enhances the SSH-based BMC reset functionality to follow secure host key verification best practices while maintaining operational flexibility for recovery scenarios. Changes: - Add knownhosts package import for secure host key verification - Implement expandPath() helper to handle tilde (~) in file paths - Update SSHResetBMC() to attempt loading ~/.ssh/known_hosts first - Fall back to InsecureIgnoreHostKey() if known_hosts is unavailable - Add logging at V(1) level to indicate which mode is being used Security improvements: - Production environments with known_hosts: Use secure verification - Development/emergency scenarios: Automatic fallback with logging - Transparent operation: No configuration changes required - Better visibility: Clear logs indicate verification mode Implementation follows the pattern from cmd/metalctl/app/console.go as recommended in PR review feedback. Related: #713
Contributor
Author
|
@copilot resolve the merge conflicts in this pull request |
atd9876
reviewed
Apr 15, 2026
atd9876
reviewed
Apr 15, 2026
xkonni
approved these changes
May 19, 2026
xkonni
left a comment
xkonni
left a comment
Contributor
There was a problem hiding this comment.
looks good, maybe rebase, fix the pipeline issues and merge it?
atd9876
approved these changes
May 19, 2026
stefanhipfel
force-pushed
the
hard_reset
branch
2 times, most recently
from
July 17, 2026 08:16
e6d0c8b to
f371c6f
Compare
afritzler
requested changes
Jul 20, 2026
afritzler
reviewed
Jul 20, 2026
afritzler
requested changes
Jul 20, 2026
adracus
reviewed
Jul 23, 2026
stefanhipfel
added a commit
that referenced
this pull request
Aug 3, 2026
- Replace sshResetQueue+worker goroutine with annotation-driven reconciliation: on Redfish 5xx, set ForceSSHResetBMC annotation and process it in the next reconcile loop iteration (no separate goroutine or channel needed; in-cluster state drives retries and survives restarts) - Add ForceSSHResetBMC constant to api/v1alpha1/constants.go - Add SSHResetBMCFunc variable to pkg/bmcutils for test injection - Add ReasonResetComplete constant to conditions.go; replace inline "ResetComplete" string at the call site - Add SSHResetTimeout flag to cmd/main.go (SSHResetWorkerTimeout removed) - Add BMC SSH Reset test suite covering success, failure, and idempotency
stefanhipfel
force-pushed
the
hard_reset
branch
from
August 3, 2026 08:22
ebccc32 to
f295f69
Compare
- Replace sshResetQueue+worker goroutine with annotation-driven reconciliation: on Redfish 5xx, set ForceSSHResetBMC annotation and process it in the next reconcile loop iteration (no separate goroutine or channel needed; in-cluster state drives retries and survives restarts) - Add ForceSSHResetBMC constant to api/v1alpha1/constants.go - Add SSHResetBMCFunc variable to pkg/bmcutils for test injection - Add ReasonResetComplete constant to conditions.go; replace inline "ResetComplete" string at the call site - Add SSHResetTimeout flag to cmd/main.go (SSHResetWorkerTimeout removed) - Add BMC SSH Reset test suite covering success, failure, and idempotency Signed-off-by: Stefan Hipfel <stefan.hipfel@sap.com>
…rEach Signed-off-by: Stefan Hipfel <stefan.hipfel@sap.com>
stefanhipfel
force-pushed
the
hard_reset
branch
from
August 3, 2026 08:58
a026525 to
a0f7acc
Compare
afritzler
requested changes
Aug 4, 2026
Signed-off-by: Stefan Hipfel <stefan.hipfel@sap.com>
stefanhipfel
force-pushed
the
hard_reset
branch
from
August 4, 2026 15:10
8c3fc18 to
92a0d64
Compare
Member
|
@coderabbitai resume |
Contributor
✅ Action performedReviews resumed. |
Contributor
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
internal/controller/bmc_controller.go (1)
519-542: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
ConditionReset=Trueis set before the attempt and is never rolled back on failure.Line 521 sets
ConditionResettoTruebefore any reset is attempted or scheduled. Three return paths then leave itTruealthough no reset started:
- Lines 539-542: no client and the error is not a 5xx.
- Line 550: a live client and a non-5xx Redfish error.
- Line 562: a live client and a non-
schemas.Errorfailure.
shouldResetBMCreturns false whileConditionResetisTrue(line 491). The condition is only cleared on line 194, which runs after a successful Redfish connection. A BMC that stays offline after a failed reset therefore keepsReset=Trueindefinitely, and the automatic reset path never retries it.Set the condition to
Trueonly on the paths that actually start or schedule a reset, or clear it toFalsebefore each failure return.🛠️ Proposed direction: clear the condition on the non-scheduling returns
if bmcClient == nil { if clientErr != nil { if httpErr, ok := errors.AsType[*schemas.Error](clientErr); ok && httpErr.HTTPReturnedStatusCode >= 500 && httpErr.HTTPReturnedStatusCode < 600 { ... return r.patchBMCStatePending(ctx, bmcObj) } } + _ = r.updateConditions(ctx, bmcObj, false, ConditionReset, corev1.ConditionFalse, ReasonConnectionFailed, "BMC reset did not start: no client connection") return errors.Join( r.patchBMCStatePending(ctx, bmcObj), fmt.Errorf("could not reset BMC %s: no client connection", bmcObj.Name), ) }Apply the same rollback before the returns on lines 550 and 562.
As per coding guidelines
internal/controller/**/*_controller.go: "Implement idempotent reconciliation logic - safe to run the same reconciliation multiple times without side effects".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/controller/bmc_controller.go` around lines 519 - 542, Update resetBMC so ConditionReset is rolled back to False before every return path where no reset is started or scheduled, including the no-client non-5xx path and live-client non-5xx or non-schemas.Error failures. Preserve ConditionReset=True for paths that schedule SSH reset or begin an actual reset, and propagate any condition-update failure appropriately so reconciliation remains idempotent.Source: Coding guidelines
🧹 Nitpick comments (6)
internal/controller/bmc_controller_test.go (4)
735-749: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRegister resource cleanup with
DeferCleanupinstead of trailing deletes.Each spec deletes its BMC and secret at the end of the body (lines 815-817, 854-856, 910-912). If an assertion fails earlier, those deletes never run. The
AfterEachthen waits the full 30 seconds on each of the twoEventuallyblocks before reporting a second failure, which obscures the original one.Move the deletes into
DeferCleanupinsidecreateBMCSecretForSSHTestandcreateBMCForSSHTest. Ginkgo then runs them regardless of spec outcome, and the list-empty waits inAfterEachbecome a fast confirmation.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/controller/bmc_controller_test.go` around lines 735 - 749, Move BMC and secret deletion into DeferCleanup within createBMCSecretForSSHTest and createBMCForSSHTest, ensuring cleanup runs even when assertions fail. Remove the trailing per-spec deletes, while retaining the AfterEach list-empty checks as confirmation of cleanup.
903-906: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winReplace the fixed sleep with
Consistently.The sleep is 500ms and
BMCResetWaitTimeis 400ms insuite_test.go, leaving a 100ms margin. Under CI load, a reconcile that wrongly triggers a second SSH reset can land after the sleep, so a regression can pass.Consistentlypolls throughout the window and states the intent directly.♻️ Proposed change
- time.Sleep(500 * time.Millisecond) - mu.Lock() - Expect(callCount).To(Equal(1)) - mu.Unlock() + Consistently(func() int { + mu.Lock() + defer mu.Unlock() + return callCount + }).WithTimeout(1 * time.Second).WithPolling(50 * time.Millisecond).Should(Equal(1))🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/controller/bmc_controller_test.go` around lines 903 - 906, Replace the fixed time.Sleep assertion in the BMC reset test with Gomega Consistently, polling callCount under the mutex for the full BMCResetWaitTime window and asserting it remains 1. Preserve the existing synchronization and ensure the check covers the entire wait period so delayed duplicate SSH resets are detected.
751-818: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winGood end-to-end coverage; the automatic reset path is not covered.
The synchronization is correct here. Every write at lines 758-766 and every read at lines 795-801 holds
mu, and the polling closure at 788-792 locks as well.All three new specs drive the fallback through the user annotation branch at
bmc_controller.goline 165. None of them exercise the automatic path at line 154, whichshouldResetBMCgates onBMCFailureResetDelay. That reconciler is configured withoutBMCFailureResetDelayinsuite_test.go, so the automatic path is disabled in this suite. Thenilargument defect I flagged atbmc_controller.goline 156 is therefore invisible to these tests.Consider a spec that sets
BMCFailureResetDelayand asserts the SSH fallback runs after repeated connection failures.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/controller/bmc_controller_test.go` around lines 751 - 818, Add an end-to-end spec alongside the existing SSH reset coverage that configures a nonzero BMCFailureResetDelay, drives repeated connection failures without the user reset annotation, and verifies the automatic shouldResetBMC path invokes SSHResetBMCFunc with the expected parameters. Ensure the test setup enables the delay in the reconciler configuration and specifically exercises the automatic fallback argument handling.
845-850: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUse the exported constants instead of string literals.
Lines 846 and 848 hardcode
"Reset"and"InternalServerError". The package already exportsConditionResetandReasonInternalError, and line 679 in this file usesReasonResetComplete. Using the constants keeps the assertions correct if a value changes. The same applies to"Reset"at lines 805 and 899, and"Ready"at lines 773, 830, and 875.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/controller/bmc_controller_test.go` around lines 845 - 850, Replace hardcoded condition type and reason strings in the assertions around the affected tests with the exported constants: use ConditionReset for “Reset”, ReasonInternalError for “InternalServerError”, and the existing exported constant for “Ready” (as used by the package). Update every cited occurrence, including the assertions near lines 805, 773, 830, 875, and 899, while preserving the assertion structure.internal/controller/bmc_controller.go (2)
164-177: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winA fully unreachable BMC drops the user reset request.
This branch deletes the operation annotation before it calls
resetBMC.resetBMConly schedules the SSH fallback whenconnErris a 5xx*schemas.Error. A BMC whose management interface is completely down returns a transport error, not a 5xx Redfish response, so no fallback is scheduled and the annotation is already removed. The user request is lost and the user must re-apply the annotation.That case is arguably the strongest reason to use SSH, since SSH may still answer when the Redfish stack is dead. Consider extending the classification in
resetBMCto also schedule the SSH fallback forbmcutils.BMCUnAvailableErrorand transport-level dial failures.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/controller/bmc_controller.go` around lines 164 - 177, The user reset path currently removes the operation annotation without scheduling SSH when resetBMC receives a transport-level failure. Update resetBMC’s connection-error classification to schedule the SSH fallback for bmcutils.BMCUnAvailableError and transport-level dial failures in addition to 5xx schemas.Error responses, preserving the existing user-reset behavior and annotation handling.
128-142: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winConsider bounding the blocking SSH reset on the reconcile worker.
resetBMCViaSSHruns synchronously here. It dials TCP and runs a remote command withSSHResetTimeout, which defaults to 2 minutes incmd/main.go. The BMC controller does not setMaxConcurrentReconciles, so it runs with the controller-runtime default of 1 worker. One unreachable BMC therefore stalls reconciliation for every other BMC for up to 2 minutes per attempt.The annotation-driven design is a deliberate replacement for the previous channel queue, so keeping the call in-reconcile is reasonable. Two options bound the impact:
- Set
MaxConcurrentReconcileson the BMC controller so a single slow SSH reset does not serialize all BMC work.- Derive a shorter context for this call than the full reconcile budget.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@internal/controller/bmc_controller.go` around lines 128 - 142, Bound the synchronous resetBMCViaSSH call in the BMC reconcile path so one unreachable BMC cannot block all reconciliation for the full SSHResetTimeout. Prefer configuring a reasonable MaxConcurrentReconciles value for the BMC controller, or derive a shorter context specifically for resetBMCViaSSH while preserving the existing annotation removal, error propagation, and requeue behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@internal/controller/bmc_controller.go`:
- Around line 154-163: Pass the existing BMC client connection error from the
automatic reset branch into resetBMC instead of nil, using the err returned by
GetBMCClientFromBMC. Keep the remaining reset behavior unchanged so resetBMC can
classify the failure and schedule the ForceSSHResetBMC fallback.
---
Outside diff comments:
In `@internal/controller/bmc_controller.go`:
- Around line 519-542: Update resetBMC so ConditionReset is rolled back to False
before every return path where no reset is started or scheduled, including the
no-client non-5xx path and live-client non-5xx or non-schemas.Error failures.
Preserve ConditionReset=True for paths that schedule SSH reset or begin an
actual reset, and propagate any condition-update failure appropriately so
reconciliation remains idempotent.
---
Nitpick comments:
In `@internal/controller/bmc_controller_test.go`:
- Around line 735-749: Move BMC and secret deletion into DeferCleanup within
createBMCSecretForSSHTest and createBMCForSSHTest, ensuring cleanup runs even
when assertions fail. Remove the trailing per-spec deletes, while retaining the
AfterEach list-empty checks as confirmation of cleanup.
- Around line 903-906: Replace the fixed time.Sleep assertion in the BMC reset
test with Gomega Consistently, polling callCount under the mutex for the full
BMCResetWaitTime window and asserting it remains 1. Preserve the existing
synchronization and ensure the check covers the entire wait period so delayed
duplicate SSH resets are detected.
- Around line 751-818: Add an end-to-end spec alongside the existing SSH reset
coverage that configures a nonzero BMCFailureResetDelay, drives repeated
connection failures without the user reset annotation, and verifies the
automatic shouldResetBMC path invokes SSHResetBMCFunc with the expected
parameters. Ensure the test setup enables the delay in the reconciler
configuration and specifically exercises the automatic fallback argument
handling.
- Around line 845-850: Replace hardcoded condition type and reason strings in
the assertions around the affected tests with the exported constants: use
ConditionReset for “Reset”, ReasonInternalError for “InternalServerError”, and
the existing exported constant for “Ready” (as used by the package). Update
every cited occurrence, including the assertions near lines 805, 773, 830, 875,
and 899, while preserving the assertion structure.
In `@internal/controller/bmc_controller.go`:
- Around line 164-177: The user reset path currently removes the operation
annotation without scheduling SSH when resetBMC receives a transport-level
failure. Update resetBMC’s connection-error classification to schedule the SSH
fallback for bmcutils.BMCUnAvailableError and transport-level dial failures in
addition to 5xx schemas.Error responses, preserving the existing user-reset
behavior and annotation handling.
- Around line 128-142: Bound the synchronous resetBMCViaSSH call in the BMC
reconcile path so one unreachable BMC cannot block all reconciliation for the
full SSHResetTimeout. Prefer configuring a reasonable MaxConcurrentReconciles
value for the BMC controller, or derive a shorter context specifically for
resetBMCViaSSH while preserving the existing annotation removal, error
propagation, and requeue behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 8d5d8b36-0058-45db-a7e7-3e100c75b9a5
📒 Files selected for processing (7)
api/v1alpha1/constants.gocmd/main.gointernal/controller/bmc_controller.gointernal/controller/bmc_controller_test.gointernal/controller/conditions.gointernal/controller/suite_test.gopkg/bmcutils/bmcutils.go
Comment on lines
154
to
163
| if r.shouldResetBMC(bmcObj) { | ||
| log.V(1).Info("BMC needs reset, resetting", "BMC", bmcObj.Name) | ||
| if err := r.resetBMC(ctx, bmcObj, bmcClient, ReasonAutoReset, bmcAutoResetMessage); err != nil { | ||
| if err := r.resetBMC(ctx, bmcObj, bmcClient, nil, ReasonAutoReset, bmcAutoResetMessage); err != nil { | ||
| return ctrl.Result{}, fmt.Errorf("failed to reset BMC: %w", err) | ||
| } | ||
| log.V(1).Info("BMC reset initiated", "BMC", bmcObj.Name) | ||
| return ctrl.Result{ | ||
| RequeueAfter: r.BMCClientRetryInterval, | ||
| }, nil | ||
| } |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
The automatic reset path never reaches the SSH fallback.
Line 156 passes nil as clientErr. In this branch bmcClient is also nil, because GetBMCClientFromBMC returned an error on line 152. Inside resetBMC, the bmcClient == nil branch only schedules the ForceSSHResetBMC annotation when clientErr is a 5xx *schemas.Error. With clientErr == nil, that check is skipped, so this path always falls through to lines 539-542 and returns "no client connection".
The automatic path is the unattended case that BMCFailureResetDelay exists for, and it fires exactly when Redfish is unresponsive. It should classify the connection error the same way the user-annotation path does on line 173.
The outer err is still in scope at line 156. Go evaluates the call arguments before the new err binding in the if statement takes effect, so passing err directly is safe here.
🐛 Proposed fix: pass the connection error into resetBMC
if r.shouldResetBMC(bmcObj) {
log.V(1).Info("BMC needs reset, resetting", "BMC", bmcObj.Name)
- if err := r.resetBMC(ctx, bmcObj, bmcClient, nil, ReasonAutoReset, bmcAutoResetMessage); err != nil {
+ connErr := err // preserve before the inner scope shadows it
+ if err := r.resetBMC(ctx, bmcObj, bmcClient, connErr, ReasonAutoReset, bmcAutoResetMessage); err != nil {
return ctrl.Result{}, fmt.Errorf("failed to reset BMC: %w", err)
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@internal/controller/bmc_controller.go` around lines 154 - 163, Pass the
existing BMC client connection error from the automatic reset branch into
resetBMC instead of nil, using the err returned by GetBMCClientFromBMC. Keep the
remaining reset behavior unchanged so resetBMC can classify the failure and
schedule the ForceSSHResetBMC fallback.
Member
There was a problem hiding this comment.
@stefanhipfel can you please check if this is a valid finding?
afritzler
approved these changes
Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed Changes
Implements a ssh based bmc reset in case the redfish api is no longer responsive.
The ssh reset is done via a separate goroutine
Summary by CodeRabbit