You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
bug: CreateVolume doesn't set PersistentVolume.spec.nodeAffinity, so a bound PVC's pod can land on any node #403
The CSI driver's CreateVolume never populates PersistentVolume.spec.nodeAffinity. Once a PVC is bound, a pod referencing it can be scheduled onto any node in the cluster — not just the node the AccessibleTopology/AllowedTopologies gate originally selected at first-consumer binding time.
Why this was harmless until now
For a plain NVMe-oF-backed volume, this genuinely didn't matter: the client just connects over the fabric from wherever the pod lands, and behaves identically from any node. AllowedTopologies + WaitForFirstConsumer only ever needed to influence the initial provisioning decision.
Why it matters now
Found while validating client-side compression/dedup via VDO (issue #277, PR #402). VDO's state (the LVM PV/VG/LV stack) is genuinely node-local — it only exists on whichever node created it. If a pod using a VDO-backed PVC gets rescheduled to a different node (a plain pod delete+recreate is enough; no drain or node failure needed), the CSI driver on that new node has no VDO stack to attach to.
Reproduction
Create a Pool with clientCompression/clientDeduplication enabled (or any StorageClass using allowedTopologies).
Create a PVC + Pod; it correctly binds and schedules onto a topology-matching node (e.g. the only node with vdo-capable=true).
Delete the pod only (leave the PVC in place) and recreate it without a nodeSelector.
The pod can be scheduled onto a different, non-matching node. In our case it landed on a node without kvdo loaded, and NodeStageVolume correctly failed loudly (modprobe: FATAL: Module kvdo not found) rather than silently corrupting anything — but a bound PVC lacking nodeAffinity is the root cause, and a less defensive code path could fail more silently.
Confirmed via kubectl get pv <name> -o yaml: spec.nodeAffinity is absent entirely.
Suggested fix
CreateVolume should populate PersistentVolume.spec.nodeAffinity from the same AccessibleTopology/AccessibilityRequirements data it already uses to decide AllowedTopologies gating — this is standard CSI driver behavior (see the CSI spec's CreateVolumeResponse.Volume.accessible_topology, which external-provisioner uses to set PersistentVolume.spec.nodeAffinity automatically when populated). If accessible_topology is already being returned correctly, this may already work for some cases — worth checking why it isn't showing up on the PV.
Scope note
This is a pre-existing, orthogonal gap — not introduced by #402, and not blocking it (the PR works around it for its own test verification by pinning nodeSelector explicitly). Affects any StorageClass using allowedTopologies today (e.g. DHCHAP's allowedNodes gate), not just the new VDO feature.
Activity
revert: scope this fix to the CSI driver only, accept multi-pool over…
fix: set PersistentVolume.spec.nodeAffinity for topology-gated volumes (
revert: scope this fix to the CSI driver only, accept multi-pool over…
test(csi): add e2e coverage for the DHCHAP allowed-node scheduling fix (
fix(operator): restart a node's CSI plugin when it gets a genuinely n…
fix(operator): make restartCSINodePod self-healing instead of best-ef…
fix: pin PersistentVolume.spec.nodeAffinity for VDO volumes too (#403)
fix: pin PersistentVolume.spec.nodeAffinity for VDO volumes too (#403)