-
Notifications
You must be signed in to change notification settings - Fork 40
docs: add off-cluster qemu-ssh provisioner guide and samples #1116
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
bkhizgiy
wants to merge
6
commits into
jumpstarter-dev:main
Choose a base branch
from
bkhizgiy:qemu-ssh-docs
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
fe29085
controller: add qemu-ssh provisioner library (SSH, quadlets, host pool)
bkhizgiy 2c081dc
fix: drop arch/slots and fix go issues
bkhizgiy ecc6fd8
fix: address code review comments
bkhizgiy cec4ff6
controller: wire qemu-ssh provisioner into reconciler
bkhizgiy c480a8d
fix: address review comments
bkhizgiy fbffd87
docs: add off-cluster qemu-ssh provisioner guide and samples
bkhizgiy File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
23 changes: 23 additions & 0 deletions
23
controller/config/samples/operator_jumpstarter_with_qemu_ssh.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # Jumpstarter operator CR — enabling both in-cluster QEMU and off-cluster | ||
| # QEMU-SSH provisioners. | ||
| # | ||
| # Apply with: kubectl apply -f operator_jumpstarter_with_qemu_ssh.yaml | ||
| apiVersion: operator.jumpstarter.dev/v1alpha1 | ||
| kind: Jumpstarter | ||
| metadata: | ||
| name: jumpstarter | ||
| namespace: jumpstarter | ||
| spec: | ||
| exporterSets: | ||
| # Container image for the exporter-set controller binary. | ||
| # All provisioners share the same binary; --provisioner selects | ||
| # the backend at startup. | ||
| image: quay.io/jumpstarter-dev/exporter-set-controller:latest | ||
| provisioners: | ||
| # In-cluster QEMU provisioner (creates Pods with QEMU sidecar) | ||
| - name: qemu.jumpstarter.dev | ||
| enabled: true | ||
| # Off-cluster QEMU provisioner (deploys containers on remote | ||
| # hosts via SSH using Podman quadlets) | ||
| - name: qemu-ssh.jumpstarter.dev | ||
| enabled: true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| # SSH credentials Secret — required by the qemu-ssh.jumpstarter.dev | ||
| # provisioner for authenticating to remote lab hosts. | ||
| # | ||
| # The credentialsSecretRef in the VirtualTargetClass points to this | ||
| # Secret. It uses the standard kubernetes.io/ssh-auth type. | ||
| # | ||
| # Note: kubernetes.io/ssh-auth only carries the private key. The SSH | ||
| # username is configured in the VirtualTargetClass parameters | ||
| # (parameters.ssh.user or parameters.host.user), not in this Secret. | ||
| # | ||
| # Create from a file (preferred): | ||
| # kubectl create secret generic lab-ssh-key \ | ||
| # --from-file=ssh-privatekey=$HOME/.ssh/id_ed25519 \ | ||
| # -n jumpstarter | ||
| # | ||
| # Or apply this manifest after base64-encoding your key: | ||
| # cat ~/.ssh/id_ed25519 | base64 -w0 | ||
| apiVersion: v1 | ||
| kind: Secret | ||
| metadata: | ||
| name: lab-ssh-key | ||
| namespace: jumpstarter | ||
| type: kubernetes.io/ssh-auth | ||
| data: | ||
| # Replace with your base64-encoded SSH private key | ||
| ssh-privatekey: <BASE64_ENCODED_SSH_PRIVATE_KEY> | ||
44 changes: 44 additions & 0 deletions
44
controller/config/samples/v1alpha1_exporterset_qemu_ssh.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| # ExporterSet — off-cluster QEMU aarch64 pool via SSH | ||
| # | ||
| # This example creates a scalable pool of aarch64 virtual targets | ||
| # running as Podman containers on remote lab hosts. The | ||
| # exporter-set controller manages the lifecycle via SSH. | ||
| # | ||
| # Uses the qemu-ssh-aarch64 VirtualTargetClass (see | ||
| # v1alpha1_virtualtargetclass_qemu_ssh.yaml). | ||
| apiVersion: virtualtarget.jumpstarter.dev/v1alpha1 | ||
| kind: ExporterSet | ||
| metadata: | ||
| name: aarch64-ssh-pool | ||
| namespace: jumpstarter | ||
| spec: | ||
| minReplicas: 0 | ||
| maxReplicas: 4 | ||
| minAvailableReplicas: 1 | ||
| scaleDownCooldown: 5m | ||
| recycleStrategy: ExitAndReplace | ||
| virtualTargetClassName: qemu-ssh-aarch64 | ||
| # Override class-level resource defaults (deep-merged) | ||
| parameters: | ||
| resources: | ||
| memory: 8Gi | ||
| selector: | ||
| matchLabels: | ||
| board: aarch64-qemu | ||
| virtual: "true" | ||
| template: | ||
| metadata: | ||
| labels: | ||
| board: aarch64-qemu | ||
| arch: aarch64 | ||
| virtual: "true" | ||
| spec: | ||
| drivers: | ||
| - name: qemu | ||
| type: jumpstarter_driver_qemu.driver.Qemu | ||
| - name: power | ||
| type: jumpstarter_driver_power.driver.QemuPower | ||
| - name: serial | ||
| type: jumpstarter_driver_serial.driver.QemuSerial | ||
| # tcp driver is auto-injected by the provisioner (ssh | ||
| # hostfwd on port 2222) — no need to specify it explicitly |
46 changes: 46 additions & 0 deletions
46
controller/config/samples/v1alpha1_virtualtargetclass_qemu_ssh.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| # VirtualTargetClass — off-cluster QEMU via SSH | ||
| # | ||
| # This example defines a virtual target profile for aarch64 targets | ||
| # running on a remote lab host (bare metal or VM with KVM). The | ||
| # exporter-set controller connects via SSH and deploys Podman | ||
| # containers (quadlets) on the host. | ||
| # | ||
| # Each ExporterSet manages one host. To use multiple hosts, create | ||
| # additional ExporterSets under this same VirtualTargetClass. | ||
| # | ||
| # Prerequisites: | ||
| # 1. Create the SSH credentials Secret: | ||
| # kubectl create secret generic lab-ssh-key \ | ||
| # --from-file=ssh-privatekey=$HOME/.ssh/id_ed25519 \ | ||
| # -n jumpstarter | ||
| # 2. Ensure the remote host has Podman and systemd installed. | ||
| # 3. Enable qemu-ssh.jumpstarter.dev provisioner in the Jumpstarter CR. | ||
| apiVersion: virtualtarget.jumpstarter.dev/v1alpha1 | ||
| kind: VirtualTargetClass | ||
| metadata: | ||
| name: qemu-ssh-aarch64 | ||
| namespace: jumpstarter | ||
| spec: | ||
| provisioner: qemu-ssh.jumpstarter.dev | ||
| credentialsSecretRef: | ||
| name: lab-ssh-key | ||
| bindingMode: Immediate | ||
| reclaimPolicy: Delete | ||
| parameters: | ||
| # SSH connection defaults (per-host overrides available in host) | ||
| ssh: | ||
| user: root | ||
| port: 22 | ||
| # Remote lab host for this ExporterSet | ||
| host: | ||
| name: lab-host-01.example.com | ||
| # Runtime container configuration | ||
| runtime: | ||
| kvm: true # pass /dev/kvm into the QEMU runtime container | ||
| # Default resource allocation per virtual target | ||
| arch: aarch64 | ||
| resources: | ||
| cpu: 4 | ||
| memory: 4Gi | ||
| storage: | ||
| size: 16Gi |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is there a standard way to pass the username here as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
kubernetes.io/ssh-authonly definesssh-privatekey, so the username should stay in the VTC parameters. I added a comment to the secret sample to make that clearer, and also updated the VTC sample to the new single-host model.