-
Notifications
You must be signed in to change notification settings - Fork 32
docs: Add stateless migration quickstart tutorial #569
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
msajidmansoori12
merged 9 commits into
migtools:main
from
msajidmansoori12:add-stateless-migration-doc
Jun 30, 2026
Merged
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
38fa275
Add stateless migration quickstart tutorial
msajidmansoori12 310487e
Merge branch 'main' into add-stateless-migration-doc
msajidmansoori12 5b60590
Update output section
msajidmansoori12 618a6c2
Update what stage is
msajidmansoori12 251fd40
update link for multistage
msajidmansoori12 ea7cc90
Add one liners about what commands do
msajidmansoori12 1de0bee
Update transform --force to be more clear
msajidmansoori12 c11ae02
Write about stage directory output from instructions.yaml file usage
msajidmansoori12 b5e2799
Fix
msajidmansoori12 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,306 @@ | ||
| # Quickstart Stateless Migration Tutorial | ||
|
|
||
| This quickstart covers a generic stateless migration workflow in Crane using the current pipeline: | ||
|
|
||
| `export -> transform -> apply -> validate` | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - `crane` CLI installed | ||
| - Kubernetes access configured for source and target contexts | ||
| - A namespace in the source cluster containing the resources you want to migrate | ||
|
|
||
| Before running any Crane commands, make sure your local `kubeconfig` already includes valid contexts for both clusters. Crane runs locally and uses the `--context` flag to talk directly to each cluster using your existing Kubernetes RBAC permissions. | ||
|
|
||
| ## 1) Set Environment Variables | ||
|
|
||
| ```bash | ||
| export SOURCE_CONTEXT=src-cluster | ||
| export TARGET_CONTEXT=tgt-cluster | ||
| export SOURCE_NAMESPACE=source-namespace | ||
| ``` | ||
|
|
||
| ## 2) Export | ||
|
|
||
| `crane export` discovers resources in the source namespace and writes them as Kubernetes manifest files on disk. | ||
|
|
||
| Run export with an explicit export directory: | ||
|
|
||
| - `-e export` tells Crane to write exported manifests under the `export/` directory. | ||
|
|
||
| ```bash | ||
| crane export \ | ||
| --context "${SOURCE_CONTEXT}" \ | ||
| -n "${SOURCE_NAMESPACE}" \ | ||
| -e export | ||
| ``` | ||
|
|
||
| Example output snippet (abbreviated): | ||
|
|
||
| ```text | ||
| INFO[0000] adding resource: secrets to the list of GVRs to be extracted | ||
| INFO[0000] adding resource: services to the list of GVRs to be extracted | ||
| INFO[0000] adding resource: deployments to the list of GVRs to be extracted | ||
| INFO[0000] No matching cluster-scoped resources found; _cluster/ directory will be empty | ||
| INFO[0000] Writing objects of resource: secrets to the output directory | ||
| INFO[0000] Writing objects of resource: services to the output directory | ||
| INFO[0000] Writing objects of resource: deployments to the output directory | ||
| ``` | ||
|
|
||
| What you can expect to see (example): | ||
|
|
||
| - Log lines showing resources discovered for extraction (for example, `secrets`, `services`, `configmaps`, `deployments`) | ||
| - A write phase with lines like `Writing objects of resource: <name> to the output directory` | ||
| - If no cluster-scoped objects match, a message that `_cluster/` will be empty | ||
| - Exported manifests written under `export/resources/${SOURCE_NAMESPACE}/` | ||
| - If extraction fails for specific objects, failure artifacts are written under `export/failures/${SOURCE_NAMESPACE}/` | ||
|
|
||
| ## 3) Transform (Default Stage) | ||
|
msajidmansoori12 marked this conversation as resolved.
|
||
|
|
||
| `crane transform` takes exported manifests (output from `crane export`), cleans and updates them in stages, and saves the results in `transform/`. | ||
|
|
||
| A stage is one step in the transform pipeline. | ||
|
|
||
| Think of it like an assembly line: | ||
| - Stage 1 takes your exported manifests and makes the first set of changes. | ||
| - Stage 2 takes Stage 1 output and applies the next changes. | ||
|
|
||
| Example: `10_KubernetesPlugin` runs first, then `25_CustomStage` runs on top of that result. | ||
|
|
||
| Run transform with explicit input and stage directories: | ||
|
|
||
| - `-e export` tells Crane to read exported resources from the `export/` directory. | ||
| - `-t transform` tells Crane to write and execute transform stages under the `transform/` directory. | ||
|
|
||
| ```bash | ||
| crane transform -e export -t transform | ||
| ``` | ||
|
|
||
| Example output snippet (abbreviated): | ||
|
|
||
| ```text | ||
| INFO[0000] No existing stages found, creating default stages for 1 plugin(s) | ||
| INFO[0000] Creating default stage for plugin: KubernetesPlugin -> 10_KubernetesPlugin | ||
| INFO[0000] Created 1 default stage(s): [10_KubernetesPlugin] | ||
| INFO[0000] Populating and executing all default stages | ||
| INFO[0000] Executing stage 1/1: 10_KubernetesPlugin | ||
| INFO[0000] Stage 10_KubernetesPlugin: loaded 10 input resource(s) | ||
| INFO[0000] Stage 10_KubernetesPlugin: produced 4 output resource(s) | ||
| INFO[0000] Successfully completed 1 stage(s) | ||
| ``` | ||
|
|
||
| What you can expect to see (example): | ||
|
|
||
| - Default stage creation logs when no stages exist yet | ||
| - Stage execution progress (for example, `Executing stage 1/1: 10_KubernetesPlugin`) | ||
| - Input and output resource counts per stage | ||
| - A success line indicating all stages completed | ||
| - Stage artifacts in `transform/10_KubernetesPlugin/`: `input/`, `patches/`, `output/`, `kustomization.yaml` | ||
|
|
||
| Stage naming uses numeric prefixes to control order. For example, `10_KubernetesPlugin` runs before `25_CustomStage`, and Crane executes stages from lowest number to highest number. | ||
|
|
||
| Expected transform directory shape (example): | ||
|
|
||
| ```text | ||
| transform/ | ||
| └── 10_KubernetesPlugin | ||
| ├── input/ | ||
| │ ├── ...ConfigMap... | ||
| │ ├── ...Deployment... | ||
| │ ├── ...Secret... | ||
| │ └── ...Service... | ||
| ├── kustomization.yaml | ||
| ├── output/ | ||
| │ └── <source-namespace>/ | ||
| │ ├── ...ConfigMap... | ||
| │ ├── ...Deployment... | ||
| │ ├── ...Secret... | ||
| │ └── ...Service... | ||
| └── patches/ | ||
| ├── ...Deployment.patch.yaml | ||
| ├── ...ConfigMap.patch.yaml | ||
| ├── ...Secret.patch.yaml | ||
| └── ...Service.patch.yaml | ||
| ``` | ||
|
|
||
| ## 4) Optional: Add Additional Stages | ||
|
|
||
| You can add a custom pass-through stage: | ||
|
|
||
| ```bash | ||
| crane transform -e export -t transform 25_CustomStage | ||
| ``` | ||
|
|
||
| What you should see (example): | ||
|
|
||
| - `transform/25_CustomStage/` with `input/`, `output/`, `kustomization.yaml` | ||
|
|
||
| Custom stages are rendered with Kustomize. After editing resources in your custom stage, you can preview the rendered manifests before `crane apply`: | ||
|
|
||
| ```bash | ||
| kubectl kustomize transform/25_CustomStage | ||
| ``` | ||
|
|
||
| In most pipelines, your custom stage is the last stage under `transform/`, so rendering that directory should show the manifests that will feed into the final apply output. | ||
|
|
||
| For a deeper explanation of stage ordering, stage structure, and multi-stage behavior, see [Multi-Stage Kustomize Transform Pipeline](./multistage-pipeline.md). | ||
|
|
||
| If a custom stage already contains edits, rerun with `--force` to regenerate: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. regenerate what?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed with more clear info |
||
|
|
||
| ```bash | ||
| crane transform -e export -t transform --force | ||
| ``` | ||
|
|
||
| ## 5) Apply (Render Final Manifests) | ||
|
|
||
| `crane apply` renders the final manifests from transform stages into deployable output files. | ||
|
|
||
| Run apply with explicit transform and output directories: | ||
|
|
||
| - `-t transform` tells Crane which staged transform directory to read. | ||
| - `-o output` tells Crane where to write the final rendered manifests. | ||
|
|
||
| ```bash | ||
| crane apply -t transform -o output | ||
| ``` | ||
|
|
||
| Example output snippet (abbreviated): | ||
|
|
||
| ```text | ||
| INFO[0000] Applying all stages... | ||
| INFO[0000] Applying final stage: 10_KubernetesPlugin | ||
| INFO[0000] Successfully applied final stage to .../output/output.yaml | ||
| ``` | ||
|
|
||
| What you can expect to see (example): | ||
|
|
||
| - `output/output.yaml` generated as one combined file containing all rendered resources (useful for a single `kubectl apply -f`). | ||
| - `output/resources/` generated as separate files per resource (useful when you want to review, diff, or apply resources selectively). | ||
|
|
||
| For namespace-only/non-admin scenarios: | ||
|
|
||
| ```bash | ||
| crane apply \ | ||
| --skip-cluster-scoped | ||
| ``` | ||
|
|
||
| ## 6) Validate (Optional, Recommended) | ||
|
|
||
| `crane validate` checks whether the rendered manifests are compatible with the target cluster API. | ||
|
|
||
| Run live validation against the target cluster API. This step is optional, but strongly recommended before promoting manifests across environments: | ||
|
|
||
| - `-i output` tells Crane which rendered manifest directory to validate. | ||
| - `--validate-dir validate` tells Crane where to write validation reports and failure artifacts. | ||
|
|
||
| ```bash | ||
| crane validate \ | ||
| --context "${TARGET_CONTEXT}" \ | ||
| -i output \ | ||
| --validate-dir validate | ||
| ``` | ||
|
|
||
| Example output snippet (abbreviated): | ||
|
|
||
| ```text | ||
| INFO[0000] Scanned 3 distinct GVK+namespace tuples | ||
| INFO[0000] Validating in live mode against context "tgt" | ||
| Mode: live (context: tgt) | ||
|
coderabbitai[bot] marked this conversation as resolved.
Outdated
msajidmansoori12 marked this conversation as resolved.
Outdated
|
||
| ... | ||
| Summary: 3 scanned, 3 compatible, 0 incompatible | ||
| Result: PASSED — all resources compatible with target cluster | ||
| INFO[0000] Wrote validation report to validate/report.json | ||
| ``` | ||
|
msajidmansoori12 marked this conversation as resolved.
|
||
|
|
||
| What you can expect to see (example): | ||
|
|
||
| - Terminal output includes `Mode: live` and a compatibility summary/result | ||
| - For successful validation, `Result: PASSED` with all scanned resources marked compatible | ||
| - Validation report generated at `validate/report.json` | ||
| - If incompatibilities are found, failure artifacts are written under `validate/failures/` | ||
|
|
||
| Example report structure: | ||
|
|
||
| ```json | ||
| { | ||
| "mode": "live", | ||
| "clusterContext": "tgt-cluster", | ||
| "results": [ | ||
| { | ||
| "apiVersion": "apps/v1", | ||
| "kind": "Deployment", | ||
| "namespace": "target-namespace", | ||
| "resourcePlural": "deployments", | ||
| "status": "OK" | ||
| }, | ||
| { | ||
| "apiVersion": "v1", | ||
| "kind": "Secret", | ||
| "namespace": "target-namespace", | ||
| "resourcePlural": "secrets", | ||
| "status": "OK" | ||
| }, | ||
| { | ||
| "apiVersion": "v1", | ||
| "kind": "Service", | ||
| "namespace": "target-namespace", | ||
| "resourcePlural": "services", | ||
| "status": "OK" | ||
| } | ||
| ], | ||
| "totalScanned": 3, | ||
| "compatible": 3, | ||
| "incompatible": 0 | ||
| } | ||
| ``` | ||
|
|
||
| ## 7) Optional: Instructions-File Driven Transform | ||
|
|
||
| For repeatable pipelines, drive stage behavior with an instructions file: | ||
|
|
||
| Example `instructions.yaml`: | ||
|
|
||
| ```yaml | ||
| stages: | ||
| - KubernetesPlugin | ||
| - CustomStage | ||
| ``` | ||
|
|
||
| ```bash | ||
| crane transform \ | ||
| --instructions-file ./instructions.yaml | ||
| ``` | ||
|
|
||
| What you should see (example): | ||
|
|
||
| - Stage directories defined by the instructions file are created in `transform/` | ||
|
istein1 marked this conversation as resolved.
|
||
| - Transform runs exactly the stages listed in `instructions.yaml`, in the same order they are provided | ||
|
|
||
| ## 8) Apply Cleaned Manifests to Target Cluster | ||
|
|
||
| After validation passes and you are satisfied with the cleaned manifests, apply them to the target cluster: | ||
|
|
||
| ```bash | ||
| kubectl --context "${TARGET_CONTEXT}" apply -f output/output.yaml | ||
| ``` | ||
|
|
||
| Make sure the target namespace already exists before applying manifests, so namespace-scoped resources do not fail during apply. | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Export directory already exists | ||
|
|
||
| Use `--overwrite` with `crane export`. | ||
|
|
||
| ### Apply or validate directory already exists | ||
|
|
||
| Use `--overwrite` with `crane apply` or `crane validate`. | ||
|
|
||
| ### Existing custom stage blocks rerun | ||
|
|
||
| Use `crane transform --force` to regenerate stage directories. | ||
|
|
||
| ### Validation shows incompatibilities | ||
|
|
||
| - Check `validate/report.json` for `apiVersion`/`kind` mismatches | ||
| - Update transforms, then rerun `crane apply` and `crane validate` | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.