Skip to content

Commit 51370f3

Browse files
committed
fix(workshop): Keep the backgrounded EKS apply and its wait in one shell
A CodeBuild build spec runs each list item in its own shell, so the cluster apply started with `&` in one item could not be waited on in a later item: `wait <pid>` returned non-zero immediately and failed the build (the cluster apply was also orphaned at the command boundary). Move the background apply, the amd64-image wait, the GGUF start, and the `wait` into a single `- |` block so they share one shell.
1 parent 54ccdd5 commit 51370f3

1 file changed

Lines changed: 14 additions & 16 deletions

File tree

‎examples/k8s-autoscaling/workshop/infrastructure/buildspec.yaml‎

Lines changed: 14 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -99,29 +99,28 @@ phases:
9999
# The model bucket is created with the VPC target's siblings; make sure it
100100
# exists now so the GGUF conversion can start before the cluster is up.
101101
- terraform apply -input=false -auto-approve -target=aws_s3_bucket.models -target=aws_s3_bucket_server_side_encryption_configuration.models -target=aws_s3_bucket_public_access_block.models -target=aws_s3_bucket_versioning.models
102-
# Start the cluster apply in the background (it holds the state lock for
103-
# 12+ minutes) and, while it runs, wait for the amd64 image and start the
104-
# GGUF conversion. The conversion needs only the bucket (applied above)
105-
# and the amd64 gguf-convert image (it runs on the x86 fleet), not the
106-
# cluster and not the arm64 image. `terraform output` reads state without
107-
# taking the lock, so it is safe while the apply runs. Ordering the wait
108-
# after the apply starts is deliberate: a first attempt put the amd64
109-
# wait before the EKS apply and delayed the cluster by nine minutes.
102+
# Apply the cluster and, in parallel, start the GGUF conversion as soon as
103+
# the amd64 gguf-convert image exists. The conversion needs only the model
104+
# bucket (applied above) and that image (it runs on the x86 fleet), not the
105+
# cluster or the arm64 build, so it must not wait on the 12-minute cluster
106+
# apply. This whole block is ONE shell command: a CodeBuild build spec runs
107+
# each list item in a separate shell, so a background job started in one
108+
# item cannot be waited on in another. The cluster apply runs in the
109+
# background here and is waited on at the end of the same block.
110110
- |
111111
terraform apply -input=false -auto-approve -target=module.eks > /tmp/eks-apply.log 2>&1 &
112-
echo $! > /tmp/eks-apply.pid
113-
echo "EKS apply started in the background (pid $(cat /tmp/eks-apply.pid))"
114-
- |
112+
EKS_PID=$!
113+
echo "EKS apply started in the background (pid ${EKS_PID}); starting the GGUF conversion meanwhile"
115114
if [ -n "${GGUF_BUILD:-}" ] && [ -f /tmp/image-builds ]; then
116115
read -r AMD_ID ARM_ID < /tmp/image-builds
117-
echo "Waiting for the amd64 image build before starting the GGUF conversion"
116+
echo "Waiting for the amd64 image build"
118117
while true; do
119118
STATUS=$(aws codebuild batch-get-builds --ids "${AMD_ID}" --query 'builds[0].buildStatus' --output text)
120119
echo " $(date +%H:%M:%S) amd64 ${STATUS}"
121120
case "${STATUS}" in
122121
IN_PROGRESS) sleep 20 ;;
123122
SUCCEEDED) break ;;
124-
*) echo "amd64 image build failed: ${STATUS}"; exit 1 ;;
123+
*) echo "amd64 image build failed: ${STATUS}"; kill "${EKS_PID}" 2>/dev/null; exit 1 ;;
125124
esac
126125
done
127126
MODEL_BUCKET=$(terraform output -raw model_bucket)
@@ -131,9 +130,8 @@ phases:
131130
echo "gguf build started: ${GGUF_ID}"
132131
echo "${GGUF_ID}" > /tmp/gguf-build
133132
fi
134-
- |
135-
echo "Waiting for the EKS apply"
136-
if wait "$(cat /tmp/eks-apply.pid)"; then
133+
echo "Waiting for the EKS apply (pid ${EKS_PID})"
134+
if wait "${EKS_PID}"; then
137135
tail -n 3 /tmp/eks-apply.log
138136
else
139137
echo "EKS apply failed:"; tail -n 60 /tmp/eks-apply.log; exit 1

0 commit comments

Comments
 (0)