Skip to content

feat(storage): separate read and hedging thread pools - #16389

Open
ajayky-os wants to merge 1 commit into
googleapis:mainfrom
ajayky-os:perf/hedging_thread_pool
Open

feat(storage): separate read and hedging thread pools#16389
ajayky-os wants to merge 1 commit into
googleapis:mainfrom
ajayky-os:perf/hedging_thread_pool

feat(storage): separate read and hedging thread pools

edddb77
Select commit
Loading
Failed to load commit list.
Google Cloud Build / ubsan-storage-pr (cloud-cpp-testing-resources) succeeded Aug 28, 2026 in 8m 27s

Summary

Build Information

Trigger ubsan-storage-pr
Build 79f99bce-bd5a-46df-bf55-7ed79b04b119
Start 2026-08-28T04:39:06-07:00
Duration 7m17.402s
Status SUCCESS

Steps

Step Status Duration
kaniko-build SUCCESS 3m1.022s
download-runner-image SUCCESS 49.63s
build.sh SUCCESS 3m4.75s
remove-image SUCCESS 3.09s
cancel-in-progress-builds-for-PR SUCCESS 46.862s

Details

starting build "79f99bce-bd5a-46df-bf55-7ed79b04b119"

FETCHSOURCE
From https://github.com/googleapis/google-cloud-cpp
 * branch              edddb774b3e8c7ca9876b04897dd10c390771135 -> FETCH_HEAD
Updating files:   1% (313/23195)
Updating files:   2% (464/23195)
Updating files:   3% (696/23195)
Updating files:   4% (928/23195)
Updating files:   5% (1160/23195)
Updating files:   6% (1392/23195)
Updating files:   7% (1624/23195)
Updating files:   8% (1856/23195)
Updating files:   9% (2088/23195)
Updating files:  10% (2320/23195)
Updating files:  11% (2552/23195)
Updating files:  12% (2784/23195)
Updating files:  13% (3016/23195)
Updating files:  14% (3248/23195)
Updating files:  15% (3480/23195)
Updating files:  16% (3712/23195)
Updating files:  17% (3944/23195)
Updating files:  18% (4176/23195)
Updating files:  19% (4408/23195)
Updating files:  20% (4639/23195)
Updating files:  21% (4871/23195)
Updating files:  22% (5103/23195)
Updating files:  23% (5335/23195)
Updating files:  24% (5567/23195)
Updating files:  25% (5799/23195)
Updating files:  26% (6031/23195)
Updating files:  27% (6263/23195)
Updating files:  28% (6495/23195)
Updating files:  29% (6727/23195)
Updating files:  30% (6959/23195)
Updating files:  31% (7191/23195)
Updating files:  32% (7423/23195)
Updating files:  33% (7655/23195)
Updating files:  34% (7887/23195)
Updating files:  35% (8119/23195)
Updating files:  36% (8351/23195)
Updating files:  37% (8583/23195)
Updating files:  38% (8815/23195)
Updating files:  39% (9047/23195)
Updating files:  40% (9278/23195)
Updating files:  41% (9510/23195)
Updating files:  42% (9742/23195)
Updating files:  43% (9974/23195)
Updating files:  44% (10206/23195)
Updating files:  45% (10438/23195)
Updating files:  46% (10670/23195)
Updating files:  47% (10902/23195)
Updating files:  48% (11134/23195)
Updating files:  49% (11366/23195)
Updating files:  50% (11598/23195)
Updating files:  51% (11830/23195)
Updating files:  52% (12062/23195)
Updating files:  53% (12294/23195)
Updating files:  54% (12526/23195)
Updating files:  55% (12758/23195)
Updating files:  56% (12990/23195)
Updating files:  57% (13222/23195)
Updating files:  58% (13454/23195)
Updating files:  59% (13686/23195)
Updating files:  60% (13917/23195)
Updating files:  61% (14149/23195)
Updating files:  62% (14381/23195)
Updating files:  63% (14613/23195)
Updating files:  64% (14845/23195)
Updating files:  65% (15077/23195)
Updating files:  66% (15309/23195)
Updating files:  67% (15541/23195)
Updating files:  68% (15773/23195)
Updating files:  69% (16005/23195)
Updating files:  70% (16237/23195)
Updating files:  71% (16469/23195)
Updating files:  72% (16701/23195)
Updating files:  73% (16933/23195)
Updating files:  74% (17165/23195)
Updating files:  75% (17397/23195)
Updating files:  75% (17424/23195)
Updating files:  76% (17629/23195)
Updating files:  77% (17861/23195)
Updating files:  78% (18093/23195)
Updating files:  79% (18325/23195)
Updating files:  80% (18556/23195)
Updating files:  81% (18788/23195)
Updating files:  82% (19020/23195)
Updating files:  83% (19252/23195)
Updating files:  84% (19484/23195)
Updating files:  85% (19716/23195)
Updating files:  86% (19948/23195)
Updating files:  87% (20180/23195)
Updating files:  88% (20412/23195)
Updating files:  89% (20644/23195)
Updating files:  90% (20876/23195)
Updating files:  91% (21108/23195)
Updating files:  92% (21340/23195)
Updating files:  93% (21572/23195)
Updating files:  94% (21804/23195)
Updating files:  95% (22036/23195)
Updating files:  96% (22268/23195)
Updating files:  97% (22500/23195)
Updating files:  98% (22732/23195)
Updating files:  99% (22964/23195)
Updating files: 100% (23195/23195)
Updating files: 100% (23195/23195), done.
HEAD is now at edddb774 feat(storage): separate read and hedging thread pools
GitCommit:
edddb774b3e8c7ca9876b04897dd10c390771135
BUILD
Starting Step #4 - "cancel-in-progress-builds-for-PR"
Starting Step #0 - "kaniko-build"
Step #0 - "kaniko-build": Pulling image: gcr.io/kaniko-project/executor:v1.24.0-debug@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #4 - "cancel-in-progress-builds-for-PR": Pulling image: gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574: Pulling from google.com/cloudsdktool/cloud-sdk
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Pulling fs layer
Step #0 - "kaniko-build": gcr.io/kaniko-project/executor@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd: Pulling from kaniko-project/executor
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Download complete
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Download complete
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Download complete
Step #0 - "kaniko-build": 2fc842204170: Pulling fs layer
Step #0 - "kaniko-build": 8d1674b25e7b: Pulling fs layer
Step #0 - "kaniko-build": 9bdd0371dbb4: Pulling fs layer
Step #0 - "kaniko-build": 9aeeda79b717: Pulling fs layer
Step #0 - "kaniko-build": aaab912f1a4e: Pulling fs layer
Step #0 - "kaniko-build": 0006a7b536fb: Pulling fs layer
Step #0 - "kaniko-build": 3d08e2063109: Pulling fs layer
Step #0 - "kaniko-build": fc4517ec89af: Pulling fs layer
Step #0 - "kaniko-build": b892afb8bf07: Pulling fs layer
Step #0 - "kaniko-build": cdc2b9345183: Pulling fs layer
Step #0 - "kaniko-build": 3934701e2c3d: Pulling fs layer
Step #0 - "kaniko-build": 46be974dd378: Pulling fs layer
Step #0 - "kaniko-build": 2b61f8ddf6d9: Pulling fs layer
Step #0 - "kaniko-build": 3d08e2063109: Waiting
Step #0 - "kaniko-build": 3934701e2c3d: Waiting
Step #0 - "kaniko-build": b892afb8bf07: Waiting
Step #0 - "kaniko-build": cdc2b9345183: Waiting
Step #0 - "kaniko-build": 2b61f8ddf6d9: Waiting
Step #0 - "kaniko-build": 9aeeda79b717: Waiting
Step #0 - "kaniko-build": 0006a7b536fb: Waiting
Step #0 - "kaniko-build": 9bdd0371dbb4: Verifying Checksum
Step #0 - "kaniko-build": 9bdd0371dbb4: Download complete
Step #0 - "kaniko-build": 2fc842204170: Verifying Checksum
Step #0 - "kaniko-build": 2fc842204170: Pull complete
Step #0 - "kaniko-build": 8d1674b25e7b: Verifying Checksum
Step #0 - "kaniko-build": 8d1674b25e7b: Download complete
Step #0 - "kaniko-build": 8d1674b25e7b: Pull complete
Step #0 - "kaniko-build": 9bdd0371dbb4: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Download complete
Step #0 - "kaniko-build": 9aeeda79b717: Verifying Checksum
Step #0 - "kaniko-build": 9aeeda79b717: Download complete
Step #0 - "kaniko-build": aaab912f1a4e: Download complete
Step #0 - "kaniko-build": 0006a7b536fb: Verifying Checksum
Step #0 - "kaniko-build": 0006a7b536fb: Download complete
Step #0 - "kaniko-build": 9aeeda79b717: Pull complete
Step #0 - "kaniko-build": fc4517ec89af: Verifying Checksum
Step #0 - "kaniko-build": fc4517ec89af: Download complete
Step #0 - "kaniko-build": 3d08e2063109: Verifying Checksum
Step #0 - "kaniko-build": 3d08e2063109: Download complete
Step #0 - "kaniko-build": aaab912f1a4e: Pull complete
Step #0 - "kaniko-build": 46be974dd378: Download complete
Step #0 - "kaniko-build": 3934701e2c3d: Verifying Checksum
Step #0 - "kaniko-build": 3934701e2c3d: Download complete
Step #0 - "kaniko-build": 0006a7b536fb: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Download complete
Step #0 - "kaniko-build": 3d08e2063109: Pull complete
Step #0 - "kaniko-build": b892afb8bf07: Verifying Checksum
Step #0 - "kaniko-build": b892afb8bf07: Download complete
Step #0 - "kaniko-build": 2b61f8ddf6d9: Verifying Checksum
Step #0 - "kaniko-build": 2b61f8ddf6d9: Download complete
Step #0 - "kaniko-build": fc4517ec89af: Pull complete
Step #0 - "kaniko-build": cdc2b9345183: Verifying Checksum
Step #0 - "kaniko-build": cdc2b9345183: Download complete
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Pull complete
Step #0 - "kaniko-build": b892afb8bf07: Pull complete
Step #0 - "kaniko-build": cdc2b9345183: Pull complete
Step #0 - "kaniko-build": 3934701e2c3d: Pull complete
Step #0 - "kaniko-build": 46be974dd378: Pull complete
Step #0 - "kaniko-build": 2b61f8ddf6d9: Pull complete
Step #0 - "kaniko-build": Digest: sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": Status: Downloaded newer image for gcr.io/kaniko-project/executor@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": gcr.io/kaniko-project/executor:v1.24.0-debug@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Using dockerignore file: /workspace/ci/.dockerignore"
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Retrieving image fedora:40 from registry index.docker.io"
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Built cross stage deps: map[]"
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Executing 0 build triggers"
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Building stage 'fedora:40' [idx: '0', base-idx: '-1']"
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel/cache:34931edfd93bb824614aba9c723b34ef8db9a46a57dc352d0d8fd52896be0fb2..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Using caching version of cmd: RUN dnf install -y clang diffutils findutils gcc-c++ git jq lcov libcxx-devel         libcxxabi-devel libasan libubsan libtsan llvm patch python python3         python-pip tar unzip w3m wget which zip zlib-devel"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel/cache:5c06271b174395c7aaa648efa4a014359b480ceeffbd5b02612a719079db39fe..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y python3-devel"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel/cache:24994684b2c5b5b76a2f7b99cefbf726f05835d0ee77b1a7159d90be4ea3934e..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Using caching version of cmd: RUN pip3 install --upgrade pip"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel/cache:6af306fe29573ca00b915f75404759a5b32170ebe1959be491bc0ee19e89e72b..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Using caching version of cmd: RUN pip3 install setuptools wheel"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel/cache:b4dc53fdbf870eea2e48651c01453f76bacd8a48da20fc6869b2cd1bf754d631..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y java-latest-openjdk-devel"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel/cache:ef6c64df8688ff309fbb5665f29bfa6f88c12281bdf87a447b62cd297e5f52f8..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:26Z" level=info msg="Using caching version of cmd: RUN echo 'root:cloudcxx' | chpasswd"
Step #0 - "kaniko-build": time="2026-08-28T11:39:26Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel/cache:fbe000c8e80dd45f318baf402949b617263436e40c1064b001fa94f5895ab873..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:26Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y python3.10"
Step #0 - "kaniko-build": time="2026-08-28T11:39:26Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel/cache:17c1d7dae57b5da1708f11dfc71d1637f23d2f197c790087882d3104567cee23..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:26Z" level=info msg="Using caching version of cmd: RUN /var/tmp/ci/install-cloud-sdk.sh"
Step #0 - "kaniko-build": time="2026-08-28T11:39:26Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel/cache:8d0b3f88e82778fc9f8dad13f2e3b5d44f2a1c249b34171d9f0eb0a0875bf15c..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:26Z" level=info msg="Using caching version of cmd: RUN curl -o /usr/bin/bazelisk -sSL \"https://github.com/bazelbuild/bazelisk/releases/download/v1.28.1/bazelisk-linux-${ARCH}\" &&     chmod +x /usr/bin/bazelisk &&     ln -s /usr/bin/bazelisk /usr/bin/bazel"
Step #0 - "kaniko-build": time="2026-08-28T11:39:26Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel/cache:a70b0ca34659fc70526f8fab4f676e8c6f4de18099aaff7818a59e1c5ac34426..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:26Z" level=info msg="Using caching version of cmd: RUN wget -O go.tgz https://go.dev/dl/go1.25.4.linux-amd64.tar.gz"
Step #0 - "kaniko-build": time="2026-08-28T11:39:26Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel/cache:39ce4bd71ee2d154840fbb5bf2731021b93e5445822c5204b8f0eb9852192d47..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Using caching version of cmd: RUN tar -C /usr/local/ -xzf go.tgz"
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel/cache:a0bbcdb494f118ca6399af3a90ee90af95c3fda499c55771d29d41f5117510e1..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Using caching version of cmd: RUN go version"
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel/cache:3642b0feec738437872da4868d6cad1bcc39a6f25f989a4bada5b3eb8e078549..."
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Download complete
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Using caching version of cmd: RUN curl -fsSL https://github.com/googleapis/cloud-bigtable-clients-test/archive/v0.0.4.tar.gz |     tar -xzf - --strip-components=1"
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Unpacking rootfs as cmd COPY . /var/tmp/ci requires it."
Step #0 - "kaniko-build": time="2026-08-28T11:39:30Z" level=info msg="ARG NCPU=4"
Step #0 - "kaniko-build": time="2026-08-28T11:39:30Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:39:30Z" level=info msg="ARG ARCH=amd64"
Step #0 - "kaniko-build": time="2026-08-28T11:39:30Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:39:30Z" level=info msg="RUN dnf install -y clang diffutils findutils gcc-c++ git jq lcov libcxx-devel         libcxxabi-devel libasan libubsan libtsan llvm patch python python3         python-pip tar unzip w3m wget which zip zlib-devel"
Step #0 - "kaniko-build": time="2026-08-28T11:39:30Z" level=info msg="Found cached layer, extracting to filesystem"
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": Digest: sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": Status: Downloaded newer image for gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": + test -z 16389
Step #4 - "cancel-in-progress-builds-for-PR": ++ gcloud builds describe --region us-east1 --format 'value(create_time)' 79f99bce-bd5a-46df-bf55-7ed79b04b119
Step #4 - "cancel-in-progress-builds-for-PR": + ctime=2026-08-28T11:38:12.228215Z
Step #4 - "cancel-in-progress-builds-for-PR": + query=tags=pr
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND tags=16389'
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND substitutions.COMMIT_SHA != edddb774b3e8c7ca9876b04897dd10c390771135'
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND create_time < 2026-08-28T11:38:12.228215Z'
Step #4 - "cancel-in-progress-builds-for-PR": + gcloud builds list --region us-east1 --ongoing '--format=value(id)' --filter 'tags=pr AND tags=16389 AND substitutions.COMMIT_SHA != edddb774b3e8c7ca9876b04897dd10c390771135 AND create_time < 2026-08-28T11:38:12.228215Z'
Step #4 - "cancel-in-progress-builds-for-PR": + xargs -r -t gcloud builds cancel --region us-east1
Step #4 - "cancel-in-progress-builds-for-PR": WARNING: The following filter keys were not present in any resource : create_time, substitutions.COMMIT_SHA, tags
Finished Step #4 - "cancel-in-progress-builds-for-PR"
Step #0 - "kaniko-build": time="2026-08-28T11:40:16Z" level=info msg="RUN dnf makecache && dnf install -y python3-devel"
Step #0 - "kaniko-build": time="2026-08-28T11:40:16Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:40:18Z" level=info msg="RUN pip3 install --upgrade pip"
Step #0 - "kaniko-build": time="2026-08-28T11:40:18Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:40:19Z" level=info msg="RUN pip3 install setuptools wheel"
Step #0 - "kaniko-build": time="2026-08-28T11:40:19Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:40:20Z" level=info msg="RUN dnf makecache && dnf install -y java-latest-openjdk-devel"
Step #0 - "kaniko-build": time="2026-08-28T11:40:20Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="RUN echo 'root:cloudcxx' | chpasswd"
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="WORKDIR /var/tmp/build"
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="Cmd: workdir"
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="Changed working directory to /var/tmp/build"
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="Creating directory /var/tmp/build with uid -1 and gid -1"
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="Taking snapshot of files..."
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="COPY . /var/tmp/ci"
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="Taking snapshot of files..."
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="WORKDIR /var/tmp/downloads"
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="Cmd: workdir"
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="Changed working directory to /var/tmp/downloads"
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="Creating directory /var/tmp/downloads with uid -1 and gid -1"
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="Taking snapshot of files..."
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="RUN dnf makecache && dnf install -y python3.10"
Step #0 - "kaniko-build": time="2026-08-28T11:40:49Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:40:53Z" level=info msg="ENV CLOUDSDK_PYTHON=python3.10"
Step #0 - "kaniko-build": time="2026-08-28T11:40:53Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:40:53Z" level=info msg="RUN /var/tmp/ci/install-cloud-sdk.sh"
Step #0 - "kaniko-build": time="2026-08-28T11:40:53Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:00Z" level=info msg="ENV CLOUD_SDK_LOCATION=/usr/local/google-cloud-sdk"
Step #0 - "kaniko-build": time="2026-08-28T11:42:00Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:42:00Z" level=info msg="ENV PATH=${CLOUD_SDK_LOCATION}/bin:${PATH}"
Step #0 - "kaniko-build": time="2026-08-28T11:42:00Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:42:00Z" level=info msg="RUN curl -o /usr/bin/bazelisk -sSL \"https://github.com/bazelbuild/bazelisk/releases/download/v1.28.1/bazelisk-linux-${ARCH}\" &&     chmod +x /usr/bin/bazelisk &&     ln -s /usr/bin/bazelisk /usr/bin/bazel"
Step #0 - "kaniko-build": time="2026-08-28T11:42:00Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:00Z" level=info msg="WORKDIR /var/tmp/downloads"
Step #0 - "kaniko-build": time="2026-08-28T11:42:00Z" level=info msg="Cmd: workdir"
Step #0 - "kaniko-build": time="2026-08-28T11:42:00Z" level=info msg="Changed working directory to /var/tmp/downloads"
Step #0 - "kaniko-build": time="2026-08-28T11:42:00Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:42:00Z" level=info msg="RUN wget -O go.tgz https://go.dev/dl/go1.25.4.linux-amd64.tar.gz"
Step #0 - "kaniko-build": time="2026-08-28T11:42:00Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:02Z" level=info msg="RUN tar -C /usr/local/ -xzf go.tgz"
Step #0 - "kaniko-build": time="2026-08-28T11:42:02Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:13Z" level=info msg="ENV GO_LOCATION=/usr/local/go"
Step #0 - "kaniko-build": time="2026-08-28T11:42:13Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:42:13Z" level=info msg="ENV PATH=${GO_LOCATION}/bin:${PATH}"
Step #0 - "kaniko-build": time="2026-08-28T11:42:13Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:42:13Z" level=info msg="RUN go version"
Step #0 - "kaniko-build": time="2026-08-28T11:42:13Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:13Z" level=info msg="WORKDIR /var/tmp/downloads/cloud-bigtable-clients-test"
Step #0 - "kaniko-build": time="2026-08-28T11:42:13Z" level=info msg="Cmd: workdir"
Step #0 - "kaniko-build": time="2026-08-28T11:42:13Z" level=info msg="Changed working directory to /var/tmp/downloads/cloud-bigtable-clients-test"
Step #0 - "kaniko-build": time="2026-08-28T11:42:13Z" level=info msg="Creating directory /var/tmp/downloads/cloud-bigtable-clients-test with uid -1 and gid -1"
Step #0 - "kaniko-build": time="2026-08-28T11:42:13Z" level=info msg="Taking snapshot of files..."
Step #0 - "kaniko-build": time="2026-08-28T11:42:13Z" level=info msg="RUN curl -fsSL https://github.com/googleapis/cloud-bigtable-clients-test/archive/v0.0.4.tar.gz |     tar -xzf - --strip-components=1"
Step #0 - "kaniko-build": time="2026-08-28T11:42:13Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:14Z" level=info msg="Pushing image to us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel:79f99bce-bd5a-46df-bf55-7ed79b04b119"
Step #0 - "kaniko-build": time="2026-08-28T11:42:14Z" level=info msg="Pushed us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel@sha256:9c099193430ca97799ba63d483c47be36c038a52979f4f950596e9dc25c76e2c"
Finished Step #0 - "kaniko-build"
Starting Step #1 - "download-runner-image"
Step #1 - "download-runner-image": Pulling image: us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel:79f99bce-bd5a-46df-bf55-7ed79b04b119
Step #1 - "download-runner-image": 79f99bce-bd5a-46df-bf55-7ed79b04b119: Pulling from cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel
Step #1 - "download-runner-image": fc4518b6c5f9: Pulling fs layer
Step #1 - "download-runner-image": e9c85922a459: Pulling fs layer
Step #1 - "download-runner-image": 95308800b832: Pulling fs layer
Step #1 - "download-runner-image": 8b2ff4c88c64: Pulling fs layer
Step #1 - "download-runner-image": bc63d92173ea: Pulling fs layer
Step #1 - "download-runner-image": be065012be5f: Pulling fs layer
Step #1 - "download-runner-image": b0edebbe18f8: Pulling fs layer
Step #1 - "download-runner-image": fb9f736a2019: Pulling fs layer
Step #1 - "download-runner-image": 77022393f332: Pulling fs layer
Step #1 - "download-runner-image": 1cd55b15b92d: Pulling fs layer
Step #1 - "download-runner-image": 09399c47c56d: Pulling fs layer
Step #1 - "download-runner-image": 54d1c0a309e4: Pulling fs layer
Step #1 - "download-runner-image": 1b9f6235d230: Pulling fs layer
Step #1 - "download-runner-image": fc6391f6e219: Pulling fs layer
Step #1 - "download-runner-image": 9f6ec92ec428: Pulling fs layer
Step #1 - "download-runner-image": 5fa5d251d40c: Pulling fs layer
Step #1 - "download-runner-image": 840c54a5c1de: Pulling fs layer
Step #1 - "download-runner-image": 75dcbe9f0eef: Pulling fs layer
Step #1 - "download-runner-image": 77022393f332: Waiting
Step #1 - "download-runner-image": 1cd55b15b92d: Waiting
Step #1 - "download-runner-image": 09399c47c56d: Waiting
Step #1 - "download-runner-image": 54d1c0a309e4: Waiting
Step #1 - "download-runner-image": 840c54a5c1de: Waiting
Step #1 - "download-runner-image": 5fa5d251d40c: Waiting
Step #1 - "download-runner-image": 75dcbe9f0eef: Waiting
Step #1 - "download-runner-image": 1b9f6235d230: Waiting
Step #1 - "download-runner-image": fc6391f6e219: Waiting
Step #1 - "download-runner-image": 9f6ec92ec428: Waiting
Step #1 - "download-runner-image": b0edebbe18f8: Waiting
Step #1 - "download-runner-image": fb9f736a2019: Waiting
Step #1 - "download-runner-image": bc63d92173ea: Verifying Checksum
Step #1 - "download-runner-image": bc63d92173ea: Download complete
Step #1 - "download-runner-image": 8b2ff4c88c64: Verifying Checksum
Step #1 - "download-runner-image": 8b2ff4c88c64: Download complete
Step #1 - "download-runner-image": b0edebbe18f8: Verifying Checksum
Step #1 - "download-runner-image": b0edebbe18f8: Download complete
Step #1 - "download-runner-image": fb9f736a2019: Download complete
Step #1 - "download-runner-image": 77022393f332: Download complete
Step #1 - "download-runner-image": 1cd55b15b92d: Verifying Checksum
Step #1 - "download-runner-image": 1cd55b15b92d: Download complete
Step #1 - "download-runner-image": 95308800b832: Verifying Checksum
Step #1 - "download-runner-image": 95308800b832: Download complete
Step #1 - "download-runner-image": 1b9f6235d230: Verifying Checksum
Step #1 - "download-runner-image": 1b9f6235d230: Download complete
Step #1 - "download-runner-image": fc4518b6c5f9: Verifying Checksum
Step #1 - "download-runner-image": fc4518b6c5f9: Download complete
Step #1 - "download-runner-image": 09399c47c56d: Verifying Checksum
Step #1 - "download-runner-image": 09399c47c56d: Download complete
Step #1 - "download-runner-image": 5fa5d251d40c: Verifying Checksum
Step #1 - "download-runner-image": 5fa5d251d40c: Download complete
Step #1 - "download-runner-image": 840c54a5c1de: Download complete
Step #1 - "download-runner-image": 75dcbe9f0eef: Verifying Checksum
Step #1 - "download-runner-image": 75dcbe9f0eef: Download complete
Step #1 - "download-runner-image": 9f6ec92ec428: Verifying Checksum
Step #1 - "download-runner-image": 9f6ec92ec428: Download complete
Step #1 - "download-runner-image": fc6391f6e219: Verifying Checksum
Step #1 - "download-runner-image": fc6391f6e219: Download complete
Step #1 - "download-runner-image": be065012be5f: Verifying Checksum
Step #1 - "download-runner-image": be065012be5f: Download complete
Step #1 - "download-runner-image": e9c85922a459: Verifying Checksum
Step #1 - "download-runner-image": e9c85922a459: Download complete
Step #1 - "download-runner-image": fc4518b6c5f9: Pul
...
[Logs truncated due to log size limitations. For full logs, see https://storage.cloud.google.com/cloud-cpp-community-publiclogs/logs/google-cloud-cpp/16389/edddb774b3e8c7ca9876b04897dd10c390771135/fedora-latest-bazel-ubsan-__default__/log-79f99bce-bd5a-46df-bf55-7ed79b04b119.txt.]
...
gle/cloud/oslogin/quickstart\,google/cloud/asset/quickstart\,google/cloud/configdelivery/quickstart\,google/cloud/workstations/quickstart\,google/cloud/datastream/quickstart\,google/cloud/containeranalysis/quickstart\,google/cloud/ids/quickstart\,google/cloud/language/quickstart\,google/cloud/logging/quickstart\,google/cloud/vectorsearch/quickstart\,google/cloud/gkeconnect/quickstart\,google/cloud/vision/quickstart\,google/cloud/vmmigration/quickstart\,google/cloud/oracledatabase/quickstart\,google/cloud/datafusion/quickstart\,google/cloud/accessapproval/quickstart\,google/cloud/notebooks/quickstart\,google/cloud/discoveryengine/quickstart\,google/cloud/translate/quickstart\,google/cloud/parametermanager/quickstart\,google/cloud/scheduler/quickstart\,google/cloud/migrationcenter/quickstart\,google/cloud/serviceusage/quickstart\,google/cloud/sql/quickstart\,google/cloud/functions/quickstart\,google/cloud/apiregistry/quickstart\,google/cloud/tpu/quickstart\,google/cloud/maintenance/quickstart\,google/cloud/bigquerycontrol/quickstart\,google/cloud/spanner/quickstart\,google/cloud/aiplatform/quickstart\,google/cloud/domains/quickstart\,google/cloud/metastore/quickstart\,google/cloud/managedidentities/quickstart\,google/cloud/apikeys/quickstart\,google/cloud/lustre/quickstart\,google/cloud/billing/quickstart\,google/cloud/contentwarehouse/quickstart\,google/cloud/agentidentity/quickstart\,google/cloud/config/quickstart\,google/cloud/essentialcontacts/quickstart\,google/cloud/talent/quickstart\,google/cloud/cloudquotas/quickstart\,google/cloud/privilegedaccessmanager/quickstart\,google/cloud/composer/quickstart\,google/cloud/gkehub/quickstart\,google/cloud/iap/quickstart\,google/cloud/geminidataanalytics/quickstart\,google/cloud/cloudsecuritycompliance/quickstart\,google/cloud/gkemulticloud/quickstart\,google/cloud/shell/quickstart\,google/cloud/hypercomputecluster/quickstart\,google/cloud/telcoautomation/quickstart\,google/cloud/profiler/quickstart\,google/cloud/gkerecommender/quickstart\,google/cloud/auditmanager/quickstart\,google/cloud/chronicle/quickstart\,google/cloud/automl/quickstart\,google/cloud/binaryauthorization/quickstart\,google/cloud/workloadmanager/quickstart\,google/cloud/alloydb/quickstart\,google/cloud/visionai/quickstart\,google/cloud/trace/quickstart\,google/cloud/policytroubleshooter/quickstart\,google/cloud/artifactregistry/quickstart\,google/cloud/beyondcorp/quickstart\,google/cloud/certificatemanager/quickstart\,google/cloud/channel/quickstart\,google/cloud/commerce/quickstart\,google/cloud/recommender/quickstart\,google/cloud/networkmanagement/quickstart\,google/cloud/licensemanager/quickstart\,google/cloud/iam/quickstart\,google/cloud/gkebackup/quickstart\,google/cloud/datacatalog/quickstart\,google/cloud/support/quickstart\,google/cloud/dataform/quickstart\,google/cloud/advisorynotifications/quickstart\,google/cloud/batch/quickstart\,google/cloud/videointelligence/quickstart\,google/cloud/confidentialcomputing/quickstart\,google/cloud/universe_domain/demo\,google/cloud/dialogflow_es/quickstart\,google/cloud/servicehealth/quickstart\,google/cloud/secretmanager/quickstart\,google/cloud/recaptchaenterprise/quickstart\,google/cloud/kms/quickstart\,google/cloud/workflows/quickstart\,google/cloud/tasks/quickstart\,google/cloud/publicca/quickstart\,google/cloud/resourcemanager/quickstart\,google/cloud/managedkafka/quickstart\,google/cloud/oauth2/quickstart\,google/cloud/ces/quickstart\,google/cloud/networkservices/quickstart\,google/cloud/cloudcontrolspartner/quickstart\,google/cloud/baremetalsolution/quickstart\,google/cloud/memcache/quickstart\,google/cloud/privateca/quickstart\,google/cloud/securitycenter/quickstart\,google/cloud/apphub/quickstart\,google/cloud/devicestreaming/quickstart\,google/cloud/bigtable/quickstart\,google/cloud/servicemanagement/quickstart\,google/cloud/financialservices/quickstart\,google/cloud/storagecontrol/quickstart\,google/cloud/servicedirectory/quickstart\,google/cloud/redis/quickstart\,google/cloud/vpcaccess/quickstart\,google/cloud/storage/quickstart\,google/cloud/documentai/quickstart\,google/cloud/networksecurity/quickstart\,google/cloud/websecurityscanner/quickstart\,google/cloud/speech/quickstart\,google/cloud/orgpolicy/quickstart\,google/cloud/memorystore/quickstart\,google/cloud/apigeeconnect/quickstart\,google/cloud/contactcenterinsights/quickstart\,google/cloud/opentelemetry/quickstart\,google/cloud/ftp/quickstart\,google/cloud/eventarc/quickstart\,google/cloud/backupdr/quickstart\,google/cloud/datamigration/quickstart\,google/cloud/edgenetwork/quickstart\,google/cloud/connectors/quickstart\,google/cloud/appengine/quickstart\,google/cloud/dlp/quickstart\,google/cloud/policysimulator/quickstart\,google/cloud/apigateway/quickstart\,google/cloud/deploy/quickstart --flaky_test_attempts=3 --test_env=GOOGLE_CLOUD_PROJECT=cloud-cpp-testing-resources --test_env=GOOGLE_CLOUD_CPP_TEST_REGION=us-central1 --test_env=GOOGLE_CLOUD_CPP_NON_US_TEST_REGION=asia-southeast1 --test_env=GOOGLE_CLOUD_CPP_US_EAST_TEST_REGION=us-east4 --test_env=GOOGLE_CLOUD_CPP_TEST_ZONE=us-central1-a --test_env=GOOGLE_CLOUD_CPP_TEST_ORGANIZATION=1006341795026 --test_env=GOOGLE_CLOUD_CPP_TEST_SERVICE_ACCOUNT_KEYFILE=/workspace/ci/etc/invalidated-keyfile.json --test_env=GOOGLE_CLOUD_CPP_AUTO_RUN_EXAMPLES=yes --test_env=GOOGLE_CLOUD_CPP_EXPERIMENTAL_LOG_CONFIG=lastN\,1024\,WARNING --test_env=GOOGLE_CLOUD_CPP_ENABLE_TRACING=rpc\,rpc-streams --test_env=GOOGLE_CLOUD_CPP_TRACING_OPTIONS=single_line_mode=off\,truncate_string_field_longer_than=512 --test_env=CLOUD_STORAGE_ENABLE_TRACING=raw-client\,rpc\,rpc-streams --test_env=HOME=/h --test_env=GOOGLE_CLOUD_CPP_IAM_CREDENTIALS_TEST_SERVICE_ACCOUNT=iam-credentials-test-sa@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_IAM_TEST_SERVICE_ACCOUNT=iam-test-sa@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_IAM_INVALID_TEST_SERVICE_ACCOUNT=invalid-test-account@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_IAM_QUOTA_LIMITED_INTEGRATION_TESTS= --test_env=GOOGLE_CLOUD_CPP_IAM_QUOTA_LIMITED_SAMPLES= --test_env=GOOGLE_CLOUD_CPP_BIGTABLE_TEST_INSTANCE_ID=test-instance --test_env=GOOGLE_CLOUD_CPP_BIGTABLE_TEST_CLUSTER_ID=test-instance-c1 --test_env=GOOGLE_CLOUD_CPP_BIGTABLE_TEST_ZONE_A=us-central1-b --test_env=GOOGLE_CLOUD_CPP_BIGTABLE_TEST_ZONE_B=us-central1-c --test_env=GOOGLE_CLOUD_CPP_BIGTABLE_TEST_SERVICE_ACCOUNT=bigtable-test-iam-sa@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=ENABLE_BIGTABLE_ADMIN_INTEGRATION_TESTS=no --test_env=GOOGLE_CLOUD_CPP_PUBSUB_TEST_IMPERSONATED_SERVICE_ACCOUNT=pubsub-impersonate-test-sa@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_REST_TEST_SIGNING_SERVICE_ACCOUNT=kokoro-run@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_BUCKET_NAME=gcs-grpc-team-cloud-cpp-testing-bucket --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_FOLDER_BUCKET_NAME=cloud-cpp-testing-folder-bucket --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_DESTINATION_BUCKET_NAME=gcs-grpc-team-cloud-cpp-testing-regional --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_REGION_ID=us-central1 --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_TOPIC_NAME=projects/cloud-cpp-testing-resources/topics/gcs-changes --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_SERVICE_ACCOUNT=storage-test-iam-sa@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_SIGNING_SERVICE_ACCOUNT=kokoro-run@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_HMAC_SERVICE_ACCOUNT=fake-service-account-hmac@example.com --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_CMEK_KEY=projects/cloud-cpp-testing-resources/locations/us/keyRings/gcs-testing-us-kr/cryptoKeys/integration-tests-key --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_SIGNING_CONFORMANCE_FILENAME=/workspace/google/cloud/storage/tests/v4_signatures.json --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_GZIP_FILENAME=/workspace/ci/data/fox.txt.gz --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_ROOTS_PEM=/dev/shm/roots.pem --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_ALTERNATIVE_HOSTS=private.googleapis.com\,restricted.googleapis.com --test_env=GOOGLE_CLOUD_CPP_SPANNER_SLOW_INTEGRATION_TESTS= --test_env=GOOGLE_CLOUD_CPP_SPANNER_TEST_INSTANCE_ID=test-instance --test_env=GOOGLE_CLOUD_CPP_SPANNER_TEST_SERVICE_ACCOUNT=spanner-iam-test-sa@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_SPANNER_DEFAULT_ENDPOINT= --test_env=GOOGLE_CLOUD_CPP_SPANNER_DEFAULT_AUTHORITY= --test_env=GOOGLE_CLOUD_CPP_BATCH_TEST_TEMPLATE_NAME=cloud-batch-sample-template --test_env=GOOGLE_CLOUD_CPP_REST_TEST_KEY_FILE_JSON=/dev/shm/key-2026-08.json --test_env=GOOGLE_CLOUD_CPP_BIGTABLE_TEST_KEY_FILE_JSON=/dev/shm/key-2026-08.json --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_KEY_FILE_JSON=/dev/shm/key-2026-08.json --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_KEY_FILE_P12=/dev/shm/key-2026-08.p12 --test_env=GOOGLE_CLOUD_CPP_CLIENT_SSL_CERT_FILE=/dev/shm/client.crt --test_env=GOOGLE_CLOUD_CPP_CLIENT_SSL_CERT_CHAIN_FILE=/dev/shm/client.chain.crt --test_env=GOOGLE_CLOUD_CPP_CLIENT_SSL_KEY_FILE=/dev/shm/client.private.pem --test_tag_filters=integration-test\,-ud-only --test_env=TESTBENCH_SHA=62d7b032a61a325e9c974181b7334879b26aca0ad4a5a2d33f4ea763ee527377\ \ - --test_env=TESTBENCH_CONTENTS_SHA=62a135c39b301f36f12cff75603cc49bef96d372e07521c354c8e6f6ad48e5fb\ \ - --test_env=CLOUD_STORAGE_EMULATOR_ENDPOINT=http://localhost:8585 --test_env=CLOUD_STORAGE_EXPERIMENTAL_GRPC_TESTBENCH_ENDPOINT=localhost:8000 --test_env=HTTPBIN_ENDPOINT=http://localhost:8585/httpbin --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_HMAC_SERVICE_ACCOUNT=fake-service-account-sign@example.com -- //google/cloud/storage/...:all -//google/cloud/storage/tests:error_injection_integration_test-default -//google/cloud/storage/tests:error_injection_integration_test-grpc-metadata -//google/cloud/storage/examples:storage_policy_doc_samples -//google/cloud/storage/examples:storage_signed_url_v2_samples -//google/cloud/storage/examples:storage_signed_url_v4_samples -//google/cloud/storage/tests:alternative_endpoint_integration_test-default -//google/cloud/storage/tests:alternative_endpoint_integration_test-grpc-metadata -//google/cloud/storage/tests:key_file_integration_test-default -//google/cloud/storage/tests:key_file_integration_test-grpc-metadata -//google/cloud/storage/tests:signed_url_integration_test-default -//google/cloud/storage/tests:signed_url_integration_test-grpc-metadata -//google/cloud/storage/tests:unified_credentials_integration_test-default -//google/cloud/storage/tests:unified_credentials_integration_test-grpc-metadata
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": INFO: Invocation ID: 9d0bc2c7-b279-4123-9cf2-07b99de2f40e
Step #2 - "build.sh": Computing main repo mapping: 
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": Loading: 
Step #2 - "build.sh": Loading: 0 packages loaded
Step #2 - "build.sh": WARNING: Build option --test_env has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
Step #2 - "build.sh": Analyzing: 349 targets (2 packages loaded, 0 targets configured)
Step #2 - "build.sh": Analyzing: 349 targets (2 packages loaded, 0 targets configured)
Step #2 - "build.sh": 
Step #2 - "build.sh": Analyzing: 349 targets (3 packages loaded, 15538 targets configured)
Step #2 - "build.sh": [2,332 / 4,939] checking cached actions
Step #2 - "build.sh": INFO: Analyzed 349 targets (3 packages loaded, 15723 targets configured).
Step #2 - "build.sh": [10,850 / 10,995] Testing //google/cloud/storage/tests:decompressive_transcoding_integration_test-default; Downloading google/cloud/storage/tests/decompressive_transcoding_integration_test-default/test.xml; 0s remote-cache ... (32 actions, 0 running)
Step #2 - "build.sh": INFO: Found 204 targets and 145 test targets...
Step #2 - "build.sh": INFO: Elapsed time: 3.411s, Critical Path: 0.60s
Step #2 - "build.sh": INFO: 146 processes: 11194 action cache hit, 183 remote cache hit.
Step #2 - "build.sh": INFO: Build completed successfully, 146 total actions
Step #2 - "build.sh": //google/cloud/storage/examples:storage_async_samples           (cached) PASSED in 44.2s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_acl_samples      (cached) PASSED in 1.7s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_autoclass_samples (cached) PASSED in 1.5s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_cors_samples     (cached) PASSED in 2.8s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_default_kms_key_samples (cached) PASSED in 1.5s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_encryption_enforcement_samples (cached) PASSED in 7.1s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_iam_samples      (cached) PASSED in 3.0s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_object_retention_samples (cached) PASSED in 2.8s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_requester_pays_samples (cached) PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_samples          (cached) PASSED in 1.5s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_soft_delete_samples (cached) PASSED in 1.6s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_client_initialization_samples (cached) PASSED in 1.8s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_client_per_operation_samples (cached) PASSED in 1.5s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_default_object_acl_samples (cached) PASSED in 1.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_event_based_hold_samples (cached) PASSED in 2.9s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_grpc_samples            (cached) PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_lifecycle_management_samples (cached) PASSED in 1.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_notification_samples    (cached) PASSED in 1.6s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_acl_samples      (cached) PASSED in 1.8s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_cmek_samples     (cached) PASSED in 1.7s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_csek_samples     (cached) PASSED in 1.7s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_file_transfer_samples (cached) PASSED in 1.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_hold_samples     (cached) PASSED in 1.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_preconditions_samples (cached) PASSED in 3.0s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_resumable_write_samples (cached) PASSED in 1.6s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_retention_samples (cached) PASSED in 1.3s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_rewrite_samples  (cached) PASSED in 1.7s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_samples          (cached) PASSED in 6.0s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_soft_delete_samples (cached) PASSED in 1.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_versioning_samples (cached) PASSED in 3.0s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_otel_samples            (cached) PASSED in 1.2s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_public_object_samples   (cached) PASSED in 1.5s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_quickstart              (cached) PASSED in 1.5s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_retention_policy_samples (cached) PASSED in 3.2s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_service_account_samples (cached) PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_website_samples         (cached) PASSED in 1.9s
Step #2 - "build.sh": //google/cloud/storage/tests:async_client_integration_test-default (cached) PASSED in 153.7s
Step #2 - "build.sh": //google/cloud/storage/tests:async_client_integration_test-grpc-metadata (cached) PASSED in 175.1s
Step #2 - "build.sh": //google/cloud/storage/tests:auto_finalize_integration_test-default (cached) PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/storage/tests:auto_finalize_integration_test-grpc-metadata (cached) PASSED in 30.3s
Step #2 - "build.sh": //google/cloud/storage/tests:bucket_acl_integration_test-default (cached) PASSED in 3.1s
Step #2 - "build.sh": //google/cloud/storage/tests:bucket_acl_integration_test-grpc-metadata (cached) PASSED in 83.3s
Step #2 - "build.sh": //google/cloud/storage/tests:bucket_integration_test-default    (cached) PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/storage/tests:bucket_integration_test-grpc-metadata (cached) PASSED in 156.4s
Step #2 - "build.sh": //google/cloud/storage/tests:create_client_integration_test-default (cached) PASSED in 3.0s
Step #2 - "build.sh": //google/cloud/storage/tests:create_client_integration_test-grpc-metadata (cached) PASSED in 3.2s
Step #2 - "build.sh": //google/cloud/storage/tests:curl_sign_blob_integration_test-default (cached) PASSED in 1.3s
Step #2 - "build.sh": //google/cloud/storage/tests:curl_sign_blob_integration_test-grpc-metadata (cached) PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/storage/tests:decompressive_transcoding_integration_test-default (cached) PASSED in 4.6s
Step #2 - "build.sh": //google/cloud/storage/tests:decompressive_transcoding_integration_test-grpc-metadata (cached) PASSED in 3.6s
Step #2 - "build.sh": //google/cloud/storage/tests:default_object_acl_integration_test-default (cached) PASSED in 1.6s
Step #2 - "build.sh": //google/cloud/storage/tests:default_object_acl_integration_test-grpc-metadata (cached) PASSED in 62.8s
Step #2 - "build.sh": //google/cloud/storage/tests:error_parsing_integration_test-default (cached) PASSED in 1.6s
Step #2 - "build.sh": //google/cloud/storage/tests:error_parsing_integration_test-grpc-metadata (cached) PASSED in 3.5s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_bucket_metadata_integration_test-default (cached) PASSED in 4.3s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_bucket_metadata_integration_test-grpc-metadata (cached) PASSED in 9.1s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_integration_test-default      (cached) PASSED in 2.5s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_integration_test-grpc-metadata (cached) PASSED in 30.2s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_acl_integration_test-default (cached) PASSED in 58.8s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_acl_integration_test-grpc-metadata (cached) PASSED in 61.7s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_media_integration_test-default (cached) PASSED in 2.9s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_media_integration_test-grpc-metadata (cached) PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_metadata_integration_test-default (cached) PASSED in 7.2s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_metadata_integration_test-grpc-metadata (cached) PASSED in 41.4s
Step #2 - "build.sh": //google/cloud/storage/tests:hmac_key_integration_test-default  (cached) PASSED in 1.4s
Step #2 - "build.sh": //google/cloud/storage/tests:hmac_key_integration_test-grpc-metadata (cached) PASSED in 1.7s
Step #2 - "build.sh": //google/cloud/storage/tests:mtls_object_basic_crud_integration_test-default (cached) PASSED in 3.1s
Step #2 - "build.sh": //google/cloud/storage/tests:mtls_object_basic_crud_integration_test-grpc-metadata (cached) PASSED in 1.8s
Step #2 - "build.sh": //google/cloud/storage/tests:notification_integration_test-default (cached) PASSED in 2.1s
Step #2 - "build.sh": //google/cloud/storage/tests:notification_integration_test-grpc-metadata (cached) PASSED in 17.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_basic_crud_integration_test-default (cached) PASSED in 3.2s
Step #2 - "build.sh": //google/cloud/storage/tests:object_basic_crud_integration_test-grpc-metadata (cached) PASSED in 62.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_checksum_integration_test-default (cached) PASSED in 1.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_checksum_integration_test-grpc-metadata (cached) PASSED in 46.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_compose_many_integration_test-default (cached) PASSED in 1.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_compose_many_integration_test-grpc-metadata (cached) PASSED in 67.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_file_integration_test-default (cached) PASSED in 3.0s
Step #2 - "build.sh": //google/cloud/storage/tests:object_file_integration_test-grpc-metadata (cached) PASSED in 89.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_file_multi_threaded_test-default (cached) PASSED in 2.1s
Step #2 - "build.sh": //google/cloud/storage/tests:object_file_multi_threaded_test-grpc-metadata (cached) PASSED in 9.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_hash_integration_test-default (cached) PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_hash_integration_test-grpc-metadata (cached) PASSED in 60.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_insert_integration_test-default (cached) PASSED in 2.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_insert_integration_test-grpc-metadata (cached) PASSED in 123.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_insert_preconditions_integration_test-default (cached) PASSED in 1.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_insert_preconditions_integration_test-grpc-metadata (cached) PASSED in 67.0s
Step #2 - "build.sh": //google/cloud/storage/tests:object_integration_test-default    (cached) PASSED in 2.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_integration_test-grpc-metadata (cached) PASSED in 171.2s
Step #2 - "build.sh": //google/cloud/storage/tests:object_list_objects_versions_integration_test-default (cached) PASSED in 2.0s
Step #2 - "build.sh": //google/cloud/storage/tests:object_list_objects_versions_integration_test-grpc-metadata (cached) PASSED in 29.8s
Step #2 - "build.sh": //google/cloud/storage/tests:object_media_integration_test-default (cached) PASSED in 44.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_media_integration_test-grpc-metadata (cached) PASSED in 143.2s
Step #2 - "build.sh": //google/cloud/storage/tests:object_parallel_upload_integration_test-default (cached) PASSED in 4.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata (cached) PASSED in 169.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_plenty_clients_serially_integration_test-default (cached) PASSED in 3.0s
Step #2 - "build.sh": //google/cloud/storage/tests:object_plenty_clients_serially_integration_test-grpc-metadata (cached) PASSED in 2.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_plenty_clients_simultaneously_integration_test-default (cached) PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_plenty_clients_simultaneously_integration_test-grpc-metadata (cached) PASSED in 1.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_headers_integration_test-default (cached) PASSED in 5.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_headers_integration_test-grpc-metadata (cached) PASSED in 9.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_large_integration_test-default (cached) PASSED in 2.5s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_large_integration_test-grpc-metadata (cached) PASSED in 15.1s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_preconditions_integration_test-default (cached) PASSED in 2.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_preconditions_integration_test-grpc-metadata (cached) PASSED in 46.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_range_integration_test-default (cached) PASSED in 1.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_range_integration_test-grpc-metadata (cached) PASSED in 67.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_stream_integration_test-default (cached) PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_stream_integration_test-grpc-metadata (cached) PASSED in 4.1s
Step #2 - "build.sh": //google/cloud/storage/tests:object_resumable_parallel_upload_integration_test-default (cached) PASSED in 2.0s
Step #2 - "build.sh": //google/cloud/storage/tests:object_resumable_parallel_upload_integration_test-grpc-metadata (cached) PASSED in 67.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_resumable_write_integration_test-default (cached) PASSED in 8.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_resumable_write_integration_test-grpc-metadata (cached) PASSED in 102.8s
Step #2 - "build.sh": //google/cloud/storage/tests:object_rewrite_integration_test-default (cached) PASSED in 3.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_rewrite_integration_test-grpc-metadata (cached) PASSED in 116.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_preconditions_integration_test-default (cached) PASSED in 1.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_preconditions_integration_test-grpc-metadata (cached) PASSED in 82.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_stream_integration_test-default (cached) PASSED in 2.2s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_stream_integration_test-grpc-metadata (cached) PASSED in 40.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_streambuf_integration_test-default (cached) PASSED in 1.5s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_streambuf_integration_test-grpc-metadata (cached) PASSED in 37.0s
Step #2 - "build.sh": //google/cloud/storage/tests:service_account_credentials_integration_test-default (cached) PASSED in 1.8s
Step #2 - "build.sh": //google/cloud/storage/tests:service_account_credentials_integration_test-grpc-metadata (cached) PASSED in 2.0s
Step #2 - "build.sh": //google/cloud/storage/tests:service_account_integration_test-default (cached) PASSED in 1.5s
Step #2 - "build.sh": //google/cloud/storage/tests:service_account_integration_test-grpc-metadata (cached) PASSED in 1.6s
Step #2 - "build.sh": //google/cloud/storage/tests:signed_url_conformance_test-default (cached) PASSED in 2.1s
Step #2 - "build.sh": //google/cloud/storage/tests:signed_url_conformance_test-grpc-metadata (cached) PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/storage/tests:slow_reader_chunk_integration_test-default (cached) PASSED in 37.2s
Step #2 - "build.sh": //google/cloud/storage/tests:slow_reader_chunk_integration_test-grpc-metadata (cached) PASSED in 62.1s
Step #2 - "build.sh": //google/cloud/storage/tests:slow_reader_stream_integration_test-default (cached) PASSED in 37.3s
Step #2 - "build.sh": //google/cloud/storage/tests:slow_reader_stream_integration_test-grpc-metadata (cached) PASSED in 46.6s
Step #2 - "build.sh": //google/cloud/storage/tests:small_reads_integration_test-default (cached) PASSED in 1.5s
Step #2 - "build.sh": //google/cloud/storage/tests:small_reads_integration_test-grpc-metadata (cached) PASSED in 40.3s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_async-default           (cached) PASSED in 15.2s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_async-grpc-metadata     (cached) PASSED in 14.7s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_grpc-default            (cached) PASSED in 14.5s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_grpc-grpc-metadata      (cached) PASSED in 13.2s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_rest-default            (cached) PASSED in 1.7s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_rest-grpc-metadata      (cached) PASSED in 1.4s
Step #2 - "build.sh": //google/cloud/storage/tests:storage_include_test-default       (cached) PASSED in 1.4s
Step #2 - "build.sh": //google/cloud/storage/tests:storage_include_test-grpc-metadata (cached) PASSED in 1.8s
Step #2 - "build.sh": //google/cloud/storage/tests:thread_integration_test-default    (cached) PASSED in 6.9s
Step #2 - "build.sh": //google/cloud/storage/tests:thread_integration_test-grpc-metadata (cached) PASSED in 169.6s
Step #2 - "build.sh": //google/cloud/storage/tests:tracing_integration_test-default   (cached) PASSED in 1.8s
Step #2 - "build.sh": //google/cloud/storage/tests:tracing_integration_test-grpc-metadata (cached) PASSED in 2.7s
Step #2 - "build.sh": //google/cloud/storage/tests:universe_domain_integration_test   (cached) PASSED in 1.9s
Step #2 - "build.sh": 
Step #2 - "build.sh": Executed 0 out of 145 tests: 145 tests pass.
Step #2 - "build.sh": There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
Step #2 - "build.sh": Killing emulator server [3987] ... done.
Step #2 - "build.sh": ==> 🕑 ubsan completed in 164.031 seconds
Finished Step #2 - "build.sh"
Starting Step #3 - "remove-image"
Step #3 - "remove-image": Already have image (with digest): gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #3 - "remove-image": Digests:
Step #3 - "remove-image": - us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel@sha256:9c099193430ca97799ba63d483c47be36c038a52979f4f950596e9dc25c76e2c
Step #3 - "remove-image": 
Step #3 - "remove-image": Tags:
Step #3 - "remove-image": - us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-bazel:79f99bce-bd5a-46df-bf55-7ed79b04b119
Step #3 - "remove-image": Delete request issued.
Step #3 - "remove-image": Waiting for operation [projects/cloud-cpp-testing-resources/locations/us-east1/operations/505d4d1b-3d4e-4e07-98b9-72d0eefd279a] to complete...
Step #3 - "remove-image": .....done.
Finished Step #3 - "remove-image"
PUSH
DONE

Build Log: https://storage.cloud.google.com/cloud-cpp-community-publiclogs/logs/google-cloud-cpp/16389/edddb774b3e8c7ca9876b04897dd10c390771135/fedora-latest-bazel-ubsan-default/log-79f99bce-bd5a-46df-bf55-7ed79b04b119.txt