feat(storage): separate read and hedging thread pools - #16389

starting build "7846a417-1671-4bfe-91bd-ca884284144e"

FETCHSOURCE
From https://github.com/googleapis/google-cloud-cpp
 * branch              edddb774b3e8c7ca9876b04897dd10c390771135 -> FETCH_HEAD
Updating files:   1% (255/23195)
Updating files:   2% (464/23195)
Updating files:   3% (696/23195)
Updating files:   4% (928/23195)
Updating files:   5% (1160/23195)
Updating files:   6% (1392/23195)
Updating files:   7% (1624/23195)
Updating files:   8% (1856/23195)
Updating files:   9% (2088/23195)
Updating files:  10% (2320/23195)
Updating files:  11% (2552/23195)
Updating files:  12% (2784/23195)
Updating files:  13% (3016/23195)
Updating files:  14% (3248/23195)
Updating files:  15% (3480/23195)
Updating files:  16% (3712/23195)
Updating files:  17% (3944/23195)
Updating files:  18% (4176/23195)
Updating files:  19% (4408/23195)
Updating files:  20% (4639/23195)
Updating files:  21% (4871/23195)
Updating files:  22% (5103/23195)
Updating files:  23% (5335/23195)
Updating files:  24% (5567/23195)
Updating files:  25% (5799/23195)
Updating files:  26% (6031/23195)
Updating files:  27% (6263/23195)
Updating files:  28% (6495/23195)
Updating files:  29% (6727/23195)
Updating files:  30% (6959/23195)
Updating files:  31% (7191/23195)
Updating files:  32% (7423/23195)
Updating files:  33% (7655/23195)
Updating files:  34% (7887/23195)
Updating files:  35% (8119/23195)
Updating files:  36% (8351/23195)
Updating files:  37% (8583/23195)
Updating files:  38% (8815/23195)
Updating files:  39% (9047/23195)
Updating files:  40% (9278/23195)
Updating files:  41% (9510/23195)
Updating files:  41% (9522/23195)
Updating files:  42% (9742/23195)
Updating files:  43% (9974/23195)
Updating files:  44% (10206/23195)
Updating files:  45% (10438/23195)
Updating files:  46% (10670/23195)
Updating files:  47% (10902/23195)
Updating files:  48% (11134/23195)
Updating files:  49% (11366/23195)
Updating files:  50% (11598/23195)
Updating files:  51% (11830/23195)
Updating files:  52% (12062/23195)
Updating files:  53% (12294/23195)
Updating files:  54% (12526/23195)
Updating files:  55% (12758/23195)
Updating files:  56% (12990/23195)
Updating files:  57% (13222/23195)
Updating files:  58% (13454/23195)
Updating files:  59% (13686/23195)
Updating files:  60% (13917/23195)
Updating files:  61% (14149/23195)
Updating files:  62% (14381/23195)
Updating files:  63% (14613/23195)
Updating files:  64% (14845/23195)
Updating files:  65% (15077/23195)
Updating files:  66% (15309/23195)
Updating files:  67% (15541/23195)
Updating files:  68% (15773/23195)
Updating files:  69% (16005/23195)
Updating files:  70% (16237/23195)
Updating files:  71% (16469/23195)
Updating files:  72% (16701/23195)
Updating files:  73% (16933/23195)
Updating files:  74% (17165/23195)
Updating files:  75% (17397/23195)
Updating files:  76% (17629/23195)
Updating files:  77% (17861/23195)
Updating files:  78% (18093/23195)
Updating files:  79% (18325/23195)
Updating files:  80% (18556/23195)
Updating files:  81% (18788/23195)
Updating files:  82% (19020/23195)
Updating files:  83% (19252/23195)
Updating files:  84% (19484/23195)
Updating files:  85% (19716/23195)
Updating files:  86% (19948/23195)
Updating files:  87% (20180/23195)
Updating files:  88% (20412/23195)
Updating files:  89% (20644/23195)
Updating files:  90% (20876/23195)
Updating files:  91% (21108/23195)
Updating files:  92% (21340/23195)
Updating files:  93% (21572/23195)
Updating files:  94% (21804/23195)
Updating files:  95% (22036/23195)
Updating files:  96% (22268/23195)
Updating files:  97% (22500/23195)
Updating files:  98% (22732/23195)
Updating files:  99% (22964/23195)
Updating files: 100% (23195/23195)
Updating files: 100% (23195/23195), done.
HEAD is now at edddb774 feat(storage): separate read and hedging thread pools
GitCommit:
edddb774b3e8c7ca9876b04897dd10c390771135
BUILD
Starting Step #4 - "cancel-in-progress-builds-for-PR"
Starting Step #0 - "kaniko-build"
Step #4 - "cancel-in-progress-builds-for-PR": Pulling image: gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #0 - "kaniko-build": Pulling image: gcr.io/kaniko-project/executor:v1.24.0-debug@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": gcr.io/kaniko-project/executor@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd: Pulling from kaniko-project/executor
Step #4 - "cancel-in-progress-builds-for-PR": gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574: Pulling from google.com/cloudsdktool/cloud-sdk
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Download complete
Step #0 - "kaniko-build": 2fc842204170: Pulling fs layer
Step #0 - "kaniko-build": 8d1674b25e7b: Pulling fs layer
Step #0 - "kaniko-build": 9bdd0371dbb4: Pulling fs layer
Step #0 - "kaniko-build": 9aeeda79b717: Pulling fs layer
Step #0 - "kaniko-build": aaab912f1a4e: Pulling fs layer
Step #0 - "kaniko-build": 0006a7b536fb: Pulling fs layer
Step #0 - "kaniko-build": 3d08e2063109: Pulling fs layer
Step #0 - "kaniko-build": fc4517ec89af: Pulling fs layer
Step #0 - "kaniko-build": b892afb8bf07: Pulling fs layer
Step #0 - "kaniko-build": cdc2b9345183: Pulling fs layer
Step #0 - "kaniko-build": 3934701e2c3d: Pulling fs layer
Step #0 - "kaniko-build": 46be974dd378: Pulling fs layer
Step #0 - "kaniko-build": 2b61f8ddf6d9: Pulling fs layer
Step #0 - "kaniko-build": 8d1674b25e7b: Waiting
Step #0 - "kaniko-build": b892afb8bf07: Waiting
Step #0 - "kaniko-build": 9bdd0371dbb4: Waiting
Step #0 - "kaniko-build": cdc2b9345183: Waiting
Step #0 - "kaniko-build": 9aeeda79b717: Waiting
Step #0 - "kaniko-build": 3934701e2c3d: Waiting
Step #0 - "kaniko-build": aaab912f1a4e: Waiting
Step #0 - "kaniko-build": 0006a7b536fb: Waiting
Step #0 - "kaniko-build": 3d08e2063109: Waiting
Step #0 - "kaniko-build": fc4517ec89af: Waiting
Step #0 - "kaniko-build": 2b61f8ddf6d9: Waiting
Step #0 - "kaniko-build": 46be974dd378: Waiting
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Download complete
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Download complete
Step #0 - "kaniko-build": 2fc842204170: Download complete
Step #0 - "kaniko-build": 2fc842204170: Pull complete
Step #0 - "kaniko-build": 9bdd0371dbb4: Verifying Checksum
Step #0 - "kaniko-build": 9bdd0371dbb4: Download complete
Step #0 - "kaniko-build": 8d1674b25e7b: Verifying Checksum
Step #0 - "kaniko-build": 8d1674b25e7b: Download complete
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Download complete
Step #0 - "kaniko-build": 8d1674b25e7b: Pull complete
Step #0 - "kaniko-build": 9aeeda79b717: Verifying Checksum
Step #0 - "kaniko-build": 9aeeda79b717: Download complete
Step #0 - "kaniko-build": 9bdd0371dbb4: Pull complete
Step #0 - "kaniko-build": aaab912f1a4e: Verifying Checksum
Step #0 - "kaniko-build": aaab912f1a4e: Download complete
Step #0 - "kaniko-build": fc4517ec89af: Verifying Checksum
Step #0 - "kaniko-build": fc4517ec89af: Download complete
Step #0 - "kaniko-build": 0006a7b536fb: Verifying Checksum
Step #0 - "kaniko-build": 0006a7b536fb: Download complete
Step #0 - "kaniko-build": 9aeeda79b717: Pull complete
Step #0 - "kaniko-build": aaab912f1a4e: Pull complete
Step #0 - "kaniko-build": 46be974dd378: Verifying Checksum
Step #0 - "kaniko-build": 46be974dd378: Download complete
Step #0 - "kaniko-build": b892afb8bf07: Verifying Checksum
Step #0 - "kaniko-build": b892afb8bf07: Download complete
Step #0 - "kaniko-build": 0006a7b536fb: Pull complete
Step #0 - "kaniko-build": cdc2b9345183: Verifying Checksum
Step #0 - "kaniko-build": cdc2b9345183: Download complete
Step #0 - "kaniko-build": 2b61f8ddf6d9: Verifying Checksum
Step #0 - "kaniko-build": 2b61f8ddf6d9: Download complete
Step #0 - "kaniko-build": 3934701e2c3d: Verifying Checksum
Step #0 - "kaniko-build": 3934701e2c3d: Download complete
Step #0 - "kaniko-build": 3d08e2063109: Pull complete
Step #0 - "kaniko-build": fc4517ec89af: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Download complete
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Pull complete
Step #0 - "kaniko-build": b892afb8bf07: Pull complete
Step #0 - "kaniko-build": cdc2b9345183: Pull complete
Step #0 - "kaniko-build": 3934701e2c3d: Pull complete
Step #0 - "kaniko-build": 46be974dd378: Pull complete
Step #0 - "kaniko-build": 2b61f8ddf6d9: Pull complete
Step #0 - "kaniko-build": Digest: sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": Status: Downloaded newer image for gcr.io/kaniko-project/executor@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": gcr.io/kaniko-project/executor:v1.24.0-debug@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Using dockerignore file: /workspace/ci/.dockerignore"
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Retrieving image fedora:40 from registry index.docker.io"
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Built cross stage deps: map[]"
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Executing 0 build triggers"
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Building stage 'fedora:40' [idx: '0', base-idx: '-1']"
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:e005ad110158bfe52f0bfb62c43052fc86e99e36d8386a62785d23e3037f90d2..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Using caching version of cmd: RUN dnf makecache &&     dnf install -y clang clang-tools-extra cmake findutils         git llvm make ninja-build openssl-devel patch python         python3 python3-devel python3-lit python-pip tar unzip which wget xz"
Step #0 - "kaniko-build": time="2026-08-28T11:39:33Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:9d5b1e125453cae51a24d3cd1cae5443fca86cc71b75234372c1dab7f670bf59..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:34Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y python3-devel"
Step #0 - "kaniko-build": time="2026-08-28T11:39:34Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:90ef07e1cd6fa920a949553451f6c11581264649e54b461dd254f4930ccd7094..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:34Z" level=info msg="Using caching version of cmd: RUN pip3 install --upgrade pip"
Step #0 - "kaniko-build": time="2026-08-28T11:39:34Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:ebc88857bacfdea4e75ba63cf341267b897a999db7dbfb1d606262d3674c2705..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:34Z" level=info msg="Using caching version of cmd: RUN pip3 install setuptools wheel"
Step #0 - "kaniko-build": time="2026-08-28T11:39:34Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:56c94ce0178764adf3fe805e8a64e1eec7e23f50333cbb6092b70202b425927a..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:34Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y java-latest-openjdk-devel"
Step #0 - "kaniko-build": time="2026-08-28T11:39:34Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:6e1ee6d2a1fb763e652088324c6f241d6c3978ac18de6f0f9380fc732625ded5..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:35Z" level=info msg="Using caching version of cmd: RUN echo 'root:cloudcxx' | chpasswd"
Step #0 - "kaniko-build": time="2026-08-28T11:39:35Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:c9c1c92f41f0a8db11af9f076f47a40be83252a23c669a699f5eb08ee43e65ef..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:35Z" level=info msg="Using caching version of cmd: RUN git clone --depth=1 --branch llvmorg-18.1.1 https://github.com/llvm/llvm-project"
Step #0 - "kaniko-build": time="2026-08-28T11:39:35Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:a7fa06631ce468534586b02b61c044c6e735d21b0f1a979f2dd1242668fa8d8c..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:35Z" level=info msg="Using caching version of cmd: RUN cmake -GNinja -S runtimes -B build     -DCMAKE_BUILD_TYPE=Release     -DLLVM_ENABLE_PROJECTS=\"clang\"     -DLLVM_ENABLE_RUNTIMES=\"libcxx;libcxxabi;libunwind\"     -DCMAKE_C_COMPILER=clang     -DCMAKE_CXX_COMPILER=clang++     -DLLVM_USE_SANITIZER=MemoryWithOrigins     -DLIBCXXABI_USE_LLVM_UNWINDER=OFF     -DCOMPILER_RT_USE_LLVM_UNWINDER=OFF     -DCMAKE_INSTALL_PREFIX=/usr"
Step #0 - "kaniko-build": time="2026-08-28T11:39:35Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:f65b1167323150bf86b8173f1ca602e9c5c458c852dccaabc74ecd0235ca7c12..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:35Z" level=info msg="Using caching version of cmd: RUN cmake --build build"
Step #0 - "kaniko-build": time="2026-08-28T11:39:35Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:e8db00a497af4cc57c853dd3fd6af51d4a14638275c7ab3fb259bce747d6fa21..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:35Z" level=info msg="Using caching version of cmd: RUN cmake --install build"
Step #0 - "kaniko-build": time="2026-08-28T11:39:35Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:cbc61717fbbfd6ac8e73048dd3f77763f4ef330ae3a80dcfc5d43046e76c49aa..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:36Z" level=info msg="Using caching version of cmd: RUN echo \"/usr/lib/x86_64-unknown-linux-gnu\" >/etc/ld.so.conf.d/msan.conf"
Step #0 - "kaniko-build": time="2026-08-28T11:39:36Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:9e44ab0a2cbbf08ec937ea005172f9f791baf66167aa559c4366837cd9a3aa71..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:36Z" level=info msg="Using caching version of cmd: RUN ldconfig"
Step #0 - "kaniko-build": time="2026-08-28T11:39:36Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:3bc1249892dcf73cb470bdbd3bb3958888bea905938b20055137bfcd8b095847..."
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Download complete
Step #0 - "kaniko-build": time="2026-08-28T11:39:36Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y python3.10"
Step #0 - "kaniko-build": time="2026-08-28T11:39:36Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:cda0e47b1bc391d27a5d470a4e90c0b8fcbd0432291328f344e8785ac2203edc..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:36Z" level=info msg="Using caching version of cmd: RUN /var/tmp/ci/install-cloud-sdk.sh"
Step #0 - "kaniko-build": time="2026-08-28T11:39:36Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:26ed640b3a705683cd7e2f987df4b430fc9a89ea3cc195d9c181bc42c670379d..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:36Z" level=info msg="Using caching version of cmd: RUN curl -o /usr/bin/bazelisk -sSL \"https://github.com/bazelbuild/bazelisk/releases/download/v1.28.1/bazelisk-linux-${ARCH}\" &&     chmod +x /usr/bin/bazelisk &&     ln -s /usr/bin/bazelisk /usr/bin/bazel"
Step #0 - "kaniko-build": time="2026-08-28T11:39:36Z" level=info msg="Unpacking rootfs as cmd COPY . /var/tmp/ci requires it."
Step #0 - "kaniko-build": time="2026-08-28T11:39:40Z" level=info msg="ARG NCPU=4"
Step #0 - "kaniko-build": time="2026-08-28T11:39:40Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:39:40Z" level=info msg="ARG ARCH=amd64"
Step #0 - "kaniko-build": time="2026-08-28T11:39:40Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:39:40Z" level=info msg="RUN dnf makecache &&     dnf install -y clang clang-tools-extra cmake findutils         git llvm make ninja-build openssl-devel patch python         python3 python3-devel python3-lit python-pip tar unzip which wget xz"
Step #0 - "kaniko-build": time="2026-08-28T11:39:40Z" level=info msg="Found cached layer, extracting to filesystem"
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": Digest: sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": Status: Downloaded newer image for gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": + test -z 16389
Step #4 - "cancel-in-progress-builds-for-PR": ++ gcloud builds describe --region us-east1 --format 'value(create_time)' 7846a417-1671-4bfe-91bd-ca884284144e
Step #4 - "cancel-in-progress-builds-for-PR": + ctime=2026-08-28T11:38:13.612075Z
Step #4 - "cancel-in-progress-builds-for-PR": + query=tags=pr
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND tags=16389'
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND substitutions.COMMIT_SHA != edddb774b3e8c7ca9876b04897dd10c390771135'
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND create_time < 2026-08-28T11:38:13.612075Z'
Step #4 - "cancel-in-progress-builds-for-PR": + gcloud builds list --region us-east1 --ongoing '--format=value(id)' --filter 'tags=pr AND tags=16389 AND substitutions.COMMIT_SHA != edddb774b3e8c7ca9876b04897dd10c390771135 AND create_time < 2026-08-28T11:38:13.612075Z'
Step #4 - "cancel-in-progress-builds-for-PR": + xargs -r -t gcloud builds cancel --region us-east1
Finished Step #4 - "cancel-in-progress-builds-for-PR"
Step #0 - "kaniko-build": time="2026-08-28T11:40:41Z" level=info msg="RUN dnf makecache && dnf install -y python3-devel"
Step #0 - "kaniko-build": time="2026-08-28T11:40:41Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:40:42Z" level=info msg="RUN pip3 install --upgrade pip"
Step #0 - "kaniko-build": time="2026-08-28T11:40:42Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:40:44Z" level=info msg="RUN pip3 install setuptools wheel"
Step #0 - "kaniko-build": time="2026-08-28T11:40:44Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:40:44Z" level=info msg="RUN dnf makecache && dnf install -y java-latest-openjdk-devel"
Step #0 - "kaniko-build": time="2026-08-28T11:40:44Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:41:24Z" level=info msg="RUN echo 'root:cloudcxx' | chpasswd"
Step #0 - "kaniko-build": time="2026-08-28T11:41:24Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:41:25Z" level=info msg="WORKDIR /var/tmp/build"
Step #0 - "kaniko-build": time="2026-08-28T11:41:25Z" level=info msg="Cmd: workdir"
Step #0 - "kaniko-build": time="2026-08-28T11:41:25Z" level=info msg="Changed working directory to /var/tmp/build"
Step #0 - "kaniko-build": time="2026-08-28T11:41:25Z" level=info msg="Creating directory /var/tmp/build with uid -1 and gid -1"
Step #0 - "kaniko-build": time="2026-08-28T11:41:25Z" level=info msg="Taking snapshot of files..."
Step #0 - "kaniko-build": time="2026-08-28T11:41:25Z" level=info msg="RUN git clone --depth=1 --branch llvmorg-18.1.1 https://github.com/llvm/llvm-project"
Step #0 - "kaniko-build": time="2026-08-28T11:41:25Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:44Z" level=info msg="WORKDIR /var/tmp/build/llvm-project"
Step #0 - "kaniko-build": time="2026-08-28T11:42:44Z" level=info msg="Cmd: workdir"
Step #0 - "kaniko-build": time="2026-08-28T11:42:44Z" level=info msg="Changed working directory to /var/tmp/build/llvm-project"
Step #0 - "kaniko-build": time="2026-08-28T11:42:44Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:42:44Z" level=info msg="RUN cmake -GNinja -S runtimes -B build     -DCMAKE_BUILD_TYPE=Release     -DLLVM_ENABLE_PROJECTS=\"clang\"     -DLLVM_ENABLE_RUNTIMES=\"libcxx;libcxxabi;libunwind\"     -DCMAKE_C_COMPILER=clang     -DCMAKE_CXX_COMPILER=clang++     -DLLVM_USE_SANITIZER=MemoryWithOrigins     -DLIBCXXABI_USE_LLVM_UNWINDER=OFF     -DCOMPILER_RT_USE_LLVM_UNWINDER=OFF     -DCMAKE_INSTALL_PREFIX=/usr"
Step #0 - "kaniko-build": time="2026-08-28T11:42:44Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:45Z" level=info msg="RUN cmake --build build"
Step #0 - "kaniko-build": time="2026-08-28T11:42:45Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:47Z" level=info msg="RUN cmake --install build"
Step #0 - "kaniko-build": time="2026-08-28T11:42:47Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:49Z" level=info msg="RUN echo \"/usr/lib/x86_64-unknown-linux-gnu\" >/etc/ld.so.conf.d/msan.conf"
Step #0 - "kaniko-build": time="2026-08-28T11:42:49Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:49Z" level=info msg="RUN ldconfig"
Step #0 - "kaniko-build": time="2026-08-28T11:42:49Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:50Z" level=info msg="COPY . /var/tmp/ci"
Step #0 - "kaniko-build": time="2026-08-28T11:42:50Z" level=info msg="Taking snapshot of files..."
Step #0 - "kaniko-build": time="2026-08-28T11:42:50Z" level=info msg="WORKDIR /var/tmp/downloads"
Step #0 - "kaniko-build": time="2026-08-28T11:42:50Z" level=info msg="Cmd: workdir"
Step #0 - "kaniko-build": time="2026-08-28T11:42:50Z" level=info msg="Changed working directory to /var/tmp/downloads"
Step #0 - "kaniko-build": time="2026-08-28T11:42:50Z" level=info msg="Creating directory /var/tmp/downloads with uid -1 and gid -1"
Step #0 - "kaniko-build": time="2026-08-28T11:42:50Z" level=info msg="Taking snapshot of files..."
Step #0 - "kaniko-build": time="2026-08-28T11:42:50Z" level=info msg="RUN dnf makecache && dnf install -y python3.10"
Step #0 - "kaniko-build": time="2026-08-28T11:42:50Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:54Z" level=info msg="ENV CLOUDSDK_PYTHON=python3.10"
Step #0 - "kaniko-build": time="2026-08-28T11:42:54Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:42:54Z" level=info msg="RUN /var/tmp/ci/install-cloud-sdk.sh"
Step #0 - "kaniko-build": time="2026-08-28T11:42:54Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:44:17Z" level=info msg="ENV CLOUD_SDK_LOCATION=/usr/local/google-cloud-sdk"
Step #0 - "kaniko-build": time="2026-08-28T11:44:17Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:44:17Z" level=info msg="ENV PATH=${CLOUD_SDK_LOCATION}/bin:${PATH}"
Step #0 - "kaniko-build": time="2026-08-28T11:44:17Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:44:17Z" level=info msg="RUN curl -o /usr/bin/bazelisk -sSL \"https://github.com/bazelbuild/bazelisk/releases/download/v1.28.1/bazelisk-linux-${ARCH}\" &&     chmod +x /usr/bin/bazelisk &&     ln -s /usr/bin/bazelisk /usr/bin/bazel"
Step #0 - "kaniko-build": time="2026-08-28T11:44:17Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:44:18Z" level=info msg="Pushing image to us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan:7846a417-1671-4bfe-91bd-ca884284144e"
Step #0 - "kaniko-build": time="2026-08-28T11:44:18Z" level=info msg="Pushed us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan@sha256:c022bb55325e144c52ec5cf23278b0f6cc5560e238338105a7d42ae5be34470f"
Finished Step #0 - "kaniko-build"
Starting Step #1 - "download-runner-image"
Step #1 - "download-runner-image": Pulling image: us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan:7846a417-1671-4bfe-91bd-ca884284144e
Step #1 - "download-runner-image": 7846a417-1671-4bfe-91bd-ca884284144e: Pulling from cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan
Step #1 - "download-runner-image": fc4518b6c5f9: Pulling fs layer
Step #1 - "download-runner-image": 5e8a8bc10713: Pulling fs layer
Step #1 - "download-runner-image": b5493f6eace4: Pulling fs layer
Step #1 - "download-runner-image": 6a4bb87f2a62: Pulling fs layer
Step #1 - "download-runner-image": 44052672a2f7: Pulling fs layer
Step #1 - "download-runner-image": 7f55e8d9f973: Pulling fs layer
Step #1 - "download-runner-image": 32621cf3b379: Pulling fs layer
Step #1 - "download-runner-image": c14c6c7620a0: Pulling fs layer
Step #1 - "download-runner-image": f43b21e3ba11: Pulling fs layer
Step #1 - "download-runner-image": 3f3e1a9e6272: Pulling fs layer
Step #1 - "download-runner-image": 5a6b79a42a18: Pulling fs layer
Step #1 - "download-runner-image": 9e4c5aa6f8f2: Pulling fs layer
Step #1 - "download-runner-image": 3561492257fe: Pulling fs layer
Step #1 - "download-runner-image": e107a1067de3: Pulling fs layer
Step #1 - "download-runner-image": 30bb58566698: Pulling fs layer
Step #1 - "download-runner-image": f53f9c45a180: Pulling fs layer
Step #1 - "download-runner-image": 9fb3ffac4da6: Pulling fs layer
Step #1 - "download-runner-image": fef9c1cebd0d: Pulling fs layer
Step #1 - "download-runner-image": a4b4078c88ca: Pulling fs layer
Step #1 - "download-runner-image": 3f3e1a9e6272: Waiting
Step #1 - "download-runner-image": 5a6b79a42a18: Waiting
Step #1 - "download-runner-image": 32621cf3b379: Waiting
Step #1 - "download-runner-image": 9e4c5aa6f8f2: Waiting
Step #1 - "download-runner-image": 3561492257fe: Waiting
Step #1 - "download-runner-image": c14c6c7620a0: Waiting
Step #1 - "download-runner-image": f43b21e3ba11: Waiting
Step #1 - "download-runner-image": e107a1067de3: Waiting
Step #1 - "download-runner-image": fef9c1cebd0d: Waiting
Step #1 - "download-runner-image": 30bb58566698: Waiting
Step #1 - "download-runner-image": a4b4078c88ca: Waiting
Step #1 - "download-runner-image": f53f9c45a180: Waiting
Step #1 - "download-runner-image": 9fb3ffac4da6: Waiting
Step #1 - "download-runner-image": 44052672a2f7: Verifying Checksum
Step #1 - "download-runner-image": 44052672a2f7: Download complete
Step #1 - "download-runner-image": 6a4bb87f2a62: Verifying Checksum
Step #1 - "download-runner-image": 6a4bb87f2a62: Download complete
Step #1 - "download-runner-image": 32621cf3b379: Verifying Checksum
Step #1 - "download-runner-image": 32621cf3b379: Download complete
Step #1 - "download-runner-image": b5493f6eace4: Verifying Checksum
Step #1 - "download-runner-image": b5493f6eace4: Download complete
Step #1 - "download-runner-image": c14c6c7620a0: Verifying Checksum
Step #1 - "download-runner-image": c14c6c7620a0: Download complete
Step #1 - "download-runner-image": 3f3e1a9e6272: Verifying Checksum
Step #1 - "download-runner-image": 3f3e1a9e6272: Download complete
Step #1 - "download-runner-image": 5a6b79a42a18: Verifying Checksum
Step #1 - "download-runner-image": 5a6b79a42a18: Download complete
Step #1 - "download-runner-image": 9e4c5aa6f8f2: Verifying Checksum
Step #1 - "download-runner-image": 9e4c5aa6f8f2: Download complete
Step #1 - "download-runner-image": 3561492257fe: Download complete
Step #1 - "download-runner-image": fc4518b6c5f9: Download complete
Step #1 - "download-runner-image": e107a1067de3: Download complete
Step #1 - "download-runner-image": 30bb58566698: Verifying Checksum
Step #1 - "download-runner-image": 30bb58566698: Download complete
St
...
[Logs truncated due to log size limitations. For full logs, see https://storage.cloud.google.com/cloud-cpp-community-publiclogs/logs/google-cloud-cpp/16389/edddb774b3e8c7ca9876b04897dd10c390771135/fedora-msan-msan-__default__/log-7846a417-1671-4bfe-91bd-ca884284144e.txt.]
...
e/cloud/bigtable/tests/instance_admin_emulator <args omitted>
Step #2 - "build.sh":     20	Cloud Bigtable emulator running on localhost:8490
Step #2 - "build.sh": cat: /instance-admin-emulator.log: No such file or directory
Step #2 - "build.sh": ================ instance-admin-emulator.log ================
Step #2 - "build.sh": 
Step #2 - "build.sh": 2026-08-28T11:57:14Z (+684s)
Step #2 - "build.sh": -------------------------------------------------
Step #2 - "build.sh": |   Running Bigtable gRPC credential examples   |
Step #2 - "build.sh": -------------------------------------------------
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": INFO: Invocation ID: 63c815ee-b603-46bb-a84a-ea6a43ee8b73
Step #2 - "build.sh": Computing main repo mapping: 
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": Loading: 
Step #2 - "build.sh": Loading: 0 packages loaded
Step #2 - "build.sh": WARNING: Build option --test_env has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
Step #2 - "build.sh": Analyzing: target //google/cloud/bigtable/examples:bigtable_grpc_credentials (0 packages loaded, 0 targets configured)
Step #2 - "build.sh": Analyzing: target //google/cloud/bigtable/examples:bigtable_grpc_credentials (0 packages loaded, 0 targets configured)
Step #2 - "build.sh": 
Step #2 - "build.sh": INFO: Analyzed target //google/cloud/bigtable/examples:bigtable_grpc_credentials (0 packages loaded, 14879 targets configured).
Step #2 - "build.sh": [8,754 / 8,755] [Sched] Testing //google/cloud/bigtable/examples:bigtable_grpc_credentials
Step #2 - "build.sh": [8,754 / 8,755] [Sched] Testing //google/cloud/bigtable/examples:bigtable_grpc_credentials; Downloading external/googleapis+/google/bigtable/admin/v2/libadmin_cc_grpc.so, 9.0 MiB / 9.7 MiB
Step #2 - "build.sh": [8,754 / 8,755] Testing //google/cloud/bigtable/examples:bigtable_grpc_credentials; 0s remote-cache, linux-sandbox
Step #2 - "build.sh": [8,754 / 8,755] Testing //google/cloud/bigtable/examples:bigtable_grpc_credentials; 3s remote-cache, linux-sandbox
Step #2 - "build.sh": INFO: Found 1 test target...
Step #2 - "build.sh": Target //google/cloud/bigtable/examples:bigtable_grpc_credentials up-to-date:
Step #2 - "build.sh":   /h/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/execroot/_main/bazel-out/k8-fastbuild/bin/google/cloud/bigtable/examples/bigtable_grpc_credentials
Step #2 - "build.sh": INFO: Elapsed time: 8.298s, Critical Path: 6.99s
Step #2 - "build.sh": INFO: 2 processes: 8762 action cache hit, 2 linux-sandbox.
Step #2 - "build.sh": INFO: Build completed successfully, 2 total actions
Step #2 - "build.sh": //google/cloud/bigtable/examples:bigtable_grpc_credentials               PASSED in 2.5s
Step #2 - "build.sh": 
Step #2 - "build.sh": Executed 1 out of 1 test: 1 test passes.
Step #2 - "build.sh": There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
Step #2 - "build.sh": 
Step #2 - "build.sh": 2026-08-28T11:57:23Z (+693s)
Step #2 - "build.sh": ------------------------------------------------------
Step #2 - "build.sh": |   Running REST integration tests (with emulator)   |
Step #2 - "build.sh": ------------------------------------------------------
Step #2 - "build.sh": 2026-08-28T11:57:23Z (+693s): Launching Cloud Storage emulator on port 8888
Step #2 - "build.sh": Successfully connected to emulator [10884]
Step #2 - "build.sh": 2026-08-28T11:57:24Z (+694s): Successfully connected to gRPC server at port 40745
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": INFO: Invocation ID: 966718fb-51af-438d-b00e-4226ff22856e
Step #2 - "build.sh": Computing main repo mapping: 
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": Loading: 
Step #2 - "build.sh": Loading: 0 packages loaded
Step #2 - "build.sh": WARNING: Build option --test_env has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
Step #2 - "build.sh": Analyzing: target //google/cloud:internal_curl_rest_client_integration_test (0 packages loaded, 0 targets configured)
Step #2 - "build.sh": Analyzing: target //google/cloud:internal_curl_rest_client_integration_test (0 packages loaded, 0 targets configured)
Step #2 - "build.sh": 
Step #2 - "build.sh": INFO: Analyzed target //google/cloud:internal_curl_rest_client_integration_test (0 packages loaded, 9557 targets configured).
Step #2 - "build.sh": INFO: Found 1 test target...
Step #2 - "build.sh": Target //google/cloud:internal_curl_rest_client_integration_test up-to-date:
Step #2 - "build.sh":   /h/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/execroot/_main/bazel-out/k8-fastbuild/bin/google/cloud/internal_curl_rest_client_integration_test
Step #2 - "build.sh": INFO: Elapsed time: 0.956s, Critical Path: 0.23s
Step #2 - "build.sh": INFO: 2 processes: 1710 action cache hit, 1 remote cache hit, 1 internal.
Step #2 - "build.sh": INFO: Build completed successfully, 2 total actions
Step #2 - "build.sh": //google/cloud:internal_curl_rest_client_integration_test       (cached) PASSED in 1.7s
Step #2 - "build.sh": 
Step #2 - "build.sh": Executed 0 out of 1 test: 1 test passes.
Step #2 - "build.sh": There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
Step #2 - "build.sh": Killing emulator server [10884] ... done.
Step #2 - "build.sh": 
Step #2 - "build.sh": 2026-08-28T11:57:26Z (+696s)
Step #2 - "build.sh": -------------------------------------------------
Step #2 - "build.sh": |   Running REST production integration tests   |
Step #2 - "build.sh": -------------------------------------------------
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": INFO: Invocation ID: 4855e562-219d-4302-9a9b-bc5288b652cc
Step #2 - "build.sh": Computing main repo mapping: 
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": Loading: 
Step #2 - "build.sh": Loading: 0 packages loaded
Step #2 - "build.sh": WARNING: Build option --test_env has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
Step #2 - "build.sh": Analyzing: target //google/cloud:internal_unified_rest_credentials_integration_test (0 packages loaded, 0 targets configured)
Step #2 - "build.sh": Analyzing: target //google/cloud:internal_unified_rest_credentials_integration_test (0 packages loaded, 0 targets configured)
Step #2 - "build.sh": 
Step #2 - "build.sh": INFO: Analyzed target //google/cloud:internal_unified_rest_credentials_integration_test (0 packages loaded, 9557 targets configured).
Step #2 - "build.sh": INFO: Found 1 test target...
Step #2 - "build.sh": Target //google/cloud:internal_unified_rest_credentials_integration_test up-to-date:
Step #2 - "build.sh":   /h/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/execroot/_main/bazel-out/k8-fastbuild/bin/google/cloud/internal_unified_rest_credentials_integration_test
Step #2 - "build.sh": INFO: Elapsed time: 0.689s, Critical Path: 0.14s
Step #2 - "build.sh": INFO: 2 processes: 1710 action cache hit, 1 remote cache hit, 1 internal.
Step #2 - "build.sh": INFO: Build completed successfully, 2 total actions
Step #2 - "build.sh": //google/cloud:internal_unified_rest_credentials_integration_test (cached) PASSED in 11.7s
Step #2 - "build.sh": 
Step #2 - "build.sh": Executed 0 out of 1 test: 1 test passes.
Step #2 - "build.sh": There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
Step #2 - "build.sh": 
Step #2 - "build.sh": 2026-08-28T11:57:26Z (+696s)
Step #2 - "build.sh": ----------------------------------------------------------------
Step #2 - "build.sh": |   Running production integration tests for other libraries   |
Step #2 - "build.sh": ----------------------------------------------------------------
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": INFO: Invocation ID: 26d5e4a5-e646-4177-8dbd-fe888456fe58
Step #2 - "build.sh": Computing main repo mapping: 
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": Loading: 
Step #2 - "build.sh": Loading: 0 packages loaded
Step #2 - "build.sh": Analyzing: 845 targets (15 packages loaded, 0 targets configured)
Step #2 - "build.sh": Analyzing: 845 targets (15 packages loaded, 2 targets configured)
Step #2 - "build.sh": 
Step #2 - "build.sh": INFO: Analyzed 845 targets (34 packages loaded, 10101 targets configured).
Step #2 - "build.sh": [5,394 / 8,731] checking cached actions
Step #2 - "build.sh": [9,691 / 9,762] 59 / 129 tests; Testing //google/cloud/compute:storage_pools_client_samples; Downloading google/cloud/compute/storage_pools_client_samples/test.xml; 0s remote-cache ... (32 actions, 0 running)
Step #2 - "build.sh": INFO: Found 716 targets and 129 test targets...
Step #2 - "build.sh": INFO: Elapsed time: 3.093s, Critical Path: 0.65s
Step #2 - "build.sh": INFO: 130 processes: 9833 action cache hit, 254 remote cache hit.
Step #2 - "build.sh": INFO: Build completed successfully, 130 total actions
Step #2 - "build.sh": //google/cloud:internal_grpc_impersonate_service_account_integration_test (cached) PASSED in 2.8s
Step #2 - "build.sh": //google/cloud/bigquery:analyticshub_v1_samples_analytics_hub_client_samples (cached) PASSED in 2.5s
Step #2 - "build.sh": //google/cloud/bigquery:biglake_v1_samples_metastore_client_samples (cached) PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/bigquery:connection_v1_samples_connection_client_samples (cached) PASSED in 2.5s
Step #2 - "build.sh": //google/cloud/bigquery:datapolicies_v1_samples_data_policy_client_samples (cached) PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/bigquery:datapolicies_v2_samples_data_policy_client_samples (cached) PASSED in 2.5s
Step #2 - "build.sh": //google/cloud/bigquery:datatransfer_v1_samples_data_transfer_client_samples (cached) PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/bigquery:migration_v2_samples_migration_client_samples (cached) PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/bigquery:reservation_v1_samples_reservation_client_samples (cached) PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/bigquery:storage_v1_samples_bigquery_read_client_samples (cached) PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/bigquery:storage_v1_samples_bigquery_write_client_samples (cached) PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/compute:accelerator_types_client_samples         (cached) PASSED in 3.1s
Step #2 - "build.sh": //google/cloud/compute:addresses_client_samples                 (cached) PASSED in 3.9s
Step #2 - "build.sh": //google/cloud/compute:autoscalers_client_samples               (cached) PASSED in 6.8s
Step #2 - "build.sh": //google/cloud/compute:backend_buckets_client_samples           (cached) PASSED in 6.0s
Step #2 - "build.sh": //google/cloud/compute:backend_services_client_samples          (cached) PASSED in 8.0s
Step #2 - "build.sh": //google/cloud/compute:disk_types_client_samples                (cached) PASSED in 3.7s
Step #2 - "build.sh": //google/cloud/compute:disks_client_samples                     (cached) PASSED in 7.4s
Step #2 - "build.sh": //google/cloud/compute:external_vpn_gateways_client_samples     (cached) PASSED in 3.5s
Step #2 - "build.sh": //google/cloud/compute:firewall_policies_client_samples         (cached) PASSED in 5.9s
Step #2 - "build.sh": //google/cloud/compute:firewalls_client_samples                 (cached) PASSED in 5.4s
Step #2 - "build.sh": //google/cloud/compute:forwarding_rules_client_samples          (cached) PASSED in 5.4s
Step #2 - "build.sh": //google/cloud/compute:global_addresses_client_samples          (cached) PASSED in 9.1s
Step #2 - "build.sh": //google/cloud/compute:global_forwarding_rules_client_samples   (cached) PASSED in 4.0s
Step #2 - "build.sh": //google/cloud/compute:global_network_endpoint_groups_client_samples (cached) PASSED in 4.1s
Step #2 - "build.sh": //google/cloud/compute:global_operations_client_samples         (cached) PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/compute:global_organization_operations_client_samples (cached) PASSED in 6.6s
Step #2 - "build.sh": //google/cloud/compute:global_public_delegated_prefixes_client_samples (cached) PASSED in 5.8s
Step #2 - "build.sh": //google/cloud/compute:health_checks_client_samples             (cached) PASSED in 3.6s
Step #2 - "build.sh": //google/cloud/compute:http_health_checks_client_samples        (cached) PASSED in 9.2s
Step #2 - "build.sh": //google/cloud/compute:https_health_checks_client_samples       (cached) PASSED in 4.0s
Step #2 - "build.sh": //google/cloud/compute:image_family_views_client_samples        (cached) PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/compute:images_client_samples                    (cached) PASSED in 3.7s
Step #2 - "build.sh": //google/cloud/compute:instance_group_manager_resize_requests_client_samples (cached) PASSED in 4.4s
Step #2 - "build.sh": //google/cloud/compute:instance_group_managers_client_samples   (cached) PASSED in 7.3s
Step #2 - "build.sh": //google/cloud/compute:instance_groups_client_samples           (cached) PASSED in 6.0s
Step #2 - "build.sh": //google/cloud/compute:instance_settings_client_samples         (cached) PASSED in 9.5s
Step #2 - "build.sh": //google/cloud/compute:instance_templates_client_samples        (cached) PASSED in 5.6s
Step #2 - "build.sh": //google/cloud/compute:instances_client_samples                 (cached) PASSED in 8.8s
Step #2 - "build.sh": //google/cloud/compute:instant_snapshots_client_samples         (cached) PASSED in 3.5s
Step #2 - "build.sh": //google/cloud/compute:interconnect_attachment_groups_client_samples (cached) PASSED in 8.7s
Step #2 - "build.sh": //google/cloud/compute:interconnect_attachments_client_samples  (cached) PASSED in 4.4s
Step #2 - "build.sh": //google/cloud/compute:interconnect_groups_client_samples       (cached) PASSED in 5.0s
Step #2 - "build.sh": //google/cloud/compute:interconnect_locations_client_samples    (cached) PASSED in 6.5s
Step #2 - "build.sh": //google/cloud/compute:interconnect_remote_locations_client_samples (cached) PASSED in 5.6s
Step #2 - "build.sh": //google/cloud/compute:interconnects_client_samples             (cached) PASSED in 8.0s
Step #2 - "build.sh": //google/cloud/compute:license_codes_client_samples             (cached) PASSED in 8.5s
Step #2 - "build.sh": //google/cloud/compute:licenses_client_samples                  (cached) PASSED in 5.1s
Step #2 - "build.sh": //google/cloud/compute:machine_images_client_samples            (cached) PASSED in 4.8s
Step #2 - "build.sh": //google/cloud/compute:machine_types_client_samples             (cached) PASSED in 3.0s
Step #2 - "build.sh": //google/cloud/compute:network_attachments_client_samples       (cached) PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/compute:network_edge_security_services_client_samples (cached) PASSED in 4.7s
Step #2 - "build.sh": //google/cloud/compute:network_endpoint_groups_client_samples   (cached) PASSED in 5.7s
Step #2 - "build.sh": //google/cloud/compute:network_firewall_policies_client_samples (cached) PASSED in 7.2s
Step #2 - "build.sh": //google/cloud/compute:network_profiles_client_samples          (cached) PASSED in 7.9s
Step #2 - "build.sh": //google/cloud/compute:networks_client_samples                  (cached) PASSED in 5.0s
Step #2 - "build.sh": //google/cloud/compute:node_groups_client_samples               (cached) PASSED in 8.4s
Step #2 - "build.sh": //google/cloud/compute:node_templates_client_samples            (cached) PASSED in 7.8s
Step #2 - "build.sh": //google/cloud/compute:node_types_client_samples                (cached) PASSED in 6.5s
Step #2 - "build.sh": //google/cloud/compute:packet_mirrorings_client_samples         (cached) PASSED in 8.1s
Step #2 - "build.sh": //google/cloud/compute:projects_client_samples                  (cached) PASSED in 6.2s
Step #2 - "build.sh": //google/cloud/compute:public_advertised_prefixes_client_samples (cached) PASSED in 5.3s
Step #2 - "build.sh": //google/cloud/compute:public_delegated_prefixes_client_samples (cached) PASSED in 8.6s
Step #2 - "build.sh": //google/cloud/compute:region_autoscalers_client_samples        (cached) PASSED in 7.0s
Step #2 - "build.sh": //google/cloud/compute:region_backend_services_client_samples   (cached) PASSED in 4.0s
Step #2 - "build.sh": //google/cloud/compute:region_commitments_client_samples        (cached) PASSED in 8.6s
Step #2 - "build.sh": //google/cloud/compute:region_disk_types_client_samples         (cached) PASSED in 9.1s
Step #2 - "build.sh": //google/cloud/compute:region_disks_client_samples              (cached) PASSED in 3.6s
Step #2 - "build.sh": //google/cloud/compute:region_health_check_services_client_samples (cached) PASSED in 6.6s
Step #2 - "build.sh": //google/cloud/compute:region_health_checks_client_samples      (cached) PASSED in 9.4s
Step #2 - "build.sh": //google/cloud/compute:region_instance_group_managers_client_samples (cached) PASSED in 8.3s
Step #2 - "build.sh": //google/cloud/compute:region_instance_groups_client_samples    (cached) PASSED in 4.9s
Step #2 - "build.sh": //google/cloud/compute:region_instance_templates_client_samples (cached) PASSED in 7.8s
Step #2 - "build.sh": //google/cloud/compute:region_instances_client_samples          (cached) PASSED in 7.5s
Step #2 - "build.sh": //google/cloud/compute:region_instant_snapshots_client_samples  (cached) PASSED in 6.4s
Step #2 - "build.sh": //google/cloud/compute:region_network_endpoint_groups_client_samples (cached) PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/compute:region_network_firewall_policies_client_samples (cached) PASSED in 3.5s
Step #2 - "build.sh": //google/cloud/compute:region_notification_endpoints_client_samples (cached) PASSED in 4.4s
Step #2 - "build.sh": //google/cloud/compute:region_operations_client_samples         (cached) PASSED in 8.6s
Step #2 - "build.sh": //google/cloud/compute:region_security_policies_client_samples  (cached) PASSED in 6.8s
Step #2 - "build.sh": //google/cloud/compute:region_ssl_certificates_client_samples   (cached) PASSED in 3.7s
Step #2 - "build.sh": //google/cloud/compute:region_ssl_policies_client_samples       (cached) PASSED in 3.1s
Step #2 - "build.sh": //google/cloud/compute:region_target_http_proxies_client_samples (cached) PASSED in 3.2s
Step #2 - "build.sh": //google/cloud/compute:region_target_https_proxies_client_samples (cached) PASSED in 8.2s
Step #2 - "build.sh": //google/cloud/compute:region_target_tcp_proxies_client_samples (cached) PASSED in 6.9s
Step #2 - "build.sh": //google/cloud/compute:region_url_maps_client_samples           (cached) PASSED in 5.2s
Step #2 - "build.sh": //google/cloud/compute:region_zones_client_samples              (cached) PASSED in 4.7s
Step #2 - "build.sh": //google/cloud/compute:regions_client_samples                   (cached) PASSED in 4.9s
Step #2 - "build.sh": //google/cloud/compute:reservation_blocks_client_samples        (cached) PASSED in 6.8s
Step #2 - "build.sh": //google/cloud/compute:reservation_sub_blocks_client_samples    (cached) PASSED in 9.1s
Step #2 - "build.sh": //google/cloud/compute:reservations_client_samples              (cached) PASSED in 7.7s
Step #2 - "build.sh": //google/cloud/compute:resource_policies_client_samples         (cached) PASSED in 8.9s
Step #2 - "build.sh": //google/cloud/compute:routers_client_samples                   (cached) PASSED in 7.1s
Step #2 - "build.sh": //google/cloud/compute:routes_client_samples                    (cached) PASSED in 4.1s
Step #2 - "build.sh": //google/cloud/compute:security_policies_client_samples         (cached) PASSED in 6.9s
Step #2 - "build.sh": //google/cloud/compute:service_attachments_client_samples       (cached) PASSED in 9.4s
Step #2 - "build.sh": //google/cloud/compute:snapshot_settings_client_samples         (cached) PASSED in 8.4s
Step #2 - "build.sh": //google/cloud/compute:snapshots_client_samples                 (cached) PASSED in 5.8s
Step #2 - "build.sh": //google/cloud/compute:ssl_certificates_client_samples          (cached) PASSED in 6.5s
Step #2 - "build.sh": //google/cloud/compute:ssl_policies_client_samples              (cached) PASSED in 6.2s
Step #2 - "build.sh": //google/cloud/compute:storage_pool_types_client_samples        (cached) PASSED in 7.5s
Step #2 - "build.sh": //google/cloud/compute:storage_pools_client_samples             (cached) PASSED in 7.5s
Step #2 - "build.sh": //google/cloud/compute:subnetworks_client_samples               (cached) PASSED in 5.3s
Step #2 - "build.sh": //google/cloud/compute:target_grpc_proxies_client_samples       (cached) PASSED in 7.2s
Step #2 - "build.sh": //google/cloud/compute:target_http_proxies_client_samples       (cached) PASSED in 6.2s
Step #2 - "build.sh": //google/cloud/compute:target_https_proxies_client_samples      (cached) PASSED in 3.1s
Step #2 - "build.sh": //google/cloud/compute:target_instances_client_samples          (cached) PASSED in 4.6s
Step #2 - "build.sh": //google/cloud/compute:target_pools_client_samples              (cached) PASSED in 4.7s
Step #2 - "build.sh": //google/cloud/compute:target_ssl_proxies_client_samples        (cached) PASSED in 4.5s
Step #2 - "build.sh": //google/cloud/compute:target_tcp_proxies_client_samples        (cached) PASSED in 3.8s
Step #2 - "build.sh": //google/cloud/compute:target_vpn_gateways_client_samples       (cached) PASSED in 7.4s
Step #2 - "build.sh": //google/cloud/compute:url_maps_client_samples                  (cached) PASSED in 8.3s
Step #2 - "build.sh": //google/cloud/compute:vpn_gateways_client_samples              (cached) PASSED in 6.6s
Step #2 - "build.sh": //google/cloud/compute:vpn_tunnels_client_samples               (cached) PASSED in 5.0s
Step #2 - "build.sh": //google/cloud/compute:zone_operations_client_samples           (cached) PASSED in 4.0s
Step #2 - "build.sh": //google/cloud/compute:zones_client_samples                     (cached) PASSED in 3.2s
Step #2 - "build.sh": //google/cloud/compute/integration_tests:compute_integration_test (cached) PASSED in 18.5s
Step #2 - "build.sh": //google/cloud/compute/samples:disk_samples                     (cached) PASSED in 14.6s
Step #2 - "build.sh": //google/cloud/iam:admin_v1_samples_iam_client_samples          (cached) PASSED in 4.4s
Step #2 - "build.sh": //google/cloud/iam:credentials_v1_samples_iam_credentials_client_samples (cached) PASSED in 2.3s
Step #2 - "build.sh": //google/cloud/iam:v1_samples_iam_policy_client_samples         (cached) PASSED in 4.9s
Step #2 - "build.sh": //google/cloud/iam:v2_samples_policies_client_samples           (cached) PASSED in 4.3s
Step #2 - "build.sh": //google/cloud/iam:v3_samples_policy_bindings_client_samples    (cached) PASSED in 5.1s
Step #2 - "build.sh": //google/cloud/iam:v3_samples_principal_access_boundary_policies_client_samples (cached) PASSED in 3.1s
Step #2 - "build.sh": //google/cloud/logging:v2_samples_config_service_v2_client_samples (cached) PASSED in 4.4s
Step #2 - "build.sh": //google/cloud/logging:v2_samples_logging_service_v2_client_samples (cached) PASSED in 5.2s
Step #2 - "build.sh": //google/cloud/logging:v2_samples_metrics_service_v2_client_samples (cached) PASSED in 5.1s
Step #2 - "build.sh": //google/cloud/logging/integration_tests:logging_integration_test (cached) PASSED in 2.9s
Step #2 - "build.sh": //google/cloud/sql/integration_tests:sql_integration_test       (cached) PASSED in 2.8s
Step #2 - "build.sh": 
Step #2 - "build.sh": Executed 0 out of 129 tests: 129 tests pass.
Step #2 - "build.sh": There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
Step #2 - "build.sh": 
Step #2 - "build.sh": 2026-08-28T11:57:29Z (+699s)
Step #2 - "build.sh": ---------------------------------------------------------
Step #2 - "build.sh": |   Running combined examples using multiple services   |
Step #2 - "build.sh": ---------------------------------------------------------
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": INFO: Invocation ID: 3c2e9a83-311a-4afb-b85d-f48d71904fa1
Step #2 - "build.sh": Computing main repo mapping: 
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": Loading: 
Step #2 - "build.sh": Loading: 0 packages loaded
Step #2 - "build.sh": WARNING: Build option --test_env has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
Step #2 - "build.sh": Analyzing: 9 targets (1 packages loaded, 0 targets configured)
Step #2 - "build.sh": Analyzing: 9 targets (1 packages loaded, 0 targets configured)
Step #2 - "build.sh": 
Step #2 - "build.sh": INFO: Analyzed 9 targets (11 packages loaded, 15583 targets configured).
Step #2 - "build.sh": [9,091 / 9,094] 1 / 3 tests; Testing //examples:api_key; Downloading examples/api_key/test.xml; 0s remote-cache ... (3 actions, 0 running)
Step #2 - "build.sh": INFO: Found 6 targets and 3 test targets...
Step #2 - "build.sh": INFO: Elapsed time: 1.675s, Critical Path: 0.37s
Step #2 - "build.sh": INFO: 4 processes: 9102 action cache hit, 6 remote cache hit.
Step #2 - "build.sh": INFO: Build completed successfully, 4 total actions
Step #2 - "build.sh": //examples:api_key                                              (cached) PASSED in 8.8s
Step #2 - "build.sh": //examples:gcs2cbt_test                                         (cached) PASSED in 2.8s
Step #2 - "build.sh": //examples:grpc_credential_types                                (cached) PASSED in 63.8s
Step #2 - "build.sh": 
Step #2 - "build.sh": Executed 0 out of 3 tests: 3 tests pass.
Step #2 - "build.sh": There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
Step #2 - "build.sh": 
Step #2 - "build.sh": 2026-08-28T11:57:34Z (+704s)
Step #2 - "build.sh": ------------------------------------------------------
Step #2 - "build.sh": |   Running generator production integration tests   |
Step #2 - "build.sh": ------------------------------------------------------
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": INFO: Invocation ID: b92b1e2d-e31f-447e-8ec5-279593852056
Step #2 - "build.sh": Computing main repo mapping: 
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": Loading: 
Step #2 - "build.sh": Loading: 0 packages loaded
Step #2 - "build.sh": WARNING: Build option --test_env has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
Step #2 - "build.sh": Analyzing: 84 targets (2 packages loaded, 0 targets configured)
Step #2 - "build.sh": Analyzing: 84 targets (2 packages loaded, 3774 targets configured)
Step #2 - "build.sh": 
Step #2 - "build.sh": INFO: Analyzed 84 targets (4 packages loaded, 14808 targets configured).
Step #2 - "build.sh": [9,424 / 9,429] Testing //generator/integration_tests:golden_v1_samples_deprecated_client_samples; 0s remote-cache ... (5 actions, 0 running)
Step #2 - "build.sh": INFO: Found 79 targets and 5 test targets...
Step #2 - "build.sh": INFO: Elapsed time: 1.585s, Critical Path: 0.36s
Step #2 - "build.sh": INFO: 6 processes: 9503 action cache hit, 10 remote cache hit.
Step #2 - "build.sh": INFO: Build completed successfully, 6 total actions
Step #2 - "build.sh": //generator/integration_tests:golden_v1_samples_deprecated_client_samples (cached) PASSED in 1.5s
Step #2 - "build.sh": //generator/integration_tests:golden_v1_samples_golden_kitchen_sink_client_samples (cached) PASSED in 1.4s
Step #2 - "build.sh": //generator/integration_tests:golden_v1_samples_golden_rest_only_client_samples (cached) PASSED in 1.4s
Step #2 - "build.sh": //generator/integration_tests:golden_v1_samples_golden_thing_admin_client_samples (cached) PASSED in 1.5s
Step #2 - "build.sh": //generator/integration_tests:golden_v1_samples_request_id_client_samples (cached) PASSED in 1.5s
Step #2 - "build.sh": 
Step #2 - "build.sh": Executed 0 out of 5 tests: 5 tests pass.
Step #2 - "build.sh": There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
Step #2 - "build.sh": 
Step #2 - "build.sh": 2026-08-28T11:57:35Z (+705s)
Step #2 - "build.sh": -------------------------------------------
Step #2 - "build.sh": |   Skipping generator integration test   |
Step #2 - "build.sh": -------------------------------------------
Step #2 - "build.sh": ==> 🕑 msan completed in 704.884 seconds
Finished Step #2 - "build.sh"
Starting Step #3 - "remove-image"
Step #3 - "remove-image": Already have image (with digest): gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #3 - "remove-image": Digests:
Step #3 - "remove-image": - us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan@sha256:c022bb55325e144c52ec5cf23278b0f6cc5560e238338105a7d42ae5be34470f
Step #3 - "remove-image": 
Step #3 - "remove-image": Tags:
Step #3 - "remove-image": - us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan:7846a417-1671-4bfe-91bd-ca884284144e
Step #3 - "remove-image": Delete request issued.
Step #3 - "remove-image": Waiting for operation [projects/cloud-cpp-testing-resources/locations/us-east1/operations/7e02c0b8-830c-466f-ab5a-0e991ce12a57] to complete...
Step #3 - "remove-image": .....done.
Finished Step #3 - "remove-image"
PUSH
DONE