Skip to content

feat(storage): separate read and hedging thread pools - #16389

Open
ajayky-os wants to merge 1 commit into
googleapis:mainfrom
ajayky-os:perf/hedging_thread_pool
Open

feat(storage): separate read and hedging thread pools#16389
ajayky-os wants to merge 1 commit into
googleapis:mainfrom
ajayky-os:perf/hedging_thread_pool

feat(storage): separate read and hedging thread pools

edddb77
Select commit
Loading
Failed to load commit list.
Google Cloud Build / msan-storage-pr (cloud-cpp-testing-resources) succeeded Aug 28, 2026 in 11m 58s

Summary

Build Information

Trigger msan-storage-pr
Build 1f24298e-5503-4be7-9c1a-9f7dff0f4e3d
Start 2026-08-28T04:39:07-07:00
Duration 10m48.326s
Status SUCCESS

Steps

Step Status Duration
kaniko-build SUCCESS 3m34.892s
download-runner-image SUCCESS 1m10.413s
build.sh SUCCESS 5m36.921s
remove-image SUCCESS 4.107s
cancel-in-progress-builds-for-PR SUCCESS 44.768s

Details

starting build "1f24298e-5503-4be7-9c1a-9f7dff0f4e3d"

FETCHSOURCE
From https://github.com/googleapis/google-cloud-cpp
 * branch              edddb774b3e8c7ca9876b04897dd10c390771135 -> FETCH_HEAD
Updating files:   1% (312/23195)
Updating files:   2% (464/23195)
Updating files:   3% (696/23195)
Updating files:   4% (928/23195)
Updating files:   5% (1160/23195)
Updating files:   6% (1392/23195)
Updating files:   7% (1624/23195)
Updating files:   8% (1856/23195)
Updating files:   9% (2088/23195)
Updating files:  10% (2320/23195)
Updating files:  11% (2552/23195)
Updating files:  12% (2784/23195)
Updating files:  13% (3016/23195)
Updating files:  14% (3248/23195)
Updating files:  15% (3480/23195)
Updating files:  16% (3712/23195)
Updating files:  17% (3944/23195)
Updating files:  18% (4176/23195)
Updating files:  19% (4408/23195)
Updating files:  20% (4639/23195)
Updating files:  21% (4871/23195)
Updating files:  22% (5103/23195)
Updating files:  23% (5335/23195)
Updating files:  24% (5567/23195)
Updating files:  25% (5799/23195)
Updating files:  26% (6031/23195)
Updating files:  27% (6263/23195)
Updating files:  28% (6495/23195)
Updating files:  29% (6727/23195)
Updating files:  30% (6959/23195)
Updating files:  31% (7191/23195)
Updating files:  32% (7423/23195)
Updating files:  33% (7655/23195)
Updating files:  34% (7887/23195)
Updating files:  35% (8119/23195)
Updating files:  36% (8351/23195)
Updating files:  37% (8583/23195)
Updating files:  38% (8815/23195)
Updating files:  39% (9047/23195)
Updating files:  40% (9278/23195)
Updating files:  41% (9510/23195)
Updating files:  42% (9742/23195)
Updating files:  43% (9974/23195)
Updating files:  44% (10206/23195)
Updating files:  45% (10438/23195)
Updating files:  46% (10670/23195)
Updating files:  47% (10902/23195)
Updating files:  48% (11134/23195)
Updating files:  49% (11366/23195)
Updating files:  50% (11598/23195)
Updating files:  51% (11830/23195)
Updating files:  52% (12062/23195)
Updating files:  53% (12294/23195)
Updating files:  54% (12526/23195)
Updating files:  55% (12758/23195)
Updating files:  56% (12990/23195)
Updating files:  57% (13222/23195)
Updating files:  58% (13454/23195)
Updating files:  59% (13686/23195)
Updating files:  60% (13917/23195)
Updating files:  61% (14149/23195)
Updating files:  62% (14381/23195)
Updating files:  63% (14613/23195)
Updating files:  64% (14845/23195)
Updating files:  65% (15077/23195)
Updating files:  66% (15309/23195)
Updating files:  67% (15541/23195)
Updating files:  68% (15773/23195)
Updating files:  69% (16005/23195)
Updating files:  70% (16237/23195)
Updating files:  71% (16469/23195)
Updating files:  72% (16701/23195)
Updating files:  73% (16933/23195)
Updating files:  74% (17165/23195)
Updating files:  75% (17397/23195)
Updating files:  76% (17629/23195)
Updating files:  77% (17861/23195)
Updating files:  78% (18093/23195)
Updating files:  79% (18325/23195)
Updating files:  80% (18556/23195)
Updating files:  81% (18788/23195)
Updating files:  82% (19020/23195)
Updating files:  83% (19252/23195)
Updating files:  84% (19484/23195)
Updating files:  85% (19716/23195)
Updating files:  86% (19948/23195)
Updating files:  87% (20180/23195)
Updating files:  88% (20412/23195)
Updating files:  89% (20644/23195)
Updating files:  90% (20876/23195)
Updating files:  91% (21108/23195)
Updating files:  92% (21340/23195)
Updating files:  93% (21572/23195)
Updating files:  94% (21804/23195)
Updating files:  95% (22036/23195)
Updating files:  96% (22268/23195)
Updating files:  97% (22500/23195)
Updating files:  98% (22732/23195)
Updating files:  99% (22964/23195)
Updating files: 100% (23195/23195)
Updating files: 100% (23195/23195), done.
HEAD is now at edddb774 feat(storage): separate read and hedging thread pools
GitCommit:
edddb774b3e8c7ca9876b04897dd10c390771135
BUILD
Starting Step #4 - "cancel-in-progress-builds-for-PR"
Starting Step #0 - "kaniko-build"
Step #4 - "cancel-in-progress-builds-for-PR": Pulling image: gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #0 - "kaniko-build": Pulling image: gcr.io/kaniko-project/executor:v1.24.0-debug@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #4 - "cancel-in-progress-builds-for-PR": gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574: Pulling from google.com/cloudsdktool/cloud-sdk
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Pulling fs layer
Step #0 - "kaniko-build": gcr.io/kaniko-project/executor@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd: Pulling from kaniko-project/executor
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Download complete
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Download complete
Step #0 - "kaniko-build": 2fc842204170: Pulling fs layer
Step #0 - "kaniko-build": 8d1674b25e7b: Pulling fs layer
Step #0 - "kaniko-build": 9bdd0371dbb4: Pulling fs layer
Step #0 - "kaniko-build": 9aeeda79b717: Pulling fs layer
Step #0 - "kaniko-build": aaab912f1a4e: Pulling fs layer
Step #0 - "kaniko-build": 0006a7b536fb: Pulling fs layer
Step #0 - "kaniko-build": 3d08e2063109: Pulling fs layer
Step #0 - "kaniko-build": fc4517ec89af: Pulling fs layer
Step #0 - "kaniko-build": b892afb8bf07: Pulling fs layer
Step #0 - "kaniko-build": cdc2b9345183: Pulling fs layer
Step #0 - "kaniko-build": 3934701e2c3d: Pulling fs layer
Step #0 - "kaniko-build": 46be974dd378: Pulling fs layer
Step #0 - "kaniko-build": 2b61f8ddf6d9: Pulling fs layer
Step #0 - "kaniko-build": 9bdd0371dbb4: Waiting
Step #0 - "kaniko-build": 0006a7b536fb: Waiting
Step #0 - "kaniko-build": 9aeeda79b717: Waiting
Step #0 - "kaniko-build": fc4517ec89af: Waiting
Step #0 - "kaniko-build": b892afb8bf07: Waiting
Step #0 - "kaniko-build": aaab912f1a4e: Waiting
Step #0 - "kaniko-build": cdc2b9345183: Waiting
Step #0 - "kaniko-build": 3d08e2063109: Waiting
Step #0 - "kaniko-build": 2b61f8ddf6d9: Waiting
Step #0 - "kaniko-build": 46be974dd378: Waiting
Step #0 - "kaniko-build": 3934701e2c3d: Waiting
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Download complete
Step #0 - "kaniko-build": 2fc842204170: Download complete
Step #0 - "kaniko-build": 9bdd0371dbb4: Verifying Checksum
Step #0 - "kaniko-build": 9bdd0371dbb4: Download complete
Step #0 - "kaniko-build": 2fc842204170: Pull complete
Step #0 - "kaniko-build": 8d1674b25e7b: Download complete
Step #0 - "kaniko-build": 8d1674b25e7b: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Download complete
Step #0 - "kaniko-build": 9bdd0371dbb4: Pull complete
Step #0 - "kaniko-build": 3d08e2063109: Verifying Checksum
Step #0 - "kaniko-build": 3d08e2063109: Download complete
Step #0 - "kaniko-build": aaab912f1a4e: Verifying Checksum
Step #0 - "kaniko-build": aaab912f1a4e: Download complete
Step #0 - "kaniko-build": 9aeeda79b717: Verifying Checksum
Step #0 - "kaniko-build": 9aeeda79b717: Download complete
Step #0 - "kaniko-build": fc4517ec89af: Verifying Checksum
Step #0 - "kaniko-build": fc4517ec89af: Download complete
Step #0 - "kaniko-build": 9aeeda79b717: Pull complete
Step #0 - "kaniko-build": 0006a7b536fb: Verifying Checksum
Step #0 - "kaniko-build": 0006a7b536fb: Download complete
Step #0 - "kaniko-build": aaab912f1a4e: Pull complete
Step #0 - "kaniko-build": 46be974dd378: Verifying Checksum
Step #0 - "kaniko-build": 46be974dd378: Download complete
Step #0 - "kaniko-build": cdc2b9345183: Verifying Checksum
Step #0 - "kaniko-build": cdc2b9345183: Download complete
Step #0 - "kaniko-build": b892afb8bf07: Verifying Checksum
Step #0 - "kaniko-build": b892afb8bf07: Download complete
Step #0 - "kaniko-build": 0006a7b536fb: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Download complete
Step #0 - "kaniko-build": 3d08e2063109: Pull complete
Step #0 - "kaniko-build": 2b61f8ddf6d9: Verifying Checksum
Step #0 - "kaniko-build": 2b61f8ddf6d9: Download complete
Step #0 - "kaniko-build": 3934701e2c3d: Verifying Checksum
Step #0 - "kaniko-build": 3934701e2c3d: Download complete
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Pull complete
Step #0 - "kaniko-build": fc4517ec89af: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Pull complete
Step #0 - "kaniko-build": b892afb8bf07: Pull complete
Step #0 - "kaniko-build": cdc2b9345183: Pull complete
Step #0 - "kaniko-build": 3934701e2c3d: Pull complete
Step #0 - "kaniko-build": 46be974dd378: Pull complete
Step #0 - "kaniko-build": 2b61f8ddf6d9: Pull complete
Step #0 - "kaniko-build": Digest: sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": Status: Downloaded newer image for gcr.io/kaniko-project/executor@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": gcr.io/kaniko-project/executor:v1.24.0-debug@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Using dockerignore file: /workspace/ci/.dockerignore"
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-08-28T11:39:24Z" level=info msg="Retrieving image fedora:40 from registry index.docker.io"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Built cross stage deps: map[]"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Executing 0 build triggers"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Building stage 'fedora:40' [idx: '0', base-idx: '-1']"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:e005ad110158bfe52f0bfb62c43052fc86e99e36d8386a62785d23e3037f90d2..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Using caching version of cmd: RUN dnf makecache &&     dnf install -y clang clang-tools-extra cmake findutils         git llvm make ninja-build openssl-devel patch python         python3 python3-devel python3-lit python-pip tar unzip which wget xz"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:9d5b1e125453cae51a24d3cd1cae5443fca86cc71b75234372c1dab7f670bf59..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y python3-devel"
Step #0 - "kaniko-build": time="2026-08-28T11:39:25Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:90ef07e1cd6fa920a949553451f6c11581264649e54b461dd254f4930ccd7094..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:26Z" level=info msg="Using caching version of cmd: RUN pip3 install --upgrade pip"
Step #0 - "kaniko-build": time="2026-08-28T11:39:26Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:ebc88857bacfdea4e75ba63cf341267b897a999db7dbfb1d606262d3674c2705..."
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Download complete
Step #0 - "kaniko-build": time="2026-08-28T11:39:26Z" level=info msg="Using caching version of cmd: RUN pip3 install setuptools wheel"
Step #0 - "kaniko-build": time="2026-08-28T11:39:26Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:56c94ce0178764adf3fe805e8a64e1eec7e23f50333cbb6092b70202b425927a..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y java-latest-openjdk-devel"
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:6e1ee6d2a1fb763e652088324c6f241d6c3978ac18de6f0f9380fc732625ded5..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Using caching version of cmd: RUN echo 'root:cloudcxx' | chpasswd"
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:c9c1c92f41f0a8db11af9f076f47a40be83252a23c669a699f5eb08ee43e65ef..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Using caching version of cmd: RUN git clone --depth=1 --branch llvmorg-18.1.1 https://github.com/llvm/llvm-project"
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:a7fa06631ce468534586b02b61c044c6e735d21b0f1a979f2dd1242668fa8d8c..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Using caching version of cmd: RUN cmake -GNinja -S runtimes -B build     -DCMAKE_BUILD_TYPE=Release     -DLLVM_ENABLE_PROJECTS=\"clang\"     -DLLVM_ENABLE_RUNTIMES=\"libcxx;libcxxabi;libunwind\"     -DCMAKE_C_COMPILER=clang     -DCMAKE_CXX_COMPILER=clang++     -DLLVM_USE_SANITIZER=MemoryWithOrigins     -DLIBCXXABI_USE_LLVM_UNWINDER=OFF     -DCOMPILER_RT_USE_LLVM_UNWINDER=OFF     -DCMAKE_INSTALL_PREFIX=/usr"
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:f65b1167323150bf86b8173f1ca602e9c5c458c852dccaabc74ecd0235ca7c12..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Using caching version of cmd: RUN cmake --build build"
Step #0 - "kaniko-build": time="2026-08-28T11:39:27Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:e8db00a497af4cc57c853dd3fd6af51d4a14638275c7ab3fb259bce747d6fa21..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:28Z" level=info msg="Using caching version of cmd: RUN cmake --install build"
Step #0 - "kaniko-build": time="2026-08-28T11:39:28Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:cbc61717fbbfd6ac8e73048dd3f77763f4ef330ae3a80dcfc5d43046e76c49aa..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:28Z" level=info msg="Using caching version of cmd: RUN echo \"/usr/lib/x86_64-unknown-linux-gnu\" >/etc/ld.so.conf.d/msan.conf"
Step #0 - "kaniko-build": time="2026-08-28T11:39:28Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:9e44ab0a2cbbf08ec937ea005172f9f791baf66167aa559c4366837cd9a3aa71..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:28Z" level=info msg="Using caching version of cmd: RUN ldconfig"
Step #0 - "kaniko-build": time="2026-08-28T11:39:28Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:3bc1249892dcf73cb470bdbd3bb3958888bea905938b20055137bfcd8b095847..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:28Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y python3.10"
Step #0 - "kaniko-build": time="2026-08-28T11:39:28Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:cda0e47b1bc391d27a5d470a4e90c0b8fcbd0432291328f344e8785ac2203edc..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:29Z" level=info msg="Using caching version of cmd: RUN /var/tmp/ci/install-cloud-sdk.sh"
Step #0 - "kaniko-build": time="2026-08-28T11:39:29Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:26ed640b3a705683cd7e2f987df4b430fc9a89ea3cc195d9c181bc42c670379d..."
Step #0 - "kaniko-build": time="2026-08-28T11:39:29Z" level=info msg="Using caching version of cmd: RUN curl -o /usr/bin/bazelisk -sSL \"https://github.com/bazelbuild/bazelisk/releases/download/v1.28.1/bazelisk-linux-${ARCH}\" &&     chmod +x /usr/bin/bazelisk &&     ln -s /usr/bin/bazelisk /usr/bin/bazel"
Step #0 - "kaniko-build": time="2026-08-28T11:39:29Z" level=info msg="Unpacking rootfs as cmd COPY . /var/tmp/ci requires it."
Step #0 - "kaniko-build": time="2026-08-28T11:39:32Z" level=info msg="ARG NCPU=4"
Step #0 - "kaniko-build": time="2026-08-28T11:39:32Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:39:32Z" level=info msg="ARG ARCH=amd64"
Step #0 - "kaniko-build": time="2026-08-28T11:39:32Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:39:32Z" level=info msg="RUN dnf makecache &&     dnf install -y clang clang-tools-extra cmake findutils         git llvm make ninja-build openssl-devel patch python         python3 python3-devel python3-lit python-pip tar unzip which wget xz"
Step #0 - "kaniko-build": time="2026-08-28T11:39:32Z" level=info msg="Found cached layer, extracting to filesystem"
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": Digest: sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": Status: Downloaded newer image for gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": + test -z 16389
Step #4 - "cancel-in-progress-builds-for-PR": ++ gcloud builds describe --region us-east1 --format 'value(create_time)' 1f24298e-5503-4be7-9c1a-9f7dff0f4e3d
Step #4 - "cancel-in-progress-builds-for-PR": + ctime=2026-08-28T11:38:13.667562Z
Step #4 - "cancel-in-progress-builds-for-PR": + query=tags=pr
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND tags=16389'
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND substitutions.COMMIT_SHA != edddb774b3e8c7ca9876b04897dd10c390771135'
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND create_time < 2026-08-28T11:38:13.667562Z'
Step #4 - "cancel-in-progress-builds-for-PR": + gcloud builds list --region us-east1 --ongoing '--format=value(id)' --filter 'tags=pr AND tags=16389 AND substitutions.COMMIT_SHA != edddb774b3e8c7ca9876b04897dd10c390771135 AND create_time < 2026-08-28T11:38:13.667562Z'
Step #4 - "cancel-in-progress-builds-for-PR": + xargs -r -t gcloud builds cancel --region us-east1
Finished Step #4 - "cancel-in-progress-builds-for-PR"
Step #0 - "kaniko-build": time="2026-08-28T11:40:14Z" level=info msg="RUN dnf makecache && dnf install -y python3-devel"
Step #0 - "kaniko-build": time="2026-08-28T11:40:14Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:40:15Z" level=info msg="RUN pip3 install --upgrade pip"
Step #0 - "kaniko-build": time="2026-08-28T11:40:15Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:40:16Z" level=info msg="RUN pip3 install setuptools wheel"
Step #0 - "kaniko-build": time="2026-08-28T11:40:16Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:40:17Z" level=info msg="RUN dnf makecache && dnf install -y java-latest-openjdk-devel"
Step #0 - "kaniko-build": time="2026-08-28T11:40:17Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:40:44Z" level=info msg="RUN echo 'root:cloudcxx' | chpasswd"
Step #0 - "kaniko-build": time="2026-08-28T11:40:44Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:40:44Z" level=info msg="WORKDIR /var/tmp/build"
Step #0 - "kaniko-build": time="2026-08-28T11:40:44Z" level=info msg="Cmd: workdir"
Step #0 - "kaniko-build": time="2026-08-28T11:40:44Z" level=info msg="Changed working directory to /var/tmp/build"
Step #0 - "kaniko-build": time="2026-08-28T11:40:44Z" level=info msg="Creating directory /var/tmp/build with uid -1 and gid -1"
Step #0 - "kaniko-build": time="2026-08-28T11:40:44Z" level=info msg="Taking snapshot of files..."
Step #0 - "kaniko-build": time="2026-08-28T11:40:44Z" level=info msg="RUN git clone --depth=1 --branch llvmorg-18.1.1 https://github.com/llvm/llvm-project"
Step #0 - "kaniko-build": time="2026-08-28T11:40:44Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:41:36Z" level=info msg="WORKDIR /var/tmp/build/llvm-project"
Step #0 - "kaniko-build": time="2026-08-28T11:41:36Z" level=info msg="Cmd: workdir"
Step #0 - "kaniko-build": time="2026-08-28T11:41:36Z" level=info msg="Changed working directory to /var/tmp/build/llvm-project"
Step #0 - "kaniko-build": time="2026-08-28T11:41:36Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:41:36Z" level=info msg="RUN cmake -GNinja -S runtimes -B build     -DCMAKE_BUILD_TYPE=Release     -DLLVM_ENABLE_PROJECTS=\"clang\"     -DLLVM_ENABLE_RUNTIMES=\"libcxx;libcxxabi;libunwind\"     -DCMAKE_C_COMPILER=clang     -DCMAKE_CXX_COMPILER=clang++     -DLLVM_USE_SANITIZER=MemoryWithOrigins     -DLIBCXXABI_USE_LLVM_UNWINDER=OFF     -DCOMPILER_RT_USE_LLVM_UNWINDER=OFF     -DCMAKE_INSTALL_PREFIX=/usr"
Step #0 - "kaniko-build": time="2026-08-28T11:41:36Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:41:36Z" level=info msg="RUN cmake --build build"
Step #0 - "kaniko-build": time="2026-08-28T11:41:36Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:41:38Z" level=info msg="RUN cmake --install build"
Step #0 - "kaniko-build": time="2026-08-28T11:41:38Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:41:40Z" level=info msg="RUN echo \"/usr/lib/x86_64-unknown-linux-gnu\" >/etc/ld.so.conf.d/msan.conf"
Step #0 - "kaniko-build": time="2026-08-28T11:41:40Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:41:40Z" level=info msg="RUN ldconfig"
Step #0 - "kaniko-build": time="2026-08-28T11:41:40Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:41:40Z" level=info msg="COPY . /var/tmp/ci"
Step #0 - "kaniko-build": time="2026-08-28T11:41:40Z" level=info msg="Taking snapshot of files..."
Step #0 - "kaniko-build": time="2026-08-28T11:41:40Z" level=info msg="WORKDIR /var/tmp/downloads"
Step #0 - "kaniko-build": time="2026-08-28T11:41:40Z" level=info msg="Cmd: workdir"
Step #0 - "kaniko-build": time="2026-08-28T11:41:40Z" level=info msg="Changed working directory to /var/tmp/downloads"
Step #0 - "kaniko-build": time="2026-08-28T11:41:40Z" level=info msg="Creating directory /var/tmp/downloads with uid -1 and gid -1"
Step #0 - "kaniko-build": time="2026-08-28T11:41:40Z" level=info msg="Taking snapshot of files..."
Step #0 - "kaniko-build": time="2026-08-28T11:41:40Z" level=info msg="RUN dnf makecache && dnf install -y python3.10"
Step #0 - "kaniko-build": time="2026-08-28T11:41:40Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:41:44Z" level=info msg="ENV CLOUDSDK_PYTHON=python3.10"
Step #0 - "kaniko-build": time="2026-08-28T11:41:44Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:41:44Z" level=info msg="RUN /var/tmp/ci/install-cloud-sdk.sh"
Step #0 - "kaniko-build": time="2026-08-28T11:41:44Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:44Z" level=info msg="ENV CLOUD_SDK_LOCATION=/usr/local/google-cloud-sdk"
Step #0 - "kaniko-build": time="2026-08-28T11:42:44Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:42:44Z" level=info msg="ENV PATH=${CLOUD_SDK_LOCATION}/bin:${PATH}"
Step #0 - "kaniko-build": time="2026-08-28T11:42:44Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-08-28T11:42:44Z" level=info msg="RUN curl -o /usr/bin/bazelisk -sSL \"https://github.com/bazelbuild/bazelisk/releases/download/v1.28.1/bazelisk-linux-${ARCH}\" &&     chmod +x /usr/bin/bazelisk &&     ln -s /usr/bin/bazelisk /usr/bin/bazel"
Step #0 - "kaniko-build": time="2026-08-28T11:42:44Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-08-28T11:42:45Z" level=info msg="Pushing image to us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan:1f24298e-5503-4be7-9c1a-9f7dff0f4e3d"
Step #0 - "kaniko-build": time="2026-08-28T11:42:46Z" level=info msg="Pushed us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan@sha256:d904cf0fd254ef1ef8f7968d2a8a3056aa1d91a8da838ec8124c8e5c6cea1b40"
Finished Step #0 - "kaniko-build"
Starting Step #1 - "download-runner-image"
Step #1 - "download-runner-image": Pulling image: us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan:1f24298e-5503-4be7-9c1a-9f7dff0f4e3d
Step #1 - "download-runner-image": 1f24298e-5503-4be7-9c1a-9f7dff0f4e3d: Pulling from cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan
Step #1 - "download-runner-image": fc4518b6c5f9: Pulling fs layer
Step #1 - "download-runner-image": 5e8a8bc10713: Pulling fs layer
Step #1 - "download-runner-image": b5493f6eace4: Pulling fs layer
Step #1 - "download-runner-image": 6a4bb87f2a62: Pulling fs layer
Step #1 - "download-runner-image": 44052672a2f7: Pulling fs layer
Step #1 - "download-runner-image": 7f55e8d9f973: Pulling fs layer
Step #1 - "download-runner-image": 32621cf3b379: Pulling fs layer
Step #1 - "download-runner-image": cbac088656a9: Pulling fs layer
Step #1 - "download-runner-image": f43b21e3ba11: Pulling fs layer
Step #1 - "download-runner-image": 3f3e1a9e6272: Pulling fs layer
Step #1 - "download-runner-image": 5a6b79a42a18: Pulling fs layer
Step #1 - "download-runner-image": 9e4c5aa6f8f2: Pulling fs layer
Step #1 - "download-runner-image": 3561492257fe: Pulling fs layer
Step #1 - "download-runner-image": e107a1067de3: Pulling fs layer
Step #1 - "download-runner-image": 8e4b31def5f3: Pulling fs layer
Step #1 - "download-runner-image": 40ceee76334a: Pulling fs layer
Step #1 - "download-runner-image": 9fb3ffac4da6: Pulling fs layer
Step #1 - "download-runner-image": fef9c1cebd0d: Pulling fs layer
Step #1 - "download-runner-image": a4b4078c88ca: Pulling fs layer
Step #1 - "download-runner-image": 32621cf3b379: Waiting
Step #1 - "download-runner-image": cbac088656a9: Waiting
Step #1 - "download-runner-image": f43b21e3ba11: Waiting
Step #1 - "download-runner-image": 3f3e1a9e6272: Waiting
Step #1 - "download-runner-image": 5a6b79a42a18: Waiting
Step #1 - "download-runner-image": 9e4c5aa6f8f2: Waiting
Step #1 - "download-runner-image": 3561492257fe: Waiting
Step #1 - "download-runner-image": e107a1067de3: Waiting
Step #1 - "download-runner-image": 8e4b31def5f3: Waiting
Step #1 - "download-runner-image": 40ceee76334a: Waiting
Step #1 - "download-runner-image": 9fb3ffac4da6: Waiting
Step #1 - "download-runner-image": fef9c1cebd0d: Waiting
Step #1 - "download-runner-image": a4b4078c88ca: Waiting
Step #1 - "download-runner-image": 44052672a2f7: Verifying Checksum
Step #1 - "download-runner-image": 44052672a2f7: Download complete
Step #1 - "download-runner-image": 6a4bb87f2a62: Verifying Checksum
Step #1 - "download-runner-image": 6a4bb87f2a62: Download complete
Step #1 - "download-runner-image": 32621cf3b379: Download complete
Step #1 - "download-runner-image": cbac088656a9: Verifying Checksum
Step #1 - "download-runner-image": cbac088656a9: Download complete
Step #1 - "download-runner-image": b5493f6eace4: Verifying Checksum
Step #1 - "download-runner-image": b5493f6eace4: Download complete
Step #1 - "download-runner-image": 3f3e1a9e6272: Verifying Checksum
Step #1 - "download-runner-image": 3f3e1a9e6272: Download complete
Step #1 - "download-runner-image": 5a6b79a42a18: Download complete
Step #1 - "download-runner-image": 9e4c5aa6f8f2: Verifying Checksum
Step #1 - "download-runner-image": 9e4c5aa6f8f2: Download complete
Step #1 - "download-runner-image": fc4518b6c5f9: Download complete
Step #1 - "download-runner-image": 3561492257fe: Verifying Checksum
Step #1 - "download-runner-image": 3561492257fe: Download complete
Step #1 - "download-runner-image": e107a1067de3: Verifying Checksum
Step #1 - "download-runner-image": e107a1067de3: Download complete
Step #1 - "download-runner-image": 40ceee76334a: Verifying Checksum
Step #1 - "download-runner-image": 40ceee76334a: Download complete
Step #1 - "download-runner-image": 8e4b31def5f3: Verifying Checksum
Step #1 - "download-runner-image": 8e
...
[Logs truncated due to log size limitations. For full logs, see https://storage.cloud.google.com/cloud-cpp-community-publiclogs/logs/google-cloud-cpp/16389/edddb774b3e8c7ca9876b04897dd10c390771135/fedora-msan-msan-__default__/log-1f24298e-5503-4be7-9c1a-9f7dff0f4e3d.txt.]
...
CPP_STORAGE_TEST_KEY_FILE_JSON=/dev/shm/key-2026-08.json --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_KEY_FILE_P12=/dev/shm/key-2026-08.p12 --test_env=GOOGLE_CLOUD_CPP_CLIENT_SSL_CERT_FILE=/dev/shm/client.crt --test_env=GOOGLE_CLOUD_CPP_CLIENT_SSL_CERT_CHAIN_FILE=/dev/shm/client.chain.crt --test_env=GOOGLE_CLOUD_CPP_CLIENT_SSL_KEY_FILE=/dev/shm/client.private.pem --test_tag_filters=integration-test\,-ud-only -- //google/cloud/storage/examples:storage_policy_doc_samples //google/cloud/storage/examples:storage_signed_url_v2_samples //google/cloud/storage/examples:storage_signed_url_v4_samples //google/cloud/storage/tests:alternative_endpoint_integration_test-default //google/cloud/storage/tests:alternative_endpoint_integration_test-grpc-metadata //google/cloud/storage/tests:key_file_integration_test-default //google/cloud/storage/tests:key_file_integration_test-grpc-metadata //google/cloud/storage/tests:signed_url_integration_test-default //google/cloud/storage/tests:signed_url_integration_test-grpc-metadata //google/cloud/storage/tests:unified_credentials_integration_test-default //google/cloud/storage/tests:unified_credentials_integration_test-grpc-metadata
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": INFO: Invocation ID: 02d3a218-98a7-44e1-b804-573bd71f606e
Step #2 - "build.sh": Computing main repo mapping: 
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": Loading: 
Step #2 - "build.sh": Loading: 0 packages loaded
Step #2 - "build.sh": WARNING: Build option --test_env has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
Step #2 - "build.sh": Analyzing: 11 targets (0 packages loaded, 0 targets configured)
Step #2 - "build.sh": Analyzing: 11 targets (0 packages loaded, 0 targets configured)
Step #2 - "build.sh": 
Step #2 - "build.sh": Analyzing: 11 targets (286 packages loaded, 15096 targets configured)
Step #2 - "build.sh": 
Step #2 - "build.sh": INFO: Analyzed 11 targets (286 packages loaded, 15111 targets configured).
Step #2 - "build.sh": [5,189 / 8,853] checking cached actions
Step #2 - "build.sh": INFO: Found 11 test targets...
Step #2 - "build.sh": INFO: Elapsed time: 3.982s, Critical Path: 0.45s
Step #2 - "build.sh": INFO: 12 processes: 8871 action cache hit, 14 remote cache hit.
Step #2 - "build.sh": INFO: Build completed successfully, 12 total actions
Step #2 - "build.sh": //google/cloud/storage/examples:storage_policy_doc_samples      (cached) PASSED in 5.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_signed_url_v2_samples   (cached) PASSED in 1.9s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_signed_url_v4_samples   (cached) PASSED in 1.9s
Step #2 - "build.sh": //google/cloud/storage/tests:alternative_endpoint_integration_test-default (cached) PASSED in 5.4s
Step #2 - "build.sh": //google/cloud/storage/tests:alternative_endpoint_integration_test-grpc-metadata (cached) PASSED in 7.9s
Step #2 - "build.sh": //google/cloud/storage/tests:key_file_integration_test-default  (cached) PASSED in 6.0s
Step #2 - "build.sh": //google/cloud/storage/tests:key_file_integration_test-grpc-metadata (cached) PASSED in 1.9s
Step #2 - "build.sh": //google/cloud/storage/tests:signed_url_integration_test-default (cached) PASSED in 4.9s
Step #2 - "build.sh": //google/cloud/storage/tests:signed_url_integration_test-grpc-metadata (cached) PASSED in 11.1s
Step #2 - "build.sh": //google/cloud/storage/tests:unified_credentials_integration_test-default (cached) PASSED in 9.1s
Step #2 - "build.sh": //google/cloud/storage/tests:unified_credentials_integration_test-grpc-metadata (cached) PASSED in 720.2s
Step #2 - "build.sh": 
Step #2 - "build.sh": Executed 0 out of 11 tests: 11 tests pass.
Step #2 - "build.sh": There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
Step #2 - "build.sh": 2026-08-28T11:49:33Z (+328s): Launching Cloud Storage emulator on port 8585
Step #2 - "build.sh": Successfully connected to emulator [3929]
Step #2 - "build.sh": 2026-08-28T11:49:34Z (+329s): Successfully connected to gRPC server at port 8000
Step #2 - "build.sh": 2026-08-28T11:49:36Z (+331s): + bazel test --test_output=errors --verbose_failures=true --keep_going --experimental_convenience_symlinks=ignore --cache_test_results=auto --copt=-DGOOGLE_CLOUD_CPP_TESTING_ENABLE_RAB --remote_cache=https://storage.googleapis.com/cloud-cpp-testing-resources_cloudbuild/bazel-cache/fedora-msan-msan --google_default_credentials --experimental_guard_against_concurrent_changes --config=msan --flaky_test_attempts=3 --test_env=GOOGLE_CLOUD_PROJECT=cloud-cpp-testing-resources --test_env=GOOGLE_CLOUD_CPP_TEST_REGION=us-central1 --test_env=GOOGLE_CLOUD_CPP_NON_US_TEST_REGION=asia-southeast1 --test_env=GOOGLE_CLOUD_CPP_US_EAST_TEST_REGION=us-east4 --test_env=GOOGLE_CLOUD_CPP_TEST_ZONE=us-central1-a --test_env=GOOGLE_CLOUD_CPP_TEST_ORGANIZATION=1006341795026 --test_env=GOOGLE_CLOUD_CPP_TEST_SERVICE_ACCOUNT_KEYFILE=/workspace/ci/etc/invalidated-keyfile.json --test_env=GOOGLE_CLOUD_CPP_AUTO_RUN_EXAMPLES=yes --test_env=GOOGLE_CLOUD_CPP_EXPERIMENTAL_LOG_CONFIG=lastN\,1024\,WARNING --test_env=GOOGLE_CLOUD_CPP_ENABLE_TRACING=rpc\,rpc-streams --test_env=GOOGLE_CLOUD_CPP_TRACING_OPTIONS=single_line_mode=off\,truncate_string_field_longer_than=512 --test_env=CLOUD_STORAGE_ENABLE_TRACING=raw-client\,rpc\,rpc-streams --test_env=HOME=/h --test_env=GOOGLE_CLOUD_CPP_IAM_CREDENTIALS_TEST_SERVICE_ACCOUNT=iam-credentials-test-sa@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_IAM_TEST_SERVICE_ACCOUNT=iam-test-sa@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_IAM_INVALID_TEST_SERVICE_ACCOUNT=invalid-test-account@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_IAM_QUOTA_LIMITED_INTEGRATION_TESTS= --test_env=GOOGLE_CLOUD_CPP_IAM_QUOTA_LIMITED_SAMPLES= --test_env=GOOGLE_CLOUD_CPP_BIGTABLE_TEST_INSTANCE_ID=test-instance --test_env=GOOGLE_CLOUD_CPP_BIGTABLE_TEST_CLUSTER_ID=test-instance-c1 --test_env=GOOGLE_CLOUD_CPP_BIGTABLE_TEST_ZONE_A=us-central1-b --test_env=GOOGLE_CLOUD_CPP_BIGTABLE_TEST_ZONE_B=us-central1-c --test_env=GOOGLE_CLOUD_CPP_BIGTABLE_TEST_SERVICE_ACCOUNT=bigtable-test-iam-sa@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=ENABLE_BIGTABLE_ADMIN_INTEGRATION_TESTS=no --test_env=GOOGLE_CLOUD_CPP_PUBSUB_TEST_IMPERSONATED_SERVICE_ACCOUNT=pubsub-impersonate-test-sa@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_REST_TEST_SIGNING_SERVICE_ACCOUNT=kokoro-run@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_BUCKET_NAME=gcs-grpc-team-cloud-cpp-testing-bucket --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_FOLDER_BUCKET_NAME=cloud-cpp-testing-folder-bucket --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_DESTINATION_BUCKET_NAME=gcs-grpc-team-cloud-cpp-testing-regional --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_REGION_ID=us-central1 --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_TOPIC_NAME=projects/cloud-cpp-testing-resources/topics/gcs-changes --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_SERVICE_ACCOUNT=storage-test-iam-sa@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_SIGNING_SERVICE_ACCOUNT=kokoro-run@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_HMAC_SERVICE_ACCOUNT=fake-service-account-hmac@example.com --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_CMEK_KEY=projects/cloud-cpp-testing-resources/locations/us/keyRings/gcs-testing-us-kr/cryptoKeys/integration-tests-key --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_SIGNING_CONFORMANCE_FILENAME=/workspace/google/cloud/storage/tests/v4_signatures.json --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_GZIP_FILENAME=/workspace/ci/data/fox.txt.gz --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_ROOTS_PEM=/dev/shm/roots.pem --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_ALTERNATIVE_HOSTS=private.googleapis.com\,restricted.googleapis.com --test_env=GOOGLE_CLOUD_CPP_SPANNER_SLOW_INTEGRATION_TESTS= --test_env=GOOGLE_CLOUD_CPP_SPANNER_TEST_INSTANCE_ID=test-instance --test_env=GOOGLE_CLOUD_CPP_SPANNER_TEST_SERVICE_ACCOUNT=spanner-iam-test-sa@cloud-cpp-testing-resources.iam.gserviceaccount.com --test_env=GOOGLE_CLOUD_CPP_SPANNER_DEFAULT_ENDPOINT= --test_env=GOOGLE_CLOUD_CPP_SPANNER_DEFAULT_AUTHORITY= --test_env=GOOGLE_CLOUD_CPP_BATCH_TEST_TEMPLATE_NAME=cloud-batch-sample-template --test_env=GOOGLE_CLOUD_CPP_REST_TEST_KEY_FILE_JSON=/dev/shm/key-2026-08.json --test_env=GOOGLE_CLOUD_CPP_BIGTABLE_TEST_KEY_FILE_JSON=/dev/shm/key-2026-08.json --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_KEY_FILE_JSON=/dev/shm/key-2026-08.json --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_KEY_FILE_P12=/dev/shm/key-2026-08.p12 --test_env=GOOGLE_CLOUD_CPP_CLIENT_SSL_CERT_FILE=/dev/shm/client.crt --test_env=GOOGLE_CLOUD_CPP_CLIENT_SSL_CERT_CHAIN_FILE=/dev/shm/client.chain.crt --test_env=GOOGLE_CLOUD_CPP_CLIENT_SSL_KEY_FILE=/dev/shm/client.private.pem --test_tag_filters=integration-test\,-ud-only --test_env=TESTBENCH_SHA=62d7b032a61a325e9c974181b7334879b26aca0ad4a5a2d33f4ea763ee527377\ \ - --test_env=TESTBENCH_CONTENTS_SHA=62a135c39b301f36f12cff75603cc49bef96d372e07521c354c8e6f6ad48e5fb\ \ - --test_env=CLOUD_STORAGE_EMULATOR_ENDPOINT=http://localhost:8585 --test_env=CLOUD_STORAGE_EXPERIMENTAL_GRPC_TESTBENCH_ENDPOINT=localhost:8000 --test_env=HTTPBIN_ENDPOINT=http://localhost:8585/httpbin --test_env=GOOGLE_CLOUD_CPP_STORAGE_TEST_HMAC_SERVICE_ACCOUNT=fake-service-account-sign@example.com -- //google/cloud/storage/...:all -//google/cloud/storage/tests:error_injection_integration_test-default -//google/cloud/storage/tests:error_injection_integration_test-grpc-metadata -//google/cloud/storage/examples:storage_policy_doc_samples -//google/cloud/storage/examples:storage_signed_url_v2_samples -//google/cloud/storage/examples:storage_signed_url_v4_samples -//google/cloud/storage/tests:alternative_endpoint_integration_test-default -//google/cloud/storage/tests:alternative_endpoint_integration_test-grpc-metadata -//google/cloud/storage/tests:key_file_integration_test-default -//google/cloud/storage/tests:key_file_integration_test-grpc-metadata -//google/cloud/storage/tests:signed_url_integration_test-default -//google/cloud/storage/tests:signed_url_integration_test-grpc-metadata -//google/cloud/storage/tests:unified_credentials_integration_test-default -//google/cloud/storage/tests:unified_credentials_integration_test-grpc-metadata
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": INFO: Invocation ID: 59b4c9ca-17e2-424c-b691-2dcbffda9e28
Step #2 - "build.sh": Computing main repo mapping: 
Step #2 - "build.sh": WARNING: Option 'experimental_guard_against_concurrent_changes' is deprecated: Use --guard_against_concurrent_changes instead
Step #2 - "build.sh": Loading: 
Step #2 - "build.sh": Loading: 0 packages loaded
Step #2 - "build.sh": WARNING: Build option --test_env has changed, discarding analysis cache (this can be expensive, see https://bazel.build/advanced/performance/iteration-speed).
Step #2 - "build.sh": Analyzing: 352 targets (3 packages loaded, 0 targets configured)
Step #2 - "build.sh": Analyzing: 352 targets (3 packages loaded, 0 targets configured)
Step #2 - "build.sh": 
Step #2 - "build.sh": Analyzing: 352 targets (4 packages loaded, 15685 targets configured)
Step #2 - "build.sh": [1,262 / 1,262] no actions running
Step #2 - "build.sh": INFO: Analyzed 352 targets (4 packages loaded, 15729 targets configured).
Step #2 - "build.sh": [10,901 / 11,013] 33 / 145 tests; Testing //google/cloud/storage/examples:storage_object_csek_samples; Downloading google/cloud/storage/examples/storage_object_csek_samples/test.xml; 0s remote-cache ... (32 actions, 0 running)
Step #2 - "build.sh": INFO: Found 207 targets and 145 test targets...
Step #2 - "build.sh": INFO: Elapsed time: 3.442s, Critical Path: 0.47s
Step #2 - "build.sh": INFO: 146 processes: 11215 action cache hit, 183 remote cache hit.
Step #2 - "build.sh": INFO: Build completed successfully, 146 total actions
Step #2 - "build.sh": //google/cloud/storage/examples:storage_async_samples           (cached) PASSED in 5.2s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_acl_samples      (cached) PASSED in 4.5s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_autoclass_samples (cached) PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_cors_samples     (cached) PASSED in 4.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_default_kms_key_samples (cached) PASSED in 4.1s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_encryption_enforcement_samples (cached) PASSED in 7.6s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_iam_samples      (cached) PASSED in 3.2s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_object_retention_samples (cached) PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_requester_pays_samples (cached) PASSED in 4.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_samples          (cached) PASSED in 4.5s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_soft_delete_samples (cached) PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_client_initialization_samples (cached) PASSED in 4.9s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_client_per_operation_samples (cached) PASSED in 3.2s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_default_object_acl_samples (cached) PASSED in 3.7s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_event_based_hold_samples (cached) PASSED in 4.3s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_grpc_samples            (cached) PASSED in 4.6s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_lifecycle_management_samples (cached) PASSED in 3.1s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_notification_samples    (cached) PASSED in 3.2s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_acl_samples      (cached) PASSED in 4.2s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_cmek_samples     (cached) PASSED in 4.5s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_csek_samples     (cached) PASSED in 4.2s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_file_transfer_samples (cached) PASSED in 4.5s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_hold_samples     (cached) PASSED in 4.2s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_preconditions_samples (cached) PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_resumable_write_samples (cached) PASSED in 4.1s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_retention_samples (cached) PASSED in 3.1s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_rewrite_samples  (cached) PASSED in 5.1s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_samples          (cached) PASSED in 11.6s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_soft_delete_samples (cached) PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_versioning_samples (cached) PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_otel_samples            (cached) PASSED in 2.9s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_public_object_samples   (cached) PASSED in 3.2s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_quickstart              (cached) PASSED in 4.1s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_retention_policy_samples (cached) PASSED in 3.9s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_service_account_samples (cached) PASSED in 4.3s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_website_samples         (cached) PASSED in 4.2s
Step #2 - "build.sh": //google/cloud/storage/tests:async_client_integration_test-default (cached) PASSED in 117.1s
Step #2 - "build.sh": //google/cloud/storage/tests:async_client_integration_test-grpc-metadata (cached) PASSED in 132.1s
Step #2 - "build.sh": //google/cloud/storage/tests:auto_finalize_integration_test-default (cached) PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/storage/tests:auto_finalize_integration_test-grpc-metadata (cached) PASSED in 5.5s
Step #2 - "build.sh": //google/cloud/storage/tests:bucket_acl_integration_test-default (cached) PASSED in 3.8s
Step #2 - "build.sh": //google/cloud/storage/tests:bucket_acl_integration_test-grpc-metadata (cached) PASSED in 5.8s
Step #2 - "build.sh": //google/cloud/storage/tests:bucket_integration_test-default    (cached) PASSED in 7.1s
Step #2 - "build.sh": //google/cloud/storage/tests:bucket_integration_test-grpc-metadata (cached) PASSED in 11.5s
Step #2 - "build.sh": //google/cloud/storage/tests:create_client_integration_test-default (cached) PASSED in 3.0s
Step #2 - "build.sh": //google/cloud/storage/tests:create_client_integration_test-grpc-metadata (cached) PASSED in 19.9s
Step #2 - "build.sh": //google/cloud/storage/tests:curl_sign_blob_integration_test-default (cached) PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/storage/tests:curl_sign_blob_integration_test-grpc-metadata (cached) PASSED in 2.0s
Step #2 - "build.sh": //google/cloud/storage/tests:decompressive_transcoding_integration_test-default (cached) PASSED in 10.4s
Step #2 - "build.sh": //google/cloud/storage/tests:decompressive_transcoding_integration_test-grpc-metadata (cached) PASSED in 20.7s
Step #2 - "build.sh": //google/cloud/storage/tests:default_object_acl_integration_test-default (cached) PASSED in 5.0s
Step #2 - "build.sh": //google/cloud/storage/tests:default_object_acl_integration_test-grpc-metadata (cached) PASSED in 6.2s
Step #2 - "build.sh": //google/cloud/storage/tests:error_parsing_integration_test-default (cached) PASSED in 3.8s
Step #2 - "build.sh": //google/cloud/storage/tests:error_parsing_integration_test-grpc-metadata (cached) PASSED in 3.5s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_bucket_metadata_integration_test-default (cached) PASSED in 4.7s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_bucket_metadata_integration_test-grpc-metadata (cached) PASSED in 68.2s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_integration_test-default      (cached) PASSED in 6.7s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_integration_test-grpc-metadata (cached) PASSED in 110.0s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_acl_integration_test-default (cached) PASSED in 3.8s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_acl_integration_test-grpc-metadata (cached) PASSED in 3.7s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_media_integration_test-default (cached) PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_media_integration_test-grpc-metadata (cached) PASSED in 3.7s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_metadata_integration_test-default (cached) PASSED in 41.3s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_metadata_integration_test-grpc-metadata (cached) PASSED in 5.0s
Step #2 - "build.sh": //google/cloud/storage/tests:hmac_key_integration_test-default  (cached) PASSED in 3.2s
Step #2 - "build.sh": //google/cloud/storage/tests:hmac_key_integration_test-grpc-metadata (cached) PASSED in 4.0s
Step #2 - "build.sh": //google/cloud/storage/tests:mtls_object_basic_crud_integration_test-default (cached) PASSED in 4.9s
Step #2 - "build.sh": //google/cloud/storage/tests:mtls_object_basic_crud_integration_test-grpc-metadata (cached) PASSED in 4.9s
Step #2 - "build.sh": //google/cloud/storage/tests:notification_integration_test-default (cached) PASSED in 3.8s
Step #2 - "build.sh": //google/cloud/storage/tests:notification_integration_test-grpc-metadata (cached) PASSED in 19.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_basic_crud_integration_test-default (cached) PASSED in 5.8s
Step #2 - "build.sh": //google/cloud/storage/tests:object_basic_crud_integration_test-grpc-metadata (cached) PASSED in 85.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_checksum_integration_test-default (cached) PASSED in 5.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_checksum_integration_test-grpc-metadata (cached) PASSED in 48.0s
Step #2 - "build.sh": //google/cloud/storage/tests:object_compose_many_integration_test-default (cached) PASSED in 5.2s
Step #2 - "build.sh": //google/cloud/storage/tests:object_compose_many_integration_test-grpc-metadata (cached) PASSED in 69.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_file_integration_test-default (cached) PASSED in 7.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_file_integration_test-grpc-metadata (cached) PASSED in 136.0s
Step #2 - "build.sh": //google/cloud/storage/tests:object_file_multi_threaded_test-default (cached) PASSED in 5.0s
Step #2 - "build.sh": //google/cloud/storage/tests:object_file_multi_threaded_test-grpc-metadata (cached) PASSED in 4.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_hash_integration_test-default (cached) PASSED in 5.5s
Step #2 - "build.sh": //google/cloud/storage/tests:object_hash_integration_test-grpc-metadata (cached) PASSED in 69.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_insert_integration_test-default (cached) PASSED in 6.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_insert_integration_test-grpc-metadata (cached) PASSED in 133.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_insert_preconditions_integration_test-default (cached) PASSED in 4.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_insert_preconditions_integration_test-grpc-metadata (cached) PASSED in 83.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_integration_test-default    (cached) PASSED in 7.1s
Step #2 - "build.sh": //google/cloud/storage/tests:object_integration_test-grpc-metadata (cached) PASSED in 166.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_list_objects_versions_integration_test-default (cached) PASSED in 4.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_list_objects_versions_integration_test-grpc-metadata (cached) PASSED in 3.8s
Step #2 - "build.sh": //google/cloud/storage/tests:object_media_integration_test-default (cached) PASSED in 45.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_media_integration_test-grpc-metadata (cached) PASSED in 133.8s
Step #2 - "build.sh": //google/cloud/storage/tests:object_parallel_upload_integration_test-default (cached) PASSED in 11.8s
Step #2 - "build.sh": //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata (cached) PASSED in 166.5s
Step #2 - "build.sh": //google/cloud/storage/tests:object_plenty_clients_serially_integration_test-default (cached) PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_plenty_clients_serially_integration_test-grpc-metadata (cached) PASSED in 2.2s
Step #2 - "build.sh": //google/cloud/storage/tests:object_plenty_clients_simultaneously_integration_test-default (cached) PASSED in 4.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_plenty_clients_simultaneously_integration_test-grpc-metadata (cached) PASSED in 2.5s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_headers_integration_test-default (cached) PASSED in 4.5s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_headers_integration_test-grpc-metadata (cached) PASSED in 84.1s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_large_integration_test-default (cached) PASSED in 4.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_large_integration_test-grpc-metadata (cached) PASSED in 21.2s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_preconditions_integration_test-default (cached) PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_preconditions_integration_test-grpc-metadata (cached) PASSED in 85.2s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_range_integration_test-default (cached) PASSED in 3.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_range_integration_test-grpc-metadata (cached) PASSED in 89.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_stream_integration_test-default (cached) PASSED in 1.8s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_stream_integration_test-grpc-metadata (cached) PASSED in 23.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_resumable_parallel_upload_integration_test-default (cached) PASSED in 4.5s
Step #2 - "build.sh": //google/cloud/storage/tests:object_resumable_parallel_upload_integration_test-grpc-metadata (cached) PASSED in 74.8s
Step #2 - "build.sh": //google/cloud/storage/tests:object_resumable_write_integration_test-default (cached) PASSED in 9.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_resumable_write_integration_test-grpc-metadata (cached) PASSED in 89.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_rewrite_integration_test-default (cached) PASSED in 4.1s
Step #2 - "build.sh": //google/cloud/storage/tests:object_rewrite_integration_test-grpc-metadata (cached) PASSED in 109.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_preconditions_integration_test-default (cached) PASSED in 2.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_preconditions_integration_test-grpc-metadata (cached) PASSED in 67.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_stream_integration_test-default (cached) PASSED in 3.0s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_stream_integration_test-grpc-metadata (cached) PASSED in 36.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_streambuf_integration_test-default (cached) PASSED in 2.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_streambuf_integration_test-grpc-metadata (cached) PASSED in 26.5s
Step #2 - "build.sh": //google/cloud/storage/tests:service_account_credentials_integration_test-default (cached) PASSED in 1.7s
Step #2 - "build.sh": //google/cloud/storage/tests:service_account_credentials_integration_test-grpc-metadata (cached) PASSED in 2.0s
Step #2 - "build.sh": //google/cloud/storage/tests:service_account_integration_test-default (cached) PASSED in 2.0s
Step #2 - "build.sh": //google/cloud/storage/tests:service_account_integration_test-grpc-metadata (cached) PASSED in 2.1s
Step #2 - "build.sh": //google/cloud/storage/tests:signed_url_conformance_test-default (cached) PASSED in 10.6s
Step #2 - "build.sh": //google/cloud/storage/tests:signed_url_conformance_test-grpc-metadata (cached) PASSED in 13.6s
Step #2 - "build.sh": //google/cloud/storage/tests:slow_reader_chunk_integration_test-default (cached) PASSED in 38.2s
Step #2 - "build.sh": //google/cloud/storage/tests:slow_reader_chunk_integration_test-grpc-metadata (cached) PASSED in 63.4s
Step #2 - "build.sh": //google/cloud/storage/tests:slow_reader_stream_integration_test-default (cached) PASSED in 37.8s
Step #2 - "build.sh": //google/cloud/storage/tests:slow_reader_stream_integration_test-grpc-metadata (cached) PASSED in 62.0s
Step #2 - "build.sh": //google/cloud/storage/tests:small_reads_integration_test-default (cached) PASSED in 2.1s
Step #2 - "build.sh": //google/cloud/storage/tests:small_reads_integration_test-grpc-metadata (cached) PASSED in 29.9s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_async-default           (cached) PASSED in 21.1s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_async-grpc-metadata     (cached) PASSED in 10.1s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_grpc-default            (cached) PASSED in 22.2s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_grpc-grpc-metadata      (cached) PASSED in 8.2s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_rest-default            (cached) PASSED in 2.1s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_rest-grpc-metadata      (cached) PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/storage/tests:storage_include_test-default       (cached) PASSED in 1.8s
Step #2 - "build.sh": //google/cloud/storage/tests:storage_include_test-grpc-metadata (cached) PASSED in 2.5s
Step #2 - "build.sh": //google/cloud/storage/tests:thread_integration_test-default    (cached) PASSED in 13.9s
Step #2 - "build.sh": //google/cloud/storage/tests:thread_integration_test-grpc-metadata (cached) PASSED in 176.8s
Step #2 - "build.sh": //google/cloud/storage/tests:tracing_integration_test-default   (cached) PASSED in 2.1s
Step #2 - "build.sh": //google/cloud/storage/tests:tracing_integration_test-grpc-metadata (cached) PASSED in 2.0s
Step #2 - "build.sh": //google/cloud/storage/tests:universe_domain_integration_test   (cached) PASSED in 2.3s
Step #2 - "build.sh": 
Step #2 - "build.sh": Executed 0 out of 145 tests: 145 tests pass.
Step #2 - "build.sh": There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
Step #2 - "build.sh": Killing emulator server [3929] ... done.
Step #2 - "build.sh": ==> 🕑 msan completed in 334.342 seconds
Finished Step #2 - "build.sh"
Starting Step #3 - "remove-image"
Step #3 - "remove-image": Already have image (with digest): gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #3 - "remove-image": Digests:
Step #3 - "remove-image": - us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan@sha256:d904cf0fd254ef1ef8f7968d2a8a3056aa1d91a8da838ec8124c8e5c6cea1b40
Step #3 - "remove-image": 
Step #3 - "remove-image": Tags:
Step #3 - "remove-image": - us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan:1f24298e-5503-4be7-9c1a-9f7dff0f4e3d
Step #3 - "remove-image": Delete request issued.
Step #3 - "remove-image": Waiting for operation [projects/cloud-cpp-testing-resources/locations/us-east1/operations/a257193a-21c8-46d7-b38f-5de5aa27f8db] to complete...
Step #3 - "remove-image": .....done.
Finished Step #3 - "remove-image"
PUSH
DONE

Build Log: https://storage.cloud.google.com/cloud-cpp-community-publiclogs/logs/google-cloud-cpp/16389/edddb774b3e8c7ca9876b04897dd10c390771135/fedora-msan-msan-default/log-1f24298e-5503-4be7-9c1a-9f7dff0f4e3d.txt