Skip to content

feat(storage): separate read and hedging thread pools - #16389

Open
ajayky-os wants to merge 2 commits into
googleapis:mainfrom
ajayky-os:perf/hedging_thread_pool
Open

feat(storage): separate read and hedging thread pools#16389
ajayky-os wants to merge 2 commits into
googleapis:mainfrom
ajayky-os:perf/hedging_thread_pool

fix(storage): address review comments on read hedging and thread pool

9160561
Select commit
Loading
Failed to load commit list.
Google Cloud Build / msan-storage-pr (cloud-cpp-testing-resources) succeeded Sep 1, 2026 in 23m 23s

Summary

Build Information

Trigger msan-storage-pr
Build 294dc22d-47e8-4b9f-998c-d87f1b21338d
Start 2026-09-01T04:08:16-07:00
Duration 22m12.19s
Status SUCCESS

Steps

Step Status Duration
kaniko-build SUCCESS 5m5.369s
download-runner-image SUCCESS 1m20.829s
build.sh SUCCESS 15m20.883s
remove-image SUCCESS 3.537s
cancel-in-progress-builds-for-PR SUCCESS 52.63s

Details

starting build "294dc22d-47e8-4b9f-998c-d87f1b21338d"

FETCHSOURCE
From https://github.com/googleapis/google-cloud-cpp
 * branch              9160561bfffc3629d9bb770a83d50d9ce8bce509 -> FETCH_HEAD
Updating files:   1% (266/23195)
Updating files:   2% (464/23195)
Updating files:   3% (696/23195)
Updating files:   4% (928/23195)
Updating files:   5% (1160/23195)
Updating files:   6% (1392/23195)
Updating files:   7% (1624/23195)
Updating files:   8% (1856/23195)
Updating files:   9% (2088/23195)
Updating files:  10% (2320/23195)
Updating files:  11% (2552/23195)
Updating files:  12% (2784/23195)
Updating files:  13% (3016/23195)
Updating files:  14% (3248/23195)
Updating files:  15% (3480/23195)
Updating files:  16% (3712/23195)
Updating files:  17% (3944/23195)
Updating files:  18% (4176/23195)
Updating files:  19% (4408/23195)
Updating files:  20% (4639/23195)
Updating files:  21% (4871/23195)
Updating files:  22% (5103/23195)
Updating files:  23% (5335/23195)
Updating files:  24% (5567/23195)
Updating files:  25% (5799/23195)
Updating files:  26% (6031/23195)
Updating files:  27% (6263/23195)
Updating files:  28% (6495/23195)
Updating files:  29% (6727/23195)
Updating files:  30% (6959/23195)
Updating files:  31% (7191/23195)
Updating files:  32% (7423/23195)
Updating files:  33% (7655/23195)
Updating files:  34% (7887/23195)
Updating files:  35% (8119/23195)
Updating files:  36% (8351/23195)
Updating files:  37% (8583/23195)
Updating files:  38% (8815/23195)
Updating files:  39% (9047/23195)
Updating files:  40% (9278/23195)
Updating files:  41% (9510/23195)
Updating files:  42% (9742/23195)
Updating files:  43% (9974/23195)
Updating files:  44% (10206/23195)
Updating files:  45% (10438/23195)
Updating files:  46% (10670/23195)
Updating files:  47% (10902/23195)
Updating files:  48% (11134/23195)
Updating files:  49% (11366/23195)
Updating files:  50% (11598/23195)
Updating files:  50% (11724/23195)
Updating files:  51% (11830/23195)
Updating files:  52% (12062/23195)
Updating files:  53% (12294/23195)
Updating files:  54% (12526/23195)
Updating files:  55% (12758/23195)
Updating files:  56% (12990/23195)
Updating files:  57% (13222/23195)
Updating files:  58% (13454/23195)
Updating files:  59% (13686/23195)
Updating files:  60% (13917/23195)
Updating files:  61% (14149/23195)
Updating files:  62% (14381/23195)
Updating files:  63% (14613/23195)
Updating files:  64% (14845/23195)
Updating files:  65% (15077/23195)
Updating files:  66% (15309/23195)
Updating files:  67% (15541/23195)
Updating files:  68% (15773/23195)
Updating files:  69% (16005/23195)
Updating files:  70% (16237/23195)
Updating files:  71% (16469/23195)
Updating files:  72% (16701/23195)
Updating files:  73% (16933/23195)
Updating files:  74% (17165/23195)
Updating files:  75% (17397/23195)
Updating files:  76% (17629/23195)
Updating files:  77% (17861/23195)
Updating files:  78% (18093/23195)
Updating files:  79% (18325/23195)
Updating files:  80% (18556/23195)
Updating files:  81% (18788/23195)
Updating files:  82% (19020/23195)
Updating files:  83% (19252/23195)
Updating files:  84% (19484/23195)
Updating files:  85% (19716/23195)
Updating files:  86% (19948/23195)
Updating files:  87% (20180/23195)
Updating files:  88% (20412/23195)
Updating files:  89% (20644/23195)
Updating files:  90% (20876/23195)
Updating files:  91% (21108/23195)
Updating files:  92% (21340/23195)
Updating files:  93% (21572/23195)
Updating files:  94% (21804/23195)
Updating files:  95% (22036/23195)
Updating files:  96% (22268/23195)
Updating files:  97% (22500/23195)
Updating files:  98% (22732/23195)
Updating files:  99% (22964/23195)
Updating files: 100% (23195/23195)
Updating files: 100% (23195/23195), done.
HEAD is now at 9160561b fix(storage): address review comments on read hedging and thread pool
GitCommit:
9160561bfffc3629d9bb770a83d50d9ce8bce509
BUILD
Starting Step #4 - "cancel-in-progress-builds-for-PR"
Starting Step #0 - "kaniko-build"
Step #4 - "cancel-in-progress-builds-for-PR": Pulling image: gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #0 - "kaniko-build": Pulling image: gcr.io/kaniko-project/executor:v1.24.0-debug@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": gcr.io/kaniko-project/executor@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd: Pulling from kaniko-project/executor
Step #4 - "cancel-in-progress-builds-for-PR": gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574: Pulling from google.com/cloudsdktool/cloud-sdk
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Pulling fs layer
Step #0 - "kaniko-build": 2fc842204170: Pulling fs layer
Step #0 - "kaniko-build": 8d1674b25e7b: Pulling fs layer
Step #0 - "kaniko-build": 9bdd0371dbb4: Pulling fs layer
Step #0 - "kaniko-build": 9aeeda79b717: Pulling fs layer
Step #0 - "kaniko-build": aaab912f1a4e: Pulling fs layer
Step #0 - "kaniko-build": 0006a7b536fb: Pulling fs layer
Step #0 - "kaniko-build": 3d08e2063109: Pulling fs layer
Step #0 - "kaniko-build": fc4517ec89af: Pulling fs layer
Step #0 - "kaniko-build": b892afb8bf07: Pulling fs layer
Step #0 - "kaniko-build": cdc2b9345183: Pulling fs layer
Step #0 - "kaniko-build": 3934701e2c3d: Pulling fs layer
Step #0 - "kaniko-build": 46be974dd378: Pulling fs layer
Step #0 - "kaniko-build": 2b61f8ddf6d9: Pulling fs layer
Step #0 - "kaniko-build": 2fc842204170: Waiting
Step #0 - "kaniko-build": 8d1674b25e7b: Waiting
Step #0 - "kaniko-build": b892afb8bf07: Waiting
Step #0 - "kaniko-build": 9bdd0371dbb4: Waiting
Step #0 - "kaniko-build": 2b61f8ddf6d9: Waiting
Step #0 - "kaniko-build": 0006a7b536fb: Waiting
Step #0 - "kaniko-build": 9aeeda79b717: Waiting
Step #0 - "kaniko-build": cdc2b9345183: Waiting
Step #0 - "kaniko-build": aaab912f1a4e: Waiting
Step #0 - "kaniko-build": 46be974dd378: Waiting
Step #0 - "kaniko-build": 3934701e2c3d: Waiting
Step #0 - "kaniko-build": fc4517ec89af: Waiting
Step #0 - "kaniko-build": 3d08e2063109: Waiting
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Download complete
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Download complete
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Download complete
Step #0 - "kaniko-build": 2fc842204170: Verifying Checksum
Step #0 - "kaniko-build": 2fc842204170: Download complete
Step #0 - "kaniko-build": 9bdd0371dbb4: Verifying Checksum
Step #0 - "kaniko-build": 9bdd0371dbb4: Download complete
Step #0 - "kaniko-build": 2fc842204170: Pull complete
Step #0 - "kaniko-build": 8d1674b25e7b: Verifying Checksum
Step #0 - "kaniko-build": 8d1674b25e7b: Download complete
Step #0 - "kaniko-build": 8d1674b25e7b: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Download complete
Step #0 - "kaniko-build": 9bdd0371dbb4: Pull complete
Step #0 - "kaniko-build": 9aeeda79b717: Verifying Checksum
Step #0 - "kaniko-build": 9aeeda79b717: Download complete
Step #0 - "kaniko-build": 3d08e2063109: Verifying Checksum
Step #0 - "kaniko-build": 3d08e2063109: Download complete
Step #0 - "kaniko-build": 9aeeda79b717: Pull complete
Step #0 - "kaniko-build": aaab912f1a4e: Verifying Checksum
Step #0 - "kaniko-build": aaab912f1a4e: Download complete
Step #0 - "kaniko-build": 0006a7b536fb: Verifying Checksum
Step #0 - "kaniko-build": 0006a7b536fb: Download complete
Step #0 - "kaniko-build": fc4517ec89af: Verifying Checksum
Step #0 - "kaniko-build": fc4517ec89af: Download complete
Step #0 - "kaniko-build": aaab912f1a4e: Pull complete
Step #0 - "kaniko-build": 46be974dd378: Verifying Checksum
Step #0 - "kaniko-build": 46be974dd378: Download complete
Step #0 - "kaniko-build": 3934701e2c3d: Verifying Checksum
Step #0 - "kaniko-build": 3934701e2c3d: Download complete
Step #0 - "kaniko-build": 0006a7b536fb: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Download complete
Step #0 - "kaniko-build": b892afb8bf07: Download complete
Step #0 - "kaniko-build": cdc2b9345183: Verifying Checksum
Step #0 - "kaniko-build": cdc2b9345183: Download complete
Step #0 - "kaniko-build": 3d08e2063109: Pull complete
Step #0 - "kaniko-build": 2b61f8ddf6d9: Verifying Checksum
Step #0 - "kaniko-build": 2b61f8ddf6d9: Download complete
Step #0 - "kaniko-build": fc4517ec89af: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Pull complete
Step #0 - "kaniko-build": b892afb8bf07: Pull complete
Step #0 - "kaniko-build": cdc2b9345183: Pull complete
Step #0 - "kaniko-build": 3934701e2c3d: Pull complete
Step #0 - "kaniko-build": 46be974dd378: Pull complete
Step #0 - "kaniko-build": 2b61f8ddf6d9: Pull complete
Step #0 - "kaniko-build": Digest: sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": Status: Downloaded newer image for gcr.io/kaniko-project/executor@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": gcr.io/kaniko-project/executor:v1.24.0-debug@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": time="2026-09-01T11:08:35Z" level=info msg="Using dockerignore file: /workspace/ci/.dockerignore"
Step #0 - "kaniko-build": time="2026-09-01T11:08:35Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-09-01T11:08:35Z" level=info msg="Retrieving image fedora:40 from registry index.docker.io"
Step #0 - "kaniko-build": time="2026-09-01T11:08:35Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-09-01T11:08:35Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-09-01T11:08:35Z" level=info msg="Built cross stage deps: map[]"
Step #0 - "kaniko-build": time="2026-09-01T11:08:35Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-09-01T11:08:35Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-09-01T11:08:35Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-09-01T11:08:35Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-09-01T11:08:35Z" level=info msg="Executing 0 build triggers"
Step #0 - "kaniko-build": time="2026-09-01T11:08:35Z" level=info msg="Building stage 'fedora:40' [idx: '0', base-idx: '-1']"
Step #0 - "kaniko-build": time="2026-09-01T11:08:35Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:e005ad110158bfe52f0bfb62c43052fc86e99e36d8386a62785d23e3037f90d2..."
Step #0 - "kaniko-build": time="2026-09-01T11:08:36Z" level=info msg="Using caching version of cmd: RUN dnf makecache &&     dnf install -y clang clang-tools-extra cmake findutils         git llvm make ninja-build openssl-devel patch python         python3 python3-devel python3-lit python-pip tar unzip which wget xz"
Step #0 - "kaniko-build": time="2026-09-01T11:08:36Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:9d5b1e125453cae51a24d3cd1cae5443fca86cc71b75234372c1dab7f670bf59..."
Step #0 - "kaniko-build": time="2026-09-01T11:08:36Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y python3-devel"
Step #0 - "kaniko-build": time="2026-09-01T11:08:36Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:90ef07e1cd6fa920a949553451f6c11581264649e54b461dd254f4930ccd7094..."
Step #0 - "kaniko-build": time="2026-09-01T11:08:36Z" level=info msg="Using caching version of cmd: RUN pip3 install --upgrade pip"
Step #0 - "kaniko-build": time="2026-09-01T11:08:36Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:ebc88857bacfdea4e75ba63cf341267b897a999db7dbfb1d606262d3674c2705..."
Step #0 - "kaniko-build": time="2026-09-01T11:08:36Z" level=info msg="Using caching version of cmd: RUN pip3 install setuptools wheel"
Step #0 - "kaniko-build": time="2026-09-01T11:08:36Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:56c94ce0178764adf3fe805e8a64e1eec7e23f50333cbb6092b70202b425927a..."
Step #0 - "kaniko-build": time="2026-09-01T11:08:36Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y java-latest-openjdk-devel"
Step #0 - "kaniko-build": time="2026-09-01T11:08:36Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:6e1ee6d2a1fb763e652088324c6f241d6c3978ac18de6f0f9380fc732625ded5..."
Step #0 - "kaniko-build": time="2026-09-01T11:08:37Z" level=info msg="Using caching version of cmd: RUN echo 'root:cloudcxx' | chpasswd"
Step #0 - "kaniko-build": time="2026-09-01T11:08:37Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:c9c1c92f41f0a8db11af9f076f47a40be83252a23c669a699f5eb08ee43e65ef..."
Step #0 - "kaniko-build": time="2026-09-01T11:08:37Z" level=info msg="Using caching version of cmd: RUN git clone --depth=1 --branch llvmorg-18.1.1 https://github.com/llvm/llvm-project"
Step #0 - "kaniko-build": time="2026-09-01T11:08:37Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:a7fa06631ce468534586b02b61c044c6e735d21b0f1a979f2dd1242668fa8d8c..."
Step #0 - "kaniko-build": time="2026-09-01T11:08:37Z" level=info msg="Using caching version of cmd: RUN cmake -GNinja -S runtimes -B build     -DCMAKE_BUILD_TYPE=Release     -DLLVM_ENABLE_PROJECTS=\"clang\"     -DLLVM_ENABLE_RUNTIMES=\"libcxx;libcxxabi;libunwind\"     -DCMAKE_C_COMPILER=clang     -DCMAKE_CXX_COMPILER=clang++     -DLLVM_USE_SANITIZER=MemoryWithOrigins     -DLIBCXXABI_USE_LLVM_UNWINDER=OFF     -DCOMPILER_RT_USE_LLVM_UNWINDER=OFF     -DCMAKE_INSTALL_PREFIX=/usr"
Step #0 - "kaniko-build": time="2026-09-01T11:08:37Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:f65b1167323150bf86b8173f1ca602e9c5c458c852dccaabc74ecd0235ca7c12..."
Step #0 - "kaniko-build": time="2026-09-01T11:08:37Z" level=info msg="Using caching version of cmd: RUN cmake --build build"
Step #0 - "kaniko-build": time="2026-09-01T11:08:37Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:e8db00a497af4cc57c853dd3fd6af51d4a14638275c7ab3fb259bce747d6fa21..."
Step #0 - "kaniko-build": time="2026-09-01T11:08:38Z" level=info msg="Using caching version of cmd: RUN cmake --install build"
Step #0 - "kaniko-build": time="2026-09-01T11:08:38Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:cbc61717fbbfd6ac8e73048dd3f77763f4ef330ae3a80dcfc5d43046e76c49aa..."
Step #0 - "kaniko-build": time="2026-09-01T11:08:38Z" level=info msg="Using caching version of cmd: RUN echo \"/usr/lib/x86_64-unknown-linux-gnu\" >/etc/ld.so.conf.d/msan.conf"
Step #0 - "kaniko-build": time="2026-09-01T11:08:38Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:9e44ab0a2cbbf08ec937ea005172f9f791baf66167aa559c4366837cd9a3aa71..."
Step #0 - "kaniko-build": time="2026-09-01T11:08:38Z" level=info msg="Using caching version of cmd: RUN ldconfig"
Step #0 - "kaniko-build": time="2026-09-01T11:08:38Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:3bc1249892dcf73cb470bdbd3bb3958888bea905938b20055137bfcd8b095847..."
Step #0 - "kaniko-build": time="2026-09-01T11:08:38Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y python3.10"
Step #0 - "kaniko-build": time="2026-09-01T11:08:38Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:cda0e47b1bc391d27a5d470a4e90c0b8fcbd0432291328f344e8785ac2203edc..."
Step #0 - "kaniko-build": time="2026-09-01T11:08:38Z" level=info msg="Using caching version of cmd: RUN /var/tmp/ci/install-cloud-sdk.sh"
Step #0 - "kaniko-build": time="2026-09-01T11:08:38Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan/cache:26ed640b3a705683cd7e2f987df4b430fc9a89ea3cc195d9c181bc42c670379d..."
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Download complete
Step #0 - "kaniko-build": time="2026-09-01T11:08:39Z" level=info msg="Using caching version of cmd: RUN curl -o /usr/bin/bazelisk -sSL \"https://github.com/bazelbuild/bazelisk/releases/download/v1.28.1/bazelisk-linux-${ARCH}\" &&     chmod +x /usr/bin/bazelisk &&     ln -s /usr/bin/bazelisk /usr/bin/bazel"
Step #0 - "kaniko-build": time="2026-09-01T11:08:39Z" level=info msg="Unpacking rootfs as cmd COPY . /var/tmp/ci requires it."
Step #0 - "kaniko-build": time="2026-09-01T11:08:42Z" level=info msg="ARG NCPU=4"
Step #0 - "kaniko-build": time="2026-09-01T11:08:42Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-09-01T11:08:42Z" level=info msg="ARG ARCH=amd64"
Step #0 - "kaniko-build": time="2026-09-01T11:08:42Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-09-01T11:08:42Z" level=info msg="RUN dnf makecache &&     dnf install -y clang clang-tools-extra cmake findutils         git llvm make ninja-build openssl-devel patch python         python3 python3-devel python3-lit python-pip tar unzip which wget xz"
Step #0 - "kaniko-build": time="2026-09-01T11:08:42Z" level=info msg="Found cached layer, extracting to filesystem"
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": Digest: sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": Status: Downloaded newer image for gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": + test -z 16389
Step #4 - "cancel-in-progress-builds-for-PR": ++ gcloud builds describe --region us-east1 --format 'value(create_time)' 294dc22d-47e8-4b9f-998c-d87f1b21338d
Step #4 - "cancel-in-progress-builds-for-PR": + ctime=2026-09-01T11:07:24.135740Z
Step #4 - "cancel-in-progress-builds-for-PR": + query=tags=pr
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND tags=16389'
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND substitutions.COMMIT_SHA != 9160561bfffc3629d9bb770a83d50d9ce8bce509'
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND create_time < 2026-09-01T11:07:24.135740Z'
Step #4 - "cancel-in-progress-builds-for-PR": + gcloud builds list --region us-east1 --ongoing '--format=value(id)' --filter 'tags=pr AND tags=16389 AND substitutions.COMMIT_SHA != 9160561bfffc3629d9bb770a83d50d9ce8bce509 AND create_time < 2026-09-01T11:07:24.135740Z'
Step #4 - "cancel-in-progress-builds-for-PR": + xargs -r -t gcloud builds cancel --region us-east1
Step #4 - "cancel-in-progress-builds-for-PR": gcloud builds cancel --region us-east1 bf58654f-c19c-43f1-b59d-c320a4db601f 74c4e820-891d-4d10-9c02-83b4b6eb236c 736c658d-4b7c-4d4d-a674-70e65f70a9f4 52f27030-2a2d-4aaf-821c-1aed0bdfbfcd 427a2045-18bc-4599-bd95-81d3a5a6c071 396c227a-59d6-4feb-90e3-e263b755584f 38e5c752-6344-421e-a0bf-dcef56a24f7d 2e0f6141-ad79-4853-8b7a-96d42dbafae1
Step #4 - "cancel-in-progress-builds-for-PR": Cancelled [https://cloudbuild.googleapis.com/v1/projects/cloud-cpp-testing-resources/locations/us-east1/builds/bf58654f-c19c-43f1-b59d-c320a4db601f].
Step #4 - "cancel-in-progress-builds-for-PR": Cancelled [https://cloudbuild.googleapis.com/v1/projects/cloud-cpp-testing-resources/locations/us-east1/builds/74c4e820-891d-4d10-9c02-83b4b6eb236c].
Step #4 - "cancel-in-progress-builds-for-PR": Cancelled [https://cloudbuild.googleapis.com/v1/projects/cloud-cpp-testing-resources/locations/us-east1/builds/736c658d-4b7c-4d4d-a674-70e65f70a9f4].
Step #4 - "cancel-in-progress-builds-for-PR": Cancelled [https://cloudbuild.googleapis.com/v1/projects/cloud-cpp-testing-resources/locations/us-east1/builds/52f27030-2a2d-4aaf-821c-1aed0bdfbfcd].
Step #4 - "cancel-in-progress-builds-for-PR": Cancelled [https://cloudbuild.googleapis.com/v1/projects/cloud-cpp-testing-resources/locations/us-east1/builds/427a2045-18bc-4599-bd95-81d3a5a6c071].
Step #4 - "cancel-in-progress-builds-for-PR": Cancelled [https://cloudbuild.googleapis.com/v1/projects/cloud-cpp-testing-resources/locations/us-east1/builds/396c227a-59d6-4feb-90e3-e263b755584f].
Step #4 - "cancel-in-progress-builds-for-PR": Cancelled [https://cloudbuild.googleapis.com/v1/projects/cloud-cpp-testing-resources/locations/us-east1/builds/38e5c752-6344-421e-a0bf-dcef56a24f7d].
Step #4 - "cancel-in-progress-builds-for-PR": Cancelled [https://cloudbuild.googleapis.com/v1/projects/cloud-cpp-testing-resources/locations/us-east1/builds/2e0f6141-ad79-4853-8b7a-96d42dbafae1].
Step #4 - "cancel-in-progress-builds-for-PR": ---
Step #4 - "cancel-in-progress-builds-for-PR": availableSecrets:
Step #4 - "cancel-in-progress-builds-for-PR":   secretManager:
Step #4 - "cancel-in-progress-builds-for-PR":   - env: CODECOV_TOKEN
Step #4 - "cancel-in-progress-builds-for-PR":     versionName: projects/cloud-cpp-testing-resources/secrets/CODECOV_TOKEN/versions/latest
Step #4 - "cancel-in-progress-builds-for-PR":   - env: UD
Step #4 - "cancel-in-progress-builds-for-PR":     versionName: projects/cloud-cpp-testing-resources/secrets/UD/versions/latest
Step #4 - "cancel-in-progress-builds-for-PR":   - env: UD_PROJECT
Step #4 - "cancel-in-progress-builds-for-PR":     versionName: projects/cloud-cpp-testing-resources/secrets/UD_PROJECT/versions/latest
Step #4 - "cancel-in-progress-builds-for-PR":   - env: UD_REGION
Step #4 - "cancel-in-progress-builds-for-PR":     versionName: projects/cloud-cpp-testing-resources/secrets/UD_REGION/versions/latest
Step #4 - "cancel-in-progress-builds-for-PR":   - env: UD_ZONE
Step #4 - "cancel-in-progress-builds-for-PR":     versionName: projects/cloud-cpp-testing-resources/secrets/UD_ZONE/versions/latest
Step #4 - "cancel-in-progress-builds-for-PR":   - env: UD_SERVICE_ACCOUNT
Step #4 - "cancel-in-progress-builds-for-PR":     versionName: projects/cloud-cpp-testing-resources/secrets/UD_SERVICE_ACCOUNT/versions/latest
Step #4 - "cancel-in-progress-builds-for-PR":   - env: UD_SERVICE_ACCOUNT_NAME
Step #4 - "cancel-in-progress-builds-for-PR":     versionName: projects/cloud-cpp-testing-resources/secrets/UD_SERVICE_ACCOUNT_NAME/versions/latest
Step #4 - "cancel-in-progress-builds-for-PR":   - env: UD_IMPERSONATED_SERVICE_ACCOUNT_NAME
Step #4 - "cancel-in-progress-builds-for-PR":     versionName: projects/cloud-cpp-testing-resources/secrets/UD_IMPERSONATED_SERVICE_ACCOUNT_NAME/versions/latest
Step #4 - "cancel-in-progress-builds-for-PR":   - env: UD_IDTOKEN_SA_IMPERSONATION_CRED
Step #4 - "cancel-in-progress-builds-for-PR":     versionName: projects/cloud-cpp-testing-resources/secrets/UD_IDTOKEN_SA_IMPERSONATION_CRED/versions/latest
Step #4 - "cancel-in-progress-builds-for-PR":   - env: UD_EXTERNAL_ACCOUNT_CRED
Step #4 - "cancel-in-progress-builds-for-PR":     versionName: projects/cloud-cpp-testing-resources/secrets/UD_EXTERNAL_ACCOUNT_CRED/versions/latest
Step #4 - "cancel-in-progress-builds-for-PR":   - env: UD_FETCH_OIDC_TOKEN
Step #4 - "cancel-in-progress-builds-for-PR":     versionName: projects/cloud-cpp-testing-resources/secrets/UD_FETCH_OIDC_TOKEN/versions/latest
Step #4 - "cancel-in-progress-builds-for-PR": buildTriggerId: ed97a195-0491-4ab3-84e5-e2353a10216a
Step #4 - "cancel-in-progress-builds-for-PR": createTime: '2026-09-01T10:59:46.878712Z'
Step #4 - "cancel-in-progress-builds-for-PR": failureInfo:
Step #4 - "cancel-in-progress-builds-for-PR":   detail: 'Build step failure: user cancelation'
Step #4 - "cancel-in-progress-builds-for-PR":   type: USER_BUILD_STEP
Step #4 - "cancel-in-progress-builds-for-PR": id: bf58654f-c19c-43f1-b59d-c320a4db601f
Step #4 - "cancel-in-progress-builds-for-PR": logUrl: https://console.cloud.google.com/cloud-build/builds;region=us-east1/bf58654f-c19c-43f1-b59d-c320a4db601f?project=936212892354
Step #4 - "cancel-in-progress-builds-for-PR": logsBucket: gs://cloud-cpp-community-publiclogs/logs/google-cloud-cpp/16389/65f06fb6899449239ff61becad953c10c906eab8/fedora-msan-msan-__default__
Step #4 - "cancel-in-progress-builds-for-PR": name: projects/936212892354/locations/us-east1/builds/bf58654f-c19c-43f1-b59d-c320a4db601f
Step #4 - "cancel-in-progress-builds-for-PR": options:
Step #4 - "cancel-in-progress-builds-for-PR":   dynamicSubstitutions: true
Step #4 - "cancel-in-progress-builds-for-PR":   env:
Step #4 - "cancel-in-progress-builds-for-PR":   - HOME=/h
Step #4 - "cancel-in-progress-builds-for-PR":   - TZ=UTC0
Step #4 - "cancel-in-progress-builds-for-PR":   - GOOGLE_CLOUD_BUILD=yes
Step #4 - "cancel-in-progress-builds-for-PR":   - PROJECT_ID=cloud-cpp-testing-resources
Step #4 - "cancel-in-progress-builds-for-PR":   - BUILD_ID=bf58654f-c19c-43f1-b59d-c320a4db601f
Step #4 - "cancel-in-progress-builds-for-PR":   - BRANCH_NAME=perf/hedging_thread_pool
Step #4 - "cancel-in-progress-builds-for-PR":   - COMMIT_SHA=65f06fb6899449239ff61becad953c10c906eab8
Step #4 - "cancel-in-progress-builds-for-PR":   - PR_NUMBER=16389
Step #4 - "cancel-in-progress-builds-for-PR":   - TRIGGER_TYPE=pr
Step #4 - "cancel-in-progress-builds-for-PR":   - GENERATE_GOLDEN_ONLY=
Step #4 - "cancel-in-progress-builds-for-PR":   - UPDATED_DISCOVERY_DOCUMENT=
Step #4 - "cancel-in-progress-builds-for-PR":   - CONSOLE_LOG_URL=https://console.cloud.google.com/cloud-build/builds;region=us-east1/bf58654f-c19c-43f1-b59d-c320a4db601f;tab=detail?project=cloud-cpp-testing-resources
Step #4 - "cancel-in-progress-builds-for-PR":   - RAW_LOG_URL=https://storage.googleapis.com/cloud-cpp-community-publiclogs/logs/google-cloud-cpp/16389/65f06fb6899449239ff61becad953c10c906eab8/fedora-msan-msan-__default__/log-bf58654f-c19c-43f1-b59d-c320a4db601f.txt
Step #4 - "cancel-in-progress-builds-for-PR":   logging: LEGACY
Step #4 - "cancel-in-progress-builds-for-PR":   pool:
Step #4 - "cancel-in-progress-builds-for-PR":     name: projects/cloud-cpp-testing-resources/locations/us-east1/workerPools/google-cloud-cpp-pool
Step #4 - "cancel-in-progress-builds-for-PR":   substitutionOption: ALLOW_LOOSE
Step #4 - "cancel-in-progress-builds-for-PR":   volumes:
Step #4 - "cancel-in-progress-builds-for-PR":   - name: home
Step #4 - "cancel-in-progress-builds-for-PR":     path: /h
Step #4 - "cancel-in-progress-builds-for-PR": projectId: cloud-cpp-testing-resources
Step #4 - "cancel-in-progress-builds-for-PR": queueTtl: 3600s
Step #4 - "cancel-in-progress-builds-for-PR": results:
Step #4 - "cancel-in-progress-builds-for-PR":   buildStepImages:
Step #4 - "cancel-in-progress-builds-for-PR":   - sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #4 - "cancel-in-progress-builds-for-PR":   - sha256:e99c070be060ee6dda1f971352a16b5c46867b6f8a35cd78b04278f24f29743b
Step #4 - "cancel-in-progress-builds-for-PR":   - sha256:e99c070be060ee6dda1f971352a16b5c46867b6f8a35cd78b04278f24f29743b
Step #4 - "cancel-in-progress-builds-for-PR":   - ''
Step #4 - "cancel-in-progress-builds-for-PR":   - sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR":   buildStepOutputs:
Step #4 - "cancel-in-progress-builds-for-PR":   - ''
Step #4 - "cancel-in-progress-builds-for-PR":   - ''
Step #4 - "cancel-in-progress-builds-for-PR":   - ''
Step #4 - "cancel-in-progress-builds-for-PR":   - ''
Step #4 - "cancel-in-progress-builds-for-PR":   - ''
Step #4 - "cancel-in-progress-builds-for-PR":   buildStepResults:
Step #4 - "cancel-in-progress-builds-for-PR":     cancel-in-progress-builds-for-PR: {}
Step #4 - "cancel-in-progress-builds-for-PR":     download-runner-image: {}
Step #4 - "cancel-in-progress-builds-for-PR":     kaniko-build: {}
Step #4 - "cancel-in-progress-builds-for-PR": source:
Step #4 - "cancel-in-progress-builds-for-PR":   gitSource:
Step #4 - "cancel-in-progress-builds-for-PR":     revision: 65f06fb6899449239ff61becad953c10c906eab8
Step #4 - "cancel-in-progress-builds-for-PR":     url: https://github.com/googleapis/google-cloud-cpp.git
Step #4 - "cancel-in-progress-builds-for-PR": sourceProvenance:
Step #4 - "cancel-in-progress-builds-for-PR":   resolvedGitSource:
Step #4 - "cancel-in-progress-builds-for-PR":     revision: 65f06fb6899449239ff61becad953c10c906eab8
Step #4 - "cancel-in-progress-builds-for-PR":     url: https://github.com/googleapis/google-cloud-cpp.git
Step #4 - "cancel-in-progress-builds-for-PR": startTime: '2026-09-01T11:00:35.341431451Z'
Step #4 - "cancel-in-progress-builds-for-PR": status: WORKING
Step #4 - "cancel-in-progress-builds-for-PR": steps:
Step #4 - "cancel-in-progress-builds-for-PR": - args:
Step #4 - "cancel-in-progress-builds-for-PR":   - -c
Step #4 - "cancel-in-progress-builds-for-PR":   - |
Step #4 - "cancel-in-progress-builds-for-PR":     status=1
Step #4 - "cancel-in-progress-builds-for-PR":     for delay in 0 30 30 30 30; do
Step #4 - "cancel-in-progress-builds-for-PR":       sleep $delay
Step #4 - "cancel-in-progress-bu
...
[Logs truncated due to log size limitations. For full logs, see https://storage.cloud.google.com/cloud-cpp-community-publiclogs/logs/google-cloud-cpp/16389/9160561bfffc3629d9bb770a83d50d9ce8bce509/fedora-msan-msan-__default__/log-294dc22d-47e8-4b9f-998c-d87f1b21338d.txt.]
...
] 45 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 15s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,914 / 11,013] 46 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 16s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,915 / 11,013] 48 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 18s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,916 / 11,013] 48 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 19s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,917 / 11,013] 49 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 21s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,918 / 11,013] 50 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 22s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,918 / 11,013] 50 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 23s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,919 / 11,013] 51 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 25s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,919 / 11,013] 52 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 27s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,920 / 11,013] 52 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 28s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,923 / 11,013] 55 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 29s remote-cache, linux-sandbox ... (32 actions, 31 running)
Step #2 - "build.sh": [10,924 / 11,013] 57 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 30s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,927 / 11,013] 59 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 31s remote-cache, linux-sandbox ... (32 actions, 31 running)
Step #2 - "build.sh": [10,928 / 11,013] 60 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 33s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,929 / 11,013] 61 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 34s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,930 / 11,013] 62 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 36s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,930 / 11,013] 63 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 39s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,932 / 11,013] 65 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 40s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,933 / 11,013] 65 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 41s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,934 / 11,013] 67 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 42s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,936 / 11,013] 69 / 145 tests; Testing //google/cloud/storage/tests:object_media_integration_test-default; 44s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,938 / 11,013] 70 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 44s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,939 / 11,013] 71 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 45s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,941 / 11,013] 74 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 47s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,942 / 11,013] 74 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 48s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,944 / 11,013] 77 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 49s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,946 / 11,013] 79 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 51s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,950 / 11,013] 82 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 52s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,951 / 11,013] 84 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 53s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,955 / 11,013] 87 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 54s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,959 / 11,013] 91 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 55s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,963 / 11,013] 95 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 56s remote-cache, linux-sandbox ... (32 actions, 29 running)
Step #2 - "build.sh": [10,966 / 11,013] 98 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 57s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,968 / 11,013] 100 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 58s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,968 / 11,013] 101 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 60s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,970 / 11,013] 102 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 61s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,971 / 11,013] 103 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 62s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,972 / 11,013] 105 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 64s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,974 / 11,013] 106 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 65s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,976 / 11,013] 108 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 67s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,978 / 11,013] 110 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 68s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,979 / 11,013] 111 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 69s remote-cache, linux-sandbox ... (32 actions running)
Step #2 - "build.sh": [10,982 / 11,013] 115 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 70s remote-cache, linux-sandbox ... (31 actions running)
Step #2 - "build.sh": [10,985 / 11,013] 117 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 71s remote-cache, linux-sandbox ... (28 actions running)
Step #2 - "build.sh": [10,987 / 11,013] 119 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 72s remote-cache, linux-sandbox ... (26 actions running)
Step #2 - "build.sh": [10,987 / 11,013] 119 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 76s remote-cache, linux-sandbox ... (26 actions running)
Step #2 - "build.sh": [10,994 / 11,013] 126 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 77s remote-cache, linux-sandbox ... (19 actions running)
Step #2 - "build.sh": [10,997 / 11,013] 129 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 79s remote-cache, linux-sandbox ... (16 actions running)
Step #2 - "build.sh": [10,997 / 11,013] 129 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 89s remote-cache, linux-sandbox ... (16 actions running)
Step #2 - "build.sh": [10,997 / 11,013] 130 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 98s remote-cache, linux-sandbox ... (16 actions running)
Step #2 - "build.sh": [10,998 / 11,013] 130 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 99s remote-cache, linux-sandbox ... (15 actions running)
Step #2 - "build.sh": [10,998 / 11,013] 130 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 110s remote-cache, linux-sandbox ... (15 actions running)
Step #2 - "build.sh": [10,998 / 11,013] 130 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 118s remote-cache, linux-sandbox ... (15 actions running)
Step #2 - "build.sh": [11,001 / 11,013] 133 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 120s remote-cache, linux-sandbox ... (12 actions running)
Step #2 - "build.sh": [11,001 / 11,013] 133 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 131s remote-cache, linux-sandbox ... (12 actions running)
Step #2 - "build.sh": [11,001 / 11,013] 134 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 139s remote-cache, linux-sandbox ... (12 actions running)
Step #2 - "build.sh": [11,003 / 11,013] 135 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 140s remote-cache, linux-sandbox ... (10 actions running)
Step #2 - "build.sh": [11,003 / 11,013] 135 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 151s remote-cache, linux-sandbox ... (10 actions running)
Step #2 - "build.sh": [11,003 / 11,013] 135 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 159s remote-cache, linux-sandbox ... (10 actions running)
Step #2 - "build.sh": [11,008 / 11,013] 140 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 161s remote-cache, linux-sandbox ... (5 actions running)
Step #2 - "build.sh": [11,008 / 11,013] 140 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 172s remote-cache, linux-sandbox ... (5 actions running)
Step #2 - "build.sh": [11,008 / 11,013] 141 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 181s remote-cache, linux-sandbox ... (5 actions running)
Step #2 - "build.sh": [11,010 / 11,013] 142 / 145 tests; Testing //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata; 182s remote-cache, linux-sandbox ... (3 actions running)
Step #2 - "build.sh": [11,012 / 11,013] 144 / 145 tests; Testing //google/cloud/storage/tests:thread_integration_test-grpc-metadata; 128s remote-cache, linux-sandbox
Step #2 - "build.sh": [11,012 / 11,013] 145 / 145 tests; Testing //google/cloud/storage/tests:thread_integration_test-grpc-metadata; 130s remote-cache, linux-sandbox
Step #2 - "build.sh": INFO: Found 207 targets and 145 test targets...
Step #2 - "build.sh": INFO: Elapsed time: 191.016s, Critical Path: 184.30s
Step #2 - "build.sh": INFO: 146 processes: 11215 action cache hit, 6 remote cache hit, 177 linux-sandbox.
Step #2 - "build.sh": INFO: Build completed successfully, 146 total actions
Step #2 - "build.sh": //google/cloud/storage/examples:storage_async_samples                    PASSED in 9.5s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_acl_samples               PASSED in 3.8s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_autoclass_samples         PASSED in 2.1s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_cors_samples              PASSED in 3.0s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_default_kms_key_samples   PASSED in 1.9s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_encryption_enforcement_samples PASSED in 6.0s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_iam_samples               PASSED in 2.0s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_object_retention_samples  PASSED in 2.8s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_requester_pays_samples    PASSED in 1.9s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_samples                   PASSED in 4.0s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_bucket_soft_delete_samples       PASSED in 2.0s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_client_initialization_samples    PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_client_per_operation_samples     PASSED in 2.1s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_default_object_acl_samples       PASSED in 3.7s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_event_based_hold_samples         PASSED in 2.0s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_grpc_samples                     PASSED in 3.7s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_lifecycle_management_samples     PASSED in 2.1s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_notification_samples             PASSED in 2.1s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_acl_samples               PASSED in 2.0s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_cmek_samples              PASSED in 1.9s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_csek_samples              PASSED in 1.8s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_file_transfer_samples     PASSED in 2.8s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_hold_samples              PASSED in 3.7s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_preconditions_samples     PASSED in 3.7s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_resumable_write_samples   PASSED in 3.8s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_retention_samples         PASSED in 2.1s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_rewrite_samples           PASSED in 3.9s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_samples                   PASSED in 7.2s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_soft_delete_samples       PASSED in 3.5s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_object_versioning_samples        PASSED in 2.1s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_otel_samples                     PASSED in 2.6s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_public_object_samples            PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_quickstart                       PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_retention_policy_samples         PASSED in 2.0s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_service_account_samples          PASSED in 1.9s
Step #2 - "build.sh": //google/cloud/storage/examples:storage_website_samples                  PASSED in 2.2s
Step #2 - "build.sh": //google/cloud/storage/tests:async_client_integration_test-default       PASSED in 155.8s
Step #2 - "build.sh": //google/cloud/storage/tests:async_client_integration_test-grpc-metadata PASSED in 174.4s
Step #2 - "build.sh": //google/cloud/storage/tests:auto_finalize_integration_test-default      PASSED in 4.2s
Step #2 - "build.sh": //google/cloud/storage/tests:auto_finalize_integration_test-grpc-metadata PASSED in 62.1s
Step #2 - "build.sh": //google/cloud/storage/tests:bucket_acl_integration_test-default         PASSED in 3.0s
Step #2 - "build.sh": //google/cloud/storage/tests:bucket_acl_integration_test-grpc-metadata   PASSED in 47.3s
Step #2 - "build.sh": //google/cloud/storage/tests:bucket_integration_test-default             PASSED in 7.4s
Step #2 - "build.sh": //google/cloud/storage/tests:bucket_integration_test-grpc-metadata       PASSED in 156.9s
Step #2 - "build.sh": //google/cloud/storage/tests:create_client_integration_test-default      PASSED in 2.3s
Step #2 - "build.sh": //google/cloud/storage/tests:create_client_integration_test-grpc-metadata PASSED in 21.6s
Step #2 - "build.sh": //google/cloud/storage/tests:curl_sign_blob_integration_test-default     PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/storage/tests:curl_sign_blob_integration_test-grpc-metadata PASSED in 1.8s
Step #2 - "build.sh": //google/cloud/storage/tests:decompressive_transcoding_integration_test-default PASSED in 7.9s
Step #2 - "build.sh": //google/cloud/storage/tests:decompressive_transcoding_integration_test-grpc-metadata PASSED in 31.9s
Step #2 - "build.sh": //google/cloud/storage/tests:default_object_acl_integration_test-default PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/storage/tests:default_object_acl_integration_test-grpc-metadata PASSED in 73.2s
Step #2 - "build.sh": //google/cloud/storage/tests:error_parsing_integration_test-default      PASSED in 1.9s
Step #2 - "build.sh": //google/cloud/storage/tests:error_parsing_integration_test-grpc-metadata PASSED in 23.9s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_bucket_metadata_integration_test-default PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_bucket_metadata_integration_test-grpc-metadata PASSED in 40.5s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_integration_test-default               PASSED in 3.9s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_integration_test-grpc-metadata         PASSED in 38.6s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_acl_integration_test-default    PASSED in 10.5s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_acl_integration_test-grpc-metadata PASSED in 44.5s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_media_integration_test-default  PASSED in 18.0s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_media_integration_test-grpc-metadata PASSED in 3.3s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_metadata_integration_test-default PASSED in 4.0s
Step #2 - "build.sh": //google/cloud/storage/tests:grpc_object_metadata_integration_test-grpc-metadata PASSED in 27.7s
Step #2 - "build.sh": //google/cloud/storage/tests:hmac_key_integration_test-default           PASSED in 3.5s
Step #2 - "build.sh": //google/cloud/storage/tests:hmac_key_integration_test-grpc-metadata     PASSED in 1.9s
Step #2 - "build.sh": //google/cloud/storage/tests:mtls_object_basic_crud_integration_test-default PASSED in 3.0s
Step #2 - "build.sh": //google/cloud/storage/tests:mtls_object_basic_crud_integration_test-grpc-metadata PASSED in 3.9s
Step #2 - "build.sh": //google/cloud/storage/tests:notification_integration_test-default       PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/storage/tests:notification_integration_test-grpc-metadata PASSED in 19.8s
Step #2 - "build.sh": //google/cloud/storage/tests:object_basic_crud_integration_test-default  PASSED in 4.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_basic_crud_integration_test-grpc-metadata PASSED in 66.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_checksum_integration_test-default    PASSED in 2.5s
Step #2 - "build.sh": //google/cloud/storage/tests:object_checksum_integration_test-grpc-metadata PASSED in 104.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_compose_many_integration_test-default PASSED in 3.2s
Step #2 - "build.sh": //google/cloud/storage/tests:object_compose_many_integration_test-grpc-metadata PASSED in 4.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_file_integration_test-default        PASSED in 3.0s
Step #2 - "build.sh": //google/cloud/storage/tests:object_file_integration_test-grpc-metadata  PASSED in 147.2s
Step #2 - "build.sh": //google/cloud/storage/tests:object_file_multi_threaded_test-default     PASSED in 2.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_file_multi_threaded_test-grpc-metadata PASSED in 41.2s
Step #2 - "build.sh": //google/cloud/storage/tests:object_hash_integration_test-default        PASSED in 4.8s
Step #2 - "build.sh": //google/cloud/storage/tests:object_hash_integration_test-grpc-metadata  PASSED in 67.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_insert_integration_test-default      PASSED in 3.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_insert_integration_test-grpc-metadata PASSED in 173.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_insert_preconditions_integration_test-default PASSED in 4.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_insert_preconditions_integration_test-grpc-metadata PASSED in 71.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_integration_test-default             PASSED in 5.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_integration_test-grpc-metadata       PASSED in 181.1s
Step #2 - "build.sh": //google/cloud/storage/tests:object_list_objects_versions_integration_test-default PASSED in 2.0s
Step #2 - "build.sh": //google/cloud/storage/tests:object_list_objects_versions_integration_test-grpc-metadata PASSED in 28.0s
Step #2 - "build.sh": //google/cloud/storage/tests:object_media_integration_test-default       PASSED in 43.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_media_integration_test-grpc-metadata PASSED in 154.0s
Step #2 - "build.sh": //google/cloud/storage/tests:object_parallel_upload_integration_test-default PASSED in 6.2s
Step #2 - "build.sh": //google/cloud/storage/tests:object_parallel_upload_integration_test-grpc-metadata PASSED in 182.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_plenty_clients_serially_integration_test-default PASSED in 1.8s
Step #2 - "build.sh": //google/cloud/storage/tests:object_plenty_clients_serially_integration_test-grpc-metadata PASSED in 1.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_plenty_clients_simultaneously_integration_test-default PASSED in 3.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_plenty_clients_simultaneously_integration_test-grpc-metadata PASSED in 1.7s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_headers_integration_test-default PASSED in 4.6s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_headers_integration_test-grpc-metadata PASSED in 50.5s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_large_integration_test-default  PASSED in 4.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_large_integration_test-grpc-metadata PASSED in 5.5s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_preconditions_integration_test-default PASSED in 2.2s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_preconditions_integration_test-grpc-metadata PASSED in 6.5s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_range_integration_test-default  PASSED in 2.1s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_range_integration_test-grpc-metadata PASSED in 81.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_stream_integration_test-default PASSED in 2.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_read_stream_integration_test-grpc-metadata PASSED in 44.0s
Step #2 - "build.sh": //google/cloud/storage/tests:object_resumable_parallel_upload_integration_test-default PASSED in 2.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_resumable_parallel_upload_integration_test-grpc-metadata PASSED in 24.5s
Step #2 - "build.sh": //google/cloud/storage/tests:object_resumable_write_integration_test-default PASSED in 11.1s
Step #2 - "build.sh": //google/cloud/storage/tests:object_resumable_write_integration_test-grpc-metadata PASSED in 114.1s
Step #2 - "build.sh": //google/cloud/storage/tests:object_rewrite_integration_test-default     PASSED in 3.9s
Step #2 - "build.sh": //google/cloud/storage/tests:object_rewrite_integration_test-grpc-metadata PASSED in 101.5s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_preconditions_integration_test-default PASSED in 4.2s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_preconditions_integration_test-grpc-metadata PASSED in 78.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_stream_integration_test-default PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_stream_integration_test-grpc-metadata PASSED in 22.1s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_streambuf_integration_test-default PASSED in 2.3s
Step #2 - "build.sh": //google/cloud/storage/tests:object_write_streambuf_integration_test-grpc-metadata PASSED in 4.1s
Step #2 - "build.sh": //google/cloud/storage/tests:service_account_credentials_integration_test-default PASSED in 3.1s
Step #2 - "build.sh": //google/cloud/storage/tests:service_account_credentials_integration_test-grpc-metadata PASSED in 1.7s
Step #2 - "build.sh": //google/cloud/storage/tests:service_account_integration_test-default    PASSED in 1.9s
Step #2 - "build.sh": //google/cloud/storage/tests:service_account_integration_test-grpc-metadata PASSED in 2.0s
Step #2 - "build.sh": //google/cloud/storage/tests:signed_url_conformance_test-default         PASSED in 4.4s
Step #2 - "build.sh": //google/cloud/storage/tests:signed_url_conformance_test-grpc-metadata   PASSED in 7.8s
Step #2 - "build.sh": //google/cloud/storage/tests:slow_reader_chunk_integration_test-default  PASSED in 38.0s
Step #2 - "build.sh": //google/cloud/storage/tests:slow_reader_chunk_integration_test-grpc-metadata PASSED in 71.5s
Step #2 - "build.sh": //google/cloud/storage/tests:slow_reader_stream_integration_test-default PASSED in 38.1s
Step #2 - "build.sh": //google/cloud/storage/tests:slow_reader_stream_integration_test-grpc-metadata PASSED in 67.8s
Step #2 - "build.sh": //google/cloud/storage/tests:small_reads_integration_test-default        PASSED in 3.7s
Step #2 - "build.sh": //google/cloud/storage/tests:small_reads_integration_test-grpc-metadata  PASSED in 45.8s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_async-default                    PASSED in 18.9s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_async-grpc-metadata              PASSED in 20.3s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_grpc-default                     PASSED in 21.7s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_grpc-grpc-metadata               PASSED in 33.8s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_rest-default                     PASSED in 1.7s
Step #2 - "build.sh": //google/cloud/storage/tests:smoke_test_rest-grpc-metadata               PASSED in 2.8s
Step #2 - "build.sh": //google/cloud/storage/tests:storage_include_test-default                PASSED in 2.7s
Step #2 - "build.sh": //google/cloud/storage/tests:storage_include_test-grpc-metadata          PASSED in 1.9s
Step #2 - "build.sh": //google/cloud/storage/tests:thread_integration_test-default             PASSED in 11.2s
Step #2 - "build.sh": //google/cloud/storage/tests:thread_integration_test-grpc-metadata       PASSED in 130.4s
Step #2 - "build.sh": //google/cloud/storage/tests:tracing_integration_test-default            PASSED in 1.6s
Step #2 - "build.sh": //google/cloud/storage/tests:tracing_integration_test-grpc-metadata      PASSED in 3.4s
Step #2 - "build.sh": //google/cloud/storage/tests:universe_domain_integration_test            PASSED in 1.9s
Step #2 - "build.sh": 
Step #2 - "build.sh": Executed 145 out of 145 tests: 145 tests pass.
Step #2 - "build.sh": There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
Step #2 - "build.sh": Killing emulator server [5253] ... done.
Step #2 - "build.sh": ==> 🕑 msan completed in 918.957 seconds
Finished Step #2 - "build.sh"
Starting Step #3 - "remove-image"
Step #3 - "remove-image": Already have image (with digest): gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #3 - "remove-image": Digests:
Step #3 - "remove-image": - us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan@sha256:e335940606c484c284d5a27e5395c22e01d3232f040b014bdfcdf9542196cb68
Step #3 - "remove-image": 
Step #3 - "remove-image": Tags:
Step #3 - "remove-image": - us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-msan:294dc22d-47e8-4b9f-998c-d87f1b21338d
Step #3 - "remove-image": Delete request issued.
Step #3 - "remove-image": Waiting for operation [projects/cloud-cpp-testing-resources/locations/us-east1/operations/53585b10-a557-454e-85b9-03820ba3cd69] to complete...
Step #3 - "remove-image": .....done.
Finished Step #3 - "remove-image"
PUSH
DONE

Build Log: https://storage.cloud.google.com/cloud-cpp-community-publiclogs/logs/google-cloud-cpp/16389/9160561bfffc3629d9bb770a83d50d9ce8bce509/fedora-msan-msan-default/log-294dc22d-47e8-4b9f-998c-d87f1b21338d.txt