Skip to content

feat(storage): separate read and hedging thread pools - #16389

Open
ajayky-os wants to merge 2 commits into
googleapis:mainfrom
ajayky-os:perf/hedging_thread_pool
Open

feat(storage): separate read and hedging thread pools#16389
ajayky-os wants to merge 2 commits into
googleapis:mainfrom
ajayky-os:perf/hedging_thread_pool

fix(storage): address review comments on read hedging and thread pool

9160561
Select commit
Loading
Failed to load commit list.
Google Cloud Build / cxx20-pr (cloud-cpp-testing-resources) succeeded Sep 1, 2026 in 22m 14s

Summary

Build Information

Trigger cxx20-pr
Build 4435a1d3-e023-436b-9c73-88df818f0ab2
Start 2026-09-01T04:08:38-07:00
Duration 20m49.027s
Status SUCCESS

Steps

Step Status Duration
kaniko-build SUCCESS 3m42.93s
download-runner-image SUCCESS 53.204s
build.sh SUCCESS 15m53.156s
remove-image SUCCESS 3.354s
cancel-in-progress-builds-for-PR SUCCESS 49.205s

Details

starting build "4435a1d3-e023-436b-9c73-88df818f0ab2"

FETCHSOURCE
From https://github.com/googleapis/google-cloud-cpp
 * branch              9160561bfffc3629d9bb770a83d50d9ce8bce509 -> FETCH_HEAD
Updating files:   1% (261/23195)
Updating files:   2% (464/23195)
Updating files:   3% (696/23195)
Updating files:   4% (928/23195)
Updating files:   5% (1160/23195)
Updating files:   6% (1392/23195)
Updating files:   7% (1624/23195)
Updating files:   8% (1856/23195)
Updating files:   9% (2088/23195)
Updating files:  10% (2320/23195)
Updating files:  11% (2552/23195)
Updating files:  12% (2784/23195)
Updating files:  13% (3016/23195)
Updating files:  14% (3248/23195)
Updating files:  15% (3480/23195)
Updating files:  16% (3712/23195)
Updating files:  17% (3944/23195)
Updating files:  18% (4176/23195)
Updating files:  19% (4408/23195)
Updating files:  20% (4639/23195)
Updating files:  21% (4871/23195)
Updating files:  22% (5103/23195)
Updating files:  23% (5335/23195)
Updating files:  24% (5567/23195)
Updating files:  25% (5799/23195)
Updating files:  26% (6031/23195)
Updating files:  27% (6263/23195)
Updating files:  28% (6495/23195)
Updating files:  29% (6727/23195)
Updating files:  30% (6959/23195)
Updating files:  31% (7191/23195)
Updating files:  32% (7423/23195)
Updating files:  33% (7655/23195)
Updating files:  34% (7887/23195)
Updating files:  35% (8119/23195)
Updating files:  36% (8351/23195)
Updating files:  37% (8583/23195)
Updating files:  38% (8815/23195)
Updating files:  39% (9047/23195)
Updating files:  40% (9278/23195)
Updating files:  41% (9510/23195)
Updating files:  42% (9742/23195)
Updating files:  43% (9974/23195)
Updating files:  44% (10206/23195)
Updating files:  45% (10438/23195)
Updating files:  46% (10670/23195)
Updating files:  46% (10790/23195)
Updating files:  47% (10902/23195)
Updating files:  48% (11134/23195)
Updating files:  49% (11366/23195)
Updating files:  50% (11598/23195)
Updating files:  51% (11830/23195)
Updating files:  52% (12062/23195)
Updating files:  53% (12294/23195)
Updating files:  54% (12526/23195)
Updating files:  55% (12758/23195)
Updating files:  56% (12990/23195)
Updating files:  57% (13222/23195)
Updating files:  58% (13454/23195)
Updating files:  59% (13686/23195)
Updating files:  60% (13917/23195)
Updating files:  61% (14149/23195)
Updating files:  62% (14381/23195)
Updating files:  63% (14613/23195)
Updating files:  64% (14845/23195)
Updating files:  65% (15077/23195)
Updating files:  66% (15309/23195)
Updating files:  67% (15541/23195)
Updating files:  68% (15773/23195)
Updating files:  69% (16005/23195)
Updating files:  70% (16237/23195)
Updating files:  71% (16469/23195)
Updating files:  72% (16701/23195)
Updating files:  73% (16933/23195)
Updating files:  74% (17165/23195)
Updating files:  75% (17397/23195)
Updating files:  76% (17629/23195)
Updating files:  77% (17861/23195)
Updating files:  78% (18093/23195)
Updating files:  79% (18325/23195)
Updating files:  80% (18556/23195)
Updating files:  81% (18788/23195)
Updating files:  82% (19020/23195)
Updating files:  83% (19252/23195)
Updating files:  84% (19484/23195)
Updating files:  85% (19716/23195)
Updating files:  86% (19948/23195)
Updating files:  87% (20180/23195)
Updating files:  88% (20412/23195)
Updating files:  89% (20644/23195)
Updating files:  90% (20876/23195)
Updating files:  91% (21108/23195)
Updating files:  92% (21340/23195)
Updating files:  93% (21572/23195)
Updating files:  94% (21804/23195)
Updating files:  95% (22036/23195)
Updating files:  96% (22268/23195)
Updating files:  97% (22500/23195)
Updating files:  98% (22732/23195)
Updating files:  99% (22964/23195)
Updating files: 100% (23195/23195)
Updating files: 100% (23195/23195), done.
HEAD is now at 9160561b fix(storage): address review comments on read hedging and thread pool
GitCommit:
9160561bfffc3629d9bb770a83d50d9ce8bce509
BUILD
Starting Step #4 - "cancel-in-progress-builds-for-PR"
Starting Step #0 - "kaniko-build"
Step #4 - "cancel-in-progress-builds-for-PR": Pulling image: gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #0 - "kaniko-build": Pulling image: gcr.io/kaniko-project/executor:v1.24.0-debug@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": gcr.io/kaniko-project/executor@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd: Pulling from kaniko-project/executor
Step #4 - "cancel-in-progress-builds-for-PR": gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574: Pulling from google.com/cloudsdktool/cloud-sdk
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Pulling fs layer
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Pulling fs layer
Step #0 - "kaniko-build": 2fc842204170: Pulling fs layer
Step #0 - "kaniko-build": 8d1674b25e7b: Pulling fs layer
Step #0 - "kaniko-build": 9bdd0371dbb4: Pulling fs layer
Step #0 - "kaniko-build": 9aeeda79b717: Pulling fs layer
Step #0 - "kaniko-build": aaab912f1a4e: Pulling fs layer
Step #0 - "kaniko-build": 0006a7b536fb: Pulling fs layer
Step #0 - "kaniko-build": 3d08e2063109: Pulling fs layer
Step #0 - "kaniko-build": fc4517ec89af: Pulling fs layer
Step #0 - "kaniko-build": b892afb8bf07: Pulling fs layer
Step #0 - "kaniko-build": 2fc842204170: Waiting
Step #0 - "kaniko-build": cdc2b9345183: Pulling fs layer
Step #0 - "kaniko-build": 3934701e2c3d: Pulling fs layer
Step #0 - "kaniko-build": 8d1674b25e7b: Waiting
Step #0 - "kaniko-build": 46be974dd378: Pulling fs layer
Step #0 - "kaniko-build": 2b61f8ddf6d9: Pulling fs layer
Step #0 - "kaniko-build": 9bdd0371dbb4: Waiting
Step #0 - "kaniko-build": fc4517ec89af: Waiting
Step #0 - "kaniko-build": 0006a7b536fb: Waiting
Step #0 - "kaniko-build": 9aeeda79b717: Waiting
Step #0 - "kaniko-build": 3d08e2063109: Waiting
Step #0 - "kaniko-build": 46be974dd378: Waiting
Step #0 - "kaniko-build": aaab912f1a4e: Waiting
Step #0 - "kaniko-build": 2b61f8ddf6d9: Waiting
Step #0 - "kaniko-build": b892afb8bf07: Waiting
Step #0 - "kaniko-build": 3934701e2c3d: Waiting
Step #0 - "kaniko-build": cdc2b9345183: Waiting
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Download complete
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Download complete
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Download complete
Step #0 - "kaniko-build": 9bdd0371dbb4: Download complete
Step #0 - "kaniko-build": 2fc842204170: Verifying Checksum
Step #0 - "kaniko-build": 2fc842204170: Download complete
Step #0 - "kaniko-build": 8d1674b25e7b: Verifying Checksum
Step #0 - "kaniko-build": 8d1674b25e7b: Download complete
Step #0 - "kaniko-build": 2fc842204170: Pull complete
Step #0 - "kaniko-build": 8d1674b25e7b: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Download complete
Step #0 - "kaniko-build": 9bdd0371dbb4: Pull complete
Step #0 - "kaniko-build": aaab912f1a4e: Verifying Checksum
Step #0 - "kaniko-build": aaab912f1a4e: Download complete
Step #0 - "kaniko-build": 3d08e2063109: Download complete
Step #0 - "kaniko-build": fc4517ec89af: Download complete
Step #0 - "kaniko-build": 0006a7b536fb: Verifying Checksum
Step #0 - "kaniko-build": 0006a7b536fb: Download complete
Step #0 - "kaniko-build": 3934701e2c3d: Verifying Checksum
Step #0 - "kaniko-build": 3934701e2c3d: Download complete
Step #0 - "kaniko-build": 9aeeda79b717: Verifying Checksum
Step #0 - "kaniko-build": 9aeeda79b717: Download complete
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Download complete
Step #0 - "kaniko-build": 46be974dd378: Verifying Checksum
Step #0 - "kaniko-build": 46be974dd378: Download complete
Step #0 - "kaniko-build": 9aeeda79b717: Pull complete
Step #0 - "kaniko-build": b892afb8bf07: Verifying Checksum
Step #0 - "kaniko-build": b892afb8bf07: Download complete
Step #0 - "kaniko-build": 2b61f8ddf6d9: Verifying Checksum
Step #0 - "kaniko-build": 2b61f8ddf6d9: Download complete
Step #0 - "kaniko-build": cdc2b9345183: Verifying Checksum
Step #0 - "kaniko-build": cdc2b9345183: Download complete
Step #4 - "cancel-in-progress-builds-for-PR": 6310eb16bf42: Pull complete
Step #0 - "kaniko-build": aaab912f1a4e: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 488f94d8fb5f: Pull complete
Step #0 - "kaniko-build": 0006a7b536fb: Pull complete
Step #0 - "kaniko-build": 3d08e2063109: Pull complete
Step #0 - "kaniko-build": fc4517ec89af: Pull complete
Step #0 - "kaniko-build": b892afb8bf07: Pull complete
Step #0 - "kaniko-build": cdc2b9345183: Pull complete
Step #0 - "kaniko-build": 3934701e2c3d: Pull complete
Step #0 - "kaniko-build": 46be974dd378: Pull complete
Step #0 - "kaniko-build": 2b61f8ddf6d9: Pull complete
Step #0 - "kaniko-build": Digest: sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": Status: Downloaded newer image for gcr.io/kaniko-project/executor@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": gcr.io/kaniko-project/executor:v1.24.0-debug@sha256:2562c4fe551399514277ffff7dcca9a3b1628c4ea38cb017d7286dc6ea52f4cd
Step #0 - "kaniko-build": time="2026-09-01T11:09:00Z" level=info msg="Using dockerignore file: /workspace/ci/.dockerignore"
Step #0 - "kaniko-build": time="2026-09-01T11:09:00Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-09-01T11:09:00Z" level=info msg="Retrieving image fedora:40 from registry index.docker.io"
Step #0 - "kaniko-build": time="2026-09-01T11:09:00Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-09-01T11:09:00Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Built cross stage deps: map[]"
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Retrieving image manifest fedora:40"
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Returning cached image manifest"
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Executing 0 build triggers"
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Building stage 'fedora:40' [idx: '0', base-idx: '-1']"
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:fbf2b68a76c5fcc9d82023155286f102b005ee9a9694c9fb2f529bfb71907f07..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Using caching version of cmd: RUN dnf makecache &&     dnf install -y autoconf automake         clang cmake diffutils findutils gcc-c++ git         make ninja-build patch python3         python-pip tar unzip wget which zip zlib-devel"
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:262ee5a781d9b85a2493794aaa8fa595102bc39b580fde3d0a5538d86ffb133f..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Using caching version of cmd: RUN dnf makecache &&     dnf install -y libnghttp2-devel openssl-devel"
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:4dd78ff0b6cc24f43a00fba2bf4ae63dd160da9099f866a6a863ddecd2313d28..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y python3-devel"
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:27c31638e1679dad1bfa32f20518d369bdfd0d9921cc396ad542c68b4223ea3f..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Using caching version of cmd: RUN pip3 install --upgrade pip"
Step #0 - "kaniko-build": time="2026-09-01T11:09:01Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:310fa67422b77a9eac06e992f28fca32f3d9a17bb7bdae2a283f36274d60e263..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:02Z" level=info msg="Using caching version of cmd: RUN pip3 install setuptools wheel"
Step #0 - "kaniko-build": time="2026-09-01T11:09:02Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:c7f692c3afbb965adae14a658fcb5a74af94d9180169a65d99c24225c382161c..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:02Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y java-latest-openjdk-devel"
Step #0 - "kaniko-build": time="2026-09-01T11:09:02Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:2b50aa326f120d6ce3e4e2cdb8b9511e4452de3450bb1bafa4d95570f59b8f72..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:02Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y glibc glibc-devel"
Step #0 - "kaniko-build": time="2026-09-01T11:09:02Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:aee1b0f8c896b81dd7ed9db4610c564693b38af9f6706311c12ea20e37a5853d..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:02Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y clang-tools-extra"
Step #0 - "kaniko-build": time="2026-09-01T11:09:02Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:d77ef481996c4553fa57710b773407cc5bcc332126ee468cf7745797ea9d94ba..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:02Z" level=info msg="Using caching version of cmd: RUN echo 'root:cloudcxx' | chpasswd"
Step #0 - "kaniko-build": time="2026-09-01T11:09:02Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:9f6d07e4dad75af1de80bd813f2f28720751b6c7cab8bf6a5057d694b1b4e66c..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:02Z" level=info msg="Using caching version of cmd: RUN curl -fsSL https://distfiles.ariadne.space/pkgconf/pkgconf-2.2.0.tar.gz |     tar -xzf - --strip-components=1 &&     ./configure --prefix=/usr --with-system-libdir=/lib64:/usr/lib64 --with-system-includedir=/usr/include &&     make -j ${NCPU:-4} &&     make install &&     ldconfig && cd /var/tmp && rm -fr build"
Step #0 - "kaniko-build": time="2026-09-01T11:09:02Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:b13e5c369fd44005044aac3c86abf5a80e176b17fceaed007cc192dbc1cfb63d..."
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Verifying Checksum
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Download complete
Step #0 - "kaniko-build": time="2026-09-01T11:09:03Z" level=info msg="Using caching version of cmd: RUN curl -fsSL https://github.com/curl/curl/releases/download/curl-8_7_1/curl-8.7.1.tar.gz |     tar -xzf - --strip-components=1 &&     cmake         -DCMAKE_BUILD_TYPE=Release         -DCMAKE_CXX_STANDARD=20         -DBUILD_SHARED_LIBS=ON         -DCURL_USE_OPENSSL=ON         -DBUILD_CURL_EXE=OFF         -DBUILD_TESTING=OFF         -GNinja -S . -B cmake-out &&     cmake --build cmake-out --target install &&     ldconfig && cd /var/tmp && rm -fr build"
Step #0 - "kaniko-build": time="2026-09-01T11:09:03Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:f3ad22042695eff32a01ecf59bd35343eb1e229635296650870bbcb5491b495f..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:03Z" level=info msg="Using caching version of cmd: RUN curl -fsSL https://github.com/abseil/abseil-cpp/archive/20250814.2.tar.gz |     tar -xzf - --strip-components=1 &&     cmake       -DCMAKE_CXX_STANDARD=20       -DCMAKE_BUILD_TYPE=Release       -DABSL_BUILD_TESTING=OFF       -DBUILD_SHARED_LIBS=yes       -GNinja -S . -B cmake-out &&     cmake --build cmake-out && cmake --install cmake-out &&     ldconfig &&     cd /var/tmp && rm -fr build"
Step #0 - "kaniko-build": time="2026-09-01T11:09:03Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:adbd14ab3a5e19d64705c95811690f0ec78c3807dfea1bb6acd694f315abfcde..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:03Z" level=info msg="Using caching version of cmd: RUN curl -fsSL https://github.com/google/googletest/archive/v1.17.0.tar.gz |     tar -xzf - --strip-components=1 &&     cmake       -DCMAKE_CXX_STANDARD=20       -DCMAKE_BUILD_TYPE=Release       -DBUILD_SHARED_LIBS=yes       -GNinja -S . -B cmake-out &&     cmake --build cmake-out && cmake --install cmake-out &&     ldconfig &&     cd /var/tmp && rm -fr build"
Step #0 - "kaniko-build": time="2026-09-01T11:09:03Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:495c2d07b48091ed6e2500099add94a92e6ab6033eb03d68e7393badb14d7dbf..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:03Z" level=info msg="Using caching version of cmd: RUN curl -fsSL https://github.com/google/benchmark/archive/v1.9.5.tar.gz |     tar -xzf - --strip-components=1 &&     cmake         -DCMAKE_CXX_STANDARD=20         -DCMAKE_BUILD_TYPE=Release         -DBUILD_SHARED_LIBS=yes         -DBENCHMARK_ENABLE_TESTING=OFF         -S . -B cmake-out &&     cmake --build cmake-out && cmake --install cmake-out &&     ldconfig &&     cd /var/tmp && rm -fr build"
Step #0 - "kaniko-build": time="2026-09-01T11:09:03Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:160658dc53da9e8cacdc43ab6c3b20db5f0732c7a2cb14bcc8fd20c7663cff83..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:03Z" level=info msg="Using caching version of cmd: RUN curl -fsSL https://github.com/nlohmann/json/archive/v3.11.3.tar.gz |     tar -xzf - --strip-components=1 &&     cmake       -DCMAKE_CXX_STANDARD=20       -DCMAKE_BUILD_TYPE=Release       -DBUILD_SHARED_LIBS=yes       -DBUILD_TESTING=OFF       -DJSON_BuildTests=OFF       -GNinja -S . -B cmake-out &&     cmake --build cmake-out && cmake --install cmake-out &&     ldconfig &&     cd /var/tmp && rm -fr build"
Step #0 - "kaniko-build": time="2026-09-01T11:09:03Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:18e290d1dc588046177ecf542aea363ec475970b30c23b9540fb94f6bc3dc675..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:04Z" level=info msg="Using caching version of cmd: RUN curl -fsSL https://github.com/protocolbuffers/protobuf/archive/v36.0.tar.gz |     tar -xzf - --strip-components=1 &&     cmake         -DCMAKE_CXX_STANDARD=20         -DCMAKE_BUILD_TYPE=Release         -DBUILD_SHARED_LIBS=yes         -Dprotobuf_BUILD_TESTS=OFF         -Dprotobuf_ABSL_PROVIDER=package         -GNinja -S . -B cmake-out &&     cmake --build cmake-out && cmake --install cmake-out &&     ldconfig &&     cd /var/tmp && rm -fr build"
Step #0 - "kaniko-build": time="2026-09-01T11:09:04Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:7f75076cb7aafff05846522152837f15d4a05a4b6d691f7a8fefa594f70d8236..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:04Z" level=info msg="Using caching version of cmd: RUN curl -fsSL https://github.com/open-telemetry/opentelemetry-cpp/archive/v1.24.0.tar.gz |     tar -xzf - --strip-components=1 &&     cmake         -DCMAKE_CXX_STANDARD=20         -DCMAKE_BUILD_TYPE=Release         -DCMAKE_POSITION_INDEPENDENT_CODE=TRUE         -DBUILD_SHARED_LIBS=ON         -DWITH_EXAMPLES=OFF         -DWITH_STL=CXX20         -DBUILD_TESTING=OFF         -DOPENTELEMETRY_INSTALL=ON         -DOPENTELEMETRY_ABI_VERSION_NO=2     -GNinja -S . -B cmake-out &&     cmake --build cmake-out && cmake --install cmake-out &&     ldconfig && cd /var/tmp && rm -fr build"
Step #0 - "kaniko-build": time="2026-09-01T11:09:04Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:7c7ef8395be7c82a4adfbf9b99cf9f3cc285e0a1ff0111b1810324394559a8d1..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:04Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y c-ares-devel re2-devel"
Step #0 - "kaniko-build": time="2026-09-01T11:09:04Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:39a2b36fa07f80e3db455bef79034cfe23a809c0ae3bea8deb8e55d39137f557..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:04Z" level=info msg="Using caching version of cmd: RUN curl -fsSL https://github.com/grpc/grpc/archive/v1.83.0.tar.gz |     tar -xzf - --strip-components=1 &&     cmake         -DCMAKE_CXX_STANDARD=20         -DCMAKE_BUILD_TYPE=Release         -DBUILD_SHARED_LIBS=ON         -DgRPC_INSTALL=ON         -DgRPC_BUILD_TESTS=OFF         -DgRPC_ABSL_PROVIDER=package         -DgRPC_CARES_PROVIDER=package         -DgRPC_PROTOBUF_PROVIDER=package         -DgRPC_RE2_PROVIDER=package         -DgRPC_SSL_PROVIDER=package         -DgRPC_ZLIB_PROVIDER=package         -DgRPC_OPENTELEMETRY_PROVIDER=package         -DgRPC_BUILD_GRPCPP_OTEL_PLUGIN=ON         -GNinja -S . -B cmake-out &&     cmake --build cmake-out && cmake --install cmake-out &&     ldconfig &&     cd /var/tmp && rm -fr build"
Step #0 - "kaniko-build": time="2026-09-01T11:09:04Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:66e8b74e3313dbd2c74020b0c05b4fc5bd7ebadc6adadd28a8b4a0736bd02dd7..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:04Z" level=info msg="Using caching version of cmd: RUN curl -fsSL https://github.com/mozilla/sccache/releases/download/v0.10.0/sccache-v0.10.0-x86_64-unknown-linux-musl.tar.gz |     tar -zxf - --strip-components=1 &&     mkdir -p /usr/local/bin &&     mv sccache /usr/local/bin/sccache &&     chmod +x /usr/local/bin/sccache"
Step #0 - "kaniko-build": time="2026-09-01T11:09:04Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:bd1f8e91a5b294474bb900d1ad8c38512d128b2eac2e193be62a12594bf275fd..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:05Z" level=info msg="Using caching version of cmd: RUN dnf makecache && dnf install -y python3.10"
Step #0 - "kaniko-build": time="2026-09-01T11:09:05Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:47518e41ad43e5c06876094c9336dda578b21778af113c63b5158cbf82d4a154..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:05Z" level=info msg="Using caching version of cmd: RUN /var/tmp/ci/install-cloud-sdk.sh"
Step #0 - "kaniko-build": time="2026-09-01T11:09:05Z" level=info msg="Checking for cached layer us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20/cache:18dc1c3399aad7c9d0ec4487e5e9e813e56e3d37806f0e16b66cb791f5a342c3..."
Step #0 - "kaniko-build": time="2026-09-01T11:09:05Z" level=info msg="Using caching version of cmd: RUN ldconfig /usr/local/lib*"
Step #0 - "kaniko-build": time="2026-09-01T11:09:05Z" level=info msg="Unpacking rootfs as cmd COPY . /var/tmp/ci requires it."
Step #0 - "kaniko-build": time="2026-09-01T11:09:09Z" level=info msg="ARG NCPU=4"
Step #0 - "kaniko-build": time="2026-09-01T11:09:09Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-09-01T11:09:09Z" level=info msg="ARG ARCH=amd64"
Step #0 - "kaniko-build": time="2026-09-01T11:09:09Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026-09-01T11:09:09Z" level=info msg="RUN dnf makecache &&     dnf install -y autoconf automake         clang cmake diffutils findutils gcc-c++ git         make ninja-build patch python3         python-pip tar unzip wget which zip zlib-devel"
Step #0 - "kaniko-build": time="2026-09-01T11:09:09Z" level=info msg="Found cached layer, extracting to filesystem"
Step #4 - "cancel-in-progress-builds-for-PR": a257e756f7a6: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 60070b3b968e: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 4d1f9d77cfa0: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": 301958339266: Pull complete
Step #4 - "cancel-in-progress-builds-for-PR": Digest: sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": Status: Downloaded newer image for gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #4 - "cancel-in-progress-builds-for-PR": + test -z 16389
Step #4 - "cancel-in-progress-builds-for-PR": ++ gcloud builds describe --region us-east1 --format 'value(create_time)' 4435a1d3-e023-436b-9c73-88df818f0ab2
Step #4 - "cancel-in-progress-builds-for-PR": + ctime=2026-09-01T11:07:24.127289Z
Step #4 - "cancel-in-progress-builds-for-PR": + query=tags=pr
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND tags=16389'
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND substitutions.COMMIT_SHA != 9160561bfffc3629d9bb770a83d50d9ce8bce509'
Step #4 - "cancel-in-progress-builds-for-PR": + query+=' AND create_time < 2026-09-01T11:07:24.127289Z'
Step #4 - "cancel-in-progress-builds-for-PR": + gcloud builds list --region us-east1 --ongoing '--format=value(id)' --filter 'tags=pr AND tags=16389 AND substitutions.COMMIT_SHA != 9160561bfffc3629d9bb770a83d50d9ce8bce509 AND create_time < 2026-09-01T11:07:24.127289Z'
Step #4 - "cancel-in-progress-builds-for-PR": + xargs -r -t gcloud builds cancel --region us-east1
Step #4 - "cancel-in-progress-builds-for-PR": WARNING: The following filter keys were not present in any resource : create_time, substitutions.COMMIT_SHA, tags
Finished Step #4 - "cancel-in-progress-builds-for-PR"
Step #0 - "kaniko-build": time="2026-09-01T11:09:56Z" level=info msg="RUN dnf makecache &&     dnf install -y libnghttp2-devel openssl-devel"
Step #0 - "kaniko-build": time="2026-09-01T11:09:56Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-09-01T11:09:58Z" level=info msg="RUN dnf makecache && dnf install -y python3-devel"
Step #0 - "kaniko-build": time="2026-09-01T11:09:58Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-09-01T11:10:03Z" level=info msg="RUN pip3 install --upgrade pip"
Step #0 - "kaniko-build": time="2026-09-01T11:10:03Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-09-01T11:10:04Z" level=info msg="RUN pip3 install setuptools wheel"
Step #0 - "kaniko-build": time="2026-09-01T11:10:04Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-09-01T11:10:05Z" level=info msg="RUN dnf makecache && dnf install -y java-latest-openjdk-devel"
Step #0 - "kaniko-build": time="2026-09-01T11:10:05Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-09-01T11:10:44Z" level=info msg="RUN dnf makecache && dnf install -y glibc glibc-devel"
Step #0 - "kaniko-build": time="2026-09-01T11:10:44Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-09-01T11:10:46Z" level=info msg="RUN dnf makecache && dnf install -y clang-tools-extra"
Step #0 - "kaniko-build": time="2026-09-01T11:10:46Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-09-01T11:10:53Z" level=info msg="RUN echo 'root:cloudcxx' | chpasswd"
Step #0 - "kaniko-build": time="2026-09-01T11:10:53Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-09-01T11:10:54Z" level=info msg="WORKDIR /var/tmp/build/pkgconf"
Step #0 - "kaniko-build": time="2026-09-01T11:10:54Z" level=info msg="Cmd: workdir"
Step #0 - "kaniko-build": time="2026-09-01T11:10:54Z" level=info msg="Changed working directory to /var/tmp/build/pkgconf"
Step #0 - "kaniko-build": time="2026-09-01T11:10:54Z" level=info msg="Creating directory /var/tmp/build/pkgconf with uid -1 and gid -1"
Step #0 - "kaniko-build": time="2026-09-01T11:10:54Z" level=info msg="Taking snapshot of files..."
Step #0 - "kaniko-build": time="2026-09-01T11:10:54Z" level=info msg="RUN curl -fsSL https://distfiles.ariadne.space/pkgconf/pkgconf-2.2.0.tar.gz |     tar -xzf - --strip-components=1 &&     ./configure --prefix=/usr --with-system-libdir=/lib64:/usr/lib64 --with-system-includedir=/usr/include &&     make -j ${NCPU:-4} &&     make install &&     ldconfig && cd /var/tmp && rm -fr build"
Step #0 - "kaniko-build": time="2026-09-01T11:10:54Z" level=info msg="Found cached layer, extracting to filesystem"
Step #0 - "kaniko-build": time="2026-09-01T11:10:54Z" level=info msg="ENV PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib64/pkgconfig"
Step #0 - "kaniko-build": time="2026-09-01T11:10:54Z" level=info msg="No files changed in this command, skipping snapshotting."
Step #0 - "kaniko-build": time="2026
...
[Logs truncated due to log size limitations. For full logs, see https://storage.cloud.google.com/cloud-cpp-community-publiclogs/logs/google-cloud-cpp/16389/9160561bfffc3629d9bb770a83d50d9ce8bce509/fedora-latest-cxx20-cxx20-__default__/log-4435a1d3-e023-436b-9c73-88df818f0ab2.txt.]
...
n_test
Step #2 - "build.sh":     Start 720: pubsub_subscriber_integration_test
Step #2 - "build.sh":     Start 721: pubsub_subscription_admin_integration_test
Step #2 - "build.sh":     Start 722: pubsub_topic_admin_integration_test
Step #2 - "build.sh": 1/8 Test #718: pubsub_generated_subscription_admin_integration_test ...   Passed    0.43 sec
Step #2 - "build.sh": 2/8 Test #722: pubsub_topic_admin_integration_test ....................   Passed    0.45 sec
Step #2 - "build.sh": 3/8 Test #717: pubsub_blocking_publisher_integration_test .............   Passed    0.50 sec
Step #2 - "build.sh": 4/8 Test #719: pubsub_publisher_integration_test ......................   Passed    0.53 sec
Step #2 - "build.sh": 5/8 Test #721: pubsub_subscription_admin_integration_test .............   Passed    0.57 sec
Step #2 - "build.sh": 6/8 Test #714: pubsub_throughput ......................................   Passed    1.46 sec
Step #2 - "build.sh": 7/8 Test #720: pubsub_subscriber_integration_test .....................   Passed    7.93 sec
Step #2 - "build.sh": 8/8 Test #713: pubsub_endurance .......................................   Passed   10.37 sec
Step #2 - "build.sh": 
Step #2 - "build.sh": 100% tests passed, 0 tests failed out of 8
Step #2 - "build.sh": 
Step #2 - "build.sh": Label Time Summary:
Step #2 - "build.sh": integration-test             =  22.26 sec*proc (8 tests)
Step #2 - "build.sh": integration-test-emulator    =  22.26 sec*proc (8 tests)
Step #2 - "build.sh": 
Step #2 - "build.sh": Total Test time (real) =  10.41 sec
Step #2 - "build.sh": Killing Pub/Sub Emulator [309531] +. done.
Step #2 - "build.sh": 
Step #2 - "build.sh": 2026-09-01T11:27:10Z (+819s)
Step #2 - "build.sh": ---------------------------------------------------------
Step #2 - "build.sh": |   Running Storage integration tests (with emulator)   |
Step #2 - "build.sh": ---------------------------------------------------------
Step #2 - "build.sh": 2026-09-01T11:27:10Z (+819s): Launching Cloud Storage emulator on port 0
Step #2 - "build.sh": Successfully connected to emulator [313132]
Step #2 - "build.sh": 2026-09-01T11:27:12Z (+821s): Successfully connected to gRPC server at port 35655
Step #2 - "build.sh": Test project /workspace/cmake-out
Step #2 - "build.sh":       Start 375: storage_async_client_integration_test
Step #2 - "build.sh":       Start 376: storage_auto_finalize_integration_test
Step #2 - "build.sh":       Start 377: storage_bucket_acl_integration_test
Step #2 - "build.sh":       Start 378: storage_bucket_integration_test
Step #2 - "build.sh":       Start 379: storage_create_client_integration_test
Step #2 - "build.sh":       Start 380: storage_curl_sign_blob_integration_test
Step #2 - "build.sh":       Start 381: storage_decompressive_transcoding_integration_test
Step #2 - "build.sh":       Start 382: storage_default_object_acl_integration_test
Step #2 - "build.sh":       Start 383: storage_error_injection_integration_test
Step #2 - "build.sh":       Start 384: storage_error_parsing_integration_test
Step #2 - "build.sh":       Start 385: storage_grpc_bucket_metadata_integration_test
Step #2 - "build.sh":       Start 386: storage_grpc_integration_test
Step #2 - "build.sh":       Start 387: storage_grpc_object_acl_integration_test
Step #2 - "build.sh":       Start 388: storage_grpc_object_media_integration_test
Step #2 - "build.sh":       Start 389: storage_grpc_object_metadata_integration_test
Step #2 - "build.sh":       Start 390: storage_hmac_key_integration_test
Step #2 - "build.sh":       Start 393: storage_notification_integration_test
Step #2 - "build.sh":       Start 394: storage_object_basic_crud_integration_test
Step #2 - "build.sh":       Start 395: storage_object_checksum_integration_test
Step #2 - "build.sh":       Start 396: storage_object_compose_many_integration_test
Step #2 - "build.sh":       Start 397: storage_object_file_integration_test
Step #2 - "build.sh":       Start 398: storage_object_file_multi_threaded_test
Step #2 - "build.sh":       Start 399: storage_object_hash_integration_test
Step #2 - "build.sh":       Start 400: storage_object_insert_integration_test
Step #2 - "build.sh":       Start 401: storage_object_insert_preconditions_integration_test
Step #2 - "build.sh":       Start 402: storage_object_integration_test
Step #2 - "build.sh":       Start 403: storage_object_list_objects_versions_integration_test
Step #2 - "build.sh":       Start 404: storage_object_media_integration_test
Step #2 - "build.sh":       Start 405: storage_object_parallel_upload_integration_test
Step #2 - "build.sh":       Start 406: storage_object_plenty_clients_serially_integration_test
Step #2 - "build.sh":       Start 407: storage_object_plenty_clients_simultaneously_integration_test
Step #2 - "build.sh":       Start 408: storage_object_read_headers_integration_test
Step #2 - "build.sh":  1/61 Test #380: storage_curl_sign_blob_integration_test .........................   Passed    0.08 sec
Step #2 - "build.sh":       Start 409: storage_object_read_large_integration_test
Step #2 - "build.sh":  2/61 Test #406: storage_object_plenty_clients_serially_integration_test .........   Passed    0.05 sec
Step #2 - "build.sh":       Start 410: storage_object_read_preconditions_integration_test
Step #2 - "build.sh":  3/61 Test #388: storage_grpc_object_media_integration_test ......................   Passed    0.23 sec
Step #2 - "build.sh":       Start 411: storage_object_read_range_integration_test
Step #2 - "build.sh":  4/61 Test #390: storage_hmac_key_integration_test ...............................   Passed    0.29 sec
Step #2 - "build.sh":       Start 412: storage_object_read_stream_integration_test
Step #2 - "build.sh":  5/61 Test #384: storage_error_parsing_integration_test ..........................   Passed    0.46 sec
Step #2 - "build.sh":       Start 413: storage_object_resumable_parallel_upload_integration_test
Step #2 - "build.sh":  6/61 Test #393: storage_notification_integration_test ...........................   Passed    0.82 sec
Step #2 - "build.sh":       Start 414: storage_object_resumable_write_integration_test
Step #2 - "build.sh":  7/61 Test #387: storage_grpc_object_acl_integration_test ........................   Passed    1.01 sec
Step #2 - "build.sh":       Start 415: storage_object_rewrite_integration_test
Step #2 - "build.sh":  8/61 Test #379: storage_create_client_integration_test ..........................   Passed    1.22 sec
Step #2 - "build.sh":       Start 416: storage_object_write_preconditions_integration_test
Step #2 - "build.sh":  9/61 Test #412: storage_object_read_stream_integration_test .....................   Passed    0.98 sec
Step #2 - "build.sh":       Start 417: storage_object_write_stream_integration_test
Step #2 - "build.sh": 10/61 Test #389: storage_grpc_object_metadata_integration_test ...................   Passed    1.62 sec
Step #2 - "build.sh":       Start 418: storage_object_write_streambuf_integration_test
Step #2 - "build.sh": 11/61 Test #398: storage_object_file_multi_threaded_test .........................   Passed    1.94 sec
Step #2 - "build.sh":       Start 419: storage_service_account_credentials_integration_test
Step #2 - "build.sh": 12/61 Test #419: storage_service_account_credentials_integration_test ............   Passed    0.04 sec
Step #2 - "build.sh":       Start 420: storage_service_account_integration_test
Step #2 - "build.sh": 13/61 Test #409: storage_object_read_large_integration_test ......................   Passed    2.01 sec
Step #2 - "build.sh":       Start 421: storage_signed_url_conformance_test
Step #2 - "build.sh": 14/61 Test #385: storage_grpc_bucket_metadata_integration_test ...................   Passed    2.16 sec
Step #2 - "build.sh":       Start 423: storage_slow_reader_chunk_integration_test
Step #2 - "build.sh": 15/61 Test #396: storage_object_compose_many_integration_test ....................   Passed    2.26 sec
Step #2 - "build.sh":       Start 424: storage_slow_reader_stream_integration_test
Step #2 - "build.sh": 16/61 Test #403: storage_object_list_objects_versions_integration_test ...........   Passed    2.34 sec
Step #2 - "build.sh":       Start 425: storage_small_reads_integration_test
Step #2 - "build.sh": 17/61 Test #421: storage_signed_url_conformance_test .............................   Passed    0.59 sec
Step #2 - "build.sh":       Start 426: storage_smoke_test_async
Step #2 - "build.sh": 18/61 Test #417: storage_object_write_stream_integration_test ....................   Passed    1.54 sec
Step #2 - "build.sh":       Start 427: storage_smoke_test_grpc
Step #2 - "build.sh": 19/61 Test #376: storage_auto_finalize_integration_test ..........................   Passed    3.00 sec
Step #2 - "build.sh":       Start 428: storage_smoke_test_rest
Step #2 - "build.sh": 20/61 Test #382: storage_default_object_acl_integration_test .....................   Passed    3.22 sec
Step #2 - "build.sh":       Start 429: storage_storage_include_test
Step #2 - "build.sh": 21/61 Test #429: storage_storage_include_test ....................................   Passed    0.03 sec
Step #2 - "build.sh":       Start 430: storage_thread_integration_test
Step #2 - "build.sh": 22/61 Test #394: storage_object_basic_crud_integration_test ......................   Passed    3.25 sec
Step #2 - "build.sh":       Start 431: storage_tracing_integration_test
Step #2 - "build.sh": 23/61 Test #428: storage_smoke_test_rest .........................................   Passed    0.27 sec
Step #2 - "build.sh":       Start 433: storage_aggregate_download_throughput_benchmark
Step #2 - "build.sh": 24/61 Test #413: storage_object_resumable_parallel_upload_integration_test .......   Passed    2.81 sec
Step #2 - "build.sh":       Start 434: storage_aggregate_upload_throughput_benchmark
Step #2 - "build.sh": 25/61 Test #420: storage_service_account_integration_test ........................   Passed    1.44 sec
Step #2 - "build.sh":       Start 435: storage_create_dataset
Step #2 - "build.sh": 26/61 Test #377: storage_bucket_acl_integration_test .............................   Passed    3.45 sec
Step #2 - "build.sh":       Start 436: storage_storage_file_transfer_benchmark
Step #2 - "build.sh": 27/61 Test #418: storage_object_write_streambuf_integration_test .................   Passed    1.86 sec
Step #2 - "build.sh":       Start 437: storage_storage_parallel_uploads_benchmark
Step #2 - "build.sh": 28/61 Test #431: storage_tracing_integration_test ................................   Passed    0.72 sec
Step #2 - "build.sh":       Start 438: storage_storage_throughput_vs_cpu_benchmark
Step #2 - "build.sh": 29/61 Test #433: storage_aggregate_download_throughput_benchmark .................   Passed    0.79 sec
Step #2 - "build.sh":       Start 439: storage_throughput_experiment_test
Step #2 - "build.sh": 30/61 Test #425: storage_small_reads_integration_test ............................   Passed    1.84 sec
Step #2 - "build.sh": 31/61 Test #411: storage_object_read_range_integration_test ......................   Passed    4.38 sec
Step #2 - "build.sh": 32/61 Test #401: storage_object_insert_preconditions_integration_test ............   Passed    5.25 sec
Step #2 - "build.sh": 33/61 Test #410: storage_object_read_preconditions_integration_test ..............   Passed    5.20 sec
Step #2 - "build.sh": 34/61 Test #438: storage_storage_throughput_vs_cpu_benchmark .....................   Passed    1.31 sec
Step #2 - "build.sh": 35/61 Test #436: storage_storage_file_transfer_benchmark .........................   Passed    2.01 sec
Step #2 - "build.sh": 36/61 Test #416: storage_object_write_preconditions_integration_test .............   Passed    4.74 sec
Step #2 - "build.sh": 37/61 Test #386: storage_grpc_integration_test ...................................   Passed    6.06 sec
Step #2 - "build.sh": 38/61 Test #408: storage_object_read_headers_integration_test ....................   Passed    6.28 sec
Step #2 - "build.sh": 39/61 Test #383: storage_error_injection_integration_test ........................   Passed    6.51 sec
Step #2 - "build.sh": 40/61 Test #437: storage_storage_parallel_uploads_benchmark ......................   Passed    3.03 sec
Step #2 - "build.sh": 41/61 Test #399: storage_object_hash_integration_test ............................   Passed    6.52 sec
Step #2 - "build.sh": 42/61 Test #395: storage_object_checksum_integration_test ........................   Passed    6.67 sec
Step #2 - "build.sh": 43/61 Test #397: storage_object_file_integration_test ............................   Passed    7.52 sec
Step #2 - "build.sh": 44/61 Test #378: storage_bucket_integration_test .................................   Passed    7.71 sec
Step #2 - "build.sh": 45/61 Test #415: storage_object_rewrite_integration_test .........................   Passed    7.56 sec
Step #2 - "build.sh": 46/61 Test #434: storage_aggregate_upload_throughput_benchmark ...................   Passed    5.33 sec
Step #2 - "build.sh": 47/61 Test #400: storage_object_insert_integration_test ..........................   Passed   10.13 sec
Step #2 - "build.sh": 48/61 Test #402: storage_object_integration_test .................................   Passed   10.79 sec
Step #2 - "build.sh": 49/61 Test #407: storage_object_plenty_clients_simultaneously_integration_test ...   Passed   11.15 sec
Step #2 - "build.sh": 50/61 Test #414: storage_object_resumable_write_integration_test .................   Passed   13.07 sec
Step #2 - "build.sh": 51/61 Test #405: storage_object_parallel_upload_integration_test .................   Passed   14.62 sec
Step #2 - "build.sh": 52/61 Test #439: storage_throughput_experiment_test ..............................   Passed   15.22 sec
Step #2 - "build.sh": 53/61 Test #426: storage_smoke_test_async ........................................   Passed   19.93 sec
Step #2 - "build.sh": 54/61 Test #427: storage_smoke_test_grpc .........................................   Passed   19.78 sec
Step #2 - "build.sh": 55/61 Test #435: storage_create_dataset ..........................................   Passed   23.68 sec
Step #2 - "build.sh": 56/61 Test #430: storage_thread_integration_test .................................   Passed   24.26 sec
Step #2 - "build.sh": 57/61 Test #381: storage_decompressive_transcoding_integration_test ..............   Passed   30.05 sec
Step #2 - "build.sh": 58/61 Test #423: storage_slow_reader_chunk_integration_test ......................   Passed   36.81 sec
Step #2 - "build.sh": 59/61 Test #424: storage_slow_reader_stream_integration_test .....................   Passed   36.79 sec
Step #2 - "build.sh": 60/61 Test #404: storage_object_media_integration_test ...........................   Passed   46.28 sec
Step #2 - "build.sh": 61/61 Test #375: storage_async_client_integration_test ...........................   Passed   84.21 sec
Step #2 - "build.sh": 
Step #2 - "build.sh": 100% tests passed, 0 tests failed out of 61
Step #2 - "build.sh": 
Step #2 - "build.sh": Label Time Summary:
Step #2 - "build.sh": integration-test             = 525.14 sec*proc (61 tests)
Step #2 - "build.sh": integration-test-emulator    = 525.14 sec*proc (61 tests)
Step #2 - "build.sh": integration-test-gha         =  39.97 sec*proc (3 tests)
Step #2 - "build.sh": 
Step #2 - "build.sh": Total Test time (real) =  84.25 sec
Step #2 - "build.sh": Killing emulator server [313132] ... done.
Step #2 - "build.sh": 
Step #2 - "build.sh": 2026-09-01T11:28:37Z (+906s)
Step #2 - "build.sh": ---------------------------------------------------------
Step #2 - "build.sh": |   Running Spanner integration tests (with emulator)   |
Step #2 - "build.sh": ---------------------------------------------------------
Step #2 - "build.sh": 2026-09-01T11:28:37Z (+906s): Launching Cloud Spanner emulator in the background
Step #2 - "build.sh": 2026-09-01T11:28:38Z (+907s): Spanner emulator started at localhost:8787
Step #2 - "build.sh": 2026-09-01T11:28:38Z (+907s): Launching Cloud Spanner HTTP emulator in the background
Step #2 - "build.sh": 2026-09-01T11:28:40Z (+909s): Spanner HTTP emulator started at http://localhost:8788
Step #2 - "build.sh": Test project /workspace/cmake-out
Step #2 - "build.sh":     Start 776: spanner_admin_backup_extra_integration_test
Step #2 - "build.sh":     Start 777: spanner_admin_backup_integration_test
Step #2 - "build.sh":     Start 778: spanner_admin_database_admin_integration_test
Step #2 - "build.sh":     Start 779: spanner_admin_instance_admin_integration_test
Step #2 - "build.sh":     Start 782: spanner_benchmarks_config_test
Step #2 - "build.sh":     Start 783: spanner_multiple_rows_cpu_benchmark
Step #2 - "build.sh":     Start 784: spanner_single_row_throughput_benchmark
Step #2 - "build.sh": 1/7 Test #782: spanner_benchmarks_config_test ..................   Passed    0.03 sec
Step #2 - "build.sh": 2/7 Test #777: spanner_admin_backup_integration_test ...........   Passed    0.04 sec
Step #2 - "build.sh": 3/7 Test #776: spanner_admin_backup_extra_integration_test .....   Passed    0.07 sec
Step #2 - "build.sh": 4/7 Test #778: spanner_admin_database_admin_integration_test ...   Passed    0.20 sec
Step #2 - "build.sh": 5/7 Test #779: spanner_admin_instance_admin_integration_test ...   Passed    0.25 sec
Step #2 - "build.sh": 6/7 Test #784: spanner_single_row_throughput_benchmark .........   Passed    4.17 sec
Step #2 - "build.sh": 7/7 Test #783: spanner_multiple_rows_cpu_benchmark .............   Passed   25.41 sec
Step #2 - "build.sh": 
Step #2 - "build.sh": 100% tests passed, 0 tests failed out of 7
Step #2 - "build.sh": 
Step #2 - "build.sh": Label Time Summary:
Step #2 - "build.sh": integration-test             =  30.16 sec*proc (7 tests)
Step #2 - "build.sh": integration-test-emulator    =  30.16 sec*proc (7 tests)
Step #2 - "build.sh": 
Step #2 - "build.sh": Total Test time (real) =  25.48 sec
Step #2 - "build.sh": Killing Spanner Emulator [315711] +. done.
Step #2 - "build.sh": ================ emulator.log ================
Step #2 - "build.sh":      1	WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
Step #2 - "build.sh":      2	I0000 00:00:1788262117.621006  315711 emulator_main.cc:39] Cloud Spanner Emulator running.
Step #2 - "build.sh":      3	I0000 00:00:1788262117.621059  315711 emulator_main.cc:40] Server address: localhost:8787
Step #2 - "build.sh":      4	external/com_googlesource_code_re2/re2/re2.cc:842: DFA out of memory: pattern length 41, program size 117214, list count 29752, bytemap range 106
Step #2 - "build.sh":      5	I0000 00:00:1788262120.860050  316370 schema_updater.cc:789] INTERNAL: ZETASQL_RET_CHECK failure (backend/schema/updater/schema_updater.cc:789) false Unsupported ddl statement: 61
Step #2 - "build.sh":      6	external/com_googlesource_code_re2/re2/re2.cc:842: DFA out of memory: pattern length 27, program size 117373, list count 29802, bytemap range 105
Step #2 - "build.sh": ==============================================
Step #2 - "build.sh": Killing Spanner HTTP Emulator [315752] +. done.
Step #2 - "build.sh": ============== http_emulator.log =============
Step #2 - "build.sh":      1	WARNING: proto: file "google/rpc/status.proto" is already registered
Step #2 - "build.sh":      2		previously from: "google.golang.org/genproto/googleapis/rpc/status"
Step #2 - "build.sh":      3		currently from:  "unknown"
Step #2 - "build.sh":      4	See https://protobuf.dev/reference/go/faq#namespace-conflict
Step #2 - "build.sh":      5	
Step #2 - "build.sh":      6	WARNING: proto: file "google/rpc/status.proto" has a name conflict over google.rpc.Status
Step #2 - "build.sh":      7		previously from: "google.golang.org/genproto/googleapis/rpc/status"
Step #2 - "build.sh":      8		currently from:  "unknown"
Step #2 - "build.sh":      9	See https://protobuf.dev/reference/go/faq#namespace-conflict
Step #2 - "build.sh":     10	
Step #2 - "build.sh":     11	WARNING: proto: message google.rpc.Status is already registered
Step #2 - "build.sh":     12		previously from: "google.golang.org/genproto/googleapis/rpc/status"
Step #2 - "build.sh":     13		currently from:  "unknown"
Step #2 - "build.sh":     14	See https://protobuf.dev/reference/go/faq#namespace-conflict
Step #2 - "build.sh":     15	
Step #2 - "build.sh":     16	WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
Step #2 - "build.sh":     17	I0000 00:00:1788262118.651169  315762 emulator_main.cc:39] Cloud Spanner Emulator running.
Step #2 - "build.sh":     18	I0000 00:00:1788262118.651205  315762 emulator_main.cc:40] Server address: localhost:8789
Step #2 - "build.sh":     19	2026/09/01 11:28:39 gateway.go:144: Cloud Spanner emulator running.
Step #2 - "build.sh":     20	2026/09/01 11:28:39 gateway.go:145: REST server listening at localhost:8788
Step #2 - "build.sh":     21	2026/09/01 11:28:39 gateway.go:146: gRPC server listening at localhost:8789
Step #2 - "build.sh": ==============================================
Step #2 - "build.sh": 
Step #2 - "build.sh": 2026-09-01T11:29:06Z (+935s)
Step #2 - "build.sh": ----------------------------------------------------------
Step #2 - "build.sh": |   Running Bigtable integration tests (with emulator)   |
Step #2 - "build.sh": ----------------------------------------------------------
Step #2 - "build.sh": 2026-09-01T11:29:06Z (+935s): Launching Cloud Bigtable emulators in the background
Step #2 - "build.sh": 2026-09-01T11:29:06Z (+935s): Connected to the Cloud Bigtable emulator on localhost:8480
Step #2 - "build.sh": 2026-09-01T11:29:06Z (+935s): Connected to the Cloud Bigtable emulator on localhost:8490
Step #2 - "build.sh": Test project /workspace/cmake-out
Step #2 - "build.sh":       Start 622: bigtable_apply_read_latency_benchmark
Step #2 - "build.sh":       Start 623: bigtable_endurance_benchmark
Step #2 - "build.sh":       Start 624: bigtable_mutation_batcher_throughput_benchmark
Step #2 - "build.sh":       Start 625: bigtable_read_sync_vs_async_benchmark
Step #2 - "build.sh":       Start 626: bigtable_scan_async_throughput_benchmark
Step #2 - "build.sh":       Start 627: bigtable_scan_throughput_benchmark
Step #2 - "build.sh":       Start 628: bigtable_data_async_future_integration_test
Step #2 - "build.sh":       Start 629: bigtable_data_integration_test
Step #2 - "build.sh":       Start 630: bigtable_filters_integration_test
Step #2 - "build.sh":       Start 631: bigtable_instance_admin_integration_test
Step #2 - "build.sh":       Start 632: bigtable_mutations_integration_test
Step #2 - "build.sh":       Start 633: bigtable_observability_integration_test
Step #2 - "build.sh":       Start 636: bigtable_table_admin_integration_test
Step #2 - "build.sh":       Start 637: bigtable_table_sample_rows_integration_test
Step #2 - "build.sh":  1/14 Test #631: bigtable_instance_admin_integration_test .........   Passed    0.25 sec
Step #2 - "build.sh":  2/14 Test #633: bigtable_observability_integration_test ..........   Passed    0.31 sec
Step #2 - "build.sh":  3/14 Test #628: bigtable_data_async_future_integration_test ......   Passed    0.57 sec
Step #2 - "build.sh":  4/14 Test #624: bigtable_mutation_batcher_throughput_benchmark ...   Passed    0.63 sec
Step #2 - "build.sh":  5/14 Test #632: bigtable_mutations_integration_test ..............   Passed    0.65 sec
Step #2 - "build.sh":  6/14 Test #637: bigtable_table_sample_rows_integration_test ......   Passed    0.97 sec
Step #2 - "build.sh":  7/14 Test #623: bigtable_endurance_benchmark .....................   Passed    1.12 sec
Step #2 - "build.sh":  8/14 Test #636: bigtable_table_admin_integration_test ............   Passed    1.10 sec
Step #2 - "build.sh":  9/14 Test #630: bigtable_filters_integration_test ................   Passed    2.96 sec
Step #2 - "build.sh": 10/14 Test #622: bigtable_apply_read_latency_benchmark ............   Passed    3.38 sec
Step #2 - "build.sh": 11/14 Test #625: bigtable_read_sync_vs_async_benchmark ............   Passed    3.41 sec
Step #2 - "build.sh": 12/14 Test #629: bigtable_data_integration_test ...................   Passed    3.58 sec
Step #2 - "build.sh": 13/14 Test #627: bigtable_scan_throughput_benchmark ...............   Passed    9.27 sec
Step #2 - "build.sh": 14/14 Test #626: bigtable_scan_async_throughput_benchmark .........   Passed   13.52 sec
Step #2 - "build.sh": 
Step #2 - "build.sh": 100% tests passed, 0 tests failed out of 14
Step #2 - "build.sh": 
Step #2 - "build.sh": Label Time Summary:
Step #2 - "build.sh": integration-test             =  41.74 sec*proc (14 tests)
Step #2 - "build.sh": integration-test-emulator    =  41.74 sec*proc (14 tests)
Step #2 - "build.sh": 
Step #2 - "build.sh": Total Test time (real) =  13.60 sec
Step #2 - "build.sh": 2026-09-01T11:29:19Z (+948s): Killing Bigtable Emulators...
Step #2 - "build.sh": EMULATOR_PID=318384 .+.
Step #2 - "build.sh": INSTANCE_ADMIN_EMULATOR_PID=318385 .+.
Step #2 - "build.sh": ================ emulator.log ================
Step #2 - "build.sh": cat: /h/emulator.log: No such file or directory
Step #2 - "build.sh":      1	Cloud Bigtable emulator running on 127.0.0.1:8480
Step #2 - "build.sh": ================ emulator.log ================
Step #2 - "build.sh": cat: /h/emulator.log: No such file or directory
Step #2 - "build.sh":      1	Cloud Bigtable emulator running on 127.0.0.1:8480
Step #2 - "build.sh": ================ emulator.log ================
Step #2 - "build.sh": ================ instance-admin-emulator.log ================
Step #2 - "build.sh": cat: /h/instance-admin-emulator.log: No such file or directory
Step #2 - "build.sh":      1	env -C /workspace /workspace/cmake-out/google/cloud/bigtable/tests/instance_admin_emulator 8490
Step #2 - "build.sh":      2	Cloud Bigtable emulator running on localhost:8490
Step #2 - "build.sh":      3	ListInstances() request=parent: "projects/emulated"
Step #2 - "build.sh":      4	
Step #2 - "build.sh":      5	CreateInstance() request=parent: "projects/cloud-cpp-testing-resources"
Step #2 - "build.sh":      6	instance_id: "it-2026-09-01-9gkvesw6v3lu"
Step #2 - "build.sh":      7	instance {
Step #2 - "build.sh":      8	  display_name: "IT it-2026-09-01-9gkvesw6v3lu"
Step #2 - "build.sh":      9	  type: DEVELOPMENT
Step #2 - "build.sh":     10	}
Step #2 - "build.sh": ================ instance-admin-emulator.log ================
Step #2 - "build.sh": cat: /h/instance-admin-emulator.log: No such file or directory
Step #2 - "build.sh":    234	  value {
Step #2 - "build.sh":    235	    location: "projects/cloud-cpp-testing-resources/locations/us-central1-c"
Step #2 - "build.sh":    236	    serve_nodes: 3
Step #2 - "build.sh":    237	    default_storage_type: HDD
Step #2 - "build.sh":    238	  }
Step #2 - "build.sh":    239	}
Step #2 - "build.sh":    240	
Step #2 - "build.sh":    241	ListClusters() request=parent: "projects/cloud-cpp-testing-resources/instances/it-2026-09-01-1rqvv0mlctdg"
Step #2 - "build.sh":    242	
Step #2 - "build.sh":    243	D================ instance-admin-emulator.log ================
Step #2 - "build.sh": 
Step #2 - "build.sh": 2026-09-01T11:29:20Z (+949s)
Step #2 - "build.sh": ------------------------------------------------------
Step #2 - "build.sh": |   Running REST integration tests (with emulator)   |
Step #2 - "build.sh": ------------------------------------------------------
Step #2 - "build.sh": 2026-09-01T11:29:20Z (+949s): Launching Cloud Storage emulator on port 0
Step #2 - "build.sh": Successfully connected to emulator [320857]
Step #2 - "build.sh": 2026-09-01T11:29:21Z (+950s): Successfully connected to gRPC server at port 35939
Step #2 - "build.sh": Test project /workspace/cmake-out
Step #2 - "build.sh":     Start 177: common_internal_curl_rest_client_integration_test
Step #2 - "build.sh": 1/1 Test #177: common_internal_curl_rest_client_integration_test ...   Passed    0.93 sec
Step #2 - "build.sh": 
Step #2 - "build.sh": 100% tests passed, 0 tests failed out of 1
Step #2 - "build.sh": 
Step #2 - "build.sh": Label Time Summary:
Step #2 - "build.sh": integration-test             =   0.93 sec*proc (1 test)
Step #2 - "build.sh": integration-test-emulator    =   0.93 sec*proc (1 test)
Step #2 - "build.sh": 
Step #2 - "build.sh": Total Test time (real) =   0.97 sec
Step #2 - "build.sh": Killing emulator server [320857] ... done.
Step #2 - "build.sh": 2026-09-01T11:29:23Z (+952s): ===> sccache stats
Step #2 - "build.sh": Compile requests                    6243
Step #2 - "build.sh": Compile requests executed           6243
Step #2 - "build.sh": Cache hits                          6243
Step #2 - "build.sh": Cache hits (C/C++)                  6243
Step #2 - "build.sh": Cache misses                           0
Step #2 - "build.sh": Cache hits rate                   100.00 %
Step #2 - "build.sh": Cache hits rate (C/C++)           100.00 %
Step #2 - "build.sh": Cache timeouts                         0
Step #2 - "build.sh": Cache read errors                      0
Step #2 - "build.sh": Forced recaches                        0
Step #2 - "build.sh": Cache write errors                     0
Step #2 - "build.sh": Cache errors                           0
Step #2 - "build.sh": Compilations                           0
Step #2 - "build.sh": Compilation failures                   0
Step #2 - "build.sh": Non-cacheable compilations             0
Step #2 - "build.sh": Non-cacheable calls                    0
Step #2 - "build.sh": Non-compilation calls                  0
Step #2 - "build.sh": Unsupported compiler calls             0
Step #2 - "build.sh": Average cache write                0.000 s
Step #2 - "build.sh": Average compiler                   0.000 s
Step #2 - "build.sh": Average cache read hit             0.058 s
Step #2 - "build.sh": Failed distributed compilations        0
Step #2 - "build.sh": Cache location                  gcs, name: cloud-cpp-testing-resources_cloudbuild, prefix: /sccache/fedora-latest-cxx20-cxx20/
Step #2 - "build.sh": Version (client)                0.10.0
Step #2 - "build.sh": ==> 🕑 cxx20 completed in 951.333 seconds
Finished Step #2 - "build.sh"
Starting Step #3 - "remove-image"
Step #3 - "remove-image": Already have image (with digest): gcr.io/google.com/cloudsdktool/cloud-sdk@sha256:2e480d7977829f1534665da1fffd98170f5345d0a6322b369b0fe08a932ce574
Step #3 - "remove-image": Digests:
Step #3 - "remove-image": - us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20@sha256:686ef04213a49a71f234b6db929faf515d95ca04212acaad752d4e179ee70831
Step #3 - "remove-image": 
Step #3 - "remove-image": Tags:
Step #3 - "remove-image": - us-east1-docker.pkg.dev/cloud-cpp-testing-resources/gcb/cloudbuild/fedora-latest-cxx20:4435a1d3-e023-436b-9c73-88df818f0ab2
Step #3 - "remove-image": Delete request issued.
Step #3 - "remove-image": Waiting for operation [projects/cloud-cpp-testing-resources/locations/us-east1/operations/c5a8a047-cf36-42f0-8208-d5adfdc79d81] to complete...
Step #3 - "remove-image": .....done.
Finished Step #3 - "remove-image"
PUSH
DONE

Build Log: https://storage.cloud.google.com/cloud-cpp-community-publiclogs/logs/google-cloud-cpp/16389/9160561bfffc3629d9bb770a83d50d9ce8bce509/fedora-latest-cxx20-cxx20-default/log-4435a1d3-e023-436b-9c73-88df818f0ab2.txt