Skip to content

Review fixes for the zenoh-flat transition - #696

Merged
milyin merged 4 commits into
zenoh-flat-transitionfrom
fix/review-zenoh-flat-transition
Aug 10, 2026
Merged

Review fixes for the zenoh-flat transition#696
milyin merged 4 commits into
zenoh-flat-transitionfrom
fix/review-zenoh-flat-transition

Conversation

@milyin

@milyin milyin commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Fixes found reviewing #669 end to end (build + jvmTest against the sibling
zenoh-flat-jni / zenoh-flat commits CI pins).

ZBytes publication race

ZBytes.bytes is double-checked locking over a non-volatile field:

private var eager: ByteArray?
get() = eager ?: synchronized(this) { eager ?: run { … eager = b; b } }

The fast path reads eager outside the monitor, so there is no
happens-before edge to the write inside it. A received ZBytes is exactly
the object that crosses threads — the native callback thread builds the
Sample, a consumer thread pulls it off a Channel and touches the payload
— so a reader can observe the array reference while its copied contents are
still not visible. eager is now @Volatile, which restores the edge.
It moves out of the constructor parameter list because Kotlin only accepts
the annotation on a body property.

ReplyKeyExpr declaration order

The constants were swapped to MATCHING_QUERY(1), ANY(0), with a comment
saying the order "deliberately differs" from the wire values. But the wire
values are carried in value and fromInt looks them up by value, so the
declaration order buys nothing — while reordering an enum silently changes
ordinal() and the order of values()/entries for every consumer, which
is a real source- and behaviour-level break for anyone who persisted or
switched on them. Restored to ANY(0), MATCHING_QUERY(1), with a test in
SelectorTest that pins it.

Build script indentation

Two lines in build.gradle.kts lost their indentation when the
rust-android-gradle entries next to them were removed.

Stale test comment

AdvancedPublisherTest's KDoc says AdvancedPubSubTest "remains
@Ignore'd until Round 2". It is not ignored and it passes. Replaced with
what the file actually contributes: the same advanced publisher exercised
against a plain subscriber, which AdvancedPubSubTest does not cover.


Verification: ./gradlew jvmTest -PuseLocalFlatJni=true with
zenoh-flat-jni@6f81eb8 and zenoh-flat@81feb94 — 122 tests, all green
(121 before, plus the new one).

Three defects found reviewing #669, plus one stale comment.

- `ZBytes.bytes` is double-checked locking over a non-volatile field. A
  received ZBytes is materialized lazily under a lock but read outside it,
  and it routinely crosses threads (a sample piped through a Channel), so a
  reader could see the array reference published before its copied contents.
  `eager` becomes `@Volatile`; it moves out of the constructor parameter list
  because Kotlin only allows the annotation on a body property.

- `ReplyKeyExpr` had its constants swapped so that MATCHING_QUERY is
  declared first. The wire values live in `value`, so the declaration order
  buys nothing — and reordering an enum silently changes `ordinal()` and the
  order of `values()`/`entries` for every consumer. Restored, with a test
  that pins it.

- Two lines in `build.gradle.kts` lost their indentation when the
  rust-android-gradle entries next to them were deleted.

- AdvancedPublisherTest's KDoc claimed AdvancedPubSubTest "remains
  @ignore'd"; it is not ignored and it passes. Replaced with what the file
  actually adds: the same advanced publisher against a *plain* subscriber.

Verified with `./gradlew jvmTest -PuseLocalFlatJni=true` against the
zenoh-flat-jni / zenoh-flat commits CI pins: 122 tests, all green.
@milyin

milyin commented Aug 10, 2026

Copy link
Copy Markdown
Contributor Author

Reviewed the complete PR at 43b132f5f87000d0452f23f2651a5efd8bcee221.

No blocking findings.

The ZBytes change makes the existing double-checked lazy materialization valid on the JVM: the volatile write publishes the copied array contents to fast-path readers, while the synchronized path still ensures only one thread consumes and closes the native handle. Moving the cache from a private constructor property to an initialized body property does not change the public API.

ReplyKeyExpr now matches the established ANY, MATCHING_QUERY order from main, while the explicit value field and fromInt/JNI conversion preserve the flat wire mapping independently of ordinal. The added test appropriately pins the consumer-visible order. The indentation and test-documentation cleanups are correct.

All current CI checks pass, including the JVM suites on Ubuntu and macOS.

— Codex (GPT-5)

milyin added 2 commits August 10, 2026 13:03
- `com.google.guava:guava` is dead weight in `commonMain`. Nothing in this
  repo references `TypeToken`, and since #675 serialization runs through the
  pure-Kotlin `SerializationCodec`. The only `TypeToken` user is
  zenoh-flat-jni's own `io.zenoh.jni.test.Serialization`, which declares
  guava in its `jvmTest` set — so it was never ours to declare. Removed;
  `jvmTest` stays at 122 green.

- The `jvmAndAndroidMain` rationale said the source set "carries
  kotlin-reflect", but the same change removed that dependency. `typeOf<T>()`
  and `KClass.qualifiedName` resolve through `kotlin.jvm.internal.Reflection`
  in the JVM stdlib — which is a real JVM/Android-only constraint, just not
  the stated one. Corrected in both the build script and KTypeSerde's KDoc.

- `ZENOH_FLAT_TRANSITION.md` still pointed at the `ZettaScaleLabs` forks
  (README and settings.gradle.kts were already corrected in #695), listed
  constituent PRs only up to #668, described the superseded
  KType-over-JNI serializer as the approach taken, and listed advanced
  pub/sub as planned. Table synced with the branch, serializer note
  rewritten, and the remaining-work list is now what is actually left.
  Also: `prebindgen` resolves from crates.io since #692, and the error
  model has had two sink channels since #673.
The file was always meant to be deleted when this branch merges, and keeping
it current in the meantime costs a refresh on every constituent PR — the
previous commit is proof. Nothing in the repository links to it; the umbrella
PR #669 carries the same architecture, error-model and constituent-PR
descriptions in its body, which is where they stay accurate for free.
@milyin

milyin commented Aug 10, 2026

Copy link
Copy Markdown
Contributor Author

Concurrency analysis: why this is a zenoh-kotlin ZBytes fix

The fix is correct, but the callback-to-Channel example is slightly imprecise. The actual defect is the double-checked cache inside the public zenoh-kotlin wrapper:

private var eager: ByteArray?

get() = eager ?: synchronized(this) {
    eager ?: run {
        val b = handle!!.toBytes(...)
        eager = b
        ...
        b
    }
}

Thread A writes eager while holding the monitor, but thread B reads it on the fast path without acquiring that monitor. Without @Volatile, there is no guaranteed happens-before relationship between those operations. A reader may observe the published reference without being guaranteed visibility of the preceding byte-array initialization. The volatile write/read pair makes the double-checked pattern valid:

native byte copy
    -> volatile write of eager
    -> volatile read by another thread
    -> copied contents are visible

Why this relates specifically to ZBytes

It is not because only payloads cross threads. Samples, queries, and other SDK objects cross threads too. ZBytes is currently unique because it introduces a two-representation lazy state machine:

received native handle
        -> first access copies bytes and closes the handle
        -> later accesses use an unlocked ByteArray cache

Other values are generally either built eagerly as Kotlin values or retained as generated native handles whose operations use synchronized handle locking. In the current public layer, ZBytes is the only wrapper using this unsynchronized-fast-path/synchronized-initialization pattern. The rule is general, though: any future wrapper using the same pattern would require equivalent safe publication.

Why the fix belongs in the external SDK layer

There are two distinct objects:

io.zenoh.jni.bytes.ZBytes       generated native handle
                -> wrapped by
io.zenoh.bytes.ZBytes           public SDK value + lazy ByteArray cache

The public zenoh-kotlin wrapper chooses to retain the handle, defer the copy, cache the array, close the handle after extraction, and serve subsequent public operations from that cache. toBytes(), string conversion, equality, hashing, and outgoing publication all eventually use it. The layer that creates this shared cached state is therefore the layer that must publish it safely. Moving the private constructor property to a volatile body property changes no public source or binary API.

Why zenoh-flat-jni does not already solve it

The generated handle protects a different boundary. It keeps the native pointer volatile, synchronizes close() and take(), and locks the handle around toBytes(). Those guarantees prevent native handle lifecycle races and ensure an individual JNI call returns a completed ByteArray.

They cannot establish visibility for io.zenoh.bytes.ZBytes.eager, because that downstream cache does not exist in zenoh-flat-jni. Volatility attaches to a field, not to an object returned from a method, and the generated handle volatile ptr is unrelated to publication of the wrapper cache.

The generated layer could avoid this only through a broader semantic change, such as eagerly copying every callback payload or moving caching into generated handles. That would undermine the deliberately lazy hot-path policy and could reintroduce its measured throughput cost.

Qualification

A single callback-to-Channel handoff followed by access from one consumer normally safely publishes the object and does not, by itself, constitute this race. The problematic case is the same ZBytes instance being accessed by multiple threads around or after lazy initialization without another synchronization edge. The channel path explains why these objects naturally escape the callback thread, but concurrent access is the condition that makes the defective double-check observable.

The fix also does not make the returned ByteArray immutable. Concurrent caller mutation of that array remains outside its scope.

— Codex (GPT-5)

@milyin
milyin merged commit febdc68 into zenoh-flat-transition Aug 10, 2026
11 checks passed
milyin added a commit that referenced this pull request Aug 10, 2026
* chore: add branch placeholder zbobr_fix-72-make-zenoh-kotlin-depend-on-zenoh-jni-runtime

* chore: add zenoh-java as git submodule on common-jni branch

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* build: update Gradle build to depend on zenoh-jni-runtime, remove Rust wiring

- Remove :zenoh-jni from settings.gradle.kts, add gated composite build
  for zenoh-java submodule
- Remove rust-android-gradle plugin from root build.gradle.kts
- Rewrite zenoh-kotlin/build.gradle.kts: add zenoh-jni-runtime dependency,
  add jvmAndAndroidMain source set, remove all Cargo/NDK task wiring

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: move ZSerialize/ZDeserialize to jvmAndAndroidMain source set

Kotlin reflection (typeOf<T>()) is not available on Kotlin/Native or
Kotlin/JS. Moving these functions to jvmAndAndroidMain mirrors the
intentional design in zenoh-jni-runtime where JNIZBytesKotlin is also
in jvmAndAndroidMain.

Update call sites to use runtime's JNIZBytesKotlin instead of the
deleted local JNIZBytes adapter.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: delete zenoh-kotlin JNI adapter layer and duplicate shared classes

Delete all JNI adapter files (except JNILiveliness.kt which will be
adapted to use runtime's public JNISession methods):
- All callbacks in jni/callbacks/
- JNISession, JNIPublisher, JNISubscriber, JNIQuerier, etc.
- JNIZBytes (replaced by runtime's JNIZBytesKotlin)

Delete duplicate classes that conflict with zenoh-jni-runtime:
- exceptions/ZError.kt (runtime provides io.zenoh.exceptions.ZError)
- jvmMain/Target.kt (runtime provides io.zenoh.Target)
- jvmMain/Zenoh.kt and androidMain/Zenoh.kt (actual ZenohLoad impls,
  runtime provides io.zenoh.ZenohLoad)
- commonMain/Zenoh.kt expect ZenohLoad declaration will be removed next

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor: adapt Zenoh.kt, Config.kt, KeyExpr.kt, ZenohId.kt, Logger.kt to runtime API

- Zenoh.kt: remove expect ZenohLoad declaration; adapt scout methods to use
  JNIScout.scout(Int, JNIScoutCallback, JNIOnCloseCallback, JNIConfig?)
- Config.kt: loadDefault/loadFromFile/loadFromJson/loadFromYaml replace old
  methods; wrap runtime calls in runCatching where needed
- KeyExpr.kt: tryFrom/autocanonize now return String from runtime, wrap in
  runCatching { KeyExpr(...) }; intersects/includes/relationTo/join/concat
  updated to pass JNIKeyExpr? primitives explicitly
- ZenohId.kt: toStringViaJNI → toString (runtime method)
- Logger.kt: replace private external fun with JNILogger.startLogs(filter)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: remove zenoh-jni Rust crate and rust-toolchain.toml

All JNI functionality is now provided by zenoh-jni-runtime from the
zenoh-java submodule. The local Rust crate is no longer needed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* build: remove CompileZenohJNI task and cargo wiring from examples

The examples no longer need to build a local Rust JNI library since
zenoh-jni-runtime is now the JNI provider via Maven dependency.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* ci: remove Rust build steps, simplify workflows to use zenoh-jni-runtime

- ci.yml: add submodules: recursive checkout, remove cargo fmt/clippy/build steps
- publish-jvm.yml: remove 6-platform cross-compilation matrix, simplify to
  single publish job that depends on zenoh-jni-runtime via Maven
- publish-android.yml: remove NDK setup, Rust cross-compilation for Android ABIs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: correct ntpValue call, ReplyKeyExpr ordinal, and KeyExpr undeclare

- Fix ntpValue property access to ntpValue() function call in Query.kt
- Reorder ReplyKeyExpr enum so MATCHING_QUERY=0, ANY=1 to match Rust mapping
- Fix Session.undeclare(KeyExpr) to null out jniKeyExpr after undeclaring
  and return failure when already undeclared (prevents double-free SIGABRT)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: remove zenoh-jni references from bump-and-tag script and README

The zenoh-jni Rust crate was deleted; update the release automation script
to stop editing the now-nonexistent Cargo.toml and update README to reflect
that zenoh-kotlin now depends on zenoh-jni-runtime rather than building its
own native JNI library.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: decouple publish from local submodule, add separate runtime version property

- Gate includeBuild("zenoh-java") behind zenoh.useLocalJniRuntime property so
  publication resolves against the published Maven artifact, not a local submodule build
- Add zenohJniRuntimeVersion in gradle.properties to independently track the
  zenoh-jni-runtime release version, decoupling it from zenoh-kotlin's own version.txt
- Remove submodule checkout and Rust toolchain from publish-jvm and publish-android
  workflows; publish path no longer needs them
- Pass -Pzenoh.useLocalJniRuntime=true in ci.yml test step so CI still builds against
  the local submodule, and remove now-unnecessary Rust toolchain install step

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs: update README to distinguish Maven vs local submodule test modes

Clarify that the default build/test path resolves zenoh-jni-runtime from
Maven and requires no Rust toolchain, while the opt-in local submodule
path (-Pzenoh.useLocalJniRuntime=true) builds from source and does require
one. Fixes the documentation to match the actual opt-in build wiring.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: use runtime acceptReplies value in Query and fail-fast on selector params

- Query now stores acceptReplies from the JNI callback instead of inferring from selector parameters
- resolveQueryable passes ReplyKeyExpr.entries[acceptReplies] to Query constructor
- Selector param parsing now uses getOrThrow() so malformed data surfaces as a failure

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: fail fast when useLocalJniRuntime=true but submodule is absent

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: remove stale publish-crates-github job from release workflow

zenoh-kotlin no longer contains any Rust crates (zenoh-jni/ directory
was removed as part of the migration to zenoh-jni-runtime). The
publish-github job that invoked publish-crates-github@main is now
obsolete and would fail at release time.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update zenoh-java submodule reference

Update to latest common-jni commit with markdownlint fix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update zenoh-java submodule to eclipse-zenoh/zenoh-java:common-jni HEAD

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test fix attempt

* Open the zenoh-flat transition integration branch

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Record PR #668 in the transition table

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Port zenoh-kotlin to zenoh-flat-jni generated bindings (#668)

* Port zenoh-kotlin to zenoh-flat-jni generated bindings

Replace the external zenoh-jni-runtime dependency (zenoh-java common-jni,
git submodule + conditional composite build) with the generated zenoh-flat-jni
bindings, consumed via an unconditional Gradle composite build against a
sibling checkout — the architecture proven by zenoh-java's
zenoh-flat-transition branch.

The flat bindings never throw: every fallible generated wrapper takes a
trailing error sink and returns onError.run(...) on failure. Since
zenoh-kotlin's public API is Result-based, errors become Result.failure
directly inside the sink (exceptions/ResultHandlers.kt zCall* helpers,
born-closed sentinel handles on the error path) — zero try/catch and zero
runCatching on the JNI path.

Value models follow zenoh-java #484-#489: Encoding stays a pure JVM value
with the (id, schema) selector block; KeyExpr is string-backed except
declared handles; received ZBytes materialize lazily; ZenohId renders via the
shared-tier zidString codec; callbacks arrive value-decomposed in one JNI
crossing (FlatCallbacks.kt + fromParts factories). Query.reply* now closes
the native query (leak fix), Config.fromJson5 actually parses JSON5 (bugfix),
CongestionControl.BLOCK_FIRST added for receive-path totality.

Advanced pub/sub is not yet exposed by zenoh-flat/zenoh-flat-jni: public
signatures are kept but Session.declareAdvanced* return Result.failure and
AdvancedPubSubTest is disabled. zSerialize/zDeserialize bridge KType to
java.lang.reflect.Type (top-level primitives boxed); the Kotlin-specific
unsigned/Pair/Triple types are disabled pending a KType-aware serializer in
the shared tier.

CI mirrors zenoh-java's: sibling checkouts of zenoh-flat-jni@dbb1f8c and
zenoh-flat@6d22091 (same pins), prebindgen from git main, cargo build, then
gradle jvmTest. 113 tests, 0 failed, 12 skipped by design.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Querier: invalid-querier error is a Result.failure, not a throw

performGet returns Result<R>, but an invalid (undeclared) querier threw an
unchecked ZError — a pre-existing inconsistency carried over from the old
runtime-based code. Align it with the Result-based API: Publisher and Session
already report their closed-handle state as Result.failure.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Preserve the Result contract for JVM-side exceptions

The zCall* helpers captured only native errors (reported via the sink) and
let exceptions thrown by the block itself escape from Result-returning
functions — unlike main, whose runCatching also converted argument
preparation, user IntoZBytes.into() conversions, and native-library loading
failures into Result.failure. Concretely, scout(whatAmI = emptySet()) threw
from reduce despite returning Result.

Run each helper block inside runCatching (the sink capture takes precedence
and is still exception-free for native errors), move scout's whatAmI
reduction inside the captured block, make the zDeserialize cast mapCatching,
rebuild KeyExpr.fromProbe on zCall, and add a regression test for the
empty-whatAmI case. ZENOH_FLAT_TRANSITION.md now states the two-channel
contract precisely.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Session.undeclare: detach the key-expr handle even when undeclare fails

The generated wrapper consumes the handle even when the native undeclare
errors (the Rust side takes it by value), and on a pre-call guard failure the
handle instead stays live. Clearing jniKeyExpr only on success left a dead
handle attached: every later operation selected the closed handle and failed
instead of degrading to the string form. Close (no-op if consumed) and detach
unconditionally; regression-tested by undeclaring through the wrong session.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* RX path: never fault on attacker-controlled selector parameters

In the Rust layer a selector's parameters are an unvalidated string view —
any string is valid and forwarded untouched. The Kotlin queryable path
instead parsed them with the strict Parameters.from, whose failures
(duplicated names, invalid percent-encoding — both trivially producible by
any remote client) threw inside the JNI upcall: the exception is swallowed by
the callback bridge, but the query's owned payload/attachment buffers (no GC
backstop) leaked and the query was silently dropped — a remotely repeatable
leak, strictly weaker than the Rust layer.

Parse leniently on receive (Parameters.fromLenient: first duplicate wins,
split on the first '=', undecodable values kept verbatim), and add
defense-in-depth to queryCallbackOf: if decomposition ever throws, free the
owned native leaves and finalize the query before rethrowing. E2E-tested by
sending 'a=1;a=2;bad=%zz' through the raw bindings, as a remote non-Kotlin
client would.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* Parameters becomes a thin facade over the shared string-backed implementation (#670)

* Parameters becomes a thin facade over the shared string-backed implementation

The per-SDK map-backed parser (strict from + fromLenient) duplicated
zenoh-java's and diverged from Rust: it percent-decoded values, rejected
duplicated keys, and normalized eagerly. io.zenoh.query.Parameters now
delegates every operation to the shared io.zenoh.jni.query.Parameters
(zenoh-flat-jni), a pure-Kotlin string-backed mirror of Rust's
zenoh-protocol parameters.rs — construction is infallible on any (remote,
attacker-controlled) input with zero JNI crossings.

Behavior changes, all aligning with Rust:
- values are no longer percent-decoded (the URL-encoding claim was a
  zenoh-jni legacy; docs updated),
- from(String) never fails (the Result signature is kept for source
  compatibility and is always success); duplicated keys are accepted with
  first-match-wins get,
- toString round-trips the stored string verbatim; insert/remove normalize,
- equality is string equality.

fromLenient is deleted (superseded — the shared parse is inherently total).
CI pins bump to zenoh-flat#4 (native parameters oracle) and zenoh-flat-jni#9
(shared implementation + oracle declarations).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Bump zenoh-flat-jni pin: trailing-separator trim in shared Parameters

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Bump pins: parameters API reworked as regular zenoh-flat API

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci: pin zenoh-flat to merged main (parameters API, zenoh-flat#4)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* Migrate to the split error-handler API (zenoh-flat-jni #45) (#673)

* Migrate to the split error-handler API (zenoh-flat-jni #45)

prebindgen split the generated JNI error callback into two channels: the
binding `JniErrorHandler.run(je)` (unchanged) and the typed domain
`ErrorHandler.run(message)` (`je` removed, called only on a domain `Err`).
A fallible flat wrapper now takes both — `onBindingError` then `onError`.

`ResultHandlers.kt`:
- `zCall` / `zCallUnit` blocks change from `(ErrorHandler<T>) -> T` to
  `(JniErrorHandler<T>, ErrorHandler<T>) -> T`; the helper supplies both
  handlers (each records into the shared `err` local; both produce the
  sentinel where the return type demands one).
- `throwZError` becomes the 1-arg domain handler
  (`ErrorHandler { message -> throw ZError(message) }`).
- `zCall0` / `zCallUnit0` / `throwZError0` (binding-only) are unchanged.

Call sites: every `zCall`/`zCallUnit` block threads both handlers into its
flat wrapper call across Config/Session/Zenoh/KeyExpr/Liveliness/Publisher/
Querier/Query; `KeyExpr.fromProbe` gains the binding param and its callers
(`join`/`concat`/`tryFrom`/`autocanonize`/`withHandle`) thread both. The two
direct `throwZError` passes (`KeyExpr.withHandle`, a `commonTest` reply) gain
`throwZError0`. Binding-only calls (`zCall0`/`throwZError0`/`newClone`/algebra
ops) are untouched.

Builds against the local composite zenoh-flat-jni (split-error-handler);
117 jvmTest tests pass (12 skipped), including the Result error paths.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci: repoint zenoh-flat-jni pin to the split regen + ownership marker

The split error-handler regeneration (zenoh-flat-jni #10) plus the merged
Kotlin ownership-marker fix (zenoh-flat-jni #11, 249fe9e on shared-parameters).
CI pinned the pre-split 757cc6a, whose single-channel
`ErrorHandler.run(je, message)` wrappers are incompatible with this branch's
two-caller `zCall`/`zCallUnit` sites; #11's marker is also required for the
composite `cargo build` to regenerate at all (write_kotlin refuses a non-empty
output root without it). Point CI at the merged commit so the composite build
both matches the source and regenerates.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* Serialize via the pure-Kotlin SerializationCodec (no JNI) (#675)

* Enable unsigned + Pair/Triple serialization via the KType path

zenoh-flat-jni now exposes a KType-aware serializer. Switch the
zSerialize/zDeserialize bridge from the erased `.javaType`
(`serializeViaJNI`/`deserializeViaJNI`) to passing the full `KType`
(`serializeViaJNIKType`/`deserializeViaJNIKType`), so `UByte`/`UShort`/
`UInt`/`ULong`/`Pair`/`Triple` work. Delete the `javaBoxedType()` bridge
(the KType is passed directly now) and the transition TODOs; update the
supported-types KDoc.

Un-`@Ignore` the 9 ZBytesTest cases (unsigned/Pair/Triple/nested). Full
jvmTest: 117 pass (advanced pub/sub still the only skipped suite).

Depends on the zenoh-flat-jni KType-serializer commit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci: bump zenoh-flat-jni pin to the KType-serializer commit

The unsigned/Pair/Triple serialization enabled here needs zenoh-flat-jni's
new KType externs (serializeViaJNIKType/deserializeViaJNIKType, ZettaScaleLabs/
zenoh-flat-jni#12, 9503f73). Point CI at that commit so the composite build
has the KType serializer.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Serialize via the pure-Kotlin SerializationCodec (no JNI)

Replace the per-element JNI serialization (serializeViaJNIKType) with the shared
pure-Kotlin SerializationCodec. `serdeTypeOf(KType)` (jvmAndAndroidMain,
kotlin-reflect) builds a SerializationCodec.SerdeType from the KType classifier —
recognizing the unsigned value classes and Pair/Triple — and zSerialize/
zDeserialize call the codec through the SAME zCall0 error-sink wiring used for
generated wrappers (the codec takes a JniErrorHandler and never throws), so the
hand-written serializer is indistinguishable from a generated one at the call site.

Adds SerializationCorrespondenceTest (Parameters-style): the pure output is
asserted byte-identical to the native serializeViaJNIKType oracle across scalars,
unsigned, strings/bytes, containers, tuples and nested types, plus RFC golden
vectors, plus a perf comparison. Measured pure-Kotlin speedup on small payloads:
~156x (Int), ~320x (List<Int>(4)), ~279x (Map<String,Int>(2)). Full jvmTest passes.

Depends on the zenoh-flat-jni SerializationCodec commit. The native KType
serializer (serializeViaJNIKType) is kept temporarily as the correspondence oracle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci: bump zenoh-flat-jni pin to the pure-Kotlin serializer commit

The pure-Kotlin SerializationCodec this branch delegates to lives in
zenoh-flat-jni#13 (db4fb2d). Point CI at it so the composite build has the
shared codec.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci: repoint zenoh-flat-jni pin to the rebased serializer commit

The zenoh-flat-jni serialization PR was rebased onto main (conflict resolution),
changing the SerializationCodec commit SHA. Repoint CI at the current commit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Drop SerializationCorrespondenceTest moved down to zenoh-flat-jni

The serialization correspondence test (pure SerializationCodec vs the native
serializer oracle) moved into zenoh-flat-jni's own test suite. The native
serializer externs relocated to the internal io.zenoh.jni.test package; SDK
production uses the pure io.zenoh.jni.bytes.SerializationCodec, unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci: bump zenoh-flat-jni pin to the test-package/self-verify commit

zenoh-flat-jni moved its native oracle to the internal io.zenoh.jni.test
package and added self-verifying correspondence tests. Point CI at that commit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci: bump zenoh-flat-jni pin to merged main (71736f2)

zenoh-flat-jni #13 squash-merged to main as 71736f2. Re-pin from the
pre-merge branch commit 2755c06 (an orphan once the pure-kotlin-serde
branch is deleted) to the permanent main commit, which also carries the
final merged SerializationCodec (strict UTF-8).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* Add Gradle wrapper (8.12.1), like zenoh-java (#676)

zenoh-kotlin had no committed Gradle wrapper (gradlew/gradle-wrapper.jar
were gitignored), so CI installed gradle via setup-gradle's gradle-version
pin and the publish workflows generated a throwaway wrapper with
`gradle wrapper` before calling ./gradlew. Commit the wrapper (copied
verbatim from zenoh-java, Gradle 8.12.1) and drop those workarounds:

- .gitignore: stop ignoring gradle/, gradlew, gradlew.bat (keep .gradle cache).
- Add gradlew, gradlew.bat, gradle/wrapper/{gradle-wrapper.jar,.properties}.
- Workflows: drop `gradle-version: 8.12.1` (wrapper supplies it) and the
  `gradle wrapper` generation steps; invoke ./gradlew everywhere (ci.yml and
  publish-dokka switched from bare `gradle`). Matches zenoh-java's CI.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* Advanced pub/sub: AdvancedPublisher/Subscriber, matching + sample-miss listeners (#678)

* Enable advanced publisher + matching listener (Round 1)

Un-stub AdvancedPublisher and MatchingListener over the generated
io.zenoh.jni.pubsub bindings from zenoh-flat-jni's advanced-pubsub branch.

- Session.declareAdvancedPublisher now calls resolveAdvancedPublisher, lowering
  the pure-Kotlin config holders (MissDetectionConfig heartbeat mode, CacheConfig
  maxSamples, publisherDetection) to the generated scalar params.
- AdvancedPublisher: put/delete/getMatchingStatus and matching listeners
  (callback/handler/channel + background variants) over the never-throw sink
  (zCall/zCallUnit), mirroring the regular Publisher.
- MatchingListener wraps the generated handle (null for background listeners).
- AdvancedPublisherTest (Round 1): advanced publisher + matching listener vs a
  regular subscriber on a single loopback session — put/delete round-trip,
  matching status, encoding fallback. 4 tests pass.

The advanced subscriber / sample-miss / detect-publishers stubs and the full
AdvancedPubSubTest remain for Round 2. Requires zenoh-flat + zenoh-flat-jni
advanced-pubsub branches.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Enable advanced subscriber + sample-miss (Round 2)

Un-stub the subscriber side of advanced pub/sub over the generated
io.zenoh.jni.pubsub bindings from zenoh-flat-jni (advanced pub/sub, #14), and
re-align the Round 1 publisher lowering to the current typed data-class holders.

- Session: declareAdvancedSubscriber (callback/handler/channel) now wires a new
  private resolveAdvancedSubscriber, mirroring resolveSubscriber (zCall +
  sampleCallbackOf + strong-ref registration). HistoryConfig / RecoveryConfig /
  subscriberDetection lower to the generated HistoryConfig / RecoveryConfig
  holders (queryTimeout is not surfaced). resolveAdvancedPublisher is re-aligned
  to build the generated MissDetectionConfig / CacheConfig(RepliesConfig) holders
  — #14 replaced the old scalar params — which also restores CacheConfig.repliesQoS
  (Round 1 dropped it). Removed the advancedUnsupported stub error.
- AdvancedSubscriber: carries the JNI handle; the 12 declared methods fold into
  two private resolvers (detect-publishers → Subscriber, sample-miss →
  SampleMissListener), foreground/background as in Round 1's matching listener.
- SampleMiss is modernized to `SampleMiss(source: EntityGlobalId, missedCount)`
  (was four raw Longs), matching how Sample.sourceId already exposes the identity;
  FlatCallbacks gains a sampleMissCallbackOf bridge (generated ZenohId → SDK
  EntityGlobalId). SampleMissListener mirrors MatchingListener.
- AdvancedPubSubTest: removed @ignore; initialize receivedSamples before the
  subscriber that appends to it; sleep for publisher/subscriber-detection
  propagation.
- CI: pin zenoh-flat-jni -> d57837a (main, #14) and zenoh-flat -> b6b0ecf
  (main, #5 advanced pub/sub).

Verified against zenoh-flat-jni main + zenoh-flat main (composite build,
Gradle 8.12.1): AdvancedPubSubTest 3/3 + AdvancedPublisherTest 4/4 pass; full
jvmTest 121/121 (no regressions).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Rename SampleMiss data type to Miss (follow zenoh naming)

zenoh-flat renamed its sample-miss report struct to `Miss` (zenoh's own name)
carrying an `EntityGlobalId` source. Track that in the SDK data type.

- pubsub/SampleMiss.kt -> pubsub/Miss.kt: `data class Miss(source:
  EntityGlobalId, nb: Long)` (field missedCount -> nb, matching zenoh_ext::Miss).
- FlatCallbacks.sampleMissCallbackOf now returns the generated `MissCallback` and
  reads the nested `miss.source.zid`/`miss.source.eid`.
- The SampleMiss* handler/callback surface keeps its names (SampleMissListener is
  zenoh's own name); only the payload type changes to `Miss`.

The rename stays a data-type change: declareSampleMissListener and the listener
class are unchanged. Compiles; AdvancedPubSubTest 3/3 + AdvancedPublisherTest 4/4
+ full jvmTest 121/121 pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* Realign with zenoh-flat HEAD; a timestamp carries its clock's id

zenoh-flat renamed part of its surface and moved several handle types to
value forms (ZettaScaleLabs/zenoh-flat-jni#16). Mostly mechanical, with one
change that reaches the public API. Mirrors the same realignment in
zenoh-java.

## A reply timestamp needs a real originating-node id

`Query.reply(timestamp = ...)` took an Apache commons-net `TimeStamp`: an
NTP64 instant and nothing else. Zenoh's timestamp is the pair `(ntp64, id)`,
and it orders and de-duplicates on the pair — so the missing half was supplied
by the JNI layer, which built one with a **random** id:

    Timestamp::new(NTP64(timestamp_ntp_64 as u64), ID::rand())   // zenoh-jni/src/query.rs:82

The time looked right while the id was untethered from any node. zenoh-flat
removed that (ZettaScaleLabs/zenoh-flat#47) and `query_reply_success` now takes
a whole `Timestamp`, so there is no longer anywhere for a fabricated id to
come from — which is the point.

`io.zenoh.time.Timestamp` is that pair, mirroring `zenoh::time::Timestamp`.
The reply/replyDel `timestamp` parameters and `Sample.timestamp` carry it
instead of the commons-net type. **This is source-breaking**: a caller now
writes

    query.reply(keyExpr, payload, timestamp = Timestamp.ofNtp64(ntp64, zid))

`ofNtp64` exists because the primary constructor takes `ULong`, which Java
cannot express; it takes the same 64 bits `TimeStamp(long)` does, so
`TimeStamp.getCurrentTime().ntpValue()` still feeds it. The id is the
replying session's — the reply does originate here — and `Sample.timestamp`
now surfaces the sender's id, which was previously discarded on receive.

The library no longer depends on commons-net; the tests and the ZQueryable
example still use it, as an NTP64 clock.

## Mechanical

* `keyexpr_get_str` -> `asStr`, `zbytes_as_bytes` -> `toBytes`.
* `session_get` takes a whole `Selector`, so the folded (key_expr, parameters)
  pair is gone. A Selector holds an owned key-expr handle with no string arm,
  so `KeyExpr.intoJniHandle()` materializes one for a string-backed key
  expression — the slot trio cannot express this case.
* An encoding schema is raw bytes: zenoh transmits it verbatim and does not
  require UTF-8. This SDK's `Encoding` carries a String, so it is encoded at
  the boundary on send and decoded lossily on receive, rather than throwing on
  a received message.
* A sample's source and a reply's replier arrive as whole `SourceInfo` /
  `EntityGlobalId` values, so optionality lives on the value rather than on a
  leading zid leaf.
* `RecoveryMode` is a data-carrying enum, so the choice and its payload are one
  value rather than a `(period, flag)` pair in which only one of the two was
  ever meaningful. `RecoveryConfig.retentionPeriod` has no counterpart on the
  SDK config and stays absent, as it effectively was before.
* `config_new_from_json` is gone as an invented constructor (base zenoh has no
  `from_json`). `Config.fromJson` parses via JSON5, of which JSON is a subset,
  so every input accepted before still parses to the same config.

CI pins move to zenoh-flat 3f431b6b and zenoh-flat-jni 5e0ae509.

121 JVM tests pass; examples compile.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* CI: pin zenoh-flat-jni to the merged realignment

ZettaScaleLabs/zenoh-flat-jni#16 merged as 498ba26, so the pin moves off the
PR branch commit it was tracking.

The squash merge's tree is byte-identical to the branch tip this was verified
against, and 498ba26 pins zenoh-flat at the same 3f431b6b already pinned here,
so nothing but the SHA changes. 121 JVM tests pass against it; examples
compile.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* CI: build zenoh-flat-jni against published prebindgen (#692)

* CI: track zenoh-flat-jni onto the split prebindgen crates

zenoh-flat-jni#23 replaces its single `prebindgen` dependency with the
three crates the #371 split produced, so this workflow's two hardcoded
per-crate rewrites no longer match anything: the path deps would survive
into a checkout that has no sibling `../prebindgen`. One regex over every
`../prebindgen/*` dep covers the split and whatever it grows next.

The pinned checkouts also move from the stale ZettaScaleLabs forks (last
pushed in July) to the eclipse-zenoh repositories the work actually lives
in, at the two open PR heads.

No Kotlin change: zenoh-flat-jni regenerates byte-identical Kotlin over
the split, and all 121 jvmTest cases pass against it locally.

* CI: drop the prebindgen path rewrite, it is published

prebindgen 0.5.0 is on crates.io, so zenoh-flat-jni and zenoh-flat
declare version constraints and nothing has to be rewritten before the
build. Re-pin both checkouts to the PRs that make that switch.

* CI: re-pin zenoh-flat to main, zenoh-flat-jni to zenoh-flat-jni#24

eclipse-zenoh/zenoh-flat#80 merged as 81feb94. zenoh-flat-jni#25 landed
on the #24 branch rather than main, so that checkout tracks #24 now.

* CI: re-pin zenoh-flat-jni to main

eclipse-zenoh/zenoh-flat-jni#24 merged as 6f81eb8; both sibling
checkouts now sit on their repositories' main.

* Release preparation: repair the release path and document publishing (#695)

* Release preparation: repair the release path and document publishing

zenoh-kotlin no longer contains Rust, but its release path still assumed it
did, and its dependency on zenoh-flat-jni was unreleasable as written.

- Resolve zenoh-flat-jni from Maven Central by default, at a version held in
  gradle.properties, instead of hardcoding 1.9.0 and always substituting a
  sibling checkout. The composite build becomes opt-in (-PuseLocalFlatJni=true);
  CI passes it explicitly, a release must not.
- Declare the Central snapshot repository conditionally, so a rehearsal can
  build against a zenoh-flat-jni snapshot while a release provably cannot.
- Merge publish-jvm.yml and publish-android.yml into one publish.yml: both
  coordinates now come from a single Gradle invocation, so they share one
  staging repository and are released together or not at all.
- Teach bump-and-tag.bash the zenoh-flat-jni version, refusing a snapshot on a
  live run; drop the BUMP_DEPS_* inputs, dead since the Cargo crate was removed.
- Add PUBLISHING.md, and correct the README where it described the composite
  build as unconditional.

* Correct the JVM coordinate in README.md

The publications set artifactId explicitly, so the JVM one is zenoh-kotlin,
not the zenoh-kotlin-jvm that plain Kotlin Multiplatform naming would give.
Every JVM copy/paste snippet named the latter, which does not exist.

* Review fixes for the zenoh-flat transition (#696)

* Review fixes: ZBytes publication race, ReplyKeyExpr order, build script

Three defects found reviewing #669, plus one stale comment.

- `ZBytes.bytes` is double-checked locking over a non-volatile field. A
  received ZBytes is materialized lazily under a lock but read outside it,
  and it routinely crosses threads (a sample piped through a Channel), so a
  reader could see the array reference published before its copied contents.
  `eager` becomes `@Volatile`; it moves out of the constructor parameter list
  because Kotlin only allows the annotation on a body property.

- `ReplyKeyExpr` had its constants swapped so that MATCHING_QUERY is
  declared first. The wire values live in `value`, so the declaration order
  buys nothing — and reordering an enum silently changes `ordinal()` and the
  order of `values()`/`entries` for every consumer. Restored, with a test
  that pins it.

- Two lines in `build.gradle.kts` lost their indentation when the
  rust-android-gradle entries next to them were deleted.

- AdvancedPublisherTest's KDoc claimed AdvancedPubSubTest "remains
  @ignore'd"; it is not ignored and it passes. Replaced with what the file
  actually adds: the same advanced publisher against a *plain* subscriber.

Verified with `./gradlew jvmTest -PuseLocalFlatJni=true` against the
zenoh-flat-jni / zenoh-flat commits CI pins: 122 tests, all green.

* Address the #669 review: drop dead guava, refresh the transition doc

- `com.google.guava:guava` is dead weight in `commonMain`. Nothing in this
  repo references `TypeToken`, and since #675 serialization runs through the
  pure-Kotlin `SerializationCodec`. The only `TypeToken` user is
  zenoh-flat-jni's own `io.zenoh.jni.test.Serialization`, which declares
  guava in its `jvmTest` set — so it was never ours to declare. Removed;
  `jvmTest` stays at 122 green.

- The `jvmAndAndroidMain` rationale said the source set "carries
  kotlin-reflect", but the same change removed that dependency. `typeOf<T>()`
  and `KClass.qualifiedName` resolve through `kotlin.jvm.internal.Reflection`
  in the JVM stdlib — which is a real JVM/Android-only constraint, just not
  the stated one. Corrected in both the build script and KTypeSerde's KDoc.

- `ZENOH_FLAT_TRANSITION.md` still pointed at the `ZettaScaleLabs` forks
  (README and settings.gradle.kts were already corrected in #695), listed
  constituent PRs only up to #668, described the superseded
  KType-over-JNI serializer as the approach taken, and listed advanced
  pub/sub as planned. Table synced with the branch, serializer note
  rewritten, and the remaining-work list is now what is actually left.
  Also: `prebindgen` resolves from crates.io since #692, and the error
  model has had two sink channels since #673.

* Remove ZENOH_FLAT_TRANSITION.md

The file was always meant to be deleted when this branch merges, and keeping
it current in the meantime costs a refresh on every constituent PR — the
previous commit is proof. Nothing in the repository links to it; the umbrella
PR #669 carries the same architecture, error-model and constituent-PR
descriptions in its body, which is where they stay accurate for free.

* CI: pin zenoh-flat-jni to the timestamp node-id fix (#698)

zenoh-flat carried a timestamp's node id trimmed to its significant
bytes while `ZenohId` carries the full zero-padded width, so a received
sample's `timestamp.id` compared unequal to the session zid that stamped
it whenever the identifier had a high-order zero byte - about once in
256 sessions, both sides rendering identically. That is the flaky
`QueryableTest.queryable_runsWithCallback` failure; the visible
difference in its message, the encoding, is cosmetic (`Encoding.equals`
ignores the description).

Fixed in eclipse-zenoh/zenoh-flat#86 and picked up by
eclipse-zenoh/zenoh-flat-jni#35, which this pins.

Drop the zenoh-flat checkout: nothing reads it, since zenoh-flat-jni
resolves zenoh-flat from git and its Cargo.lock is the rev that decides
- a pin that pinned nothing while reading as if it did.

* CI: track zenoh-flat-jni main, and let Gradle drive its native build (#699)

* CI: let Gradle drive the zenoh-flat-jni native build

The workflow added rustfmt and clippy to a hardcoded 1.93.0 toolchain
and ran `cargo build` in the zenoh-flat-jni checkout. Neither is needed
here: zenoh-flat-jni pins its own toolchain in rust-toolchain.toml, and
the composite build's test task already depends on its native build, so
Gradle drives cargo. Verified by deleting the built dylib and running
`jvmTest`, which rebuilt it and passed.

Naming a toolchain version this repo does not own is also how the same
step broke in zenoh-java: the components landed on 1.93.0 while the
checks ran on the pinned 1.97.1. `rustup show`, run from the
zenoh-flat-jni directory, installs whatever that repo pins.

* CI: pin zenoh-flat-jni to the merged commit

#698 pinned the PR branch commit, which is reachable but not on main.
eclipse-zenoh/zenoh-flat-jni#35 has since merged as e75529c with an
identical tree, so this only makes the pin name a commit that main
actually carries.

* CI: track zenoh-flat-jni main instead of a pinned commit

A pinned SHA has to be hand-edited for every upstream fix, and that hop -
zenoh-flat-jni to this SDK - is the one no bot covers. The timestamp
node-id flake is what that costs: the fix sat in zenoh-flat for a day
while CI kept testing the commit the pin named.

Below zenoh-flat-jni the chain is automatic (eclipse-zenoh/ci#465 puts it
and zenoh-flat on the lockfile sync), so following its default branch
makes the whole chain automatic. The trade is deliberate: a run is no
longer reproducible from this repository's commit alone, and a broken
zenoh-flat-jni main breaks CI here - which, for a branch whose entire
purpose is to track those bindings, is the signal we want.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant