Skip to content

sync: SVS v4 (mhash, PARTIAL, publish+pull) #190

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Taranum01 wants to merge 17 commits into
base: psvs
Choose a base branch
from
Open

sync: SVS v4 (mhash, PARTIAL, publish+pull) #190

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
b88a364
sync: SVS v4 large-group sync (mhash, PARTIAL, publish+pull)
Jul 21, 2026
fe5586b
test: add SVS v4 unit tests
Jul 21, 2026
e9522cc
docs: SVS v4 specification
Jul 21, 2026
1002285
fix(e2e): use AIMD congestion window in segment fetcher
Jul 22, 2026
5d8b1be
sync: address review comments on SVS v4
Jul 22, 2026
9340c00
object: bump metadata fetch retries 3 to 5 and lifetime 1s to 2s
Jul 22, 2026
56c2af0
docs+code: address easy review comments
Jul 24, 2026
67d93c2
docs+code: reword 'X not Y' anti-patterns
Jul 24, 2026
aed3420
sync: tighten v4 wire-format validation
Jul 24, 2026
7f04d17
sync: drop announce wording from SVS internals
Jul 24, 2026
d27e022
fix(e2e): retry ExpressR on transient NackReasonNoRoute/Congestion
Jul 24, 2026
1872c64
docs(svs-v4): address spec review comments
Jul 24, 2026
5f3a477
sync: address v4 review comments on PARTIAL selection and SvsData flow
Jul 24, 2026
ea79483
sync: replace VectorType with distinct FullStateVector/PartialStateVe…
Jul 24, 2026
e6ce78e
lint+refactor: fix goimports and rename svs v3 package to v4
Jul 24, 2026
bbce410
e2e: bump DV convergence + put-then-cat windows; revert client retry …
Jul 24, 2026
a9a0de7
sync: tighten direct-form exclusivity and hoist mhash in PARTIAL encode
Jul 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
463 changes: 463 additions & 0 deletions docs/svs-v4.md
View file
Open in desktop
Comment thread
a-thieme marked this conversation as resolved.

Large diffs are not rendered by default.

21 changes: 21 additions & 0 deletions std/ndn/svs/v3/definitions.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,22 @@ package svs

import (
enc "github.com/named-data/ndnd/std/encoding"
"github.com/named-data/ndnd/std/types/optional"
)

// VectorType values for inline SvsData (TLV 0xCD).
const (
VectorTypeFull uint64 = 0
VectorTypePartial uint64 = 1
)

type SvsData struct {
//+field:binary:optional
MemberSetHash []byte `tlv:"0xcb"`
//+field:natural:optional
VectorType optional.Optional[uint64] `tlv:"0xcd"`
//+field:name
SvsDataRef enc.Name `tlv:"0x07"`
//+field:struct:StateVector
StateVector *StateVector `tlv:"0xc9"`
}
Expand All @@ -29,6 +42,14 @@ type SeqNoEntry struct {
SeqNo uint64 `tlv:"0xd6"`
}

// MembershipTuple is one (Name, BootstrapTime) pair used to compute MemberSetHash.
type MembershipTuple struct {
//+field:name
Name enc.Name `tlv:"0x07"`
//+field:natural
BootstrapTime uint64 `tlv:"0xd4"`
}

// +tlv-model:nocopy
type PassiveState struct {
//+field:sequence:[]byte:binary:[]byte
Expand Down
261 changes: 261 additions & 0 deletions std/ndn/svs/v3/zz_generated.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions std/object/client_consume.go
View file
Open in desktop

@a-thieme a-thieme Jul 20, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was 3 retries not enough in testing?

@Taranum01 Taranum01 Jul 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — and 3 isn't quite enough on the 53-node sprint topology. After this thread, the e2e test in CI started failing intermittently with 'fetch metadata failed with result: Timeout' (4.5s after the cat fired). Locally it passes reliably because DV's startup converges quickly, but CI's slower startup stretches the window in which a DV startup-Reset can wipe a producer prefix from a consumer's RIB just after the convergence check returns.

So in a follow-up commit (9340c00), I bumped fetchMetadata and fetchDataByPrefix to Retries: 5 and Lifetime: 2s (~10s budget). That gives the consumer enough slack for DV to finish its post-startup re-announce cycle. Verified locally: all three scenarios (NDNd, NFD, NDNd-replay) pass. Verified in CI: 9340c00 is green.

The seg fetcher (client_consume_seg.go) is still at maxRetries: 3 because the segment-fetch flake that originally motivated the bump is solved by the AIMD congestion window in 1002285, not by retry budget.

@a-thieme a-thieme Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's a DV convergence issue, the CI should be updated have a long enough wait time for convergence, instead of modifying the underlying library.

Original file line number Diff line number Diff line change
Expand Up @@ -117,9 +117,9 @@ func (c *Client) fetchMetadata(
Config: &ndn.InterestConfig{
CanBePrefix: true,
MustBeFresh: true,
Lifetime: optional.Some(time.Millisecond * 1000),
Lifetime: optional.Some(time.Millisecond * 2000),
},
Retries: 3, // TODO: configurable
Retries: 5, // TODO: configurable (sprint e2e needs ~10s budget for DV startup reset storm)
TryStore: utils.If(tryStore, c.store, nil),
Callback: func(args ndn.ExpressCallbackArgs) {
if args.Result == ndn.InterestResultError {
Expand Down Expand Up @@ -174,9 +174,9 @@ func (c *Client) fetchDataByPrefix(
Config: &ndn.InterestConfig{
CanBePrefix: true,
MustBeFresh: true,
Lifetime: optional.Some(time.Millisecond * 1000),
Lifetime: optional.Some(time.Millisecond * 2000),
},
Retries: 3, // TODO: configurable
Retries: 5, // TODO: configurable (sprint e2e needs ~10s budget for DV startup reset storm)
TryStore: utils.If(tryStore, c.store, nil),
Callback: func(args ndn.ExpressCallbackArgs) {
if args.Result == ndn.InterestResultError {
Expand Down
Loading
Loading