Repository navigation
Developer Experience with CBOR on the wire #19
Description
Activity
WG discussion:
- Notation of CBOR encoded structures #34 and Developer Experience with CBOR on the wire #19 both needed, and should just be tried out.
- @MasterKale volunteered to help with this and review this approach
@c2bo @MasterKale I am wondering if you could provide examples for the CBOR / CDDL used in #29 and check what works best?
Please post the examples in this Issue for discussion/review by WG.
@c2bo @MasterKale I am wondering if you could provide examples for the CBOR / CDDL used in #29 and check what works best?
That was my proposal in last call -> that is definitely the plan. I am on vacation this week, so i am not sure if i will make progress this week from my side.
Here's an initial exploration at combining RFC 9165's
.featureto define a helper notation to colocate type annotations between CBOR and JSON encodings, with RFC 9741 for some JSON-friendler datatypes.BTW, since the various structures in #29 are defined for CBOR as the initial/"default" encoding scheme, I think the "
JC<>" notation for expressing cross-encoding datatypes should probably be "CJ<>" instead. Accordingly,CJ<..., ...>is used below to indicate "this datatype in a CBOR domain, this other datatype in a JSON domain":Examples
Using CDDL's
.featureto define a "CBOR type, JSON type"...typeCJ<C,J> = C .feature "cbor" / J .feature "json"Values using CDDL datatype primitives get the
CJ<>treatmentlabel = CJ<int, int> / CJ<tstr,tstr> values = CJ<any, any> payload = CJ<bstr, b64u> empty_or_serialized_map = CJ<bstr .cbor header_map, TODO> / CJ<bstr .size 0, TODO>Values referring to "complex" CDDL datatypes don't get the
CJ<>treatmentreader_auth = COSE_Sign COSE_Sign = [ Headers, payload : CJ<bstr, b64u> / CJ<nil, nil>, signatures : [+ COSE_Signature] ] COSE_Signature = [ Headers, signature : CJ<bstr, b64u> ] Headers = ( protected : empty_or_serialized_map, unprotected : header_map ) header_map = { Generic_Headers, * label => values } Generic_Headers = ( ? 1 => CJ<int, int> / CJ<tstr, tstr> ; algorithm identifier ? 2 => [+label], ; criticality ? 3 => CJ<tstr, tstr> / CJ<int, int>, ; content type ? 4 => CJ<bstr, b64u>, ; key identifier ? ( 5 => CJ<bstr, b64u> // ; IV 6 => CJ<bstr, b64u> ) ; Partial IV )Experimenting with adding
CJ<>notation to dchp/pull/29Pulling out some sections from #29 to update with this syntax
Request Structure
The top-level
CredentialRequestcontains:Field Key Type Presence Description payload1CJ<bytes, b64u>M CBOR-encoded RequestPayloadreader_auth2COSE_SignO Optional reader authentication signature The
RequestPayloadcontains:Field Key Type Presence Description ScenarioSets1{ + ScenarioRef => ScenarioSet }M Map of scenarios the reader is requesting, keyed by ScenarioRefCredentialQueries2{ + CredentialRef => CredentialQuery }M Dictionary of all requested credential definitions encryption_context3EncryptionContextC Primary response encryption context; absent when ISO/IEC 18013-5 session encryption is used additional_encryption_contexts4{ + CJ<int, int> => EncryptionContext }O Additional encryption contexts for multi-key routing, keyed by integer Credential Queries
A
CredentialQuerydefines the exact credential requirements for a single credential type.Field Key Type Presence Description credential_format1CJ<tstr, tstr>M Format identifier (e.g., "mso_mdoc","vc+sd-jwt")credential_type2CJ<tstr, tstr>M Credential type (e.g., "org.iso.18013.5.1.mDL")elements_dict3{ + ElementRef => DataElementDef }M Dictionary of requested data elements requested_elements4ElementLogicM Boolean logic tree defining which elements are required general_extensions5generalExtensionsO Protocol-level extensions applicable across formats format_extensions6formatExtensionsO Format-specific extensions (e.g., mdocExtensions,sdjwtExtensions)encryption_ref7CJ<int, int>O Reference to an entry in additional_encryption_contexts; absent means the mainencryption_contextis usedMy Opinions
- Any use of CDDL primitive datatypes (e.g.
bstr,tstr,int) should get theCJ<>treatment, no matter how obvious it might seem a datatype maps to a JavaScript type - Custom CDDL types (e.g.
COSE_Signature,Headers,Generic_Headers) can probably go withoutCJ<>annotations - CDDL
ArraysandMapshave matching types in JavaScript so there's seemingly no need to give them theCJ<>treatment either
I think after going through this exercise I've come to the opinion that the
CJ<..., ...>notation style seems reasonable to me. I think I'd personally prefer to read this style over having to Cmd+F and jump between CBOR-specific and JSON-specific structs throughout an already complex document.- Any use of CDDL primitive datatypes (e.g.
And I apologize, I'm pretty sure I'm using "CBOR" and "CDDL" a bit too interchangeably above. I don't often talk about data structures using CDDL terminology so bear with me while I learn from the WG through osmosis 😅
Make sure it is clear that we use CBOR over the wire, but have a clear definition of JSON representations for configuration, debugging, etc.
I was mulling over my exploration above and agree it'd be necessary, around whatever such notation we ultimately land on, to emphasize that the JSON datatype definitions for any given data structure is only for representing an instance of that data structure in isolation. That is, there'd be no actual JSON-shaped values within the deeper parts of the protocol which expects everything to use CBOR encoding 🤔
WG discussion:
- heavy annotated CDN might be an alternative; CDN is superset of JSON, e.g., can do comments.
- for examples, no JSON notation; JSON notation is for debug/ output mostly.
- in a lot of places types are the same in JSON and CBOR; only in a few places they differ, e.g., bytestr vs b64u. we could agree on this.
- CDN and JC serve different purposes.
- CDDL is well supported, e.g., Rust, which can also do JSON/CBOR conversion.
- Need some experimenting with libraries.
- Next step could be to apply this to the examples of the initial document PR.
- Not a blocker for the remaining specification.
Discussed on today's call.
Christian is experimenting with the proposed example format in rust, autogenerating data structures from the CDDL and it seems to be working so far, he will report back once he's experimented more. We'd need to check the socket syntax used in #57 is supported by CDDL parsers.
Below our two cents trying this out in Rust.
We had to perform some text subsitutions, no brainer, so that it can be parsed :
- An entry like
payload: 1 => bstrcarries two member keys, the readable name and the actual integer key, where the grammar allows only one,
-> kept1 =>and moved the name to a comment (it should work the other way). * int / tstr => anyuses a type choice as a key without parentheses.
-> wrote* (int / tstr) => any.
With those two edits,
cddl0.10.7 reports the document conformant.
Bothcddletcddl-catstop at the same construct, and RFC 8610 agrees with them, so this is about how the CDDL is written rather than about the tools.Once it parses, the answer is conditional
We wrote a
RequestPayloadby hand, encoded it withciborium, andcddl-cat0.7.1 validates it against the fixed Request CDDL.That is the part that works. Generation does not.
No Rust crate we found generates examples.
Across the eight crates surveyed on crates.io (2026-09-08), none of them emits a conforming instance from a schema.
Andanweiss/cddllists it as an unchecked goal in itsREADME.
The only generator we located iswww/src/sample-gen.jsin that same repository, serving the browser playground, not reachable from the crate.
We did not look at the Python, Java or Go ecosystems, not used in our project.At the end what it produced does not survive the trip back to CBOR land, for two reasons :
- every map in the document ends with an RFU wildcard, which the generator satisfies by inserting a junk entry everywhere,
- and a
bstrcomes out as a base64-like text string, which is no longer abstronce re-encoded.
Neither looks fixable by writing the CDDL differently.
Dropping the RFU wildcards would address the first, at the cost of the extensibility they exist for.
The second is a property of JSON, and no rewriting reaches it.We could not get a Rust validator to enforce the
CJ<>notation as defined in #19 (comment)We tested the
CJ<C,J> = C .feature "cbor" / J .feature "json"form againstcddl0.10.7 andcddl-cat0.7.1.On a member key it parses and then does not validate.
On a value, in that exact form, validation stops applying : an integer passes against
CJ<bstr, tstr>.Two things behind this:
cddl-cat0.7.1 does not implement.featureat all, so there is no second opinion in Rust..featureis not a selector incddl: enabling or disabling a feature does not change what it accepts. Invalidator/cbor.rsa not enabled feature is pushed ontodisabled_featuresand the target skipped, with no error raised.
Sockets, as used in #57
To @jogu's question :
-
cddl0.10.7 accepts socket syntax in the schema, and validates instances against it correctly : it rejects a value matching no socket member, and a socket that was never extended. -
cddl-cat0.7.1 does not accept the syntax at all, so it never gets to validating anything. That is a feature it has not implemented, not a limit of CDDL.
So the fixed document parses under
cddlonly.But,
cddlgets also two constructs wrong, and the document is built on both :- a table whose value is a named map containing a wildcard,
- a table whose key is a named type alias.
That is why our hand-writtenRequestPayloadfails there. We have not filed these two issues yet.
As things stand, no Rust tool validates the Request CDDL as written and supports the sockets.
What this leaves
Today's Rust tooling gives you CBOR validation and type generation, both usable in CI.
It does not give you example generation, and we could not get one CDDL to validate both encodings with the precision
CJ<>is aiming for.
A plain(1 / "1") => (bstr / tstr)choice validates both, but it is a union : it documents the two encodings rather than constraining either.
SoCJ<>would carry its meaning to readers rather than to tools, at least until an implementation catches up.We can produce a script to reproduce this if you need.
- An entry like
I've played a bit around with how the different options feel to me. Example CDDL from the current draft:
EncryptionContext = { key: 1 => COSE_Key, nonce: 2 => bstr, algorithms: 3 => [ + int ], ; COSE algorithm identifiers (e.g. HPKE ciphersuites) * int / tstr => any ; RFU / application-specific extensions }As long as always keep naming the keys, even if we use a scalar value in CBOR, that seems to be sufficient to me and we could add an implementation consideration to use those strings directly to create a JSON representation if necessary for debugging and configuration.
That would provide clear guidance and not make the CDDL harder to read. Given the complexity that seems to be added if we want to fully define a JSON encoding, that currently feels like the leaner and more practical approach to me instead of going the full CDDL for both formats way.
Reacted by Matthew MillerThat would provide clear guidance and not make the CDDL harder to read. Given the complexity that seems to be added if we want to fully define a JSON encoding, that currently feels like the leaner and more practical approach to me instead of going the full CDDL for both formats way.
I could support this outcome, paired with an opinionated suggestion to use a single string encoding when
bstrvalues need to be conveyed as a string (i.e. encode to base64url or hex.) The latter would be to encourage consistent implementations/examples/documentation/etc... for all the heterogenous implementations.Reacted by Christian Bormann
#5 is discussing the general serialization format, but also mixes in the dev ux related discussion points.
My take-away from the discussions at IETF 126 was that people were pointing towards CDDL and defining CBOR and JSON in one CDDL. Make sure it is clear that we use CBOR over the wire, but have a clear definition of JSON representations for configuration, debugging, etc.
Copied over from the other issue:
If we use integer based maps in CBOR, we could in theory define in CDDL values that are different between CBOR and JSON representation which would allow us to define keys in the map that are different from CBOR (integer) and JSON (string). Taken from cbor-wg Wiki:
JC<"v",2> defines the value as
"v"for JSON and2for CBOR.RFC9741 introduces control operators that allows to express things like base64 encoding in CDDL: