Verify GBLv4 hashes and manifest signature

[yarrrly]

pygbl 1.2.0 parses every GBLv4 file I have without an unknown tag and round-trips them byte for byte. What it cannot do yet for v4 is the checking that GBL3Image.verify_signature does for v3, and there is no decompress for v4 memory sections. I would like to add both and can send a PR; this issue is to agree on the shape first.

The files are the 18 IKEA KAJPLATS Matter OTA images listed in the CSA DCL (vendor 4476, product ids 36865 to 36889 that have an otaUrl). They are Series 3 GBLv4, LZMA, unencrypted and public, so a test can download them instead of relying on a local corpus. Two of them:

https://ota.matter.ikea.com/files/4476_36871_16908288_18482475-a249-4ca5-8fd0-a7f70c23c1b1.ota
  812204 bytes, sha256 b0a5c16b73f7c92121e11fcbb7412d582fe3d9f52edd33c912a5af70469b4b6a
https://ota.matter.ikea.com/files/4476_36866_16908288_9844be7c-e3c9-4a98-80c0-34b77615b817.ota
  797428 bytes, sha256 61a4563a7c3baf65d1abb282eec988cb25c58d92bf68d5779c4be9adb5b8c68f

The GBL starts at offset 84, after the Matter OTA header. That wrapper is Matter's, not GBL's, so I am not proposing to handle it here.

What the files establish, checked on all 18 with the script below (pygbl 1.2.0, cryptography 50.0.0):

• CONTENT_HASH is SHA-256 over everything after the MANIFEST tag, as the format page says.
• UpdateMemorySection.hash is SHA-256 over the whole MEMORY_SECTION_INFO tag, 8-byte header included.
• MemorySectionInfo.final_image_hash[:32] is SHA-256 of the decompressed image, and plain_image_size is its length. The docs mark that field reserved; the files fill it.
• The manifest signature (type 2) is ECDSA P-256 over SHA-256 of the manifest children after the SIGNATURE tag, that is MANIFEST_INFO, BUNDLE_VERSION, CONTENT_HASH and UPDATE_PROCESS with their headers, verified with the key in CERTIFICATE. The public page does not say what the signature covers; that range verifies on 18 of 18 files. You may be able to confirm it against the SDK parser.
• PAD is 1 to 3 bytes of 0xFF and appears exactly when the root tag would otherwise not end on a 4-byte boundary (15 of 18 files). pygbl already keeps it opaque, which is right.

Not part of the GBL, but related: the same 136-byte certificate sits inside the decompressed image at the address ApplicationProperties_t.cert points to, followed by a 64-byte ECDSA signature over the image up to signatureLocation (pointers relative to target_address). 18 of 18 verify.

Proposal, mirroring the v3 API:

• GBL4Image.verify_content_hash() -> bool
• GBL4Image.verify_memory_sections() -> bool: each UpdateMemorySection.hash against its MEMORY_SECTION_INFO, and final_image_hash against the decompressed blob when the section is not encrypted
• GBL4Image.verify_signature(public_key=None) -> bool: default to the key from the CERTIFICATE tag, raise KeyError when unsigned, like v3
• GBL4Image.decompress() -> GBL4Image, or a plain_image(section) accessor, since decompressing in place invalidates the hashes
• a hash_type enum with SHA256 = 1 on HashValue, since the SDK's union size is not what is on the wire

For tests, a small JSON with the two URLs, sizes and digests above and a download-on-demand fixture that skips offline, next to the local Hue corpus.

Script, pygbl 1.2.0 API only:

import hashlib, pathlib, sys
from cryptography.hazmat.primitives.asymmetric import ec
from pygbl import (GBL4Certificate, GBL4ContentHash, GBL4Image, GBL4Manifest,
                   GBL4MemorySectionBlob, GBL4MemorySectionInfo, GBL4Signature,
                   GBL4UpdateMemorySection)
from pygbl.compression import lzma_decompress
from pygbl.gbl4 import serialize_tag
from pygbl.crypto import verify_digest

data = pathlib.Path(sys.argv[1]).read_bytes()[84:]   # skip the 84-byte Matter OTA header
image, trailing = GBL4Image.deserialize(data)
manifest = image.get_first_tag(GBL4Manifest)
manifest_bytes = serialize_tag(manifest)
manifest_end = 8 + len(manifest_bytes)                # root header + manifest tag

# CONTENT_HASH: SHA-256 of everything after the manifest
content_hash = image.get_first_tag(GBL4ContentHash).hash.value
print("content hash ok:", hashlib.sha256(data[manifest_end:]).digest() == content_hash)

# UpdateMemorySection.hash: SHA-256 of the whole MEMORY_SECTION_INFO tag, header included
info = image.get_first_tag(GBL4MemorySectionInfo)
update = image.get_first_tag(GBL4UpdateMemorySection)
print("memory section hash ok:", hashlib.sha256(serialize_tag(info)).digest() == update.hash.value)

# final_image_hash[:32]: SHA-256 of the decompressed image
plain = lzma_decompress(image.get_first_tag(GBL4MemorySectionBlob).data)
print("plain image size ok:", len(plain) == update.plain_image_size)
print("final image hash ok:", hashlib.sha256(plain).digest() == info.final_image_hash[:32])

# manifest signature: ECDSA P-256 over SHA-256 of the manifest children after the SIGNATURE tag
cert = image.get_first_tag(GBL4Certificate)
sig = image.get_first_tag(GBL4Signature)
children = manifest.children
signed = b"".join(serialize_tag(t) for t in children[children.index(sig) + 1:])
key = ec.EllipticCurvePublicNumbers(int.from_bytes(cert.key[:32], "big"),
                                    int.from_bytes(cert.key[32:], "big"), ec.SECP256R1()).public_key()
r, s = sig.signature.value[:32], sig.signature.value[32:]
print("manifest signature ok:", verify_digest(key, hashlib.sha256(signed).digest(), r, s))

Output on each of the 18 files: five True.

Context: universal-silabs-flasher is moving to pygbl (NabuCasa/universal-silabs-flasher#146); with these methods it could check a Series 3 image before flashing it. A standalone, stdlib-only version of the same checks with the offsets written out is at https://github.com/yarrrly/gblv4.

[puddly]

Sure, feel free to open a PR and we can discuss the details there.