Skip to content

Decode typeless Tika matches and honor masked rule offsets #179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jeremy merged 2 commits into from
Aug 24, 2026
+337 −53
Merged

Decode typeless Tika matches and honor masked rule offsets #179

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0227d59
Decode typeless Tika matches and honor masked rule offsets
jeremy Aug 24, 2026
33c67c6
Address review: bound shifted mask offsets, pin retirements
jeremy Aug 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 0 additions & 16 deletions data/custom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -243,20 +243,4 @@

<sub-class-of type="application/xml" />
</mime-type>
<!-- Tika 4.0.0 rebuilt the CMS/DER magics (pkcs7, pkcs12, timestamped-data) on typeless
<match value="0x..."> elements. Tika defaults an absent type to "string" and
hex-decodes the value; the generator only decodes when type="string" is explicit, so
those upstream rules generate as inert literal-text matchers. That leaves
application/timestamped-data, whose pre-4.0 magic was a working byte match, without
content detection. Carry upstream's rule (same shape and priority, from
tika-mimetypes.xml @ 4.0.0) with the types spelled out: SEQUENCE tag, then the
id-ct-timestampedData OID (1.2.840.113549.1.9.16.1.31) scanned across the DER
length forms. Remove once the generator decodes typeless matches like Tika does. -->
<mime-type type="application/timestamped-data">
<magic priority="60">
<match value="0x30" type="string" offset="0">
<match value="0x060B2A864886F70D010910011F" type="string" offset="2:6" />
</match>
</magic>
</mime-type>
</mime-info>
32 changes: 11 additions & 21 deletions lib/marcel/tables.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2777,7 +2777,7 @@ module Marcel
['video/webm', [[0, b["\032E\337\243"], [[4..4096, b["B\202"], [[4..4096, b['webm'], [[4..4096, b['V_VP8']], [4..4096, b['V_VP9']], [4..4096, b['V_AV1']]]]]]]]]],
['video/x-matroska', [[0, b["\032E\337\243\223B\202\210matroska"]]]],
['video/x-flv', [[0, b['FLV']]]],
['audio/mpeg', [[0, b["\377\362"]], [0, b["\377\363"]], [0, b["\377\364"]], [0, b["\377\365"]], [0, b["\377\366"]], [0, b["\377\367"]], [0, b["\377\372"]], [0, b["\377\373"]], [0, b["\377\374"]], [0, b["\377\375"]], [0, b["\377\343"]], [0, b['ID3']]]],
['audio/mpeg', [[0, b["\377\362"]], [0, b["\377\363"]], [0, b["\377\364"]], [0, b["\377\365"]], [0, b["\377\366"]], [0, b["\377\367"]], [0, b["\377\372"]], [0, b["\377\373"]], [0, b["\377\374"]], [0, b["\377\375"]], [0, b["\377\343"]]]],
['application/pdf', [[0, b['%PDF-']], [0, b["\357\273\277%PDF-"]]]],
['application/msword', [[2080, b['Microsoft Word 6.0 Document']], [2080, b['Documento Microsoft Word 6']], [2112, b['MSWordDoc']], [0, b["1\276\000\000"]], [0, b['PO^Q`']], [0, b["\3767\000#"]], [0, b["\333\245-\000\000\000"]], [0, b["\224\246."]], [0..8, b["\320\317\021\340\241\261\032\341"], [[1152..4096, b["W\000o\000r\000d\000D\000o\000c\000u\000m\000e\000n\000t"]]]]]],
['application/vnd.openxmlformats-officedocument.wordprocessingml.document', [[0, b["PK\003\004"], [[30..65536, b['[Content_Types].xml'], [[0..4096, b['word/']]]], [30, b['_rels/.rels'], [[0..4096, b['word/']]]]]]]],
Expand All @@ -2786,7 +2786,6 @@ module Marcel
['application/x-tika-msworks-spreadsheet', [[0..8, b["\320\317\021\340\241\261\032\341"], [[1152..4096, b["W\000k\000s\000S\000S\000W\000o\000r\000k\000B\000o\000o\000k"]]]]]],
['application/vnd.ms-excel', [[2080, b['Microsoft Excel 5.0 Worksheet']], [2080, b['Foglio di lavoro Microsoft Exce']], [2114, b['Biff5']], [2121, b['Biff5']], [0..8, b["\320\317\021\340\241\261\032\341"], [[1152..4096, b["W\000o\000r\000k\000b\000o\000o\000k"]]]]]],
['application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', [[0, b["PK\003\004"], [[30..65536, b['[Content_Types].xml'], [[0..4096, b['xl/']]]], [30, b['_rels/.rels'], [[0..4096, b['xl/']]]]]]]],
['image/x-tga', [[1, b["\001\001\000\000"]], [1, b["\000\002\000\000"]], [1, b["\000\003\000\000"]]]],
['application/x-endnote-refer', [[0..50, b['%A '], [[0..1000, b["\n%D "], [[0..1000, b["\n%T "]]]]]]]],
['application/x-tmx', [[0..256, b['<tmx']]]],
['image/svg+xml', [[0, b['<?xml'], [[21..512, b['<svg']]]]]],
Expand All @@ -2799,7 +2798,6 @@ module Marcel
['image/svg+xml', [[0, b['<!--'], [[4..1024, b['<svg']]]]]],
['application/cbor', [[0, b["\331\331\367"]]]],
['application/coreldraw', [[0, b['RIFF'], [[8, b['CDR']], [8, b['cdr']], [8, b['DES']], [8, b['des']]]]]],
['application/timestamped-data', [[0, b['0x30'], [[2..6, b["\006\v*\206H\206\367\r\001\t\020\001\037"]]]]]],
['application/timestamped-data', [[0, b['0'], [[2..6, b["\006\v*\206H\206\367\r\001\t\020\001\037"]]]]]],
['application/vnd.etsi.asic-e+zip', [[0, b["PK\003\004"], [[30, b['mimetypeapplication/vnd.etsi.asic-e+zip']]]]]],
['application/vnd.etsi.asic-s+zip', [[0, b["PK\003\004"], [[30, b['mimetypeapplication/vnd.etsi.asic-s+zip']]]]]],
Expand Down Expand Up @@ -2888,7 +2886,7 @@ module Marcel
['application/x-mach-o-executable', [[0, b["\376\355\372\316"], [[12, b["\000\000\000\002"]]]], [0, b["\316\372\355\376"], [[12, b["\002\000\000\000"]]]], [0, b["\376\355\372\317"], [[12, b["\000\000\000\002"]]]], [0, b["\317\372\355\376"], [[12, b["\002\000\000\000"]]]]]],
['application/x-mach-o-fvmlib', [[0, b["\376\355\372\316"], [[12, b["\000\000\000\003"]]]], [0, b["\316\372\355\376"], [[12, b["\003\000\000\000"]]]], [0, b["\376\355\372\317"], [[12, b["\000\000\000\003"]]]], [0, b["\317\372\355\376"], [[12, b["\003\000\000\000"]]]]]],
['application/x-mach-o-kext-bundle', [[0, b["\376\355\372\316"], [[12, b["\000\000\000\v"]]]], [0, b["\316\372\355\376"], [[12, b["\v\000\000\000"]]]], [0, b["\376\355\372\317"], [[12, b["\000\000\000\v"]]]], [0, b["\317\372\355\376"], [[12, b["\v\000\000\000"]]]]]],
['application/x-mach-o-object', [[0, b["\376\355\372\316"], [[12, b["\000\000\000\001"]]]], [0, b["\316\372\355\376"], [[12, b["\001\000\000\000"]]]], [0, b["\376\355\372\317"], [[12, b["\000\000\000\001"]]]], [0, b['0xCFFAEDFE'], [[12, b["\001\000\000\000"]]]]]],
['application/x-mach-o-object', [[0, b["\376\355\372\316"], [[12, b["\000\000\000\001"]]]], [0, b["\316\372\355\376"], [[12, b["\001\000\000\000"]]]], [0, b["\376\355\372\317"], [[12, b["\000\000\000\001"]]]], [0, b["\317\372\355\376"], [[12, b["\001\000\000\000"]]]]]],
['application/x-mach-o-preload', [[0, b["\376\355\372\316"], [[12, b["\000\000\000\005"]]]], [0, b["\316\372\355\376"], [[12, b["\005\000\000\000"]]]], [0, b["\376\355\372\317"], [[12, b["\000\000\000\005"]]]], [0, b["\317\372\355\376"], [[12, b["\005\000\000\000"]]]]]],
['application/x-msdownload;format=pe', [[0, b['MZ'], [[128, b["PE\000\000"]], [176, b["PE\000\000"]], [208, b["PE\000\000"]], [240, b["PE\000\000"]]]]]],
['text/x-robots', [[0, nil, [[0, b['user-agent:']], [0, b['allow:']], [0, b['disallow:']], [0, b['sitemap:']], [0..1000, b["\nuser-agent:"]], [0..1000, b["\nallow:"]], [0..1000, b["\ndisallow:"]], [0..1000, b["\nsitemap:"]]]]]],
Expand All @@ -2906,16 +2904,13 @@ module Marcel
['application/msword5', [[0, b["\3767"]]]],
['application/octet-stream', [[10, b['# This is a shell archive']], [0, b["\037\036"]], [0, b["\037\037"]], [0, b["\377\037"]], [0, b["\377\037"]], [0, b["\005\313"]]]],
['application/ogg', [[0, b['OggS']]]],
['application/onenote;format=one', [[0, b["\344R\\{"], [[4, b["\214\330"], [[6, b["\247M"], [[8, b['0xAEB15378D02996D3']]]]]]]]]],
['application/onenote;format=onetoc2', [[0, b["\241/\377C"], [[4, b["\331\357"], [[6, b['vL'], [[8, b['0x9EE210EA5722765F']]]]]]]]]],
['application/onenote;format=one', [[0, b["\344R\\{"], [[4, b["\214\330"], [[6, b["\247M"], [[8, b["\256\261Sx\320)\226\323"]]]]]]]]]],
['application/onenote;format=onetoc2', [[0, b["\241/\377C"], [[4, b["\331\357"], [[6, b['vL'], [[8, b["\236\342\020\352W\"v_"]]]]]]]]]],
['application/pgp-encrypted', [[0, b["\205"], [[3, b["\003"]]]]]],
['application/pkcs7-mime', [[0, b['0x30'], [[0, b["\006\v*\206H\206\367\r\001\t\020\001"]]]]]],
['application/pkcs7-signature', [[0, b['-----BEGIN PKCS7']], [0, b['0x30'], [[0, b["\006\t*\206H\206\367\r\001\a"], [[11, b["\240"]]]]]]]],
['application/pkcs7-mime', [[0, b['0'], [[2..6, b["\006\v*\206H\206\367\r\001\t\020\001"]]]]]],
['application/pkcs7-signature', [[0, b['-----BEGIN PKCS7']]]],
['application/postscript', [[0, b['%!']], [0, b["\004%!"]], [0, b["\305\320\323\306"]], [0, b['%!PS-Adobe-3.0 EPSF-3.0']]]],
['application/rtf', [[0, b["{\\rtf"]]]],
['application/sereal;version=1', [[0, b['=srl']]]],
['application/sereal;version=2', [[0, b['=srl']]]],
['application/sereal;version=3', [[0, b["=\363rl"]]]],
['application/vnd.android.axml', [[0, b["\003\000\b\000"], [[8, b["\001\000"]]]]]],
['application/vnd.apple.mpegurl', [[0, b['#EXTM3U']]]],
['application/vnd.digilite.prolights', [[0, b["\177\fD+"]]]],
Expand Down Expand Up @@ -2975,7 +2970,7 @@ module Marcel
['application/x-archive', [[0, b['=<ar>']], [0, b["!<arch>\n"]]]],
['application/x-arj', [[0, b["`\352"]]]],
['application/x-asprs', [[0, b['LASF'], [[24, b["\001\001"]], [24, b["\001\002"]]]]]],
['application/x-atari-floppy-disk-image', [[0, b['0x9602'], [[4, b['0x8000'], [[11, b['0x00000000']]]], [4, b['0x0001'], [[11, b['0x00000000']]]]]]]],
['application/x-atari-floppy-disk-image', [[0, b["\226\002"], [[4, b["\200\000"], [[11, b["\000\000\000\000"]]]], [4, b["\000\001"], [[11, b["\000\000\000\000"]]]]]]]],
['application/x-bat', [[0, b['@echo off']], [0, b['rem ']]]],
['application/x-berkeley-db;format=btree', [[0, b["b1\005\000"]], [0, b["\000\0051b"]], [0, b["b1\005\000"]], [12, b["b1\005\000"]], [12, b["\000\0051b"]], [12, b["b1\005\000"]]]],
['application/x-berkeley-db;format=hash', [[0, b["a\025\006\000"]], [0, b["\000\006\025a"]], [0, b["a\025\006\000"]], [12, b["a\025\006\000"]], [12, b["\000\006\025a"]], [12, b["a\025\006\000"]]]],
Expand All @@ -2995,7 +2990,6 @@ module Marcel
['application/x-endnote-style', [[0, b["\000\b"], [[4, b["\000\000"], [[8, b['RSFTSTYL']], [8, b['ENDNENFT']]]]]]]],
['application/x-erdas-hfa', [[0, b['EHFA_HEADER_TAG']]]],
['application/x-executable', [[0, b["\177ELF"], [[16, b["\002\000"]], [16, b["\000\002"]]]]]],
['application/x-fat-diskimage', [[0, b["\\xEB"], [[2, b["\\x90"]]]]]],
['application/x-filemaker', [[14, b["\300HBAM7"], [[525, b["HBAM2101OCT99\301\002H\aPro 7.0\300\300"]]]]]],
['application/x-foxmail', [[0, b["\020\020\020\020\020\020\020\021\021\021\021\021\021S"]]]],
['application/x-gnumeric', [[39, b['=<gmr:Workbook']]]],
Expand All @@ -3020,7 +3014,7 @@ module Marcel
['application/x-mach-o', [[0, b["\376\355\372\316"]], [0, b["\316\372\355\376"]], [0, b["\376\355\372\317"]], [0, b["\317\372\355\376"]]]],
['application/x-matlab-data', [[0, b['MATLAB']]]],
['application/x-mmm-digisonde', [[0, b["\t<\000"], [[24, b["\000\000"]]]]]],
['application/x-ms-compress-szdd', [[0, b['0x535A444488F0273341']]]],
['application/x-ms-compress-szdd', [[0, b["SZDD\210\360'3A"]]]],
['application/x-msdownload', [[0, b['MZ']]]],
['application/x-mswrite', [[0, b["1\276\000\000"]], [0, b["2\276\000\000"]]]],
['application/x-nesrom', [[0, b["NES\032"]]]],
Expand All @@ -3029,7 +3023,7 @@ module Marcel
['application/x-ole-storage', [[0..8, b["\320\317\021\340\241\261\032\341"]]]],
['application/x-parquet', [[0, b['PAR1']]]],
['application/x-pds', [[0, b['PDS_VERSION_ID']]]],
['application/x-pkcs12', [[0, b['0x30'], [[2..6, b['0x02010330'], [[0, b["\006\t*\206H\206\367\r\001\a"]]]]]]]],
['application/x-pkcs12', [[0, b['0'], [[2..6, b["\002\001\0030"], [[7..15, b["\006\t*\206H\206\367\r\001\a"]]]]]]]],
['application/x-project', [[0, b['MPX,Microsoft Project for Windows,']]]],
['application/x-prt', [[8, b['0M3C']]]],
['application/x-quattro-pro;version=1+5', [[0, b["\000\000\002\000\001\020"]]]],
Expand All @@ -3054,22 +3048,19 @@ module Marcel
['application/x-tex-virtual-font', [[0, b["\367\312"], [[11, b["\363\000"], [[17, b["\000\020"]]]]]]]],
['application/x-texinfo', [[0, b["\\input texinfo"]]]],
['application/x-tika-ooxml', [[0, b["PK\003\004"], [[30, b['[Content_Types].xml']], [30, b['_rels/.rels']]]]]],
['application/x-touhou', [[0, b['t1']]]],
['application/x-uc2-compressed', [[0, b["UC2\032"]]]],
['application/x-vhd', [[0, b['conectix']]]],
['application/x-x509-cert;format=pem', [[0, b['-----BEGIN CERTIFICATE-----']]]],
['application/x-x509-dsa-parameters', [[0, b['-----BEGIN DSA PARAMETERS-----']]]],
['application/x-x509-ec-parameters', [[0, b['-----BEGIN EC PARAMETERS-----']]]],
['application/x-x509-key;format=pem', [[0, b['-----BEGIN PRIVATE KEY-----']], [0, b['-----BEGIN PUBLIC KEY-----']], [0, b['-----BEGIN KEY-----']], [0, b['-----BEGIN RSA KEY-----']], [0, b['-----BEGIN RSA PRIVATE KEY-----']], [0, b['-----BEGIN DSA KEY-----']], [0, b['-----BEGIN DSA PRIVATE KEY-----']]]],
['application/x-xz', [[0, b["\3757zXZ\000"]]]],
['application/x-zim', [[0, b['0x5A494D04']]]],
['application/x-zim', [[0, b["ZIM\004"]]]],
['application/x-zoo', [[20, b["\334\247\304\375"]]]],
['application/xml', [[0, b['<?xml']], [0, b['<?XML']], [0, b["\357\273\277<?xml"]], [0, b["\377\376<\000?\000x\000m\000l\000"]], [0, b["\376\377\000<\000?\000x\000m\000l"]]]],
['application/zip', [[0, b["PK\003\004"]], [0, b["PK\005\006"]], [0, b["PK\a\b"]]]],
['application/zstd', [[0, b["(\265/\375"]]]],
['audio/ac3', [[0, b["\vw"]]]],
['audio/amr-wb', [[0, b["#!AMR-WB\n"]]]],
['audio/eac3', [[0, b["\vw"]]]],
['audio/prs.sid', [[0, b['PSID']]]],
['audio/webm', [[0, b["\032E\337\243"], [[4..4096, b["B\202"], [[4..4096, b['webm'], [[4..4096, b['A_VORBIS']], [4..4096, b['A_OPUS']]]]]]]]]],
['audio/x-flac', [[0, b['fLaC']]]],
Expand Down Expand Up @@ -3169,10 +3160,9 @@ module Marcel
['application/x-tika-msoffice', [[0..8, b["\320\317\021\340\241\261\032\341"]]]],
['audio/ac3', [[0, b["\vw"]]]],
['audio/amr', [[0, b["#!AMR\n"]], [0, b['#!AMR']]]],
['audio/x-aac', [[0, b['ID3']]]],
['image/vnd.zbrush.pcx', [[0, b["\n"], [[1, b["\000"]], [1, b["\002"]], [1, b["\003"]], [1, b["\004"]], [1, b["\005"]]]]]],
['message/rfc822', [[0..1000, b["\nMessage-ID:"]]]],
['text/vtt', [[0, b["WEBVTT\r"]], [0, b["WEBVTT\n"]], [0, b['0xfeff'], [[2, b["WEBVTT\r"]], [2, b["WEBVTT\n"]]]], [0, b['0xfeff'], [[2, b["WEBVTT\r"]], [2, b["WEBVTT\n"]]]], [0, b['0xefbbbf'], [[3, b["WEBVTT\r"]], [3, b["WEBVTT\n"]]]], [0, b["WEBVTT FILE\r"]], [0, b["WEBVTT FILE\n"]]]],
['text/vtt', [[0, b["WEBVTT\r"]], [0, b["WEBVTT\n"]], [0, b["\376\377"], [[2, b["WEBVTT\r"]], [2, b["WEBVTT\n"]]]], [0, b["\376\377"], [[2, b["WEBVTT\r"]], [2, b["WEBVTT\n"]]]], [0, b["\357\273\277"], [[3, b["WEBVTT\r"]], [3, b["WEBVTT\n"]]]], [0, b["WEBVTT FILE\r"]], [0, b["WEBVTT FILE\n"]]]],
['application/inf', [[0, b['[version]']], [0, b['[strings]']]]],
['application/x-bibtex-text-file', [[0, b['%'], [[2..128, b["\n@article{"]], [2..128, b["\n@book{"]], [2..128, b["\n@inbook{"]], [2..128, b["\n@incollection{"]], [2..128, b["\n@inproceedings{"]], [2..128, b["\n@manual{"]], [2..128, b["\n@misc{"]], [2..128, b["\n@preamble{"]], [2..128, b["\n@phdthesis{"]], [2..128, b["\n@string{"]], [2..128, b["\n@techreport{"]], [2..128, b["\n@unpublished{"]]]]]],
['application/xml', [[0, b['<!--']]]],
Expand Down
68 changes: 57 additions & 11 deletions script/generate_tables.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,11 +180,14 @@ def self.namespace_uri(value, source)
end

class UnsupportedRules
# Tika currently contains 60 unsupported XML rules: 59 magic matches plus one root-XML
# rule. Their pretty-printed warnings span 114 physical lines, so pin the canonical rule
# set rather than stderr layout.
EXPECTED_COUNT = 60
EXPECTED_SHA256 = "61d54e5e5b543d3d3721e4d056e43e2f928fb08b36e94c8b1da88a416a18bb48"
# Tika currently contains 77 unsupported XML rules: 76 magic matches plus one root-XML
# rule. Relative to the pre-propagation set of 60: 15 newly recorded parents whose
# children were all unsupported, 2 multi-segment masks at range offsets, and 2 regex
# parents reclassified as no-supported-children (17 children-kind entries in all).
# Pretty-printed warnings span multiple physical lines per rule, so pin the canonical
# rule set rather than stderr layout.
EXPECTED_COUNT = 77
EXPECTED_SHA256 = "146db7b1c0ccf9f095140912b6514c999528eb0b696403b7832d5e0182239c16"

def initialize
@signatures = []
Expand Down Expand Up @@ -298,13 +301,28 @@ def get_matches(mime_type, parent, unsupported_rules)
parent.elements.map {|match|
children = get_matches(mime_type, match, unsupported_rules)

# A match with children requires some child to match alongside it, so a parent that
# lost every child to an unsupported rule must be skipped too: emitting it alone would
# broaden the rule to the bare parent (e.g. classifying every DER SEQUENCE as
# application/pkcs7-signature). Surviving siblings elsewhere are unaffected.
if children.empty? && !match.elements.empty?
unsupported_rules.skip "children", mime_type, match,
"#{mime_type}: match with no supported children: #{match.to_s}"
next nil
end

type = match['type']
value = match['value']
offset = match['offset'] || '0'
offset = MimeData.offset(offset, mime_type)

mask = match['mask']

# Tika defaults an absent type to string, hex-decoding 0x… values and unescaping
# backslash escapes. Structural containers (minShouldMatch) carry no value at all
# and keep their nil type: they pass through the nil branch below unchanged.
type = 'string' if type.nil? && value

# We only support masks of whole bytes against a string type
if mask && (!mask.match?(/\A0x(FF|00)*\z/) || type != 'string')
unsupported_rules.skip "mask", mime_type, match, "#{mime_type}: unsupported mask #{match.to_s}"
Expand All @@ -331,20 +349,48 @@ def get_matches(mime_type, parent, unsupported_rules)
if mask
segments = []
mask.scan(/(?:FF)+/) do
match = $~
match_offset = match.offset(0)
mask_offset = (match_offset[0] - 2) / 2
mask_length = (match_offset[1] - match_offset[0]) / 2
scan = $~
scan_offset = scan.offset(0)
mask_offset = (scan_offset[0] - 2) / 2
mask_length = (scan_offset[1] - scan_offset[0]) / 2
segments << [mask_offset, mask_length]
end

# An all-zero mask matches any content, which can't be a deliberate rule.
if segments.empty?
unsupported_rules.skip "mask", mime_type, match, "#{mime_type}: unsupported all-zero mask #{match.to_s}"
next nil
end

# A range offset slides the whole match, but each emitted segment anchors at an
# absolute offset, so segments can't stay mutually aligned while sliding. A single
# segment simply shifts the range; more than one is inexpressible.
if Range === offset && segments.size > 1
unsupported_rules.skip "mask", mime_type, match,
"#{mime_type}: unsupported multi-segment mask at range offset #{match.to_s}"
next nil
end

chain = children
segments.reverse_each do |(mask_offset, mask_length)|
masked_value = value[mask_offset, mask_length]
segment_offset = if Range === offset
(offset.begin + mask_offset)..(offset.end + mask_offset)
else
offset + mask_offset
end

# The match's own offset was validated, but shifting by the mask position can
# push a segment past the bound the offset validation enforces.
segment_bound = Range === segment_offset ? segment_offset.end : segment_offset
if segment_bound > MimeData::MAX_MAGIC_OFFSET
raise ArgumentError,
"Masked segment offset exceeds #{MimeData::MAX_MAGIC_OFFSET} in #{mime_type}: #{match.to_s}"
end
if chain.empty?
chain = [[mask_offset, masked_value]]
chain = [[segment_offset, masked_value]]
else
chain = [[mask_offset, masked_value, chain]]
chain = [[segment_offset, masked_value, chain]]
end
end
next chain[0]
Expand Down
Loading