test(docx-core): corpus differential-testing harness + LibreOffice cross-process lock - #876
Draft
stevenobiajulu wants to merge 1 commit into
Draft
test(docx-core): corpus differential-testing harness + LibreOffice cross-process lock#876stevenobiajulu wants to merge 1 commit into
stevenobiajulu wants to merge 1 commit into
Conversation
…ice cross-process lock Investigation deliverables for corpus-scale differential/fuzz testing. No behavior change to shipping engine code; adds an opt-in corpus harness, a cross-process LibreOffice oracle lock, and minimized synthetic reproductions. Harness (scripts/corpus/, all opt-in, no default CI job, no document bytes committed): - differential-corpus-manifest.json: 520 SHA-256-pinned entries across five sources with a derived OOXML feature index (the only committed corpus content). - classify_docx_features.mjs: OOXML feature classifier producing the derived index. - fetch_differential_corpus.mjs: pins-and-fetches into a local cache, extending the prepare_real_comparison_corpus.mjs pattern (gated like SAFE_DOCX_REAL_CORPUS_DIR). - generate_oom_repro.mjs: deterministic synthetic reproduction for #874. - README.md + INVESTIGATION_REPORT.md: per-source licensing determination and a privacy-safe run summary. LibreOffice oracle (packages/docx-core): - Add acquireGlobalSofficeLock: a machine-wide lockfile serializing every soffice launch (probe + oracle) so parallel workers/sessions cannot spawn concurrent headless LibreOffice — the amplification vector behind #627. Stale locks whose holder PID is dead are stolen. Path/poll are parameterized for unit testing. - libreoffice-oracle-lock.test.ts: pure unit coverage (exclusive acquire, contention, stale-steal) with no soffice launch. Findings filed from the run: #874 (O(n*m) LCS matrix OOM), #875 (BOM-prefixed document.xml raw ParseError), mechanism for #742 (rebuild unwraps pre-tracked changes), and live #627 leaked-process evidence. Ref: #874 Ref: #875 Ref: #627
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| } | ||
|
|
||
| record.strict = | ||
| doc.includes(W_STRICT_NS) || (contentTypes ?? '').includes('purl.oclc.org/ooxml'); |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DRAFT — investigation output. Do not auto-merge.
Corpus-scale differential/fuzz testing of safe-docx over a 520-document SHA-256-pinned corpus (open-agreements CC-BY, docx-platform-tests Apache-2.0, dotnet/Open-XML-SDK MIT, SuperDoc docx-corpus ODC-BY, LibreOffice fuzzer seeds). No document bytes are committed; the manifest holds only hashes, URLs, licenses, and a derived OOXML feature index.
What this PR adds (no shipping-engine behavior change)
scripts/corpus/— opt-in harness: pinned manifest + fetch script + feature classifier + a deterministic synthetic OOM reproduction, plus a per-source licensing determination and a privacy-safe investigation report. Gated likeSAFE_DOCX_REAL_CORPUS_DIR; no default CI job.packages/docx-core—acquireGlobalSofficeLock: a machine-wide lockfile serializing every LibreOffice oracle/probe launch so parallel workers/sessions can't spawn concurrent headless soffice (the Investigate LibreOffice macOS headless startup crashes and parallel probe amplification #627 amplification vector). Pure unit test included (no soffice needed).Findings from the run (filed; not fixed here)
computeAtomLcsallocates an O(n·m) DP matrix → heap OOM/SIGABRT on a single large paragraph (~12k atoms @ 2 GB; a real fuzzer seed @ 4 GB)document.xml(ISO-Strict exports, 56/121 MS files) throws a raw xmldomParseErrorfrom load + compare instead of loading/failing closed<w:delText>(Word-unreadable)Invariant results
Every
inplaceself-comparison and metamorphic pair satisfied reject→original / accept→revised across the corpus; all substantive mismatches were rebuild-only and reduced to #742/#875-class. Aspose oracle was licensed and verified non-watermarked after pinningaspose-words==25.10.0(the swallowed-license defect inaspose_compare.pyis documented in the report).See
scripts/corpus/INVESTIGATION_REPORT.mdfor full counts, licensing determination, and the recommended next queue.