fix(1688): extract detail images from shadow DOM with lazy-render scrolling - #2272
Open
HARRY-BEAR wants to merge 1 commit into
Open
fix(1688): extract detail images from shadow DOM with lazy-render scrolling#2272HARRY-BEAR wants to merge 1 commit into
HARRY-BEAR wants to merge 1 commit into
Conversation
…olling The product detail section lives inside the shadow root of a custom element (v-detail-e with class html-description). Plain CSS selectors like `.html-description img` cannot pierce shadowRoot, so the detail group never matched any element and detail_images was always empty. Fix by collecting all img/source elements (walking shadow roots) and checking ancestry through the shadow host chain with closest(), plus scrolling further and settling on the detail container so its lazy content renders before extraction. Adds a jsdom regression test covering shadow-root detail images, light-DOM main gallery images, and plain-class detail containers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The 1688
assets/downloadcommands returndetail_count: 0on real offer pages, even though the page contains 7–16 detail images. Only main images and videos were extracted.Root cause
Two compounding issues:
Shadow DOM pierce failure (structural). The product detail section lives inside the shadow root of a custom element (
v-detail-e, classhtml-description). The old code queried selectors like.html-description imgthroughqueryAllDeep, which runs the same selector inside each shadowRoot — but.html-descriptionis the host element in light DOM, so the selector can never match inside the shadow root. The detail group was always empty.Lazy rendering (timing). The detail content renders only after the user scrolls near it. The old
autoScroll({ times: 3, delayMs: 400 })is not enough for long pages (~13k–18k px), so the lazy images never materialized before extraction.Fix
img/sourceelements while walking shadow roots, then decide membership by walking the shadow host chain withclosest()against the known detail containers (.html-description,.de-description-detail,#detailContentContainer,.desc-lazyload-container).autoScroll6+4 × 500ms), thenscrollIntoViewthe detail container and settle for 3s before extraction.Verification
Real-browser results before → after:
downloadend-to-end lands*_detail_NN.jpgfiles; main/video counts unchanged (no regression).Added a jsdom regression test (
assets.test.js) covering:v-detail-eshadow root → detected as detailAll 15 tests in
clis/1688pass;npx tsc --noEmitis clean.