Skip to content

fix(1688): extract detail images from shadow DOM with lazy-render scrolling - #2272

Open
HARRY-BEAR wants to merge 1 commit into
jackwener:mainfrom
HARRY-BEAR:fix/1688-detail-images-shadow-dom
Open

fix(1688): extract detail images from shadow DOM with lazy-render scrolling#2272
HARRY-BEAR wants to merge 1 commit into
jackwener:mainfrom
HARRY-BEAR:fix/1688-detail-images-shadow-dom

Conversation

@HARRY-BEAR

Copy link
Copy Markdown

Problem

The 1688 assets / download commands return detail_count: 0 on real offer pages, even though the page contains 7–16 detail images. Only main images and videos were extracted.

Root cause

Two compounding issues:

  1. Shadow DOM pierce failure (structural). The product detail section lives inside the shadow root of a custom element (v-detail-e, class html-description). The old code queried selectors like .html-description img through queryAllDeep, which runs the same selector inside each shadowRoot — but .html-description is the host element in light DOM, so the selector can never match inside the shadow root. The detail group was always empty.

  2. Lazy rendering (timing). The detail content renders only after the user scrolls near it. The old autoScroll({ times: 3, delayMs: 400 }) is not enough for long pages (~13k–18k px), so the lazy images never materialized before extraction.

Fix

  • Collect all img / source elements while walking shadow roots, then decide membership by walking the shadow host chain with closest() against the known detail containers (.html-description, .de-description-detail, #detailContentContainer, .desc-lazyload-container).
  • Scroll more aggressively (autoScroll 6+4 × 500ms), then scrollIntoView the detail container and settle for 3s before extraction.

Verification

Real-browser results before → after:

offer detail before detail after
997412525461 0 16
954578900038 0 13
871254382620 0 7

download end-to-end lands *_detail_NN.jpg files; main/video counts unchanged (no regression).

Added a jsdom regression test (assets.test.js) covering:

  • images inside the v-detail-e shadow root → detected as detail
  • light-DOM main gallery images → not detected as detail
  • light-DOM plain-class detail containers → detected as detail

All 15 tests in clis/1688 pass; npx tsc --noEmit is clean.

…olling

The product detail section lives inside the shadow root of a custom
element (v-detail-e with class html-description). Plain CSS selectors
like `.html-description img` cannot pierce shadowRoot, so the detail
group never matched any element and detail_images was always empty.

Fix by collecting all img/source elements (walking shadow roots) and
checking ancestry through the shadow host chain with closest(), plus
scrolling further and settling on the detail container so its lazy
content renders before extraction.

Adds a jsdom regression test covering shadow-root detail images,
light-DOM main gallery images, and plain-class detail containers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant