Regression: Versions >= v5.3.2 are unable to parse specific link

[stalgiag]

I work for a project that validates its links using this library. One link that is frequently validated is the HTML spec at https://html.spec.whatwg.org/. This page has one of the bigger HTML files on the web but node-html-parser was able to parse it well in approximately 23 seconds on my local machine until release 5.3.2.

Consider this example:

const HTMLParser = require('node-html-parser');
const nFetch = require('node-fetch');

async function parseHTMLSpec() {
  try {
    const response = await nFetch('https://html.spec.whatwg.org/');
    const html = await response.text();

    console.log('Fetched HTML. Attempting to parse...');
    console.time('parseHTMLSpec');
    const parsedHTML = HTMLParser.parse(html);
    console.timeEnd('parseHTMLSpec');

    console.log('HTML parsed successfully.');
    console.log('Title:', parsedHTML.querySelector('title').text);
  } catch (error) {
    console.error('Error occurred:', error);
  }
}

parseHTMLSpec();

With node-html-parser 5.3.1, this outputs the following:

Fetched HTML. Attempting to parse...
parseHTMLSpec: 23.415s
HTML parsed successfully.
Title: HTML Standard

With node-html-parser 5.3.2, this hangs indefinitely; only outputting the following even after running for hours:

console.log('Fetched HTML. Attempting to parse...');

[taoqf]

Sorry for the bad experience.
I release a beta version node-html-parser@6.1.15-0
but I could not test it due to large memory usage. Could you test it for me? thank you.

[stalgiag]

Thanks for attempting to find a fix!

I tested node-html-parser@6.1.15-0 and I also ran out of memory with this error:
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory.

I tested on both Node 20.10.0 and Node 18.18.1. Note that this does not happen on <v5.3.2 using the same machine.

[taoqf]

Indeed, this lib is getting slower and slower...

[stalgiag]

Thanks for your work!

I truly appreciate the added test. Unfortunately the test doesn't quite represent the problem accurately. The file saved to the repo for testing, "HTML Standard.html" is different than the file served at that URL. Perhaps a utility was used to save it that did some processing? I made a fork of your library that additionally uses the HTML fetched from that URL in that same test. You can see in my actions run that the fetched HTML is correctly parsed by 5.3.1 but hangs indefinitely with newer versions.

[taoqf]

This is the test log of rust wasm version (357.771ms), slower then 531(341.132): the js bridge calls result in performance loss, And, the rust wasm made all attributes to methods, cause a lot issues.
Maybe v5.3.1 is the best way to use in your case.

But, if you can use node-html-parser-rs in rust way, it is really fast.

issue 280 Regression: Versions >= v5.3.2 are unable to parse specific link
Fetched HTML. Attempting to parse...
parseHTMLSpec: 357.771ms
Title: [Function: text]
✔ Versions: dev (513ms)
Fetched HTML. Attempting to parse...
parseHTMLSpec531: 341.132ms
Title: HTML Standard
✔ Versions: v5.3.1 (362ms)
Fetched HTML. Attempting to parse...
parseHTMLSpec6113: 827.272ms
Title: HTML Standard
✔ Versions: v6.1.13 (846ms)