Tree-sitter fixes, 1.133.0 edition - #1577
Conversation
…for instance: * Interpolated strings * Comparison operators * `new` keyword * `readonly` and `async` modifiers * Function attributes Also fixed some bugs: * Instance methods were being highlighted like variables * Generic method calls were being highlighted like types rather than methods
…so that the TextMate grammar doesn't get preferred over the Tree-sitter grammar, all else being equal.
…by adopting the new signature for `Parser::parse`. This has been broken since (yeesh!) 1.127, when we upgraded to `web-tree-sitter` version 0.25.3. That version tried to harmonize the API differences between `node-tree-sitter` and `web-tree-sitter`… and those changes were more disruptive than I realized. The old way of setting a parsing timeout was by calling `Parser::setTimeoutMicros`. The TSDoc for that method marks it as `@deprecated`, but it's actually a no-op now! Its replacement is a `progressCallback` option that should return `true` when the time budget is exceeded. Furthermore, `parse` used to throw an error when its time budget was exceeded; now it returns `null` in this scenario instead of returning a tree. This means that all code parsing since 1.127 has been synchronous. This was not caught because it's hard to write specs around this behavior! Parsing in `web-tree-sitter` is _very_ fast, so for most use cases, the difference would not have been noticed. But the experience of editing large files would've been seriously degraded, with possible multi-second waits between keystrokes. Editing a large JS file is not a cake-walk in Pulsar anyway, but this made it much worse. After this change, buffer updates to large files should be instantaneous, even if the syntax highlighting update may be slightly delayed. There also will be much less UI lock-up when first opening a large file.
|
Major fix added! @mauricioszabo, with Claude's help, noticed that we've been using An individual attempt to parse a file is supposed to take no more than 3 milliseconds; that's the limit we want to set when parsing. If parsing takes longer than that, it's supposed to go async — and keep making progress on the parsing job in 3ms increments, yielding in between each one, until parsing is finished. This hasn't worked right since version 1.126; all parsing has accidentally been synchronous because the behavior of the After this change, original async behavior has been restored. It will take at least as long for parsing to finish and for syntax highlighting to update, but you are free to edit the buffer as much as you like without waiting for syntax highlighting to finish. That's how it's supposed to work, and how it did work before 1.127. A spec might have caught this — but for a spec to catch this, I'd have to ask it to parse a file large enough that it would reliably take much longer than 3ms to parse no matter which machine it ran on. I don't want to check in a file several megabytes large simply to act as a spec fixture. So if I do write a spec, it'll be a bit more elaborate than that — perhaps dynamically generating a fixture that repeats the contents of a smaller fixture file 1,000 times and writing that to disk temporarily. I may or may not update this PR with that spec. |
1 participant
The opening of #1576 is the occasion that triggers another Tree-sitter fixes PR!