Skip to content

✨ Add TypeScript and JavaScript comment type support (issue #69) #82

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
arnoox wants to merge 24 commits into
base: main
Choose a base branch
from
Open

✨ Add TypeScript and JavaScript comment type support (issue #69) #82

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
d569bff
✨ NEW: Add TypeScript support for discovery and analyse (#69)
Jun 22, 2026
61468bf
👌 IMPROVE: Resolve merge conflicts with main and keep ts/go/jsonc sup…
Jun 22, 2026
4e9ef51
✨ Update changelog: Add TypeScript comment type support for source di…
Jun 22, 2026
4d85094
✨ Update changelog: Add TypeScript support details for source discove…
Jun 22, 2026
28e12e1
🐛 FIX: Resolve CI pre-commit and docs build failures
Jun 22, 2026
5e7d48b
✨ NEW: Add CLAUDE.md for project guidance and command usage
Jul 13, 2026
b36c271
✨ Add TypeScript support for TSX files and enhance related tests
Jul 13, 2026
5fbc0e7
Merge branch 'main' of https://github.com/useblocks/sphinx-codelinks …
Jul 13, 2026
089816f
Merge branch 'main' of https://github.com/useblocks/sphinx-codelinks …
arnoox Jul 28, 2026
6b726c6
Merge branch 'main' into issue/69-typescript-support
ubmarco Aug 7, 2026
852e50c
🧪 TEST: Add TypeScript declarative extraction fixture cases
ubmarco Aug 7, 2026
0330c0b
📚 DOCS: Trace TypeScript support (FE_TS feature and impl markers)
ubmarco Aug 7, 2026
2ceb5b4
🔧 MAINTAIN: Clarify tsx fixture comment and complete README lang list
ubmarco Aug 7, 2026
279c935
✨ NEW: Widen ts comment type to full TS/JS family
ubmarco Aug 9, 2026
41fb7a9
🐛 FIX: Select TS/TSX grammar per file suffix, not TSX for all
ubmarco Aug 10, 2026
0cc642d
🐛 FIX: Capture legacy html_comment nodes in the ts extraction query
ubmarco Aug 18, 2026
4ba26b7
🐛 FIX: Exclude generated/vendored output from source discovery by def…
ubmarco Aug 18, 2026
22322cf
🧪 TEST: Add html_comment declarative extraction fixture for .js
ubmarco Aug 18, 2026
f16b61c
📚 DOCS: Document the new exclude default and two JSDoc caveats
ubmarco Aug 18, 2026
45d6d1e
🐛 FIX: Scope the ts exclude default to comment_type, not every language
ubmarco Aug 18, 2026
90ca459
🐛 FIX: correct marker field order in TypeScript demo files
ubmarco Aug 19, 2026
dd7554c
📚 DOCS: replace non-existent LANGUAGE_ANALYZERS with real architecture
ubmarco Aug 19, 2026
2da4c49
📚 DOCS: change changelog heading from 'Under development' to 'Unrelea…
ubmarco Aug 19, 2026
de4dfda
📚 DOCS: document .d.ts behavior and qualify JSX comment support
ubmarco Aug 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/components/analyse.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Limitations

**Current Limitations:**

- **Language Support**: C/C++ (``//``, ``/* */``), C# (``//``, ``/* */``, ``///``), Python (``#``), YAML (``#``), Rust (``//``, ``/* */``, ``///``), Go (``//``, ``/* */``) and JSONC (``//``, ``/* */``) comment styles are supported
- **Language Support**: C/C++ (``//``, ``/* */``), C# (``//``, ``/* */``, ``///``), TypeScript (``//``, ``/* */``), Python (``#``), YAML (``#``), Rust (``//``, ``/* */``, ``///``), Go (``//``, ``/* */``) and JSONC (``//``, ``/* */``) comment styles are supported
- **Single Comment Style**: Each analysis run processes only one comment style at a time

Extraction Examples
Expand Down
7 changes: 6 additions & 1 deletion docs/source/components/configuration.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ Specifies the comment syntax style used in the source code files. This determine

**Type:** ``str``
**Default:** ``"cpp"``
**Supported values:** ``"cpp"``, ``"python"``, ``"cs"``, ``"yaml"``, ``"rust"``, ``"go"``, ``"jsonc"``
**Supported values:** ``"cpp"``, ``"python"``, ``"cs"``, ``"ts"``, ``"yaml"``, ``"rust"``, ``"go"``, ``"jsonc"``

.. code-block:: toml

Expand Down Expand Up @@ -304,6 +304,11 @@ Specifies the comment syntax style used in the source code files. This determine
``/* */`` (multi-line),
``///`` (XML doc comments)
- ``.cs``
* - TypeScript
- ``"ts"``
- ``//`` (single-line),
``/* */`` (multi-line)
- ``.ts``, ``.tsx``
* - YAML
- ``"yaml"``
- ``#`` (single-line)
Expand Down
10 changes: 10 additions & 0 deletions docs/source/components/discover.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,13 @@ Usage Examples
include = []
exclude = ["tests/**", "setup.py"]
comment_type = "python"

**TypeScript Project:**

.. code-block:: toml

[source_discover]
src_dir = "./frontend"
include = ["**/*.ts", "**/*.tsx"]
exclude = ["**/*.test.ts", "**/*.spec.ts"]
comment_type = "ts"
14 changes: 14 additions & 0 deletions docs/source/development/change_log.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,20 @@
Changelog
=========

Under development

@patdhlk patdhlk Aug 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heading deviates from the established convention.

Every prior pre-release cycle in this file used Unreleased — e.g. at adf35cc the in-progress section was:

Unreleased
----------

The PR description also says the entry was added "under Upcoming", so the actual heading matches neither. Suggest Unreleased for consistency with the release tooling and history.

Suggested change
Under development
Unreleased
----------

-----------------

New and Improved
................

- ✨ Added TypeScript comment type support for source discovery and analysis.

TypeScript files can now be processed using ``comment_type = "ts"``.
Source discovery supports both ``.ts`` and ``.tsx`` extensions by default.

Fixes
.....

.. _`release:1.3.0`:

1.3.0
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ dependencies = [
# https://github.com/tree-sitter/py-tree-sitter/issues/386#issuecomment-3101430799
"tree-sitter~=0.25.1",
"tree-sitter-c-sharp>=0.23.1",
"tree-sitter-typescript>=0.23.2",
"tree-sitter-yaml>=0.7.1",
"tree-sitter-rust>=0.23.0",
"tree-sitter-go>=0.23.0",
Expand Down
13 changes: 13 additions & 0 deletions src/sphinx_codelinks/analyse/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,13 @@
# @C and C++ Scope Node Types, IMPL_C_2, impl, [FE_C_SUPPORT, FE_CPP]
CommentType.cpp: {"function_definition", "class_definition"},
CommentType.cs: {"method_declaration", "class_declaration", "property_declaration"},
CommentType.ts: {
Comment thread
ubmarco marked this conversation as resolved.
"function_declaration",
"class_declaration",

@patdhlk patdhlk Aug 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Class-field arrow functions resolve to the enclosing class instead of the field.

public_field_definition is missing, so the very common React/Angular class-property handler form loses precision:

class A {
  // @m
  handler = () => {};
}

find_next_scope finds nothing (the sibling is a public_field_definition), then find_enclosing_scope walks up to class_declaration, so tagged_scope becomes the entire class body (verified: class A {\n // @m\n handler = () => {};\n…) rather than the field the comment documents. method_definition is already handled, so getters/methods are fine — this is the field-assigned-arrow gap, which is exactly the case the new _is_function_like_lexical_declaration logic was written to handle at statement level.

"method_definition",
"lexical_declaration",
Comment thread
ubmarco marked this conversation as resolved.
"variable_declaration",

@patdhlk patdhlk Aug 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JavaScript's dominant function-definition forms are not scope types, so markers on them mis-associate — and .js/.cjs/.mjs are newly auto-discovered by this PR.

The scope set covers only ES declaration syntax. Assignment-based definitions — the norm in CommonJS, which is precisely what .cjs exists for — produce assignment_expression, which is not handled at any level:

// @m
module.exports = function f(){};
function unrelated(){}            // -> tagged_scope = "function unrelated(){}"

// @m
Foo.prototype.bar = function(){};
function unrelated(){}            // -> tagged_scope = "function unrelated(){}"

Anonymous default exports (the standard shape for React/Next.js page modules) get no scope at all:

// @m
export default () => {};          // -> tagged_scope = None
// @m
export default function () {};    // -> tagged_scope = None

Widening comment_type = "ts" to the whole JS family (config.py:17) advertises support for these files; the scope table should cover their idioms, or features.rst should state the limitation explicitly.

},
# @Rust Scope Node Types, IMPL_RUST_2, impl, [FE_RUST];
CommentType.rust: {
"function_item",
Expand Down Expand Up @@ -64,6 +71,7 @@
"""
CPP_QUERY = """(comment) @comment"""
C_SHARP_QUERY = """(comment) @comment"""
TYPE_SCRIPT_QUERY = """(comment) @comment"""

@patdhlk patdhlk Aug 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sixth verbatim copy of the same query string.

CPP_QUERY, C_SHARP_QUERY, YAML_QUERY, JSONC_QUERY, BASH_QUERY and now TYPE_SCRIPT_QUERY are all exactly """(comment) @comment""". One shared constant (SIMPLE_COMMENT_QUERY) referenced by the six branches would remove the copy without losing the per-language traceability marker, which can stay on the init_tree_sitter branch.

Relatedly, the if/elif chain in init_tree_sitter is now nine branches of identical shape (import, Language(...), Query(...)); a dict[CommentType, tuple[str, str]] of (module_name, query) driven by importlib.import_module would make the whole function ~6 lines and make the "add a language" edit a single-line data change.

YAML_QUERY = """(comment) @comment"""
RUST_QUERY = """
(line_comment) @comment
Expand Down Expand Up @@ -120,6 +128,11 @@ def init_tree_sitter(comment_type: CommentType) -> tuple[Parser, Query]:

parsed_language = Language(tree_sitter_c_sharp.language())
query = Query(parsed_language, C_SHARP_QUERY)
elif comment_type == CommentType.ts:
import tree_sitter_typescript # noqa: PLC0415

parsed_language = Language(tree_sitter_typescript.language_typescript())
Comment thread
ubmarco marked this conversation as resolved.
Outdated
query = Query(parsed_language, TYPE_SCRIPT_QUERY)
elif comment_type == CommentType.yaml:
import tree_sitter_yaml # noqa: PLC0415

Expand Down
2 changes: 2 additions & 0 deletions src/sphinx_codelinks/source_discover/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
"cpp": ["c", "ci", "cpp", "cc", "cxx", "h", "hpp", "hxx", "hh", "ihl"],
"python": ["py"],
"cs": ["cs"],
"ts": ["ts", "tsx"],
"yaml": ["yml", "yaml"],
"rust": ["rs"],
"go": ["go"],
Expand All @@ -20,6 +21,7 @@ class CommentType(str, Enum):
python = "python"
cpp = "cpp"
cs = "cs"
ts = "ts"
yaml = "yaml"
# @Support Rust style comments, IMPL_RUST_1, impl, [FE_RUST];
rust = "rust"
Expand Down
42 changes: 37 additions & 5 deletions tests/data/discover_fixtures.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,9 @@
"config": {
"src_dir": "src",
"include": [],
"exclude": ["**/build/**"],
"exclude": [
"**/build/**"
],
"gitignore": false,
"comment_type": "cpp"
},
Expand All @@ -94,7 +96,9 @@
},
"config": {
"src_dir": "src",
"include": ["**/*.cpp"],
"include": [
"**/*.cpp"
],
"exclude": [],
"gitignore": false,
"comment_type": "cpp"
Expand All @@ -115,8 +119,12 @@
},
"config": {
"src_dir": "src",
"include": ["**/*.cpp"],
"exclude": ["**/test_*.cpp"],
"include": [
"**/*.cpp"
],
"exclude": [
"**/test_*.cpp"
],
"gitignore": false,
"comment_type": "cpp"
},
Expand Down Expand Up @@ -280,7 +288,9 @@
"config": {
"src_dir": "src",
"include": [],
"exclude": ["**/test_*.cpp"],
"exclude": [
"**/test_*.cpp"
],
"gitignore": true,
"comment_type": "cpp"
},
Expand Down Expand Up @@ -386,5 +396,27 @@
"expected": [
"src/Program.cs"
]
},
{
"name": "typescript_comment_type",
"description": "TypeScript comment type discovers .ts and .tsx files",
"git_init": false,
"files": {
"src/main.ts": "// main",
"src/component.tsx": "// component",
"src/main.cpp": "// not ts",
"src/util.py": "# not ts"
},
"config": {
"src_dir": "src",
"include": [],
"exclude": [],
"gitignore": false,
"comment_type": "ts"
},
"expected": [
"src/component.tsx",
"src/main.ts"
]
}
]
17 changes: 17 additions & 0 deletions tests/data/typescript/demo.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
// regular comment
function testA() {
// @type,TS_REQ_002,TypeScript one-line test

@patdhlk patdhlk Aug 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This marker is in the wrong field order for the default one-line style, so the new integration test asserts on a garbage need.

OneLineCommentStyle() defaults to needs_fields = [title, id, type(default impl), links(default [])]. // @type,TS_REQ_002,TypeScript one-line test therefore parses as:

{'title': 'type', 'id': 'TS_REQ_002', 'type': 'TypeScript one-line test', 'links': []}

(actual output from running SourceAnalyse on this file). The need type becomes the free-text string TypeScript one-line test, which is not a registered need type, and no link is produced. demo.tsx has the same problem (type: 'TypeScript JSX component test').

num_oneline_needs: 1 passes regardless, so the "integration analyse test coverage for TypeScript" never exercises the canonical marker shape that every other language fixture uses (tests/data/rust/demo.rs, tests/data/jsonc/demo.jsonc):

Suggested change
// @type,TS_REQ_002,TypeScript one-line test
// @TypeScript one-line test, TS_REQ_002, impl, [REQ_TS_001]

…and the links assertion is worth adding so the shape is actually pinned.

return 1;
}

/* regular block comment */
const testB = () => {
return 2;
};

// another comment
class Demo {
methodA() {
return 3;
}
}
96 changes: 57 additions & 39 deletions tests/test_analyse.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,101 +56,119 @@ def test_analyse(src_dir, src_paths, tmp_path, snapshot_marks):


@pytest.mark.parametrize(
"src_dir, src_paths , oneline_comment_style, result",
"case",
[
(
TEST_DIR / "data" / "dcdc",
[
{
"src_dir": TEST_DIR / "data" / "dcdc",
"src_paths": [
TEST_DIR / "data" / "dcdc" / "charge" / "demo_1.cpp",
TEST_DIR / "data" / "dcdc" / "charge" / "demo_2.cpp",
TEST_DIR / "data" / "dcdc" / "discharge" / "demo_3.cpp",
TEST_DIR / "data" / "dcdc" / "supercharge.cpp",
],
ONELINE_COMMENT_STYLE,
{
"comment_type": "cpp",
"oneline_comment_style": ONELINE_COMMENT_STYLE,
"result": {
"num_src_files": 4,
"num_uncached_files": 4,
"num_cached_files": 0,
"num_comments": 29,
"num_oneline_warnings": 0,
},
),
(
TEST_DIR / "data" / "oneline_comment_basic",
[
},
{
"src_dir": TEST_DIR / "data" / "oneline_comment_basic",
"src_paths": [
TEST_DIR / "data" / "oneline_comment_basic" / "basic_oneliners.c",
],
ONELINE_COMMENT_STYLE,
{
"comment_type": "cpp",
"oneline_comment_style": ONELINE_COMMENT_STYLE,
"result": {
"num_src_files": 1,
"num_uncached_files": 1,
"num_cached_files": 0,
"num_comments": 14,
"num_oneline_warnings": 0,
"warnings_path_exists": True,
},
),
(
TEST_DIR / "data" / "oneline_comment_default",
[
},
{
"src_dir": TEST_DIR / "data" / "oneline_comment_default",
"src_paths": [
TEST_DIR / "data" / "oneline_comment_default" / "default_oneliners.c",
],
ONELINE_COMMENT_STYLE_DEFAULT,
{
"comment_type": "cpp",
"oneline_comment_style": ONELINE_COMMENT_STYLE_DEFAULT,
"result": {
"num_src_files": 1,
"num_uncached_files": 1,
"num_cached_files": 0,
"num_comments": 5,
"num_oneline_warnings": 1,
"warnings_path_exists": True,
},
),
(
TEST_DIR / "data" / "rust",
[
},
{
"src_dir": TEST_DIR / "data" / "rust",
"src_paths": [
TEST_DIR / "data" / "rust" / "demo.rs",
],
ONELINE_COMMENT_STYLE_DEFAULT,
{
"comment_type": "rust",
"oneline_comment_style": ONELINE_COMMENT_STYLE_DEFAULT,
"result": {
"num_src_files": 1,
"num_uncached_files": 1,
"num_cached_files": 0,
"num_comments": 6,
"num_oneline_warnings": 0,
},
),
(
TEST_DIR / "data" / "jsonc",
[
},
{
"src_dir": TEST_DIR / "data" / "typescript",
"src_paths": [
TEST_DIR / "data" / "typescript" / "demo.ts",
Comment thread
ubmarco marked this conversation as resolved.
],
"comment_type": "ts",
"oneline_comment_style": ONELINE_COMMENT_STYLE_DEFAULT,
"result": {
"num_src_files": 1,
"num_uncached_files": 1,
"num_cached_files": 0,
"num_comments": 4,
"num_oneline_warnings": 0,
},
},
{
"src_dir": TEST_DIR / "data" / "jsonc",
"src_paths": [
TEST_DIR / "data" / "jsonc" / "demo.jsonc",
],
ONELINE_COMMENT_STYLE_DEFAULT,
{
"comment_type": CommentType.jsonc,
"oneline_comment_style": ONELINE_COMMENT_STYLE_DEFAULT,
"result": {
"num_src_files": 1,
"num_uncached_files": 1,
"num_cached_files": 0,
"num_comments": 4,
"num_oneline_warnings": 0,
"comment_type": CommentType.jsonc,
},
),
},
],
)
def test_analyse_oneline_needs(
tmp_path, src_dir, src_paths, oneline_comment_style, result
):
def test_analyse_oneline_needs(tmp_path, case):
src_analyse_config = SourceAnalyseConfig(
src_files=src_paths,
src_dir=src_dir,
src_files=case["src_paths"],
src_dir=case["src_dir"],
get_need_id_refs=False,
get_oneline_needs=True,
get_rst=False,
oneline_comment_style=oneline_comment_style,
comment_type=result.get("comment_type", CommentType.cpp),
oneline_comment_style=case["oneline_comment_style"],
comment_type=case["comment_type"],
)
src_analyse = SourceAnalyse(src_analyse_config)
src_analyse.run()

result = case["result"]
assert len(src_analyse.src_files) == result["num_src_files"]
assert len(src_analyse.oneline_warnings) == result["num_oneline_warnings"]
Comment thread
ubmarco marked this conversation as resolved.

Expand Down
Loading
Loading