Skip to content
Open
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
d569bff
✨ NEW: Add TypeScript support for discovery and analyse (#69)
Jun 22, 2026
61468bf
👌 IMPROVE: Resolve merge conflicts with main and keep ts/go/jsonc sup…
Jun 22, 2026
4e9ef51
✨ Update changelog: Add TypeScript comment type support for source di…
Jun 22, 2026
4d85094
✨ Update changelog: Add TypeScript support details for source discove…
Jun 22, 2026
28e12e1
🐛 FIX: Resolve CI pre-commit and docs build failures
Jun 22, 2026
5e7d48b
✨ NEW: Add CLAUDE.md for project guidance and command usage
Jul 13, 2026
b36c271
✨ Add TypeScript support for TSX files and enhance related tests
Jul 13, 2026
5fbc0e7
Merge branch 'main' of https://github.com/useblocks/sphinx-codelinks …
Jul 13, 2026
089816f
Merge branch 'main' of https://github.com/useblocks/sphinx-codelinks …
arnoox Jul 28, 2026
6b726c6
Merge branch 'main' into issue/69-typescript-support
ubmarco Aug 7, 2026
852e50c
🧪 TEST: Add TypeScript declarative extraction fixture cases
ubmarco Aug 7, 2026
0330c0b
📚 DOCS: Trace TypeScript support (FE_TS feature and impl markers)
ubmarco Aug 7, 2026
2ceb5b4
🔧 MAINTAIN: Clarify tsx fixture comment and complete README lang list
ubmarco Aug 7, 2026
279c935
✨ NEW: Widen ts comment type to full TS/JS family
ubmarco Aug 9, 2026
41fb7a9
🐛 FIX: Select TS/TSX grammar per file suffix, not TSX for all
ubmarco Aug 10, 2026
0cc642d
🐛 FIX: Capture legacy html_comment nodes in the ts extraction query
ubmarco Aug 18, 2026
4ba26b7
🐛 FIX: Exclude generated/vendored output from source discovery by def…
ubmarco Aug 18, 2026
22322cf
🧪 TEST: Add html_comment declarative extraction fixture for .js
ubmarco Aug 18, 2026
f16b61c
📚 DOCS: Document the new exclude default and two JSDoc caveats
ubmarco Aug 18, 2026
45d6d1e
🐛 FIX: Scope the ts exclude default to comment_type, not every language
ubmarco Aug 18, 2026
90ca459
🐛 FIX: correct marker field order in TypeScript demo files
ubmarco Aug 19, 2026
dd7554c
📚 DOCS: replace non-existent LANGUAGE_ANALYZERS with real architecture
ubmarco Aug 19, 2026
2da4c49
📚 DOCS: change changelog heading from 'Under development' to 'Unrelea…
ubmarco Aug 19, 2026
de4dfda
📚 DOCS: document .d.ts behavior and qualify JSX comment support
ubmarco Aug 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 94 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

This repository already has a detailed **AGENTS.md** at the repo root — read it first for
full architecture diagrams, event-handler tables, commit/PR conventions, and common-pattern
recipes (adding a language, a marker type, a CLI command, a config option). This file only
covers what's needed to get moving quickly.

## What this project is

sphinx-codelinks is a Sphinx extension providing fast source-code traceability for
Sphinx-Needs: it scans source files (C++, Python, C#, Rust, TypeScript, Go, YAML, JSON) for
marker comments via tree-sitter, and generates Sphinx-Needs items / RST that link
documentation back to exact source locations.

## Commands

All commands run through `tox` (uses `tox-uv`).

```bash
# Run default test env (py312-sphinx8-needs5)
tox

# List all test env combinations (py{312,313,314}-sphinx{7,8,9}-needs{5,6,7,8})
tox -a

# Run a specific env / file / test
tox -e py312-sphinx8-needs5
tox -e py312-sphinx8-needs5 -- tests/test_analyse.py
tox -e py312-sphinx8-needs5 -- tests/test_analyse.py::test_function_name

# Update syrupy snapshots
tox -e py312-sphinx8-needs5 -- --snapshot-update

# Type check / lint / format
tox -e mypy
tox -e ruff-check
tox -e ruff-fmt
pre-commit run --all-files

# Docs
tox -e docs-clean
tox -e docs-update
BUILDER=linkcheck tox -e docs-clean
tox -e docs-live

# End-to-end demo (analyse -> write RST -> build docs)
tox -e demo
```

The CLI itself is installed as `codelinks` (`codelinks analyse <config.toml>`,
`codelinks write rst <input.json> --outpath <file>`).

## Architecture

Pipeline: **Source Files → Discovery → Parsing → Analysis → Results (JSON) → RST Generation**

- `source_discover/` — finds source files by include/exclude patterns, respects `.gitignore`.
- `analyse/oneline_parser.py` — tree-sitter based parser extracting comment marker nodes.
- `analyse/projects.py` — per-language analyzers, registered in a `LANGUAGE_ANALYZERS` dict.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This describes an architecture that does not exist, and points future agents at a recipe that cannot be followed.

analyse/projects.py is 78 lines containing a single AnalyseProjects class. There is no LANGUAGE_ANALYZERS dict and no per-language analyzer class anywhere in the repo:

$ grep -rn LANGUAGE_ANALYZERS src/    # no matches

Line 66 then tells the reader that adding a language "follow[s] a short recipe documented in AGENTS.md ... follow those rather than inventing a new approach" — but that recipe (AGENTS.md:400-417) instructs creating a BaseAnalyzer subclass and registering it in LANGUAGE_ANALYZERS, which is fiction. This PR itself could not follow it; TypeScript support landed via SCOPE_NODE_TYPES + init_tree_sitter in analyse/utils.py, which the new file never mentions.

Also: line 13's language list omits Bash, which shipped in 1.4.0.

Since a wrong CLAUDE.md actively misdirects, it is worth either correcting these two sections against analyse/utils.py, or dropping the architecture section and deferring to AGENTS.md. Separately, adding CLAUDE.md is unrelated to issue #69 and would be easier to review as its own PR.

- `analyse/analyse.py` — orchestrates discovery + parsing + analysis into `analyse/models.py`
Pydantic result models.
- `needextend_write.py` — turns analysis JSON into RST with Sphinx-Needs `needextend`
directives.
- `config.py` — Pydantic v2 config models (`AnalyseConfig` etc.), loadable from TOML.
- `sphinx_extension/source_tracing.py` — the Sphinx extension `setup()`; wires into Sphinx
build events (`config-inited`, `builder-inited`, `env-before-read-docs`,
`html-collect-pages`, `html-page-context`, `build-finished`) to register sphinx-needs extra
options/types, generate standalone traced-source HTML pages, and inject CSS
(`sphinx_extension/ub_sct.css`). See AGENTS.md for the full event table and mermaid diagram.

Adding a new language analyzer, marker type, CLI command, or config option each follow a
short recipe documented in AGENTS.md under "Common Patterns" — follow those rather than
inventing a new approach.

## Code style

- Ruff for lint/format (strict rule set incl. `S`, `PL`, `PTH`, `SIM`, `SLF`; see
`pyproject.toml` for per-file ignores).
- Mypy strict mode (`disallow_any_*`, `disallow_untyped_*`); relaxed for `tests/*` and
`sphinx_codelinks.*` via overrides in `pyproject.toml`.
- Full type annotations everywhere; Pydantic models (frozen where possible) for config/data.
- Sphinx-style docstrings (`:param:`, `:return:`, `:raises:`), no types in docstrings.
- Prefer pure functions and immutable data structures.

## Testing

- `pytest` with fixtures in `tests/conftest.py`; test data in `tests/data/`; Sphinx
integration tests use real minimal Sphinx projects in `tests/doc_test/`.
- `syrupy` for snapshot testing of complex outputs (JSON, doctrees) — use
`snapshot.assert_match()` and re-run with `--snapshot-update` when output intentionally
changes.
- Use `@pytest.mark.parametrize` for multi-language / multi-scenario tests.
2 changes: 1 addition & 1 deletion docs/source/components/analyse.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Limitations

**Current Limitations:**

- **Language Support**: C/C++ (``//``, ``/* */``), C# (``//``, ``/* */``, ``///``), Python (``#``), YAML (``#``), Rust (``//``, ``/* */``, ``///``), Go (``//``, ``/* */``), JSONC (``//``, ``/* */``) and Bash (``#``) comment styles are supported
- **Language Support**: C/C++ (``//``, ``/* */``), C# (``//``, ``/* */``, ``///``), TypeScript/JavaScript (``//``, ``/* */``), Python (``#``), YAML (``#``), Rust (``//``, ``/* */``, ``///``), Go (``//``, ``/* */``), JSONC (``//``, ``/* */``) and Bash (``#``) comment styles are supported
- **Single Comment Style**: Each analysis run processes only one comment style at a time

Extraction Examples
Expand Down
8 changes: 7 additions & 1 deletion docs/source/components/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ Specifies the comment syntax style used in the source code files. This determine

**Type:** ``str``
**Default:** ``"cpp"``
**Supported values:** ``"cpp"``, ``"python"``, ``"cs"``, ``"yaml"``, ``"rust"``, ``"go"``, ``"jsonc"``, ``"bash"``
**Supported values:** ``"cpp"``, ``"python"``, ``"cs"``, ``"ts"``, ``"yaml"``, ``"rust"``, ``"go"``, ``"jsonc"``, ``"bash"``

.. code-block:: toml

Expand Down Expand Up @@ -304,6 +304,12 @@ Specifies the comment syntax style used in the source code files. This determine
``/* */`` (multi-line),
``///`` (XML doc comments)
- ``.cs``
* - TypeScript / JavaScript
- ``"ts"``
- ``//`` (single-line),
``/* */`` (multi-line)
- ``.ts``, ``.tsx``, ``.mts``, ``.cts``, ``.js``, ``.jsx``, ``.mjs``
and ``.cjs``
* - YAML
- ``"yaml"``
- ``#`` (single-line)
Expand Down
10 changes: 10 additions & 0 deletions docs/source/components/discover.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,13 @@ Usage Examples
include = []
exclude = ["tests/**", "setup.py"]
comment_type = "python"

**TypeScript Project:**

.. code-block:: toml

[source_discover]
src_dir = "./frontend"
include = ["**/*.ts", "**/*.tsx"]
exclude = ["**/*.test.ts", "**/*.spec.ts"]
comment_type = "ts"
28 changes: 28 additions & 0 deletions docs/source/components/features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,34 @@ Features
.. fault:: Sphinx-codelinks hallucinates traceability objects in Bash
:id: FAULT_BASH_2

.. feature:: TypeScript Language Support
:id: FE_TS

Support for defining traceability objects in TypeScript and JavaScript source
files via one-line comment annotations.

The TypeScript language parser leverages tree-sitter to accurately identify and
extract comments from TypeScript and JavaScript sources, including single-line
(``//``) and multi-line (``/* */``) comment styles. All files are parsed with
the TSX grammar — a strict superset of the TypeScript grammar, which is in turn
a superset of JavaScript — so ``.tsx`` files (including JSX comments such as
``{/* ... */}``) and plain JavaScript sources need no per-file grammar choice.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The advertised {/* … */} JSX capability does not work in its idiomatic single-line form — it produces a warning and no need.

Inside JSX children // is impossible, so {/* … */} on one line is the way to comment there. With the documented default style it silently fails (run against this branch):

const App = () => (
  <div>
    {/* @Tsx Title, IMPL_TSX, impl, [REQ_TSX] */}
  </div>
);
oneline_needs: []
warnings:      ['not_start_or_end_with_square_brackets']

The default end_sequence is \n, so the trailing */} is swallowed into the links field and the bracket check rejects it. The same applies to the plainly-documented /* */ style for .ts (analyse.rst:50): /* @Blk Title, IMPL_BLK, impl, [REQ_BLK] */ yields no need either.

tests/data/extraction/oneline.yaml acknowledges this and works around it by putting the marker on its own line inside the block, explicitly "deliberately not exercised by the shared fixtures". That leaves the newly advertised feature's most common shape both broken and untested. Either handle a block-comment terminator as an implicit end sequence, or narrow the docs to say markers must be on their own line inside a block/JSX comment.


Key capabilities:

* Detection of inline and block comments
* Association of comments with function, class, and method declarations
* ``const``/``let``/``var`` declarations count as scopes only when they assign
a function or arrow function
* File extensions ``.ts``, ``.tsx``, ``.mts``, ``.cts``, ``.js``, ``.jsx``,
``.mjs`` and ``.cjs`` auto-discovered when ``comment_type = "ts"``

.. fault:: Traceability objects are not detected in TypeScript language
:id: FAULT_TS_1

.. fault:: Sphinx-codelinks hallucinates traceability objects in TypeScript
:id: FAULT_TS_2

.. feature:: Preprocessor-Aware C/C++ Extraction
:id: FE_PREPROC

Expand Down
13 changes: 13 additions & 0 deletions docs/source/development/change_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,19 @@
Changelog
=========

Under development

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heading deviates from the established convention.

Every prior pre-release cycle in this file used Unreleased — e.g. at adf35cc the in-progress section was:

Unreleased
----------

The PR description also says the entry was added "under Upcoming", so the actual heading matches neither. Suggest Unreleased for consistency with the release tooling and history.

Suggested change
Under development
Unreleased
----------

-----------------

New and Improved
................

- ✨ Added TypeScript comment type support for source discovery and analysis.

TypeScript and JavaScript files can now be processed using ``comment_type = "ts"``,
since the TSX grammar used to parse them is a superset of both languages.
Source discovery supports ``.ts``, ``.tsx``, ``.mts``, ``.cts``, ``.js``,
``.jsx``, ``.mjs`` and ``.cjs`` extensions by default.

.. _`release:1.4.0`:

1.4.0
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ dependencies = [
# https://github.com/tree-sitter/py-tree-sitter/issues/386#issuecomment-3101430799
"tree-sitter~=0.25.1",
"tree-sitter-c-sharp>=0.23.1",
"tree-sitter-typescript>=0.23.2",
"tree-sitter-yaml>=0.7.1",
"tree-sitter-rust>=0.23.0",
"tree-sitter-go>=0.23.0",
Expand Down
61 changes: 56 additions & 5 deletions src/sphinx_codelinks/analyse/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,14 @@
# @C and C++ Scope Node Types, IMPL_C_2, impl, [FE_C_SUPPORT, FE_CPP]
CommentType.cpp: {"function_definition", "class_definition"},
CommentType.cs: {"method_declaration", "class_declaration", "property_declaration"},
# @TypeScript Scope Node Types, IMPL_TS_2, impl, [FE_TS]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SCOPE_NODE_TYPES[CommentType.ts] omits TypeScript's type-level declarations, so a marker documenting one mis-associates with the next unrelated declaration.

find_next_scope walks forward until something matches, so an unmatched declaration is not "no scope" — it is a wrong scope. Verified on this branch (find_associated_scope(comment, CommentType.ts)):

source reported tagged_scope
interface Foo {…} then function unrelated(){} function unrelated() {}
enum E { A } then function unrelated(){} function unrelated() {}
type T = number; then function unrelated(){} function unrelated() {}
abstract class Base {} then function unrelated(){} function unrelated() {}
function* gen(){} then function unrelated(){} function unrelated() {}

That is FAULT_TS_2 ("hallucinates traceability objects") for five of the most common TS declaration forms. abstract_class_declaration is the starkest: class_declaration is in the set, so class Foo {} works and abstract class Foo {} silently points at the wrong code.

Missing node types: interface_declaration, enum_declaration, type_alias_declaration, abstract_class_declaration, generator_function_declaration. Compare CommentType.rust, which does include struct_item/enum_item/trait_item.

CommentType.ts: {
Comment thread
ubmarco marked this conversation as resolved.
"function_declaration",
"class_declaration",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Class-field arrow functions resolve to the enclosing class instead of the field.

public_field_definition is missing, so the very common React/Angular class-property handler form loses precision:

class A {
  // @m
  handler = () => {};
}

find_next_scope finds nothing (the sibling is a public_field_definition), then find_enclosing_scope walks up to class_declaration, so tagged_scope becomes the entire class body (verified: class A {\n // @m\n handler = () => {};\n…) rather than the field the comment documents. method_definition is already handled, so getters/methods are fine — this is the field-assigned-arrow gap, which is exactly the case the new _is_function_like_lexical_declaration logic was written to handle at statement level.

"method_definition",
"lexical_declaration",
Comment thread
ubmarco marked this conversation as resolved.
"variable_declaration",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JavaScript's dominant function-definition forms are not scope types, so markers on them mis-associate — and .js/.cjs/.mjs are newly auto-discovered by this PR.

The scope set covers only ES declaration syntax. Assignment-based definitions — the norm in CommonJS, which is precisely what .cjs exists for — produce assignment_expression, which is not handled at any level:

// @m
module.exports = function f(){};
function unrelated(){}            // -> tagged_scope = "function unrelated(){}"

// @m
Foo.prototype.bar = function(){};
function unrelated(){}            // -> tagged_scope = "function unrelated(){}"

Anonymous default exports (the standard shape for React/Next.js page modules) get no scope at all:

// @m
export default () => {};          // -> tagged_scope = None
// @m
export default function () {};    // -> tagged_scope = None

Widening comment_type = "ts" to the whole JS family (config.py:17) advertises support for these files; the scope table should cover their idioms, or features.rst should state the limitation explicitly.

},
# @Rust Scope Node Types, IMPL_RUST_2, impl, [FE_RUST];
CommentType.rust: {
"function_item",
Expand Down Expand Up @@ -66,6 +74,8 @@
"""
CPP_QUERY = """(comment) @comment"""
C_SHARP_QUERY = """(comment) @comment"""
# @TypeScript comment query for tree-sitter, IMPL_TS_3, impl, [FE_TS]
TYPE_SCRIPT_QUERY = """(comment) @comment"""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sixth verbatim copy of the same query string.

CPP_QUERY, C_SHARP_QUERY, YAML_QUERY, JSONC_QUERY, BASH_QUERY and now TYPE_SCRIPT_QUERY are all exactly """(comment) @comment""". One shared constant (SIMPLE_COMMENT_QUERY) referenced by the six branches would remove the copy without losing the per-language traceability marker, which can stay on the init_tree_sitter branch.

Relatedly, the if/elif chain in init_tree_sitter is now nine branches of identical shape (import, Language(...), Query(...)); a dict[CommentType, tuple[str, str]] of (module_name, query) driven by importlib.import_module would make the whole function ~6 lines and make the "add a language" edit a single-line data change.

YAML_QUERY = """(comment) @comment"""
RUST_QUERY = """
(line_comment) @comment
Expand Down Expand Up @@ -107,7 +117,7 @@ def is_text_file(filepath: Path, sample_size: int = 2048) -> bool:
return False


# @Tree-sitter parser initialization for multiple languages, IMPL_LANG_1, impl, [FE_C_SUPPORT, FE_CPP, FE_PY, FE_YAML, FE_RUST, FE_GO, FE_JSONC, FE_BASH]
# @Tree-sitter parser initialization for multiple languages, IMPL_LANG_1, impl, [FE_C_SUPPORT, FE_CPP, FE_PY, FE_YAML, FE_RUST, FE_GO, FE_JSONC, FE_BASH, FE_TS]
def init_tree_sitter(comment_type: CommentType) -> tuple[Parser, Query]:
if comment_type == CommentType.cpp:
import tree_sitter_cpp # noqa: PLC0415
Expand All @@ -124,6 +134,15 @@ def init_tree_sitter(comment_type: CommentType) -> tuple[Parser, Query]:

parsed_language = Language(tree_sitter_c_sharp.language())
query = Query(parsed_language, C_SHARP_QUERY)
elif comment_type == CommentType.ts:
import tree_sitter_typescript # noqa: PLC0415

# The TSX grammar is a strict superset of the TypeScript grammar, which is
# itself a superset of JavaScript, so it also parses plain .ts and the
# whole JavaScript family fine. Use it for all of them to avoid needing a
# per-file grammar choice.
parsed_language = Language(tree_sitter_typescript.language_tsx())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The TSX grammar is not a superset of the TypeScript grammar — .ts files using angle-bracket type assertions silently lose all markers after the first cast.

TypeScript deliberately forbids <T>expr assertions in .tsx because they are ambiguous with JSX; that is exactly why tree-sitter-typescript ships two grammars. Under language_tsx() a plain-TS cast becomes an unterminated JSX element and the rest of the file collapses into one ERROR node whose contents are lexed as jsx_text — so the comments in it are never captured at all.

Reproduced against this branch:

// @m1
function a() {}

const x = <string>value;   // legal .ts, illegal .tsx

// @m2
function b() {}

// @m3
class C {}
language_tsx():         has_error=True   comments found = 1   (@m2, @m3 silently gone)
                        tree: (program (comment) (function_declaration ...) (ERROR ...))
language_typescript():  has_error=False  comments found = 3

This is silent data loss (FAULT_TS_1), not a degraded scope. The same false "strict superset" claim is repeated in source_discover/config.py:14, docs/source/components/features.rst:278, and the changelog, so it will be trusted by future readers.

Fix: pick the grammar from the file suffix — language_tsx() for .tsx/.jsx, language_typescript() for .ts/.mts/.cts/.js/.mjs/.cjs. That requires init_tree_sitter to receive the path (or SourceAnalyse to cache one parser per grammar), which is the deeper fix the "one grammar for everything" shortcut is avoiding.

query = Query(parsed_language, TYPE_SCRIPT_QUERY)
elif comment_type == CommentType.yaml:
import tree_sitter_yaml # noqa: PLC0415

Expand Down Expand Up @@ -177,14 +196,46 @@ def extract_comments(
return captures.get("comment")


TS_FUNCTION_VALUE_TYPES = {"arrow_function", "function_expression"}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TS_FUNCTION_VALUE_TYPES is too narrow — generator, class, and cast-wrapped values fall through the new guard and the comment jumps to an unrelated declaration.

The guard correctly stops a plain const from stealing the association, but everything it fails to recognise is treated as a plain const, so find_next_scope keeps walking. Verified:

// @m
const gen = function* () {};      // value type: generator_function
function unrelated() {}           // -> tagged_scope = "function unrelated() {}"

// @m
const A = class {};               // value type: class
function unrelated() {}           // -> tagged_scope = "function unrelated() {}"

// @m
const f = (() => {}) as Handler;  // value type: as_expression
function unrelated() {}           // -> tagged_scope = "function unrelated() {}"

Suggest adding generator_function and class, and unwrapping the TS expression wrappers (as_expression, satisfies_expression, parenthesized_expression, non_null_expression) before the type test — const f = (() => {}) as Handler is idiomatic typed-callback style.

Suggested change
TS_FUNCTION_VALUE_TYPES = {"arrow_function", "function_expression"}
TS_FUNCTION_VALUE_TYPES = {
"arrow_function",
"function_expression",
"generator_function",
"class",
}
# expression wrappers to look through before testing the value type
TS_VALUE_WRAPPER_TYPES = {
"as_expression",
"satisfies_expression",
"parenthesized_expression",
"non_null_expression",
}



def _is_function_like_lexical_declaration(node: TreeSitterNode) -> bool:
"""True if a TS lexical/variable declaration's declarator is a function.

``const``/``let``/``var`` declarations are only treated as scopes when they
assign a function or arrow function, so a leading comment doesn't bind to an
unrelated ``const`` that merely precedes the function it documents.
"""
for declarator in node.named_children:
if declarator.type != "variable_declarator":
continue
value = declarator.child_by_field_name("value")
if value is not None and value.type in TS_FUNCTION_VALUE_TYPES:
return True
return False


def _matches_scope(
node: TreeSitterNode, scope_types: set[str], comment_type: CommentType
) -> bool:
if node.type not in scope_types:
return False
if comment_type == CommentType.ts and node.type in {
"lexical_declaration",
"variable_declaration",
}:
return _is_function_like_lexical_declaration(node)
return True


def find_enclosing_scope(
node: TreeSitterNode, comment_type: CommentType = CommentType.cpp
) -> TreeSitterNode | None:
"""Find the enclosing scope of a comment."""
scope_types = SCOPE_NODE_TYPES.get(comment_type, SCOPE_NODE_TYPES[CommentType.cpp])
current: TreeSitterNode = node
while current:
if current.type in scope_types:
if _matches_scope(current, scope_types, comment_type):
return current
current: TreeSitterNode | None = current.parent # type: ignore[no-redef] # required for node traversal
return None
Expand All @@ -197,12 +248,12 @@ def find_next_scope(
scope_types = SCOPE_NODE_TYPES.get(comment_type, SCOPE_NODE_TYPES[CommentType.cpp])
current: TreeSitterNode = node
while current:
if current.type in scope_types:
if _matches_scope(current, scope_types, comment_type):
return current
current: TreeSitterNode | None = current.next_named_sibling # type: ignore[no-redef] # required for node traversal
if current and current.type == "block":
if current and current.type in {"block", "export_statement"}:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A TypeScript-only container type is hardcoded into the shared traversal, applied to every language.

{"block", "export_statement"} now mixes a C/C++ node type with a TS/JS one in a single set consulted for all nine comment types, and _matches_scope (line 223) hardcodes comment_type == CommentType.ts inside an otherwise language-agnostic helper. The module already has the right mechanism for this — the per-language SCOPE_NODE_TYPES table, whose header comment explains carefully which languages participate and why.

Suggest a parallel per-language table so the next language adds a dict entry instead of another or-clause in shared code, e.g. SCOPE_CONTAINER_TYPES: dict[CommentType, set[str]] (cpp → {"block"}, ts → {"block", "export_statement"}) plus SCOPE_PREDICATES: dict[CommentType, dict[str, Callable[[Node], bool]]] for the lexical_declaration refinement. As written, TS's other wrapper (ambient_declaration) is already missed by the same shortcut.

for child in current.named_children:
if child.type in scope_types:
if _matches_scope(child, scope_types, comment_type):
return child
return None

Expand Down
8 changes: 8 additions & 0 deletions src/sphinx_codelinks/source_discover/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@
"cpp": ["c", "ci", "cpp", "cc", "cxx", "h", "hpp", "hxx", "hh", "ihl"],
"python": ["py"],
"cs": ["cs"],
# ".mts"/".cts" are TypeScript's own ESM/CJS module variants. ".js"/".jsx"/
# ".mjs"/".cjs" are JavaScript, covered by the same comment type because the
# TSX grammar used to parse "ts" sources is a strict superset of the
# TypeScript grammar, which is itself a superset of JavaScript, so no
# separate grammar or comment_type value is needed.
"ts": ["ts", "tsx", "mts", "cts", "js", "jsx", "mjs", "cjs"],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.d.ts declaration files are pulled into discovery by the ts suffix but no ambient node type is a scope, so every marker in them mis-associates or gets a null scope.

SourceDiscover matches on filepath.suffix.lower(), and Path("api.d.ts").suffix == ".ts" — so declaration files are discovered with no way to opt out except a user-supplied exclude. Their entire content is ambient declarations, none of which SCOPE_NODE_TYPES[CommentType.ts] recognises:

declare function f(): void;   // ambient_declaration > function_signature -> None
declare module "x" { }        // ambient_declaration > module              -> None
interface I { doThing(): void; }  // -> None

Either add ambient_declaration / function_signature / module to the scope table, or exclude .d.ts from discovery under comment_type = "ts" — the latter mirroring the existing _json_starts_with_comment gate that keeps plain .json out of the jsonc type.

"yaml": ["yml", "yaml"],
"rust": ["rs"],
"go": ["go"],
Expand All @@ -26,6 +32,8 @@ class CommentType(str, Enum):
python = "python"
cpp = "cpp"
cs = "cs"
# @Support TypeScript style comments, IMPL_TS_1, impl, [FE_TS];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding one language now requires five unsynchronised edits, and divergence fails with a bare KeyError.

ts had to be added to: the CommentType enum (here), COMMENT_FILETYPE (line 17), SCOPE_NODE_TYPES (analyse/utils.py:31), a new *_QUERY constant (utils.py:78), and an elif branch in init_tree_sitter (utils.py:137). Nothing enforces the join:

  • COMMENT_FILETYPE is keyed by raw str, not CommentType, so an enum member without a dict entry blows up as KeyError in SourceDiscover.__init__ (line 37) with no diagnostic.
  • Conversely init_tree_sitter raises ValueError, and a missing SCOPE_NODE_TYPES entry silently falls back to the C++ scope set (utils.py:236) rather than erroring — a wrong-language default.

A single per-language record (extensions + grammar module + query + scope types), with the enum derived from it, would collapse the five edits into one and make the fallback impossible. At minimum, a test asserting set(COMMENT_FILETYPE) == {c.value for c in CommentType} and that every member has a SCOPE_NODE_TYPES entry would catch the divergence.

ts = "ts"
yaml = "yaml"
# @Support Rust style comments, IMPL_RUST_1, impl, [FE_RUST];
rust = "rust"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"needs": [
{
"id": "IMPL_TS",
"title": "Ts Title",
"type": "impl",
"links": {
"links": [
"REQ_TS"
]
},
"metadata": {},
"line": 1
}
],
"need_refs": [],
"marked_rst": [],
"warnings": []
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"needs": [
{
"id": "IMPL_TSX",
"title": "Tsx Title",
"type": "impl",
"links": {
"links": [
"REQ_TSX"
]
},
"metadata": {},
"line": 4
}
],
"need_refs": [],
"marked_rst": [],
"warnings": []
}
Loading
Loading