Skip to content

Report the line number of each search result in TypeUses and MethodCalls - #8454

Open
knutwannheden wants to merge 1 commit into
mainfrom
add-a-line-number-to-typeuses-and-friends
Open

Report the line number of each search result in TypeUses and MethodCalls#8454
knutwannheden wants to merge 1 commit into
mainfrom
add-a-line-number-to-typeuses-and-friends

Conversation

@knutwannheden

Copy link
Copy Markdown
Contributor

Motivation

A caller who runs FindTypes or FindMethods learns which files matched and what the matched code was, but not where it was. TypeUses.Row records what matched, not where, so acting on a row means going back and grepping for a position the search had already computed and thrown away. In measured agent runs, 45 of 48 file reads were preceded by a grep whose only purpose was recovering that line number.

SourcePositionService.positionOf already returns absolute line numbers, but it prints from the compilation unit per element, so using it per match would be O(matches × file). This takes the approach used by moderne-cli's SearchIndexing instead: tag the matches, print the file once, recover every position from that one pass.

Examples

TypeUses and MethodCalls each gain a Line number column, appended last so positional consumers of the CSV keep working:

sourceFile,method,className,methodName,argumentTypes,line
A.java,"new B.C().foo(bar, 123)","B$C",foo,"java.lang.String, int",3

A search populates it by handing SearchResultRows the row it wants, in place of calling SearchResult.found:

private <J2 extends TypedTree> J2 found(J2 j) {
    String code = j.printTrimmed(getCursor().getParentTreeCursor());
    return rows.found(j, line -> new TypeUses.Row(sourceFile, code, concreteType, line));
}

@Override
public @Nullable J postVisit(J tree, ExecutionContext ctx) {
    if (tree instanceof SourceFile) {
        rows.insertRows(tree, ctx);
    }
    return super.postVisit(tree, ctx);
}

Summary

  • SearchResultRows (new, rewrite-core) buffers the rows a search produces for one source file, then prints that file once with PrintOutputCapture.MarkerPrinter.FENCED, which wraps each marked element in {{markerId}} after its prefix. A PrintOutputCapture counts newlines as the file prints and places each fence as it arrives, so one print positions every match and no printed text is retained. Files with no matches never print.
  • TypeUses.Row and MethodCalls.Row gain a nullable line column.
  • FindTypes, FindMethods, FindDeprecatedMethods, and FindDistinctMethods populate it — all four recipes that write these two tables.
  • A search reports each element it matches once, tracked by element id, so two visit methods reaching the same element yield one row while a search running after another that marked the same element still reports its own match. Since Markers.add keeps out a SearchResult equal to one already present, and every search result on an element fences in the same place, a row is positioned by whichever marker the element carries.
  • Positions describe the source as printed when the search ran. When no position can be established the column is left empty rather than guessed from the first textual occurrence of the name, which would report matches on unrelated lines that merely share a word.
  • FindTypesBenchmark gains a dense-file scenario, which locates its input by walking up from the working directory. JavaCompilationUnitState resolves the repository root through Class.getResource, which is null once the JMH plugin runs from its shaded jar, so the benchmarks using that state currently fail in warmup on main.

Cost

Measured on J.java (6799 lines, ~190 java.util.List matches) with 3 forks × 10 iterations, the added cost against a search-only recipe run is +2.03ms (+19.8%) and +1.06MB (+13.2%). That is the worst case: one print per matched file, nothing for unmatched files, and UsesType already skips most files.

Removing all ~190 per-match printTrimmed calls was measured to save only 95KB and ≤0.4ms, so the whole-file print is additive here rather than a replacement — TypeUses matches are type names of a few nodes each, unlike the declarations moderne-cli anchors on. The remaining cost is the print traversal itself, where the per-node Cursor allocation in TreeVisitor.visit is the lever; that would speed up all printing and belongs in its own change.

Test plan

  • FindTypesTest.lineNumberOfEachMatch — each match gets its own line, and a javadoc mentioning the type name in prose does not attract a position, so newlines inside comments are counted
  • FindMethodsTest.lineNumberOfEachMatch — invocation, a call split across lines following a three-line comment, and a call inside a lambda
  • FindMethodsTest.reportsAMatchAlreadyMarkedByAnotherSearch — two overlapping searches over one call each report their own row, at the right line; verified failing before the fix
  • FindTypesTest.simpleName, FindTypesTest.dataTable, FindMethodsTest.datatableFormat, FindDistinctMethodsTest.markFirstOccurrenceOfMethod — existing row and CSV assertions updated with expected lines
  • ./gradlew :rewrite-core:test :rewrite-java:test :rewrite-java-test:test
  • ./gradlew licenseFormat (no changes) and :rewrite-benchmarks:jmhClasses

…dCalls`

A caller running `FindTypes` or `FindMethods` learned which files matched and
what the matched code was, but not where it was, so acting on a row meant
grepping for a position the search had already computed.

`SearchResultRows` buffers the rows a search produces for one source file and
prints that file once with `PrintOutputCapture.MarkerPrinter.FENCED`, which
wraps each marked element in `{{markerId}}` after its prefix. A
`PrintOutputCapture` counts newlines as the file prints and places each fence as
it arrives, so one print positions every match at O(file) rather than O(matches
x file), and no printed text is retained. Files with no matches never print.

A search reports each element it matches once, tracked by element id, so that
two visit methods reaching the same element yield one row while a search running
after another that marked the same element still reports its own match. Since
`Markers.add` keeps out a `SearchResult` equal to one already present, and every
search result on an element fences in the same place, a row is positioned by
whichever marker the element carries.

Positions describe the source as printed when the search ran. When no position
can be established the column is left empty rather than guessed from the first
textual occurrence of the name, which would report matches on unrelated lines
that merely share a word.

Measured on `J.java` (6799 lines, ~190 `java.util.List` matches), the added cost
is +2.03ms (+19.8%) and +1.06MB against a search-only recipe run. Removing all
~190 per-match `printTrimmed` calls was measured to save only 95KB and <=0.4ms,
so the whole-file print is additive here rather than a replacement: matches are
type names of a few nodes each.

`FindTypesBenchmark` gains a dense-file scenario. It locates its input by
walking up from the working directory, since `JavaCompilationUnitState` resolves
the repository root through `Class.getResource`, which is null once the JMH
plugin runs from its shaded jar.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant