Repository navigation
[TIKA-4408] Fix Python env shebang detection - #4
Conversation
Generated-by: Codex
|
Codex-assisted technical review of cefa669. No blocking correctness issue found in this three-file change. The new rules Independent local validation on Windows / JDK 21:
The local evidence supports the regression claim. Before an Apache PR, carry |
u8026876
left a comment
There was a problem hiding this comment.
No blocking issues found in the scoped review of cefa669.
This was a Codex-assisted walkthrough of the MIME-rule correction and regression tests. I reviewed the explained offset mismatch, expected Python/Shell behaviour, why content-only checks matter, and the supplied test results.
Validation performed by Codex:
- All 18 PythonShebangDetectionTest checks passed. Reverting only the production rule caused 7 of those checks to fail, with no test errors.
- The earlier core clean install passed: 1,377 reported, 0 failures/errors, 11 skipped; XML-provider checks, TikaIT, annotation-processor tests, style and RAT also passed.
- Two additional content-only probes used the real Tika engine with the exact original and fixed MIME registries:
| Input | Original rules | Fixed rules |
|---|---|---|
#!/usr/bin/env python3.12 with a Python body |
application/x-sh |
text/x-python |
Ordinary text x/bin/env python |
text/x-python |
text/plain |
Both fixed results match the intended behaviour. The second probe also demonstrates removal of an accidental offset-1 match.
The extra probes were isolated checks, not a fresh full-suite run. Test execution and this note were assisted by Codex; remote CI and whole-repository validation are not claimed. Complete the remaining group and tutor review before upstream submission.
Keep a single file-start match for #!/usr/bin/env python3, including version suffixes such as python3.11. Remove the single-space env rule and narrow the regression matrix and changelog to the selected scope. Generated-by: Codex
Use #!/usr/bin/env python at file offset zero to cover python and its version suffixes without adding whitespace handling. Restore the bare env python regression in both detection modes and align the changelog. Generated-by: Codex
Follow the existing direct-path Python rule style with paired no-space and single-space literals at offset zero. Restore the single-space regression case in both detection modes and the original changelog text. Generated-by: Codex
Keep both the TIKA-4408 and TIKA-4694 changelog entries. The Python MIME rules and regression tests are unchanged. Generated-by: Codex
Standard Python env shebangs such as
#!/usr/bin/env python3were detected asapplication/x-sh, even with a.pyfilename hint. The original magic rule looked for/bin/env pythonat fixed offset 1.Replace it with two literal file-start prefixes:
The single-space form follows the existing direct-path Python rule style: neighboring
/bin/python,/usr/bin/python, and/usr/local/bin/pythonrules already provide paired no-space and single-space literals.Prefix matching retains
python,python3, and version suffixes such aspython3.11. Include 18 checks covering both detection modes, direct Python paths, Shell controls, filename-only detection and file-start anchoring, plus a changelog entry. General whitespace/Tab handling, regex,env -S, and interpreter-token boundary changes are outside scope.Current branch — 1 November 2026
8152ff5eemerges groupmainat7a609bdb58851a8825e1cce3b5fc014fff174979into this feature branch. TheCHANGES.txtconflict was resolved by keeping both the original TIKA-4408 entry and the TIKA-4694 entry already on main. The Python MIME rules and regression-test file are unchanged. This group PR was subsequently merged on 1 November 2026 asf3a9d0ede. Its final diff contained three files, 80 additions and 1 deletion.Validation — 1 November 2026
Fresh Java 21 verification of the resolved merge tree committed as
8152ff5ee:tika-core: 1,377 tests, 0 failures/errors, 2 skipped.TikaITand all 8 annotation-processor tests passed.BUILD SUCCESS../mvnw -B -ntp -o clean install -pl tika-core -am -Ppedantic \ -Dtika.test.locale=random -Dmaven.repo.local="$PWD/.local_m2_repo"Recorded group-branch negative controls from 25 October 2026 on
471afbe222, with the same unchanged 18-check suite (not repeated in the 1 November group-branch run):.pyhint masks that missing rule.These are local core/reactor results, not whole-repository validation or remote CI.
Review
The Python MIME rules and regression-test file remain byte-identical to the implementation reviewed and approved by Kyubin on
cefa669. The original TIKA-4408 changelog text is preserved. That recorded approval covers the unchanged Python change; it does not imply review of the new merge commit or all intervening main changes. The full repository tree is no longer identical to the old approved tree.Related group issue: #2; upstream ticket: TIKA-4408. This group PR is merged. The change has been submitted to Apache as PR #3473, currently open and awaiting maintainer review.
Course review record
Under the A5 course requirements and corrected group review policy, internal peer review and upstream submission are separate stages. Before an Apache PR, discuss the change with the group, have at least the majority of the group look at it, and have the tutor look at the proposed PR. Project/issue-selection approval is separate from review of a proposed PR. The linked technical reviews establish only their recorded scope and dates; they do not by themselves verify every pre-submission step. This description update does not backdate or certify an unrecorded approval.
Apache submission — 1 November 2026
apache/tika#3473 contains a single commit,
435181f3a, on Apache main9f8b48425fea3630020924f6ae96e98eabb9ce12. Only the three-file TIKA-4408 patch is included; group coordination files, coursework documents and merge history are excluded.Fresh validation on the Apache base: original rule 18 tests / 7 failures / 0 errors; fixed Python suite 18 passed; full core 1,418 reported / 0 failures/errors / 2 skipped; Xerces 12 reported / 0 failures/errors / 2 skipped; TikaIT, all 8 annotation-processor tests and pedantic quality checks passed. The command above was used without
-o. Full-repository validation and successful upstream CI are not claimed.Upstream submission and status update prepared with Codex for @Lynnbb88 (Jiajun Li).