Report files that still carry http:// Plazi subjects in the export trailer - #8
Merged
Merged
Conversation
…ailer Since plazi/gg2rdf#33 every Plazi resource IRI is https://, so a treatment subject and the graph this service names after the file are the same IRI. Nothing in this repo needed to change for that: the graph prefix was https:// all along, and the n-triples are passed through as they are. What did need a home is the transition. Files generated before the switch keep loading, into a graph whose name differs from the subject by scheme only, and there was no way to see how many of them are left. The `# export complete:` trailer of /nquads and /ntriples now counts files with http:// subjects under a Plazi host, so the operator can tell when the index is worth rebuilding. The README states the contract between graph name and subject IRI, and why the prefix is not a deployment setting. Closes #5. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The implementation and tests are reviewed; the remaining README feedback is a minor, non-blocking wording nit.
Review effort: Lite
Findings: None
What changed in this PR
Adds reporting for legacy http:// Plazi subjects in export trailers and documents the migration requirements.
Changes:
- Counts affected files in
/nquadsand/ntriplesexports. - Adds tests for reporting and silent behavior.
- Documents graph/subject IRI alignment and QLever rebuild guidance.
| File | Reviewed changes |
|---|---|
src/endpoints.ts |
Detects and reports legacy Plazi subjects per file. |
src/endpoints.test.ts |
Tests counting and zero-count behavior. |
README.md |
Documents IRI contracts and migration guidance; includes a minor wording nit about the upstream change. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #5.
Companion to plazi/gg2rdf#34 (the fix — gg2rdf now mints
https://subjects; closes plazi/gg2rdf#33) and plazi/turtle-hook#13 (the other loader; closes plazi/turtle-hook#12).What changes
No behaviour of the loader changes.
graphUriPrefixwashttps://treatment.plazi.org/idall along and stays; the n-triples are passed through untouched. Once gg2rdf's change lands, a treatment subject and the graph this service names after the file are the same IRI, which is what the issue asked for.What did need a home is the transition. Files generated before the switch keep loading — into a graph whose name differs from the subject by scheme only — and there was no way to see how many of them are left. Two additions:
# export complete:trailer of/nquadsand/ntriplesnow counts files that have ahttp://subject under a Plazi host (treatment,taxon-name,taxon-concept,publication,tb), e.g.… took 312.4s, 41203 files still with http:// plazi subjects (pre-gg2rdf#33, regenerate). Silent when there are none. One regex test per line, only until the first hit per file.Verification
deno test --no-check --allow-read --allow-write --allow-env --allow-net --allow-run: 10 passed (two new tests: the count, and its absence).--no-checkbecausemainalready fails type-checking under Deno 2.9 (setTimeoutreturningTimeoutinendpoints.ts:71andintegration.test.ts:103) — unrelated to this change, not touched here.🤖 Generated with Claude Code