Skip to content

6197: Degenerate WKT polygons are dropped, losing spatial for 10% of the OpenTopography feed #6294

Description

@cody-seibert-gsa

Code repo: GSA/datagov-harvester

Problem

Zero-area WKT geometries are silently dropped, so the record's translated_spatial is NULL and it disappears from spatial search.

translate_wkt_to_geojson() (general_utils.py:1142) parses the WKT fine, then validate_geojson() (:968) rejects it with less_three_unique_nodes / exterior_not_ccw.

This hits 83 of 802 datasets (10.3%) in the live OpenTopography feed (https://portal.opentopography.org/csw/catalog30.json). That source rounds coordinates to 2 decimal places, which collapses small-extent surveys into degenerate polygons:

POLYGON((-115.63 32.49, -115.63 32.49, -115.63 32.49, -115.63 32.49, -115.63 32.49))
POLYGON((-87.88 43.34, -87.88 43.35, -87.88 43.35, -87.88 43.34, -87.88 43.34))

Why it's a bug, not bad data

munge_spatial() (:1161) already handles exactly this for v1.1 comma-separated bboxes — duplicate corners collapse to a Point, 3-unique-corner boxes to a LineString. The WKT path skips that logic entirely:

Same geometry, different notation Result
-115.63,32.49,-115.63,32.49 (v1.1 bbox) ✅ Point
-87.88,43.34,-87.88,43.35 (v1.1 bbox) ✅ LineString
POINT(-115.63 32.49) (WKT) ✅ Point
POLYGON((-115.63 32.49, …×5)) (WKT) ❌ NULL

Suggested fix

In translate_wkt_to_geojson, after shapely.wkt.loads + force_2d, reduce degenerate geometries before returning: dedupe the ring and emit Point for 1 unique vertex, LineString for 2. shapely already exposes geom.is_empty, geom.area, and geom.exterior.coords.

Acceptance criteria

  • A degenerate WKT POLYGON with 1 unique vertex translates to a GeoJSON Point
  • A degenerate WKT POLYGON with 2 unique vertices translates to a GeoJSON LineString
  • Valid non-degenerate WKT is unchanged
  • POINT EMPTY / malformed WKT still returns "" (existing tests keep passing)
  • Unit tests cover the degenerate cases — there are currently none
  • All 802 OpenTopography datasets get a non-null translated_spatial

Context

Found in #6038 (SPIKE-3). Blocks #6197.

Activity

  1. moved this to 📟 Sprint Backlog (Top Priority) in data.gov team boardon Sep 8, 2026
  2. changed the title [-]Degenerate WKT polygons are dropped, losing spatial for 10% of the OpenTopography feed[/-] [+]6197: Degenerate WKT polygons are dropped, losing spatial for 10% of the OpenTopography feed[/+] on Sep 8, 2026
  3. self-assigned this
    on Sep 9, 2026
  4. moved this from 📟 Sprint Backlog (Top Priority) to 🏗 In Progress in data.gov team boardon Sep 9, 2026
  5. added a commit that references this issue on Sep 9, 2026
    5bea0db
  6. removed their assignment
    on Sep 9, 2026
  7. added a commit that references this issue on Sep 9, 2026
    3840323
  8. moved this from 🏗 In Progress to ✔ Done in data.gov team boardon Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions