Skip to content

Fix "Hole lies outside shell" for reversed islands - #53

Open
wangi wants to merge 1 commit into
osmcode:masterfrom
wangi:hole-lies-outside-shell
Open

Fix "Hole lies outside shell" for reversed islands#53
wangi wants to merge 1 commit into
osmcode:masterfrom
wangi:hole-lies-outside-shell

Conversation

@wangi

@wangi wangi commented Aug 18, 2026

Copy link
Copy Markdown

Fixes #41.

The problem

create_polygons() calls organizePolygons() with METHOD=ONLY_CCW, so ring direction alone decides shell vs. hole, and a hole is assigned to a shell using only a bounding box test. An island whose coastline was mapped the wrong way round becomes a hole candidate, and if it happens to lie inside the bounding box of a larger landmass (but outside the landmass itself) it is attached to it as a hole. GEOS then reports

Warning 1: Hole lies outside shell at or near point ...

and the whole polygon — possibly a whole continent — is invalid and ends up in the error_lines table, or is dropped entirely.

The existing "Fixing coastlines going the wrong way" step runs after the polygons have been created and only looks at exterior rings, so it never caught this case and reported "Turned 0 polygons around".

The fix

When a polygon is invalid, each interior ring is checked against the exterior ring. The check tests the first point of the interior ring with OGRLinearRing::isPointInRing() rather than a GEOS operation, because it has to work on invalid geometries too. Rings that are outside their shell are taken out of the polygon, turned around into land polygons of their own, and reported in the error_lines table with the error direction (the same error the existing direction fix uses). They are counted in num_rings_turned_around.

While at it, invalid polygons that are not part of a multipolygon are now repaired in the same way as those that are (report an invalid error line and try Buffer(0)), instead of just being dropped.

Tests

  • invalid-direction-island-in-bbox — reversed island inside the bounding box of an L-shaped landmass (single polygon code path).
  • invalid-direction-island-in-bbox-of-many — the same with a second, correctly mapped island, so organizePolygons() returns a multipolygon (the code path hit by the case in Reversed coastline results in "Hole lies outside shell" output #41).

valid-inland-sea-with-island needed updating: the island in the inland sea is mapped hole-style and so ends up nested inside the hole formed by the sea ("Holes are nested"). That polygon is now repaired instead of discarded, so the land polygon survives and the result is a warning rather than an error. The island itself is still dropped and reported as questionable; that nested-hole limitation is unchanged.

organizePolygons() decides which rings are holes based on their
direction only and uses just a bounding box test when assigning a hole
to a polygon. An island whose coastline was mapped the wrong way round
can therefore end up as a hole of a polygon it is not inside of. GEOS
then reports "Hole lies outside shell" and the whole polygon, possibly
a whole continent, is invalid and ends up in the error_lines table.
The existing direction fix runs after the polygons have been created
and only looks at exterior rings, so it never caught this case.

Interior rings that are outside their shell are now taken out of the
polygon, turned around into land polygons of their own and reported in
the error_lines table with the error "direction".

While at it, invalid polygons that are not part of a multipolygon are
now repaired in the same way as those that are, instead of being
dropped.

Fixes osmcode#41

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@wangi

wangi commented Aug 18, 2026

Copy link
Copy Markdown
Author

I have also tested this against the files referenced on #41, which now gives following output:

/opt/opengeofiction/osmcoastline/bin/osmcoastline --verbose --srs=3857 --overwrite --output-lines --output-polygons=both --output-rings --max-points=1000 --output-database ogf-coastlines-split.db ogf-coastline-data.osm.pbf
[ 0:00] Started osmcoastline version 2.5.0 / libosmium version 2.23.0
[ 0:00] Using SRS 3857 for output. (Change with the --srs/s option.)
[ 0:00] Writing to output database 'ogf-coastlines-split.db'. (Was set with the --output-database/-o option.)
[ 0:00] Removing database output file (if it exists) (because you told me to with --overwrite/-f).
[ 0:00] Will create geometry index. (If you do not want an index use --no-index/-i.)
[ 0:00] Reading from file 'ogf-coastline-data.osm.pbf'.
[ 0:00] Reading ways (1st pass through input file)...
[ 0:01]   There are 0 nodes where the coastline is not closed.
[ 0:01]   There are 38567 coastline rings (36907 from a single closed way and 1660 others).
[ 0:01] Memory used: current: 1494 MBytes, peak: 1494 MBytes
[ 0:01] Reading nodes (2nd pass through input file)...
[ 0:02] Checking for missing locations...
[ 0:02]   All locations are there.
[ 0:02] Memory used: current: 1767 MBytes, peak: 1767 MBytes
[ 0:02] Check line segments for intersections and overlaps...
[ 0:03] Trying to close Antarctica ring...
[ 0:03]   Did not find open Antarctica ring.
[ 0:03] Close broken rings... (Use --close-distance/-c 0 if you do not want this.)
[ 0:03]   Closing if distance between nodes smaller than 1. (Set this with --close-distance/-c.)
[ 0:03]   Closed 0 rings. This left 0 nodes where the coastline could not be closed.
[ 0:03] Writing out rings... (Because you gave the --output-rings/-r option.)
[ 0:05] Create polygons...
Warning 1: Hole lies outside shell at or near point 128.80197369999999 35.702867900000001
[ 0:07] Fixing coastlines going the wrong way...
[ 0:07]   Turned 1 polygons around.
[ 0:07] Transforming polygons to EPSG 3857...
[ 0:07] Writing coastlines as lines... (Because you used --output-lines/-l)
[ 0:08] Not performing check for questionable input data, because it only works in EPSG:4326...
[ 0:08] Split polygons with more than 1000 points... (Use --max-points/-m to change this. Set to 0 not to split at all.)
[ 0:08]   Using overlap of 10 (Set this with --bbox-overlap/-b).
[ 0:23] Checking and making polygons valid...
[ 0:24] Writing out land polygons...
[ 0:25] Writing out water polygons...
[ 1:40] Memory used: current: 1767 MBytes, peak: 1767 MBytes
[ 1:40] Committing database transactions...
[ 1:40] All done.
[ 1:40] Memory used: current: 1767 MBytes, peak: 1767 MBytes
There were 1 warnings.
There were 0 errors.

Also tested as part of the complete OpenGeofiction coastline process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Reversed coastline results in "Hole lies outside shell" output

1 participant