Skip to content

fix: map geoarrow to geoarrow-types - #10659

Open
kirangadhave wants to merge 7 commits into
mainfrom
kg/geometry-host-hardening
Open

fix: map geoarrow to geoarrow-types#10659
kirangadhave wants to merge 7 commits into
mainfrom
kg/geometry-host-hardening

Conversation

@kirangadhave

@kirangadhave kirangadhave commented Aug 25, 2026

Copy link
Copy Markdown
Member

📝 Summary

Map the geoarrow import to the geoarrow-types PyPI package so the missing-package alert suggests the installable name.

Run the geometry fixture corpus in CI. Smoke the corpus through ui.table search and column summaries for GeoPandas, GeoArrow, and DuckDB. Add a subprocess check that a hand-built GeoArrow WKB table still types as geometry when geopandas and shapely cannot be imported.

Keep GeoArrow field metadata when default selection="multi" adds a row-id column. PyArrow now uses native Table.add_column instead of narwhals with_row_index, which dropped extension metadata.

Last PR of the geometry stack for MO-6362

Closes MO-7327

Made with Cursor

kirangadhave and others added 5 commits August 25, 2026 12:03
The ibis geometry missing-package alert suggested geoarrow, which is the wrong PyPI name.

Co-authored-by: Cursor <cursoragent@cursor.com>
The PR 1 blanket skip is no longer needed: duckdb spatial and ibis already degrade to skip, and dtype pins are stable on the optional tiers.

Co-authored-by: Cursor <cursoragent@cursor.com>
Host search and column summaries should return clean payloads and the expected geometry field types for each corpus fixture.

Co-authored-by: Cursor <cursoragent@cursor.com>
A subprocess import blocker proves the hand-built WKB table still types as geometry when geopandas and shapely cannot be imported.

Co-authored-by: Cursor <cursoragent@cursor.com>
Narwhals with_row_index rebuilds PyArrow tables and drops field metadata. Insert the selection column with Table.add_column instead.

Co-authored-by: Cursor <cursoragent@cursor.com>
@vercel

vercel Bot commented Aug 25, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
marimo-docs Ready Ready Preview Aug 26, 2026 6:12am

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@kirangadhave
kirangadhave requested a lite review from Copilot August 25, 2026 22:17
@kirangadhave
kirangadhave marked this pull request as ready for review August 25, 2026 22:17
@kirangadhave kirangadhave added the enhancement New feature or request label Aug 25, 2026
@kirangadhave
kirangadhave requested a review from mscolnick August 25, 2026 22:18

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files

Architecture diagram
sequenceDiagram
    participant User as User/Frontend
    participant Kernel as Marimo Runtime
    participant PkgMgr as module_name_to_pypi_name
    participant TableUI as ui.table
    participant Select as selection.py
    participant TM as TableManager
    participant Data as PyArrow/GeoData

    Note over User,Data: Flow 1: Missing Package Discovery
    User->>Kernel: import geoarrow
    Kernel->>Kernel: Import fails
    Kernel->>PkgMgr: lookup("geoarrow")
    PkgMgr-->>Kernel: NEW: "geoarrow-types"
    Kernel-->>User: Suggest "pip install geoarrow-types"

    Note over User,Data: Flow 2: Table Initialization with Row Selection
    User->>TableUI: Create table (selection=True)
    TableUI->>Select: add_selection_column(data)
    
    alt Data is PyArrow Table
        Select->>Data: CHANGED: native.add_column(index_col)
        Note right of Data: Preserves ARROW:extension metadata
    else Other DataFrames (Pandas/Polars)
        Select->>Data: Use Narwhals/Native insert
    end
    Select-->>TableUI: Data with row-id + preserved metadata
    
    TableUI->>TM: get_field_types()
    
    Note over TM,Data: Flow 3: Geometry Type Inference
    TM->>Data: Inspect schema/metadata
    
    alt Has GeoArrow Extension Metadata
        Data-->>TM: b"geoarrow.wkb" / b"geoarrow.wkt"
        TM-->>TableUI: ("geometry", "geoarrow.wkb")
    else Has GeoPandas/Shapely
        Data-->>TM: Geometry column
        TM-->>TableUI: ("geometry", "geometry")
    else DuckDB Spatial
        Data-->>TM: GEOMETRY type
        TM-->>TableUI: ("geometry", "GEOMETRY")
    end

    TableUI-->>User: Render table with Geometry formatting
Loading

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread tests/_plugins/ui/_impl/tables/test_geometry.py Outdated
Comment thread tests/_plugins/ui/_impl/tables/test_geometry.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves the geometry-table “stack” by (1) mapping import geoarrow to the installable PyPI package name geoarrow-types for missing-module guidance, and (2) hardening geometry handling/tests—especially around GeoArrow extension metadata preservation when adding the default multi-selection row-id column.

Changes:

  • Map the geoarrow module name to the geoarrow-types PyPI distribution for missing-package suggestions.
  • Preserve GeoArrow extension field metadata when adding the selection row-id column for PyArrow-backed tables.
  • Expand/enable geometry corpus coverage in CI with host-level smoke tests and an isolated subprocess test that blocks geopandas/shapely imports.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
tests/_plugins/ui/_impl/tables/test_geometry.py Adds selection-metadata regression coverage, corpus smoke tests (search + summaries), and a subprocess test ensuring GeoArrow typing/formatting works without geopandas/shapely.
tests/_plugins/ui/_impl/tables/test_geometry_fixtures.py Removes the CI skip gate so geometry characterization tests can run in CI (still guarded by pytest.mark.requires).
marimo/_runtime/packages/module_name_to_pypi_name.py Adds geoarrow -> geoarrow-types mapping for install-name suggestions.
marimo/_plugins/ui/_impl/tables/selection.py Uses native pyarrow.Table.add_column for PyArrow data to avoid losing GeoArrow extension metadata when adding the selection row-id column.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

DuckDB collect goes through PyArrow, so the host-smoke cases should skip without it. Re-raise child stderr when the no-geo subprocess fails.

Co-authored-by: Cursor <cursoragent@cursor.com>
CI Python 3.10 still pins pandas 2.x, which reports string columns as
object instead of str. Compare semantic types and treat those names as
equivalent so the corpus pins stay version-stable.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants