Skip to content

perf(table): make localized Polars exports native-speed - #10674

Merged
kirangadhave merged 4 commits into
kg/mo-7455-locale-table-exportfrom
codex/pr-10661-export-performance
Aug 27, 2026
Merged

perf(table): make localized Polars exports native-speed#10674
kirangadhave merged 4 commits into
kg/mo-7455-locale-table-exportfrom
codex/pr-10661-export-performance

Conversation

@Light2Dark

@Light2Dark Light2Dark commented Aug 27, 2026

Copy link
Copy Markdown
Member

This pull request was authored by a coding agent.

📝 Summary

This is a stacked follow-up to #10661 and targets its feature branch. The parent PR makes CSV and TSV exports locale-aware, but non-default dialects route dataframe rows through Python's csv.writer. That cost is especially visible for Polars, whose native writer is otherwise implemented in Rust.

This PR keeps pandas and Polars on their native CSV writers for ordinary numeric and text data with the common . and , decimal separators. pandas prepares only values that its native decimal option does not format. Polars falls back to the shared compatibility writer when native output would change established export text.

Complexity and compatibility tradeoff

The adapter-specific fallbacks are intentional:

  • Unicode decimal separators such as ٫ use the shared compatibility writer.
  • Polars versions without native decimal-comma support use the compatibility writer.
  • Polars temporal and binary columns use it because native Polars emits different text representations.
  • Polars float columns containing NaN use it because native Polars emits an empty field instead of nan.
  • pandas normalizes object-dtype numbers, NaN-bearing float columns, and exported numeric indexes before calling its native writer.

The ordinary path still performs one native dataframe serialization. pandas does not materialize or inspect its index unless the index is included in the export. The redundant download_as(locale=...) path and the private Narwhals separator/dialect dual interface were removed to offset the compatibility branches and leave one locale and dialect flow.

Benchmark

Local microbenchmark on Apple Silicon with 200,000 rows and five columns. Each case had one warm-up serialization followed by one timed serialization, using the same environment for the parent PR and this branch.

Export Parent PR This PR Improvement
pandas pt-BR CSV 0.724s 0.518s 1.4×
Polars pt-BR CSV 0.641s 0.005s 128×
Polars TSV 0.285s 0.004s 71×

The repository does not have a Python benchmark harness, so unit tests avoid flaky wall-clock assertions. Deterministic routing tests verify the native writer paths, while exact-output tests cover NaN-bearing floats, numeric indexes, temporal values, binary values, nested data, Unicode decimal separators, and older-Polars capability detection.

📋 Pre-Review Checklist

  • This work is a refinement of the approved locale-export work in feat(table): locale-aware CSV and TSV exports #10661.
  • Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
  • No visual evidence is needed; this changes backend serialization only.

✅ Merge Checklist

  • I have read the contributor guidelines.
  • No public documentation changes are required; internal comments describe the compatibility boundary.
  • Tests cover compatibility and native-writer routing.

Written by GPT-5 on Codex

@vercel

vercel Bot commented Aug 27, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
marimo-docs Ready Ready Preview Aug 27, 2026 4:57pm

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 10 files

You’re at about 95% of the monthly reviewed-line limit. You may want to disable incremental reviews to conserve quota. Reviews will continue until that limit is exceeded. If you need help avoiding interruptions, please contact contact@cubic.dev.

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread marimo/_plugins/ui/_impl/utils/dataframe.py
Comment thread marimo/_plugins/ui/_impl/tables/pandas_table.py Outdated
Comment thread marimo/_plugins/ui/_impl/tables/polars_table.py
Comment thread marimo/_utils/narwhals_utils.py Outdated
Comment thread tests/_plugins/ui/_impl/tables/test_polars_table.py
@Light2Dark
Light2Dark marked this pull request as ready for review August 27, 2026 18:00
@kirangadhave
kirangadhave merged commit ef964c5 into kg/mo-7455-locale-table-export Aug 27, 2026
51 of 56 checks passed
@kirangadhave
kirangadhave deleted the codex/pr-10661-export-performance branch August 27, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working team-draft

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants