diff --git a/CHANGELOG.md b/CHANGELOG.md index 25b454a..7ae85c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -42,6 +42,30 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). are written as NULL and counted into a per-column warning report; columns without a schema export as text verbatim. A columnar document opens unsaved so a later Save can never overwrite the binary source with CSV. +- **Excel `.xlsx` interoperability** (F34): read and produce Excel workbooks + without a detour through CSV, and without a formula engine. The open chooser + inspects a workbook up front — every sheet with its visibility (including + hidden and very-hidden sheets), named tables and named ranges, per-sheet used + ranges and dimensions, formula and merged-cell counts, detected header-row + candidates and a bounded preview. Import a chosen sheet, named table or named + range (optionally a specific `A1` cell range), with control over the header + row (first row, a chosen row, or none), merged cells (keep the top-left value + only, or repeat it across the region), formulas (cached result, formula text, + or blank — with a warning when a workbook has formulas but no cached results, + since CEESVEE never evaluates them) and blank-row/blank-column trimming. + Imports honour the workbook's 1900/1904 date system so dates never shift and + the Excel 1900 leap-year quirk is preserved, and leading-zero text (ZIP and + account codes) stays text — cell types are respected, never numeric-coerced. + Every import produces a fresh CEESVEE document (there is no in-place `.xlsx` + save); the original workbook is never modified. Export one sheet from one + document, or several sheets (one per open tab) into a single workbook, with + optional bold/filled header styling, a frozen header row, an autofilter, and + column widths from the grid or autofit. Typed columns (from the F31 schema) + export as real Excel numbers, booleans and dates; text stays text and cells + invalid under their schema fall back to text. Excel's 1,048,576-row × + 16,384-column limits are checked before writing (an over-limit export is + refused up front), and the workbook is committed through the atomic-save + pipeline, so a failed or cancelled export never touches an existing file. - **Row bookmarks, tags & notes** (F40): mark and annotate records without touching the source data. Star or flag a row, apply multiple named tags (a per-document tag namespace with usage counts), and attach a row note or diff --git a/README.md b/README.md index f207867..514af2c 100644 --- a/README.md +++ b/README.md @@ -58,6 +58,17 @@ and faithful on large, real-world delimited files.** explode into rows, or drop). Signed/unsigned 64-bit integers, exact decimal precision/scale, timestamps with timezone, and null-vs-empty-string all survive intact. +- **Open Excel `.xlsx`** — a chooser lists every sheet (including hidden and + very-hidden ones), named tables and ranges, used ranges and dimensions, and + formula/merged-cell counts. Pick a sheet, table or cell range; choose the + header row (first, a chosen row, or none), how merged cells are handled + (top-left value only, or repeated across the region) and how formulas are + handled (cached result, formula text, or blank — with a warning when a + workbook has formulas but no cached results), and blank-row/column trimming. + The workbook's 1900/1904 date system is honoured (no date shifting, the + 1900 leap-year quirk preserved) and leading-zero text stays text. Every + import creates a new document — there is no in-place `.xlsx` save, so the + original workbook is never modified. - Auto-detect the **delimiter** (comma, tab, semicolon, pipe) with a manual / custom override — plus an **advanced import** for preambles, comment lines, custom quoting/escaping, multi-row headers, and footers. @@ -121,6 +132,14 @@ and faithful on large, real-world delimited files.** column's declared logical type to the matching arrow type (preserving 64-bit integer widths, decimal precision/scale, and timestamp timezones); cells that can't be represented are written as NULL and reported per column. +- **Export to Excel `.xlsx`** — write one document — or several open tabs as + one sheet each — to a single workbook, with optional bold/filled header + styling, a frozen header row, an autofilter, and column widths from the grid + or autofit. Typed columns emit real Excel numbers, booleans, and dates from + the declared schema (text stays text; cells invalid under their schema fall + back to text). Excel's 1,048,576-row × 16,384-column limits are checked + before anything is written, and the workbook commits through the same atomic + save pipeline, so a failed or cancelled export never touches an existing file. **Reliability** diff --git a/src-tauri/Cargo.lock b/src-tauri/Cargo.lock index c6c1a3a..5300ead 100644 --- a/src-tauri/Cargo.lock +++ b/src-tauri/Cargo.lock @@ -428,6 +428,16 @@ dependencies = [ "num-traits", ] +[[package]] +name = "atoi_simd" +version = "0.18.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f3cdb3708a128e559a30fb830e8a77a5022ee6902806925c216658652b452a44" +dependencies = [ + "debug_unsafe", + "rustversion", +] + [[package]] name = "atomic-waker" version = "1.1.2" @@ -601,6 +611,24 @@ dependencies = [ "system-deps", ] +[[package]] +name = "calamine" +version = "0.36.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6975084f43060e56343ffba7f9731fa52a7dcf2e1cd8e2459fd4c6bf4a1bff59" +dependencies = [ + "atoi_simd", + "byteorder", + "chrono", + "codepage", + "encoding_rs", + "fast-float2", + "log", + "quick-xml 0.41.0", + "serde", + "zip 8.6.0", +] + [[package]] name = "camino" version = "1.2.2" @@ -660,6 +688,7 @@ name = "ceesvee" version = "0.4.0" dependencies = [ "arrow", + "calamine", "chardetng", "chrono", "chrono-tz", @@ -670,6 +699,7 @@ dependencies = [ "hmac", "parquet", "regex", + "rust_xlsxwriter", "serde", "serde_json", "sha2", @@ -763,6 +793,15 @@ dependencies = [ "error-code", ] +[[package]] +name = "codepage" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "48f68d061bc2828ae826206326e61251aca94c1e4a5305cf52d9138639c918b4" +dependencies = [ + "encoding_rs", +] + [[package]] name = "combine" version = "4.6.7" @@ -1006,6 +1045,12 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "debug_unsafe" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7eed2c4702fa172d1ce21078faa7c5203e69f5394d48cc436d25928394a867a2" + [[package]] name = "deranged" version = "0.5.8" @@ -1307,6 +1352,12 @@ dependencies = [ "pin-project-lite", ] +[[package]] +name = "fast-float2" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8eb564c5c7423d25c886fb561d1e4ee69f72354d16918afa32c08811f6b6a55" + [[package]] name = "fastrand" version = "2.4.1" @@ -1378,6 +1429,7 @@ checksum = "843fba2746e448b37e26a819579957415c8cef339bf08564fe8b7ddbd959573c" dependencies = [ "crc32fast", "miniz_oxide", + "zlib-rs", ] [[package]] @@ -3206,7 +3258,7 @@ checksum = "092791278e026273c1b65bbdcfbba3a300f2994c896bd01ab01da613c29c46f1" dependencies = [ "base64 0.22.1", "indexmap 2.14.0", - "quick-xml", + "quick-xml 0.39.4", "serde", "time", ] @@ -3365,6 +3417,16 @@ dependencies = [ "memchr", ] +[[package]] +name = "quick-xml" +version = "0.41.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e660451e55124f798a69a5af3f49ccfbefbd41910eefd25caf2393e1f3473ec1" +dependencies = [ + "encoding_rs", + "memchr", +] + [[package]] name = "quote" version = "1.0.45" @@ -3538,6 +3600,15 @@ dependencies = [ "windows-sys 0.52.0", ] +[[package]] +name = "rust_xlsxwriter" +version = "0.96.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dd1746025420e17b5d62528b930e550e016e857038794d74e169018126ef3d14" +dependencies = [ + "zip 7.2.0", +] + [[package]] name = "rustc-hash" version = "2.1.2" @@ -5033,6 +5104,12 @@ version = "2.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9ea3136b675547379c4bd395ca6b938e5ad3c3d20fad76e7fe85f9e0d011419c" +[[package]] +name = "typed-path" +version = "0.12.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e28f89b80c87b8fb0cf04ab448d5dd0dd0ade2f8891bae878de66a75a28600e" + [[package]] name = "typeid" version = "1.0.3" @@ -5413,7 +5490,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9c324a910fd86ebdc364a3e61ec1f11737d3b1d6c273c0239ee8ff4bc0d24b4a" dependencies = [ "proc-macro2", - "quick-xml", + "quick-xml 0.39.4", "quote", ] @@ -6414,6 +6491,40 @@ dependencies = [ "memchr", ] +[[package]] +name = "zip" +version = "7.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c42e33efc22a0650c311c2ef19115ce232583abbe80850bc8b66509ebef02de0" +dependencies = [ + "crc32fast", + "flate2", + "indexmap 2.14.0", + "memchr", + "typed-path", + "zopfli", +] + +[[package]] +name = "zip" +version = "8.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2d04a6b5381502aa6087c94c669499eb1602eb9c5e8198e534de571f7154809b" +dependencies = [ + "crc32fast", + "flate2", + "indexmap 2.14.0", + "memchr", + "typed-path", + "zopfli", +] + +[[package]] +name = "zlib-rs" +version = "0.6.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b142a20ec14a91d5bc708c1dc21b080c550113d8aa77afa29635673a65dd02c5" + [[package]] name = "zmij" version = "1.0.21" diff --git a/src-tauri/Cargo.toml b/src-tauri/Cargo.toml index 17a5b4a..64dd313 100644 --- a/src-tauri/Cargo.toml +++ b/src-tauri/Cargo.toml @@ -55,6 +55,12 @@ zip = { version = "2", default-features = false, features = ["deflate"] } # Parquet & Arrow interop (F32): typed columnar open/export. arrow = { version = "59", default-features = false, features = ["ipc"] } parquet = { version = "59", default-features = false, features = ["arrow", "snap", "zstd"] } +# Excel .xlsx interoperability (F34): calamine reads workbooks (the `dates` +# feature pulls in chrono so date cells honour the workbook epoch), and +# rust_xlsxwriter produces them (typed values via its native ExcelDateTime, so +# no extra chrono feature is needed on the writer). +calamine = { version = "0.36", features = ["dates"] } +rust_xlsxwriter = "0.96" # Single-instance is desktop-only: it forwards a second "Open with" launch # (and its file argument) to the already-running window instead of spawning a diff --git a/src-tauri/src/commands.rs b/src-tauri/src/commands.rs index 72466a4..e78d9af 100644 --- a/src-tauri/src/commands.rs +++ b/src-tauri/src/commands.rs @@ -35,12 +35,16 @@ use crate::dictionary::{ use crate::document::{ChangeSummary, Document}; use crate::dto::{ BackupPolicy, CellRect, ColumnSummary, DocumentMeta, EncodingCompatibility, - EncodingIncompatibility, ExportOptions, ExportScope, ExternalChange, FileFingerprint, - FilterGroup, FindMatch, FindOptions, IndexedOpenStart, JsonExportOptions, OpenOptions, - ReparsePreview, ReplaceResult, RowsResponse, ScopeCounts, SelectionStats, SortKey, - SplitOptions, + EncodingIncompatibility, ExcelExportOptions, ExcelSheetExport, ExportOptions, ExportScope, + ExternalChange, FileFingerprint, FilterGroup, FindMatch, FindOptions, IndexedOpenStart, + JsonExportOptions, OpenOptions, ReparsePreview, ReplaceResult, RowsResponse, ScopeCounts, + SelectionStats, SortKey, SplitOptions, }; use crate::error::{AppError, AppResult}; +use crate::excel::{ + self, ExcelImportOptions, ExcelImportPreview, ExcelInspectCache, ExcelPreviewCache, + SheetSource, WorkbookInfo, +}; use crate::follow::{self, FollowRegistry}; use crate::groupby::{self, GroupByPreview, GroupBySpec}; use crate::highlight::{ @@ -5034,3 +5038,190 @@ pub fn get_columnar_export_report( ) -> Option { reports.get(job_id) } + +// ----- Excel .xlsx interoperability (F34) -------------------------------------- + +/// Start a workbook inspection as a cancellable job (kind "scan"): sheets (with +/// visibility), named tables, named ranges, used ranges + dimensions, formula +/// and merged-cell counts, header candidates and bounded previews. Nothing is +/// opened; fetch the result with `get_excel_inspect` after `job-finished`. +#[tauri::command] +pub async fn excel_inspect( + path: String, + app: tauri::AppHandle, + jobs: State<'_, JobRegistry>, + excel_inspects: State<'_, ExcelInspectCache>, +) -> AppResult { + let sink = excel_inspects.share(); + let ctx = jobs.begin_for_app(&app, "scan", None); + let job_id = ctx.id; + tauri::async_runtime::spawn(async move { + let _ = crate::job::run_blocking(ctx, move |ctx| { + let info = excel::inspect(Path::new(&path), Some(ctx))?; + if let Ok(mut map) = sink.lock() { + map.insert(job_id, info); + } + Ok(()) + }) + .await; + }); + Ok(job_id) +} + +/// The inspection of a finished workbook scan, by its job id. +#[tauri::command] +pub fn get_excel_inspect( + job_id: u64, + excel_inspects: State<'_, ExcelInspectCache>, +) -> Option { + excel_inspects.get(job_id) +} + +/// Start an import preview of the chosen source/options as a cancellable job +/// (kind "scan"): columns with inferred types, sample rows, projected +/// dimensions and warnings (including formula cells with no cached result). +/// Fetch the result with `get_excel_import_preview` after `job-finished`. +#[tauri::command] +pub async fn excel_import_preview( + path: String, + options: Option, + app: tauri::AppHandle, + jobs: State<'_, JobRegistry>, + excel_previews: State<'_, ExcelPreviewCache>, +) -> AppResult { + let options = options.unwrap_or_default(); + let sink = excel_previews.share(); + let ctx = jobs.begin_for_app(&app, "scan", None); + let job_id = ctx.id; + tauri::async_runtime::spawn(async move { + let _ = crate::job::run_blocking(ctx, move |ctx| { + let preview = excel::preview(Path::new(&path), &options, Some(ctx))?; + if let Ok(mut map) = sink.lock() { + map.insert(job_id, preview); + } + Ok(()) + }) + .await; + }); + Ok(job_id) +} + +/// The preview of a finished Excel import scan, by its job id. +#[tauri::command] +pub fn get_excel_import_preview( + job_id: u64, + excel_previews: State<'_, ExcelPreviewCache>, +) -> Option { + excel_previews.get(job_id) +} + +/// Run an Excel import as a cancellable job (kind "derive"): the selected sheet +/// / table / named range (optionally a cell sub-range) is read under the chosen +/// merged / formula / date-system / trimming options into a NEW CEESVEE +/// document that registers under the returned doc id when the job finishes. The +/// original workbook is never modified; a failure leaves no document behind. +#[tauri::command] +pub async fn excel_import_apply( + path: String, + options: Option, + app: tauri::AppHandle, + state: Db<'_>, + jobs: State<'_, JobRegistry>, +) -> AppResult { + let options = options.unwrap_or_default(); + let doc_id = lock(&state)?.alloc_id(); + let cache_root = index_cache_root(&app)?; + + let ctx = jobs.begin_for_app(&app, "derive", Some(doc_id)); + let job_id = ctx.id; + let app_for_job = app.clone(); + tauri::async_runtime::spawn(async move { + let _ = crate::job::run_blocking(ctx, move |ctx| { + use tauri::Manager; + let doc = excel::import(Path::new(&path), &options, &cache_root, doc_id, Some(ctx))?; + let registry = app_for_job.state::>(); + registry + .lock() + .map_err(|_| AppError::Other("internal state lock error".into()))? + .insert(doc); + Ok(()) + }) + .await; + }); + Ok(IndexedOpenStart { job_id, doc_id }) +} + +/// Start an Excel `.xlsx` export as a cancellable job (kind "export"): one sheet +/// from one document, or several sheets (one per selected tab) into a single +/// workbook. Revisions, scopes, sheet names and Excel's row/column limits are +/// validated BEFORE the job spawns (and again inside it); the workbook is +/// committed through the atomic-save pipeline, so a failure or cancellation +/// never touches an existing destination. +#[tauri::command] +pub async fn excel_export( + sheets: Vec, + path: String, + options: ExcelExportOptions, + app: tauri::AppHandle, + state: Db<'_>, + jobs: State<'_, JobRegistry>, +) -> AppResult { + if sheets.is_empty() { + return Err(AppError::invalid( + "an Excel export needs at least one sheet", + )); + } + // Resolve every sheet's document handle up front. + let handles: Vec = sheets + .iter() + .map(|s| doc_handle(&state, s.doc_id)) + .collect::>()?; + // Fail fast: revisions, scopes, sheet names and Excel limits reject the + // invoke, not a background job. + { + let guards: Vec<_> = handles + .iter() + .map(|h| h.read().map_err(poisoned)) + .collect::>>()?; + let sources: Vec = guards + .iter() + .zip(&sheets) + .map(|(g, s)| SheetSource { + doc: g, + name: s.name.clone(), + scope: s.scope.clone(), + expected_revision: s.expected_revision, + grid_widths_px: s.grid_widths_px.clone(), + }) + .collect(); + excel::plan_export(&sources)?; + } + + let first_doc = sheets.first().map(|s| s.doc_id); + let ctx = jobs.begin_for_app(&app, "export", first_doc); + let job_id = ctx.id; + let dest = PathBuf::from(&path); + tauri::async_runtime::spawn(async move { + let _ = crate::job::run_blocking(ctx, move |ctx| { + let guards: Vec<_> = handles + .iter() + .map(|h| h.read().map_err(poisoned)) + .collect::>>()?; + let sources: Vec = guards + .iter() + .zip(&sheets) + .map(|(g, s)| SheetSource { + doc: g, + name: s.name.clone(), + scope: s.scope.clone(), + expected_revision: s.expected_revision, + grid_widths_px: s.grid_widths_px.clone(), + }) + .collect(); + excel::export(&sources, &dest, &options, ctx)?; + Ok(()) + }) + .await; + }); + Ok(job_id) +} diff --git a/src-tauri/src/derived.rs b/src-tauri/src/derived.rs index 5abab9d..7ce99d7 100644 --- a/src-tauri/src/derived.rs +++ b/src-tauri/src/derived.rs @@ -121,8 +121,10 @@ impl DerivedDocumentBuilder { /// Finish the build and produce the document. In-memory outputs become /// ordinary editable documents (unsaved, so closing warns); spilled - /// outputs open INDEXED over the temp file with the guard attached. - /// `progress` receives byte deltas while the spilled file is indexed. + /// outputs open INDEXED over the temp file with the guard attached and are + /// likewise marked unsaved — the temp backing is discarded on close, so the + /// document must still prompt to save (Save routes to Save As, clearing the + /// guard). `progress` receives byte deltas while the spilled file is indexed. pub fn finish( self, doc_id: u64, @@ -166,6 +168,10 @@ impl DerivedDocumentBuilder { let indexed = index::build_index(&path, &self.cache_root, &settings, progress)?; let mut doc = Document::from_index(doc_id, None, indexed); doc.set_derived_guard(guard); + // Like the in-memory branch, a freshly built derived document is + // unsaved: it has no source path and its temp backing is dropped + // on close, so closing must warn and Save must route to Save As. + doc.mark_derived_unsaved(); Ok(doc) } } @@ -239,6 +245,14 @@ mod tests { assert!(!doc.is_editable(), "spilled outputs open indexed"); assert_eq!(doc.n_rows(), 50); assert_eq!(doc.headers(), &["a", "b"]); + // A spilled derived document is a fresh, temp-backed table with no + // source path: it must start unsaved so closing warns (exactly like the + // in-memory branch) instead of silently dropping the just-built import. + assert!( + doc.is_dirty(), + "spilled derived documents start unsaved too" + ); + assert!(doc.meta().path.is_none()); // Values round-trip through the CSV spill + index. let rows = doc.fetch_rows(&[0, 49]).unwrap(); assert_eq!(rows[0][0], "row 0"); diff --git a/src-tauri/src/dto.rs b/src-tauri/src/dto.rs index 1c54384..df9c596 100644 --- a/src-tauri/src/dto.rs +++ b/src-tauri/src/dto.rs @@ -487,6 +487,64 @@ impl Default for JsonExportOptions { } } +/// How an Excel export sizes its columns (F34). +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Deserialize)] +#[serde(rename_all = "camelCase")] +pub enum ExcelColumnWidths { + /// Leave Excel's default column width. + #[default] + Default, + /// Autofit each column to its widest cell. + Autofit, + /// Use the caller's per-column grid widths (pixels). + Grid, +} + +/// Options controlling an Excel `.xlsx` export (F34). Values only — never +/// formulas. Typed emission consults the F31 schema; text stays text. +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase", default)] +pub struct ExcelExportOptions { + /// Bold + filled styling on the header row. + pub header_style: bool, + /// Freeze the header row so it stays visible while scrolling. + pub freeze_header: bool, + /// Add an autofilter over the used range (header row required). + pub autofilter: bool, + pub column_widths: ExcelColumnWidths, + /// Emit typed numbers/dates/booleans for schema-carrying columns. + pub typed: bool, + /// Backup policy for the previous destination file. + pub backup: BackupPolicy, +} + +impl Default for ExcelExportOptions { + fn default() -> ExcelExportOptions { + ExcelExportOptions { + header_style: true, + freeze_header: true, + autofilter: false, + column_widths: ExcelColumnWidths::Default, + typed: true, + backup: BackupPolicy::default(), + } + } +} + +/// One sheet of a (possibly multi-sheet) Excel export (F34): the document, the +/// sheet name, the scope and the revision it was prepared against. +#[derive(Debug, Clone, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct ExcelSheetExport { + pub doc_id: u64, + pub name: String, + pub scope: ExportScope, + pub expected_revision: u64, + /// Per-output-column pixel widths (used only with `Grid` column widths). + #[serde(default)] + pub grid_widths_px: Option>, +} + /// One cell (or header) whose text cannot be represented in a target encoding. #[derive(Debug, Clone, Serialize)] #[serde(rename_all = "camelCase")] diff --git a/src-tauri/src/excel.rs b/src-tauri/src/excel.rs new file mode 100644 index 0000000..ebee0f9 --- /dev/null +++ b/src-tauri/src/excel.rs @@ -0,0 +1,2363 @@ +//! F34: Excel `.xlsx` interoperability — reading workbooks into CEESVEE +//! documents and producing workbooks from open documents. This is NOT a +//! formula engine: formulas are read as their cached result, their text, or +//! blanked; nothing is evaluated, and there is no in-place `.xlsx` save (an +//! import always yields a fresh CEESVEE document — the UI states this). +//! +//! ## Reading (calamine) +//! +//! [`inspect`] is the OPEN CHOOSER: it lists every sheet (including hidden and +//! very-hidden ones), named tables, named ranges, per-sheet used ranges and +//! approximate dimensions, formula and merged-cell counts, detected header-row +//! candidates and a bounded preview. [`preview`] scans the SELECTED source +//! (sheet / named table / named range, optionally a cell sub-range) under the +//! chosen options and reports the resulting columns, sample rows, projected +//! dimensions and warnings. [`import`] runs the same scan and streams the cells +//! into a CEESVEE document through the shared [`DerivedDocumentBuilder`] +//! pipeline (an in-memory editable document for small results, an indexed +//! read-only one for large ones or when `forceIndexed` is set). +//! +//! Cell handling honours the workbook's declared date system +//! (`workbookPr@date1904`, surfaced by [`calamine::Xlsx::has_1904_epoch`]) — a +//! date cell carries the workbook epoch inside its [`calamine::ExcelDateTime`], +//! so 1904-epoch dates are NOT shifted and the Excel 1900 leap-year bug is +//! reproduced exactly (both are calamine's responsibility; we never re-derive +//! serial dates ourselves). Text cells stay text — a leading-zero string such +//! as a ZIP code is a string cell and is never numeric-coerced. +//! +//! Merged cells are handled per [`MergedPolicy`]: `topLeftOnly` mirrors Excel's +//! own storage (the value lives only in the top-left cell; the rest are blank), +//! `repeat` fills every cell of a merged region with the top-left value. +//! Formulas are handled per [`FormulaPolicy`]: `cachedResult` uses the cached +//! `` value (blank when Excel stored none), `formulaText` emits the formula +//! source prefixed with `=`, `blank` drops it. A workbook that has formulas but +//! no cached results is flagged so the UI can warn (evaluating them is out of +//! scope). +//! +//! ## Writing (rust_xlsxwriter) +//! +//! [`export`] writes one sheet from one document, or several sheets (one per +//! open tab) into a single workbook. Values only — never formulas. Optional +//! bold+filled header styling, a frozen header row, an autofilter over the used +//! range and column widths (from the caller's grid widths or autofit). With +//! `typed`, columns carrying an F31 schema export real typed values +//! (integers/decimals/floats as numbers, booleans as booleans, dates/datetimes +//! as Excel dates); text stays text and any cell that is invalid under its +//! declared schema falls back to text. Excel's hard limits (1,048,576 rows × +//! 16,384 columns) are validated BEFORE a byte is written, and the workbook is +//! committed through the F03 atomic-save pipeline, so a failure or cancellation +//! never touches an existing destination. + +use std::collections::HashMap; +use std::io::Write; +use std::path::Path; +use std::sync::{Arc, Mutex}; + +use calamine::{ + open_workbook, Data, DataType, Dimensions, Reader, Sheet, SheetType, SheetVisible, Xlsx, + XlsxError, +}; +use chrono::{Datelike, Timelike}; +use rust_xlsxwriter::{Color, ExcelDateTime, Format, FormatPattern, Workbook}; +use serde::{Deserialize, Serialize}; + +use crate::derived::DerivedDocumentBuilder; +use crate::document::Document; +use crate::dto::{ExcelColumnWidths, ExcelExportOptions, ExportScope}; +use crate::error::{AppError, AppResult}; +use crate::export_scope; +use crate::job::JobCtx; +use crate::save; +use crate::schema::{self, CellState, ColumnSchema, LogicalType, TypedValue}; + +/// Excel's hard row limit (a worksheet has at most this many rows). +pub const EXCEL_MAX_ROWS: u64 = 1_048_576; +/// Excel's hard column limit. +pub const EXCEL_MAX_COLS: u64 = 16_384; +/// Longest sheet name Excel accepts. +const MAX_SHEET_NAME_LEN: usize = 31; +/// Characters Excel forbids in a sheet name. +const INVALID_SHEET_CHARS: [char; 7] = ['[', ']', ':', '*', '?', '/', '\\']; + +/// Rows shown in a chooser sheet preview. +const PREVIEW_ROWS: usize = 20; +/// Columns shown in a chooser sheet preview (wide sheets are clipped). +const PREVIEW_COLS: usize = 40; +/// Rows scanned for header-row candidates. +const HEADER_SCAN_ROWS: u32 = 25; +/// Most header-row candidates reported per sheet. +const MAX_HEADER_CANDIDATES: usize = 5; +/// Data rows retained by [`preview`]. +pub const SAMPLE_ROWS: usize = 50; +/// Cooperative-cancellation cadence (cells scanned / rows written). +const CANCEL_EVERY_ROWS: usize = 1024; + +// --------------------------------------------------------------------------- +// Import options (wire DTOs, camelCase) +// --------------------------------------------------------------------------- + +/// Which row of the selected region is the header. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Deserialize)] +#[serde(tag = "type", rename_all = "camelCase")] +pub enum HeaderMode { + /// The first row of the region is the header. + #[default] + FirstRow, + /// The row at this 0-based offset within the region is the header; rows + /// above it are dropped (they are title/notes rows), rows below are data. + Row { index: u32 }, + /// No header row — columns get synthetic `Column N` names and every row is + /// data. + None, +} + +/// How merged cells are imported. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Deserialize)] +#[serde(rename_all = "camelCase")] +pub enum MergedPolicy { + /// Keep the value only in the top-left cell (Excel's own storage); the rest + /// are blank. + #[default] + TopLeftOnly, + /// Repeat the top-left value across every cell of the merged region. + Repeat, +} + +/// How formula cells are imported. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Deserialize)] +#[serde(rename_all = "camelCase")] +pub enum FormulaPolicy { + /// Use the cached result Excel stored (blank when it stored none). + #[default] + CachedResult, + /// Emit the formula source text, prefixed with `=`. + FormulaText, + /// Emit a blank cell for every formula. + Blank, +} + +/// Everything an import (preview or apply) needs to know. +#[derive(Debug, Clone, Default, Deserialize)] +#[serde(rename_all = "camelCase", default)] +pub struct ExcelImportOptions { + /// Sheet to import (required unless `table` or `namedRange` is given). + pub sheet: Option, + /// Named table to import (its parent sheet and range are resolved + /// automatically; the table's own column names become the header). + pub table: Option, + /// Named range to import (resolved to a sheet and a cell range). + pub named_range: Option, + /// A1-style cell range within `sheet` (e.g. `"B2:F100"`); ignored for a + /// table or named-range source. + pub range: Option, + pub header: HeaderMode, + pub merged: MergedPolicy, + pub formula: FormulaPolicy, + /// Drop rows that are entirely empty within the selection. + pub trim_blank_rows: bool, + /// Drop columns that are entirely empty within the selection. + pub trim_blank_columns: bool, + /// Spill straight to the indexed read-only backing instead of size-based + /// auto-selection. + pub force_indexed: bool, +} + +// --------------------------------------------------------------------------- +// Inspection DTOs (chooser) +// --------------------------------------------------------------------------- + +/// One sheet in the open chooser. +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct SheetInfo { + pub name: String, + /// `"visible"`, `"hidden"` or `"veryHidden"`. + pub visibility: String, + /// `"worksheet"`, `"dialog"`, `"macro"`, `"chart"` or `"vba"`. + pub kind: String, + /// Whether the sheet holds tabular data (only worksheets do). + pub has_data: bool, + /// 0-based row/column of the used range's top-left cell. + pub start_row: u32, + pub start_col: u32, + /// Used-range height/width (approximate dimensions). + pub used_rows: u32, + pub used_cols: u32, + pub formula_count: u64, + pub merged_count: u64, + /// Formula cells for which Excel stored no cached result. + pub formulas_without_cached_results: u64, + /// Detected header-row candidates, as 0-based offsets from the used-range + /// start. + pub header_candidates: Vec, + /// Bounded preview of the cached cell values (top-left corner). + pub preview_rows: Vec>, +} + +/// One named table in the open chooser. +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct TableInfo { + pub name: String, + pub sheet: String, + pub columns: Vec, + pub rows: u64, + /// A1 range of the table body (headers excluded), when it has any rows. + pub range: Option, +} + +/// One named range in the open chooser. +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct NamedRangeInfo { + pub name: String, + /// The raw defined-name formula (e.g. `Sheet1!$A$1:$C$9`). + pub formula: String, + /// The sheet the range resolves to, when it is a simple single-area range. + pub sheet: Option, + pub range: Option, +} + +/// Everything the OPEN CHOOSER needs to render. +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct WorkbookInfo { + /// Whether the workbook uses the 1904 date epoch. + pub has_1904_epoch: bool, + pub sheets: Vec, + pub tables: Vec, + pub named_ranges: Vec, + pub warnings: Vec, +} + +// --------------------------------------------------------------------------- +// Import preview DTOs (chosen source + options) +// --------------------------------------------------------------------------- + +/// One column of an import preview. +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct PreviewColumn { + pub name: String, + pub inferred_type: LogicalType, + /// Non-empty data cells in the column. + pub non_empty: u64, + /// Empty data cells in the column. + pub empty: u64, +} + +/// The preview of importing the selected source under the chosen options. +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct ExcelImportPreview { + pub has_1904_epoch: bool, + /// Human-readable description of what was scanned (sheet + range / table). + pub source: String, + pub has_header_row: bool, + pub columns: Vec, + pub row_count: u64, + pub column_count: usize, + pub sample_rows: Vec>, + /// Formula cells with no cached result that landed in the selection. + pub formulas_without_cached_results: u64, + pub warnings: Vec, +} + +// --------------------------------------------------------------------------- +// Export source (options live in dto.rs alongside the other export options) +// --------------------------------------------------------------------------- + +/// One sheet of an export: the document it reads, the sheet name, the scope and +/// the revision it was prepared against. +pub struct SheetSource<'a> { + pub doc: &'a Document, + pub name: String, + pub scope: ExportScope, + pub expected_revision: u64, + /// Per-output-column pixel widths (only used with [`ExcelColumnWidths::Grid`]). + pub grid_widths_px: Option>, +} + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +fn read_err(e: XlsxError) -> AppError { + AppError::Other(format!("could not read the Excel workbook: {e}")) +} + +fn write_err(e: rust_xlsxwriter::XlsxError) -> AppError { + AppError::Other(format!("could not write the Excel workbook: {e}")) +} + +fn open(path: &Path) -> AppResult>> { + open_workbook(path).map_err(read_err) +} + +fn visibility_name(v: SheetVisible) -> &'static str { + match v { + SheetVisible::Visible => "visible", + SheetVisible::Hidden => "hidden", + SheetVisible::VeryHidden => "veryHidden", + } +} + +fn sheet_kind_name(t: SheetType) -> &'static str { + match t { + SheetType::WorkSheet => "worksheet", + SheetType::DialogSheet => "dialog", + SheetType::MacroSheet => "macro", + SheetType::ChartSheet => "chart", + SheetType::Vba => "vba", + } +} + +/// The five distinguishable kinds a source cell contributes, so a column can be +/// type-inferred without losing leading-zero text to numeric coercion. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum CellKind { + Empty, + Text, + Int, + Float, + Bool, + DateTime, +} + +/// Convert a 0-based column index to A1 letters (0 → `A`, 26 → `AA`). +fn col_to_letters(mut col: u32) -> String { + let mut out = Vec::new(); + loop { + out.push(b'A' + (col % 26) as u8); + if col < 26 { + break; + } + col = col / 26 - 1; + } + out.reverse(); + String::from_utf8(out).expect("ascii") +} + +/// A1 cell reference (`(row, col)`, both 0-based). +fn a1_cell(row: u32, col: u32) -> String { + format!("{}{}", col_to_letters(col), row + 1) +} + +/// A1 range for a `(start, end)` inclusive rectangle. +fn a1_range(start: (u32, u32), end: (u32, u32)) -> String { + format!("{}:{}", a1_cell(start.0, start.1), a1_cell(end.0, end.1)) +} + +/// An inclusive `(start, end)` cell rectangle, each corner `(row, col)` 0-based. +type A1Rect = ((u32, u32), (u32, u32)); + +fn col_letters_to_index(s: &str) -> Option { + if s.is_empty() { + return None; + } + let mut idx: u32 = 0; + for ch in s.chars() { + if !ch.is_ascii_alphabetic() { + return None; + } + let v = ch.to_ascii_uppercase() as u32 - 'A' as u32 + 1; + idx = idx.checked_mul(26)?.checked_add(v)?; + } + idx.checked_sub(1) +} + +/// Parse one A1 cell (`"B2"`, `"$B$2"`) to `(row, col)`, both 0-based. +fn parse_a1_cell(s: &str) -> Option<(u32, u32)> { + let s = s.replace('$', ""); + let split = s.find(|c: char| c.is_ascii_digit())?; + let (letters, digits) = s.split_at(split); + let col = col_letters_to_index(letters)?; + let row: u32 = digits.parse().ok()?; + row.checked_sub(1).map(|r| (r, col)) +} + +/// Parse an A1 range (`"B2:F100"` or a single cell) to a normalised inclusive +/// `(start, end)` rectangle. +fn parse_a1_range(s: &str) -> AppResult { + let s = s.trim(); + let bad = || AppError::invalid(format!("\"{s}\" is not a valid A1 cell range")); + let (a, b) = match s.split_once(':') { + Some((a, b)) => (a, b), + None => (s, s), + }; + let a = parse_a1_cell(a.trim()).ok_or_else(bad)?; + let b = parse_a1_cell(b.trim()).ok_or_else(bad)?; + Ok(((a.0.min(b.0), a.1.min(b.1)), (a.0.max(b.0), a.1.max(b.1)))) +} + +/// Parse a defined-name formula into `(sheet, range)` when it is a single +/// contiguous area on one sheet (`Sheet1!$A$1:$C$9`, `'My Sheet'!$B$2`). +fn parse_defined_name(formula: &str) -> Option<(String, A1Rect)> { + let formula = formula.trim().trim_start_matches('='); + // Multiple areas (comma-separated) or names/functions are not resolvable. + if formula.contains(',') { + return None; + } + let (sheet_part, range_part) = formula.rsplit_once('!')?; + let sheet = sheet_part.trim(); + let sheet = if let Some(inner) = sheet.strip_prefix('\'').and_then(|s| s.strip_suffix('\'')) { + inner.replace("''", "'") + } else { + sheet.to_string() + }; + let range = parse_a1_range(range_part).ok()?; + Some((sheet, range)) +} + +/// Render a float cell without an exponent for whole numbers (`3.0` → `"3"`). +fn format_float(f: f64) -> String { + if f == f.trunc() && f.is_finite() && f.abs() < 1e15 { + format!("{}", f as i64) + } else { + format!("{f}") + } +} + +/// Format an Excel datetime honouring the workbook epoch, WITHOUT shifting or +/// silently "correcting" dates. The epoch (1900 vs 1904) is baked into the +/// [`calamine::ExcelDateTime`] by the reader, and [`ExcelDateTime::to_ymd_hms_milli`] +/// decomposes the serial faithfully — including Excel's phantom 1900-02-29 +/// leap-year bug (serial 60), which `chrono` cannot represent, so we format the +/// components as text directly rather than round-tripping through a `NaiveDate`. +fn format_excel_datetime(edt: &calamine::ExcelDateTime) -> String { + if edt.is_duration() { + // A `[hh]:mm:ss` elapsed-time cell: render the elapsed clock. + if let Some(d) = edt.as_duration() { + let secs = d.num_seconds().max(0); + let (h, m, s) = (secs / 3600, (secs % 3600) / 60, secs % 60); + return format!("{h}:{m:02}:{s:02}"); + } + } + let (y, mo, d, h, mi, s, _ms) = edt.to_ymd_hms_milli(); + // A pure date (no time component) renders as a date; otherwise a datetime. + if h == 0 && mi == 0 && s == 0 { + format!("{y:04}-{mo:02}-{d:02}") + } else { + format!("{y:04}-{mo:02}-{d:02} {h:02}:{mi:02}:{s:02}") + } +} + +/// Turn one calamine cell into `(text, kind)`. +fn data_to_cell(d: &Data) -> (String, CellKind) { + match d { + Data::Empty => (String::new(), CellKind::Empty), + Data::String(s) => { + let kind = if s.is_empty() { + CellKind::Empty + } else { + CellKind::Text + }; + (s.clone(), kind) + } + Data::Int(i) => (i.to_string(), CellKind::Int), + Data::Float(f) => (format_float(*f), CellKind::Float), + Data::Bool(b) => ( + if *b { "TRUE" } else { "FALSE" }.to_string(), + CellKind::Bool, + ), + Data::DateTime(edt) => (format_excel_datetime(edt), CellKind::DateTime), + // ISO date/duration strings are already text; keep them verbatim. + Data::DateTimeIso(s) => (s.clone(), CellKind::Text), + Data::DurationIso(s) => (s.clone(), CellKind::Text), + Data::Error(e) => (e.to_string(), CellKind::Text), + } +} + +// --------------------------------------------------------------------------- +// Inspection (the OPEN CHOOSER) +// --------------------------------------------------------------------------- + +/// Inspect a workbook: sheets (with visibility), tables, named ranges, used +/// ranges, formula and merged-cell counts, header candidates and a bounded +/// preview per sheet. Read-only — nothing is imported. +pub fn inspect(path: &Path, ctx: Option<&JobCtx>) -> AppResult { + let mut wb = open(path)?; + let has_1904 = wb.has_1904_epoch(); + let sheets_meta: Vec = wb.sheets_metadata().to_vec(); + let defined: Vec<(String, String)> = wb.defined_names().to_vec(); + + // Named tables (best effort: a workbook without any is fine). + let mut tables = Vec::new(); + if wb.load_tables().is_ok() { + let names: Vec = wb.table_names().into_iter().cloned().collect(); + for name in names { + if let Ok(t) = wb.table_by_name(&name) { + let data = t.data(); + let (rows, range) = match (data.start(), data.end()) { + (Some(s), Some(e)) => (data.height() as u64, Some(a1_range(s, e))), + _ => (0, None), + }; + tables.push(TableInfo { + name: t.name().to_string(), + sheet: t.sheet_name().to_string(), + columns: t.columns().to_vec(), + rows, + range, + }); + } + } + } + + let named_ranges: Vec = defined + .into_iter() + .map(|(name, formula)| { + let (sheet, range) = match parse_defined_name(&formula) { + Some((s, (start, end))) => (Some(s), Some(a1_range(start, end))), + None => (None, None), + }; + NamedRangeInfo { + name, + formula, + sheet, + range, + } + }) + .collect(); + + let mut sheets = Vec::with_capacity(sheets_meta.len()); + let mut total_uncached = 0u64; + for meta in &sheets_meta { + if let Some(ctx) = ctx { + ctx.check()?; + } + let is_worksheet = meta.typ == SheetType::WorkSheet; + let range = if is_worksheet { + wb.worksheet_range(&meta.name).ok() + } else { + None + }; + let formulas = if is_worksheet { + wb.worksheet_formula(&meta.name).ok() + } else { + None + }; + let merges = if is_worksheet { + wb.merge_cells_by_sheet_name(&meta.name).unwrap_or_default() + } else { + Vec::new() + }; + + let (start_row, start_col, used_rows, used_cols, preview_rows, header_candidates) = + match &range { + Some(r) => { + let (sr, sc) = r.start().unwrap_or((0, 0)); + ( + sr, + sc, + r.height() as u32, + r.width() as u32, + preview_of(r), + header_candidates(r), + ) + } + None => (0, 0, 0, 0, Vec::new(), Vec::new()), + }; + + let formula_count = formulas + .as_ref() + .map(|f| f.used_cells().filter(|(_, _, s)| !s.is_empty()).count() as u64) + .unwrap_or(0); + let uncached = match (&range, &formulas) { + (Some(values), Some(f)) => f + .used_cells() + .filter(|(r, c, formula)| { + !formula.is_empty() + && values + .get_value((*r as u32, *c as u32)) + .map(|d| d.is_empty()) + .unwrap_or(true) + }) + .count() as u64, + _ => 0, + }; + total_uncached += uncached; + + sheets.push(SheetInfo { + name: meta.name.clone(), + visibility: visibility_name(meta.visible).to_string(), + kind: sheet_kind_name(meta.typ).to_string(), + // A worksheet with no used cells still yields `Some(range)` whose + // extents are empty; that is not importable data, so gate on the + // used dimensions rather than the mere presence of a range. + has_data: used_rows > 0 && used_cols > 0, + start_row, + start_col, + used_rows, + used_cols, + formula_count, + merged_count: merges.len() as u64, + formulas_without_cached_results: uncached, + header_candidates, + preview_rows, + }); + } + + let mut warnings = Vec::new(); + if total_uncached > 0 { + warnings.push(format!( + "{total_uncached} formula cell(s) have no cached result; under the default formula \ + policy they import blank because CEESVEE does not evaluate formulas" + )); + } + + Ok(WorkbookInfo { + has_1904_epoch: has_1904, + sheets, + tables, + named_ranges, + warnings, + }) +} + +/// Bounded top-left preview of a sheet range. +fn preview_of(range: &calamine::Range) -> Vec> { + range + .rows() + .take(PREVIEW_ROWS) + .map(|row| { + row.iter() + .take(PREVIEW_COLS) + .map(|cell| data_to_cell(cell).0) + .collect() + }) + .collect() +} + +/// Header-row candidates: rows near the top whose non-empty cells are all text +/// (a numeric/date row is data, not a header). Offsets are relative to the +/// range start. +fn header_candidates(range: &calamine::Range) -> Vec { + let mut out = Vec::new(); + for (offset, row) in range.rows().take(HEADER_SCAN_ROWS as usize).enumerate() { + let mut any = false; + let mut all_text = true; + for cell in row { + match cell { + Data::Empty => {} + Data::String(s) if !s.is_empty() => any = true, + Data::String(_) => {} + _ => { + any = true; + all_text = false; + } + } + } + if any && all_text { + out.push(offset as u32); + if out.len() >= MAX_HEADER_CANDIDATES { + break; + } + } + } + out +} + +// --------------------------------------------------------------------------- +// Materialisation (shared by preview and import) +// --------------------------------------------------------------------------- + +/// The imported grid plus the facts preview and import both need. +struct Grid { + headers: Vec, + rows: Vec>, + has_header: bool, + col_types: Vec, + source: String, + has_1904: bool, + uncached: u64, + warnings: Vec, +} + +/// A resolved source region on one sheet, plus any table-supplied headers. +struct Region { + sheet: String, + start: (u32, u32), + end: (u32, u32), + /// `Some` when the source is a table (its column names are authoritative + /// and the header row is intrinsic). + table_headers: Option>, + source_label: String, +} + +/// Resolve the import options to a concrete sheet region. +fn resolve_region( + wb: &mut Xlsx>, + opts: &ExcelImportOptions, +) -> AppResult { + if let Some(table) = &opts.table { + wb.load_tables() + .map_err(|e| AppError::invalid(format!("this workbook has no readable tables: {e}")))?; + let t = wb.table_by_name(table).map_err(|_| { + AppError::invalid(format!("no table named \"{table}\" in this workbook")) + })?; + let sheet = t.sheet_name().to_string(); + let headers = t.columns().to_vec(); + let data = t.data(); + let (start, end) = match (data.start(), data.end()) { + // The header row sits directly above the table body. + (Some((dr, dc)), Some((er, ec))) => ((dr.saturating_sub(1), dc), (er, ec)), + // A table with no body rows: read just its header row if we can + // place it, otherwise fall back to header-only. + _ => { + return Ok(Region { + sheet, + start: (0, 0), + end: (0, 0), + table_headers: Some(headers), + source_label: format!("table \"{table}\""), + }); + } + }; + return Ok(Region { + source_label: format!("table \"{table}\""), + sheet, + start, + end, + table_headers: Some(headers), + }); + } + + if let Some(name) = &opts.named_range { + let formula = wb + .defined_names() + .iter() + .find(|(n, _)| n == name) + .map(|(_, f)| f.clone()) + .ok_or_else(|| { + AppError::invalid(format!("no named range \"{name}\" in this workbook")) + })?; + let (sheet, (start, end)) = parse_defined_name(&formula).ok_or_else(|| { + AppError::invalid(format!( + "the named range \"{name}\" ({formula}) is not a single contiguous cell range" + )) + })?; + return Ok(Region { + source_label: format!("named range \"{name}\""), + sheet, + start, + end, + table_headers: None, + }); + } + + let sheet = opts + .sheet + .clone() + .ok_or_else(|| AppError::invalid("choose a sheet, table or named range to import"))?; + let range = wb.worksheet_range(&sheet).map_err(|_| { + AppError::invalid(format!("no worksheet named \"{sheet}\" in this workbook")) + })?; + let (used_start, used_end) = match (range.start(), range.end()) { + (Some(s), Some(e)) => (s, e), + _ => return Err(AppError::invalid(format!("the sheet \"{sheet}\" is empty"))), + }; + let (start, end, label) = match &opts.range { + Some(a1) => { + let (rs, re) = parse_a1_range(a1)?; + // Clamp the requested range to the used range. + let start = (rs.0.max(used_start.0), rs.1.max(used_start.1)); + let end = (re.0.min(used_end.0), re.1.min(used_end.1)); + if start.0 > end.0 || start.1 > end.1 { + return Err(AppError::invalid(format!( + "the range {a1} does not overlap any data on sheet \"{sheet}\"" + ))); + } + (start, end, format!("sheet \"{sheet}\" range {a1}")) + } + None => (used_start, used_end, format!("sheet \"{sheet}\"")), + }; + Ok(Region { + sheet, + start, + end, + table_headers: None, + source_label: label, + }) +} + +/// The materialised cells of a region: `(cells, per-cell kinds, uncached-formula count)`. +type RegionCells = (Vec>, Vec>, u64); + +/// Read one region into a string grid, honouring the formula and merged-cell +/// policies. Returns `(cells, kinds, uncached)`. +fn read_region( + values: &calamine::Range, + formulas: Option<&calamine::Range>, + merges: &[Dimensions], + region: &Region, + opts: &ExcelImportOptions, + ctx: Option<&JobCtx>, +) -> AppResult { + let (r0, c0) = region.start; + let (r1, c1) = region.end; + let n_cols = (c1 - c0 + 1) as usize; + let n_rows = (r1 - r0 + 1) as usize; + + // Compute one cell's (text, kind), applying the formula policy but NOT the + // merged-repeat override (which is layered on afterwards). + let cell_at = |r: u32, c: u32| -> (String, CellKind, bool, bool) { + let is_formula = formulas + .and_then(|f| f.get_value((r, c))) + .map(|s| !s.is_empty()) + .unwrap_or(false); + let cached_empty = values + .get_value((r, c)) + .map(|d| d.is_empty()) + .unwrap_or(true); + if is_formula { + match opts.formula { + FormulaPolicy::FormulaText => { + let text = formulas + .and_then(|f| f.get_value((r, c))) + .cloned() + .unwrap_or_default(); + (format!("={text}"), CellKind::Text, true, cached_empty) + } + FormulaPolicy::Blank => (String::new(), CellKind::Empty, true, cached_empty), + FormulaPolicy::CachedResult => { + let (text, kind) = values + .get_value((r, c)) + .map(data_to_cell) + .unwrap_or((String::new(), CellKind::Empty)); + (text, kind, true, cached_empty) + } + } + } else { + let (text, kind) = values + .get_value((r, c)) + .map(data_to_cell) + .unwrap_or((String::new(), CellKind::Empty)); + (text, kind, false, false) + } + }; + + let mut cells: Vec> = Vec::with_capacity(n_rows); + let mut kinds: Vec> = Vec::with_capacity(n_rows); + let mut uncached = 0u64; + let mut processed = 0usize; + for r in r0..=r1 { + let mut row_cells = Vec::with_capacity(n_cols); + let mut row_kinds = Vec::with_capacity(n_cols); + for c in c0..=c1 { + let (text, kind, is_formula, cached_empty) = cell_at(r, c); + if is_formula && cached_empty { + uncached += 1; + } + row_cells.push(text); + row_kinds.push(kind); + processed += 1; + if processed.is_multiple_of(CANCEL_EVERY_ROWS) { + if let Some(ctx) = ctx { + ctx.check()?; + } + } + } + cells.push(row_cells); + kinds.push(row_kinds); + } + + // Merged repeat: overwrite every covered cell within the selection with the + // top-left cell's computed string (and kind). + if opts.merged == MergedPolicy::Repeat { + for dim in merges { + let (tr, tc) = dim.start; + // Skip regions that do not touch the selection at all. + if dim.end.0 < r0 || dim.start.0 > r1 || dim.end.1 < c0 || dim.start.1 > c1 { + continue; + } + let (text, kind, _, _) = cell_at(tr, tc); + for r in dim.start.0..=dim.end.0 { + for c in dim.start.1..=dim.end.1 { + if r < r0 || r > r1 || c < c0 || c > c1 { + continue; + } + let ri = (r - r0) as usize; + let ci = (c - c0) as usize; + cells[ri][ci] = text.clone(); + kinds[ri][ci] = kind; + } + } + } + } + + Ok((cells, kinds, uncached)) +} + +/// Infer a logical type for a column from its data-cell kinds. A single text +/// cell keeps the column text (leading-zero protection); an all-numeric column +/// is integer or float; homogeneous boolean / datetime columns get their type. +fn infer_column(kinds: &[CellKind]) -> LogicalType { + let mut non_empty = 0; + let mut saw_text = false; + let mut saw_float = false; + let mut saw_int = false; + let mut saw_bool = false; + let mut saw_dt = false; + for &k in kinds { + match k { + CellKind::Empty => {} + CellKind::Text => { + saw_text = true; + non_empty += 1; + } + CellKind::Int => { + saw_int = true; + non_empty += 1; + } + CellKind::Float => { + saw_float = true; + non_empty += 1; + } + CellKind::Bool => { + saw_bool = true; + non_empty += 1; + } + CellKind::DateTime => { + saw_dt = true; + non_empty += 1; + } + } + } + if non_empty == 0 || saw_text { + return LogicalType::Text; + } + let numeric = saw_int || saw_float; + match (numeric, saw_bool, saw_dt) { + (true, false, false) => { + if saw_float { + LogicalType::Float + } else { + LogicalType::Integer + } + } + (false, true, false) => LogicalType::Boolean, + (false, false, true) => LogicalType::Datetime, + _ => LogicalType::Text, + } +} + +/// Materialise the selected source into a [`Grid`] (the shared core of +/// [`preview`] and [`import`]). +fn materialize(path: &Path, opts: &ExcelImportOptions, ctx: Option<&JobCtx>) -> AppResult { + let mut wb = open(path)?; + let has_1904 = wb.has_1904_epoch(); + let region = resolve_region(&mut wb, opts)?; + + let values = wb + .worksheet_range(®ion.sheet) + .map_err(|_| AppError::invalid(format!("could not read sheet \"{}\"", region.sheet)))?; + let formulas = wb.worksheet_formula(®ion.sheet).ok(); + let merges = if opts.merged == MergedPolicy::Repeat { + wb.merge_cells_by_sheet_name(®ion.sheet) + .unwrap_or_default() + } else { + Vec::new() + }; + + // An empty-body table degrades to a header-only import. + let region_has_data = region.start.0 <= region.end.0 && region.start.1 <= region.end.1; + let (mut cells, mut kinds, uncached) = if region_has_data { + read_region(&values, formulas.as_ref(), &merges, ®ion, opts, ctx)? + } else { + (Vec::new(), Vec::new(), 0) + }; + + // Header extraction. + let (headers, has_header) = if let Some(table_headers) = region.table_headers { + // Tables: the top region row IS the header row; drop it, use the + // table's authoritative column names. + if region_has_data && !cells.is_empty() { + cells.remove(0); + kinds.remove(0); + } + (table_headers, true) + } else { + match opts.header { + HeaderMode::None => { + let n = cells.first().map(Vec::len).unwrap_or(0); + ((0..n).map(|i| format!("Column {}", i + 1)).collect(), false) + } + HeaderMode::FirstRow => { + if cells.is_empty() { + (Vec::new(), true) + } else { + let headers = cells.remove(0); + kinds.remove(0); + (headers, true) + } + } + HeaderMode::Row { index } => { + let index = index as usize; + if index >= cells.len() { + return Err(AppError::invalid(format!( + "the chosen header row {index} is outside the selected range" + ))); + } + let headers = cells[index].clone(); + // Drop the header row and everything above it (title/notes rows). + cells.drain(0..=index); + kinds.drain(0..=index); + (headers, true) + } + } + }; + + let mut headers = headers; + let mut n_cols = headers.len().max(cells.first().map(Vec::len).unwrap_or(0)); + // Pad every row and the header to a rectangular width. + headers.resize(n_cols, String::new()); + for row in &mut cells { + row.resize(n_cols, String::new()); + } + for row in &mut kinds { + row.resize(n_cols, CellKind::Empty); + } + + // Blank-column trimming: drop columns empty in the header AND every data row. + if opts.trim_blank_columns && n_cols > 0 { + let keep: Vec = (0..n_cols) + .map(|c| !headers[c].is_empty() || cells.iter().any(|row| !row[c].is_empty())) + .collect(); + if keep.iter().any(|&k| !k) { + headers = filter_by(&headers, &keep); + for row in &mut cells { + *row = filter_by(row, &keep); + } + for row in &mut kinds { + *row = filter_by(row, &keep); + } + n_cols = headers.len(); + } + } + + // Blank-row trimming: drop data rows that are entirely empty. + if opts.trim_blank_rows { + let mut kept_kinds = Vec::with_capacity(cells.len()); + let mut kept_cells = Vec::with_capacity(cells.len()); + for (row, krow) in cells.into_iter().zip(kinds) { + if row.iter().any(|c| !c.is_empty()) { + kept_cells.push(row); + kept_kinds.push(krow); + } + } + cells = kept_cells; + kinds = kept_kinds; + } + + if n_cols == 0 { + return Err(AppError::invalid( + "the selection has no columns to import (everything was blank or trimmed away)", + )); + } + + // Column type inference from the data cells' kinds. + let col_types: Vec = (0..n_cols) + .map(|c| infer_column(&kinds.iter().map(|row| row[c]).collect::>())) + .collect(); + + let mut warnings = Vec::new(); + if uncached > 0 { + warnings.push(format!( + "{uncached} formula cell(s) in the selection have no cached result; CEESVEE does not \ + evaluate formulas, so they import blank under the cached-result policy" + )); + } + + Ok(Grid { + headers, + rows: cells, + has_header, + col_types, + source: region.source_label, + has_1904, + uncached, + warnings, + }) +} + +fn filter_by(items: &[T], keep: &[bool]) -> Vec { + items + .iter() + .zip(keep) + .filter(|(_, &k)| k) + .map(|(v, _)| v.clone()) + .collect() +} + +// --------------------------------------------------------------------------- +// Preview +// --------------------------------------------------------------------------- + +/// Preview importing the selected source under the chosen options: columns with +/// inferred types and counts, sample rows, projected dimensions and warnings. +pub fn preview( + path: &Path, + opts: &ExcelImportOptions, + ctx: Option<&JobCtx>, +) -> AppResult { + let grid = materialize(path, opts, ctx)?; + let n_cols = grid.headers.len(); + let columns: Vec = (0..n_cols) + .map(|c| { + let non_empty = grid.rows.iter().filter(|row| !row[c].is_empty()).count() as u64; + PreviewColumn { + name: grid.headers[c].clone(), + inferred_type: grid.col_types[c], + non_empty, + empty: grid.rows.len() as u64 - non_empty, + } + }) + .collect(); + let sample_rows: Vec> = grid.rows.iter().take(SAMPLE_ROWS).cloned().collect(); + Ok(ExcelImportPreview { + has_1904_epoch: grid.has_1904, + source: grid.source, + has_header_row: grid.has_header, + columns, + row_count: grid.rows.len() as u64, + column_count: n_cols, + sample_rows, + formulas_without_cached_results: grid.uncached, + warnings: grid.warnings, + }) +} + +// --------------------------------------------------------------------------- +// Import +// --------------------------------------------------------------------------- + +/// Import the selected source into a CEESVEE document through the standard +/// derived-document pipeline (in-memory editable for small results, indexed +/// read-only for large ones or when `forceIndexed` is set). Inferred column +/// schemas are attached so typed numbers/dates round-trip; text — including +/// leading-zero codes — stays text. The document is created fresh and marked +/// unsaved (dirty); the original workbook is never modified. +pub fn import( + path: &Path, + opts: &ExcelImportOptions, + cache_root: &Path, + doc_id: u64, + ctx: Option<&JobCtx>, +) -> AppResult { + if let Some(ctx) = ctx { + ctx.set_message("reading the workbook"); + } + let grid = materialize(path, opts, ctx)?; + if let Some(ctx) = ctx { + ctx.set_total(grid.rows.len() as u64); + ctx.set_message("building the document"); + } + + let budget = if opts.force_indexed { + 0 + } else { + crate::derived::SPILL_BUDGET + }; + let mut builder = + DerivedDocumentBuilder::new(grid.headers.clone(), cache_root.to_path_buf(), budget) + .with_header_row(grid.has_header); + let mut emitted = 0u64; + for row in grid.rows { + builder.push_row(row)?; + emitted += 1; + if emitted.is_multiple_of(CANCEL_EVERY_ROWS as u64) { + if let Some(ctx) = ctx { + ctx.advance(CANCEL_EVERY_ROWS as u64)?; + } + } + } + let mut doc = builder.finish(doc_id, &mut |_| match ctx { + Some(ctx) => ctx.check(), + None => Ok(()), + })?; + + // Attach the inferred schema for non-text columns (numbers/dates/booleans). + let ids = doc.column_ids().to_vec(); + let headers = doc.headers().to_vec(); + for (i, id) in ids.iter().enumerate() { + let logical = grid.col_types.get(i).copied().unwrap_or(LogicalType::Text); + if logical != LogicalType::Text { + let name = headers.get(i).cloned().unwrap_or_default(); + doc.set_column_schema(ColumnSchema::new(id.clone(), name, logical)); + } + } + Ok(doc) +} + +// --------------------------------------------------------------------------- +// Export +// --------------------------------------------------------------------------- + +/// Validate an Excel sheet name (length, forbidden characters, apostrophe +/// edges, non-blank). Uniqueness is checked by the caller across the set. +fn validate_sheet_name(name: &str) -> AppResult<()> { + if name.trim().is_empty() { + return Err(AppError::invalid("a sheet name must not be blank")); + } + if name.chars().count() > MAX_SHEET_NAME_LEN { + return Err(AppError::invalid(format!( + "the sheet name \"{name}\" is longer than Excel's {MAX_SHEET_NAME_LEN}-character limit" + ))); + } + if let Some(bad) = name.chars().find(|c| INVALID_SHEET_CHARS.contains(c)) { + return Err(AppError::invalid(format!( + "the sheet name \"{name}\" contains the character '{bad}', which Excel forbids" + ))); + } + if name.starts_with('\'') || name.ends_with('\'') { + return Err(AppError::invalid(format!( + "the sheet name \"{name}\" must not start or end with an apostrophe" + ))); + } + Ok(()) +} + +/// The output shape of one export sheet: absolute rows, columns and whether a +/// header row is written. Computed and limit-checked before any byte is +/// produced. +struct SheetPlan { + name: String, + rows: Vec, + cols: Vec, + has_header: bool, +} + +/// Validate the whole export up front — revisions, scopes, sheet names and +/// Excel's row/column limits — so a bad request is refused BEFORE writing. +pub fn plan_export(sheets: &[SheetSource<'_>]) -> AppResult<()> { + if sheets.is_empty() { + return Err(AppError::invalid( + "an Excel export needs at least one sheet", + )); + } + let mut seen: Vec = Vec::with_capacity(sheets.len()); + for s in sheets { + s.doc.check_revision(s.expected_revision)?; + validate_sheet_name(&s.name)?; + let lower = s.name.to_lowercase(); + if seen.contains(&lower) { + return Err(AppError::invalid(format!( + "two export sheets are both named \"{}\" (Excel sheet names must be unique)", + s.name + ))); + } + seen.push(lower); + let resolved = export_scope::resolve_scope(s.doc, &s.scope)?; + let has_header = s.doc.has_header_row(); + let out_rows = resolved.rows.len() as u64 + u64::from(has_header); + let out_cols = resolved.cols.len() as u64; + if out_rows > EXCEL_MAX_ROWS { + return Err(AppError::invalid(format!( + "sheet \"{}\" would have {out_rows} rows, over Excel's limit of {EXCEL_MAX_ROWS}; \ + export fewer rows or split the document", + s.name + ))); + } + if out_cols > EXCEL_MAX_COLS { + return Err(AppError::invalid(format!( + "sheet \"{}\" would have {out_cols} columns, over Excel's limit of {EXCEL_MAX_COLS}", + s.name + ))); + } + } + Ok(()) +} + +/// A date/datetime number format so typed date cells display as dates rather +/// than serial numbers. +fn date_format() -> Format { + Format::new().set_num_format("yyyy-mm-dd") +} + +fn datetime_format() -> Format { + Format::new().set_num_format("yyyy-mm-dd hh:mm:ss") +} + +/// Header styling: bold text on a light fill. +fn header_format() -> Format { + Format::new() + .set_bold() + .set_background_color(Color::RGB(0x00D9_E1F2)) + .set_pattern(FormatPattern::Solid) +} + +/// Try to write `cell` as a typed value under `schema`; `Ok(true)` when a typed +/// value was written, `Ok(false)` when the caller should fall back to text. +fn write_typed( + ws: &mut rust_xlsxwriter::Worksheet, + row: u32, + col: u16, + cell: &str, + schema: &ColumnSchema, + date_fmt: &Format, + datetime_fmt: &Format, +) -> AppResult { + match schema::classify(Some(cell), schema) { + CellState::Valid(TypedValue::Integer(v)) => { + // Only integers f64 can hold exactly; larger ones stay text. + if v.unsigned_abs() <= (1u128 << 53) { + ws.write_number(row, col, v as f64).map_err(write_err)?; + Ok(true) + } else { + Ok(false) + } + } + CellState::Valid(TypedValue::Decimal(d)) => match d.to_plain_string().parse::() { + Ok(f) if f.is_finite() => { + ws.write_number(row, col, f).map_err(write_err)?; + Ok(true) + } + _ => Ok(false), + }, + CellState::Valid(TypedValue::Float(f)) => { + ws.write_number(row, col, f).map_err(write_err)?; + Ok(true) + } + CellState::Valid(TypedValue::Boolean(b)) => { + ws.write_boolean(row, col, b).map_err(write_err)?; + Ok(true) + } + CellState::Valid(TypedValue::Date(d)) => { + // Excel dates live in 1900..=9999. A schema-typed year outside that + // window (reachable when a custom input format parses a signed + // out-of-range year, e.g. `+67536-01-01`) must fall back to text — + // the `as u16` cast below truncates modulo 65536, which could + // otherwise pass rust_xlsxwriter's own 1900..=9999 check as a + // bogus-but-valid in-range date and silently corrupt the value. + if !(1900..=9999).contains(&d.year()) { + return Ok(false); + } + match ExcelDateTime::from_ymd(d.year() as u16, d.month() as u8, d.day() as u8) { + Ok(edt) => { + ws.write_datetime_with_format(row, col, &edt, date_fmt) + .map_err(write_err)?; + Ok(true) + } + Err(_) => Ok(false), + } + } + CellState::Valid(TypedValue::DateTime(dt)) => { + // Same 1900..=9999 guard as the date branch, before the `as u16` + // truncation, so an out-of-range year falls back to text. + if !(1900..=9999).contains(&dt.year()) { + return Ok(false); + } + let built = ExcelDateTime::from_ymd(dt.year() as u16, dt.month() as u8, dt.day() as u8) + .and_then(|d| d.and_hms(dt.hour() as u16, dt.minute() as u8, dt.second() as f64)); + match built { + Ok(edt) => { + ws.write_datetime_with_format(row, col, &edt, datetime_fmt) + .map_err(write_err)?; + Ok(true) + } + Err(_) => Ok(false), + } + } + // Null/empty/missing → leave the cell blank; invalid → fall back to text. + CellState::NullToken | CellState::Empty | CellState::Missing => Ok(true), + CellState::Invalid(_) | CellState::Valid(_) => Ok(false), + } +} + +/// Write one document's slice into a fresh worksheet. +fn write_sheet( + ws: &mut rust_xlsxwriter::Worksheet, + source: &SheetSource<'_>, + plan: &SheetPlan, + options: &ExcelExportOptions, + ctx: &JobCtx, +) -> AppResult<()> { + ws.set_name(&plan.name).map_err(write_err)?; + let headers = source.doc.headers().to_vec(); + let date_fmt = date_format(); + let datetime_fmt = datetime_format(); + let hdr_fmt = header_format(); + + // Header row. + let data_start: u32 = if plan.has_header { + for (j, &c) in plan.cols.iter().enumerate() { + let name = headers.get(c).cloned().unwrap_or_default(); + if options.header_style { + ws.write_string_with_format(0, j as u16, name, &hdr_fmt) + .map_err(write_err)?; + } else { + ws.write_string(0, j as u16, name).map_err(write_err)?; + } + } + 1 + } else { + 0 + }; + + // Per-column schemas (only consulted with `typed`). + let schemas: Vec> = plan + .cols + .iter() + .map(|&c| { + if options.typed { + source.doc.column_schema_at(c).cloned() + } else { + None + } + }) + .collect(); + + let mut written = 0u32; + let mut pending = 0u64; + let mut err: Option = None; + source.doc.visit_rows_at(&plan.rows, &mut |_, row| { + let out_row = data_start + written; + for (j, &c) in plan.cols.iter().enumerate() { + let cell = row.get(c).map(String::as_str).unwrap_or(""); + let col = j as u16; + let typed_ok = match &schemas[j] { + Some(schema) => { + match write_typed(ws, out_row, col, cell, schema, &date_fmt, &datetime_fmt) { + Ok(v) => v, + Err(e) => { + err = Some(e); + return Ok(false); + } + } + } + None => false, + }; + if !typed_ok && !cell.is_empty() { + // Default (and fallback) path: text stays text — leading zeros + // and codes are preserved exactly. + if let Err(e) = ws.write_string(out_row, col, cell) { + err = Some(write_err(e)); + return Ok(false); + } + } + } + written += 1; + pending += 1; + if pending >= CANCEL_EVERY_ROWS as u64 { + if let Err(e) = ctx.advance(pending) { + err = Some(e); + return Ok(false); + } + pending = 0; + } + Ok(true) + })?; + if let Some(e) = err { + return Err(e); + } + ctx.advance(pending)?; + + // Freeze the header row. + if options.freeze_header && plan.has_header { + ws.set_freeze_panes(1, 0).map_err(write_err)?; + } + // Autofilter over the used range (only meaningful with a header row). + if options.autofilter && plan.has_header && !plan.cols.is_empty() { + let last_row = if written == 0 { + 0 + } else { + data_start + written - 1 + }; + ws.autofilter(0, 0, last_row, (plan.cols.len() - 1) as u16) + .map_err(write_err)?; + } + // Column widths. + match options.column_widths { + ExcelColumnWidths::Default => {} + ExcelColumnWidths::Autofit => { + ws.autofit(); + } + ExcelColumnWidths::Grid => { + if let Some(widths) = &source.grid_widths_px { + for (j, w) in widths.iter().enumerate().take(plan.cols.len()) { + if *w > 0.0 { + ws.set_column_width_pixels(j as u16, *w as u32) + .map_err(write_err)?; + } + } + } + } + } + Ok(()) +} + +/// Produce a workbook from one or more document slices and commit it atomically. +/// Revisions, scopes, sheet names and Excel's limits are validated first; the +/// workbook is built entirely in memory and only the final commit touches disk, +/// so a failure or cancellation leaves any existing destination untouched. +pub fn export( + sheets: &[SheetSource<'_>], + dest: &Path, + options: &ExcelExportOptions, + ctx: &JobCtx, +) -> AppResult { + plan_export(sheets)?; + + // Resolve every sheet's plan (rows/cols) up front. + let mut plans = Vec::with_capacity(sheets.len()); + let mut total_rows = 0u64; + for s in sheets { + s.doc.check_revision(s.expected_revision)?; + let resolved = export_scope::resolve_scope(s.doc, &s.scope)?; + total_rows += resolved.rows.len() as u64; + plans.push(SheetPlan { + name: s.name.clone(), + rows: resolved.rows, + cols: resolved.cols, + has_header: s.doc.has_header_row(), + }); + } + ctx.set_total(total_rows); + + let mut workbook = Workbook::new(); + for (s, plan) in sheets.iter().zip(&plans) { + ctx.check()?; + let ws = workbook.add_worksheet(); + write_sheet(ws, s, plan, options, ctx)?; + } + + ctx.check()?; + let buffer = workbook.save_to_buffer().map_err(write_err)?; + ctx.check()?; + + // Commit the finished workbook through the atomic-save pipeline. + let bytes = save::atomic_write(dest, options.backup, |file| { + file.write_all(&buffer)?; + Ok(buffer.len() as u64) + })?; + ctx.add_bytes(bytes); + ctx.flush_progress(); + Ok(bytes) +} + +// --------------------------------------------------------------------------- +// Preview caches (fetched after the `job-finished` event) +// --------------------------------------------------------------------------- + +/// Finished workbook inspections keyed by the job id that produced them. +#[derive(Default)] +pub struct ExcelInspectCache(Arc>>); + +impl ExcelInspectCache { + pub fn share(&self) -> Arc>> { + Arc::clone(&self.0) + } + + pub fn get(&self, job_id: u64) -> Option { + self.0.lock().ok()?.get(&job_id).cloned() + } +} + +/// Finished import previews keyed by the job id that produced them. +#[derive(Default)] +pub struct ExcelPreviewCache(Arc>>); + +impl ExcelPreviewCache { + pub fn share(&self) -> Arc>> { + Arc::clone(&self.0) + } + + pub fn get(&self, job_id: u64) -> Option { + self.0.lock().ok()?.get(&job_id).cloned() + } +} + +// --------------------------------------------------------------------------- +// Tests +// --------------------------------------------------------------------------- + +#[cfg(test)] +mod tests { + use super::*; + use std::io::Read as _; + use std::path::PathBuf; + + use calamine::{ + open_workbook as cal_open, Data, DataType, ExcelDateTime as CalDateTime, ExcelDateTimeType, + Xlsx, + }; + use rust_xlsxwriter::{Format as XFormat, Formula, Table, TableColumn, Workbook}; + + use crate::document::Document; + use crate::dto::{ExcelExportOptions, ExportScope}; + use crate::job::{JobCtx, JobRegistry}; + use crate::parse::{parse, ParseSettings}; + use crate::schema::{ColumnSchema, LogicalType}; + + // ----- fixtures ----------------------------------------------------------- + + fn book( + build: impl FnOnce(&mut Workbook) -> Result<(), rust_xlsxwriter::XlsxError>, + ) -> (tempfile::TempDir, PathBuf) { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("book.xlsx"); + let mut wb = Workbook::new(); + build(&mut wb).unwrap(); + wb.save(&path).unwrap(); + (dir, path) + } + + fn sheet_opts(name: &str) -> ExcelImportOptions { + ExcelImportOptions { + sheet: Some(name.to_string()), + ..Default::default() + } + } + + fn ctx() -> (JobRegistry, JobCtx) { + let reg = JobRegistry::default(); + let ctx = reg.begin("test", None, |_| {}); + (reg, ctx) + } + + fn doc_from(csv: &str, has_header: bool) -> Document { + let parsed = parse(csv.as_bytes(), &ParseSettings::default()).unwrap(); + Document::from_parsed(1, None, parsed, has_header) + } + + /// Rewrite one entry of an xlsx (a zip) in place, for scenarios rust_xlsxwriter + /// cannot express directly (a formula with no cached ``). + fn patch_entry(src: &Path, dst: &Path, entry: &str, edit: impl Fn(&str) -> String) { + let file = std::fs::File::open(src).unwrap(); + let mut archive = zip::ZipArchive::new(file).unwrap(); + let out = std::fs::File::create(dst).unwrap(); + let mut writer = zip::ZipWriter::new(out); + for i in 0..archive.len() { + let mut f = archive.by_index(i).unwrap(); + let name = f.name().to_string(); + let mut buf = Vec::new(); + f.read_to_end(&mut buf).unwrap(); + drop(f); + writer + .start_file(name.clone(), zip::write::SimpleFileOptions::default()) + .unwrap(); + if name == entry { + let patched = edit(&String::from_utf8(buf).unwrap()); + writer.write_all(patched.as_bytes()).unwrap(); + } else { + writer.write_all(&buf).unwrap(); + } + } + writer.finish().unwrap(); + } + + fn import_grid(path: &Path, opts: &ExcelImportOptions) -> (tempfile::TempDir, Document) { + let cache = tempfile::tempdir().unwrap(); + let doc = import(path, opts, cache.path(), 1, None).unwrap(); + (cache, doc) + } + + // ----- date systems ------------------------------------------------------- + + #[test] + fn excel_1900_leap_bug_and_1904_epoch_are_honoured_without_shifting() { + // Serial 60 is Excel's phantom 1900-02-29 (the 1900 leap-year bug); 61 + // is the real 1900-03-01. + let leap = CalDateTime::new(60.0, ExcelDateTimeType::DateTime, false); + assert_eq!(format_excel_datetime(&leap), "1900-02-29"); + let after = CalDateTime::new(61.0, ExcelDateTimeType::DateTime, false); + assert_eq!(format_excel_datetime(&after), "1900-03-01"); + + // The same serial reads as one date under the 1900 epoch and a DIFFERENT + // (1462-days-later) date under the 1904 epoch — the epoch is honoured, so + // a 1904 workbook's dates are not silently shifted to the 1900 reading. + let serial = 43831.0; // 2020-01-01 under the 1900 system + let d1900 = CalDateTime::new(serial, ExcelDateTimeType::DateTime, false); + let d1904 = CalDateTime::new(serial, ExcelDateTimeType::DateTime, true); + assert_eq!(format_excel_datetime(&d1900), "2020-01-01"); + assert_eq!(format_excel_datetime(&d1904), "2024-01-02"); + assert_ne!(format_excel_datetime(&d1900), format_excel_datetime(&d1904)); + } + + /// A cell written as a raw serial number carrying a date number-format, so + /// calamine classifies it as a real `Data::DateTime` on the workbook epoch + /// (the path `write_datetime` cannot express — it refuses pre-1900 serials + /// like the leap-bug's serial 60). + fn date_cell(ws: &mut rust_xlsxwriter::Worksheet, row: u32, col: u16, serial: f64) { + let fmt = XFormat::new().set_num_format("yyyy-mm-dd"); + ws.write_number_with_format(row, col, serial, &fmt).unwrap(); + } + + #[test] + fn real_date_cell_reproduces_1900_leap_bug_through_import() { + // Serial 60 is Excel's phantom 1900-02-29. Drive it through the FULL + // read pipeline (open_workbook → worksheet_range → data_to_cell → + // materialize → import), not just the isolated formatter. + let (_dir, path) = book(|wb| { + let ws = wb.add_worksheet(); + ws.set_name("Sheet1")?; + ws.write_string(0, 0, "d")?; + date_cell(ws, 1, 0, 60.0); + date_cell(ws, 2, 0, 61.0); + Ok(()) + }); + let (_cache, doc) = import_grid(&path, &sheet_opts("Sheet1")); + assert_eq!(doc.headers(), &["d"]); + let rows = doc.fetch_rows(&[0, 1]).unwrap(); + assert_eq!(rows[0][0], "1900-02-29", "phantom leap day survives import"); + assert_eq!(rows[1][0], "1900-03-01"); + // The column is inferred as a datetime and carries that schema. + assert_eq!( + doc.column_schema_at(0).map(|s| s.logical_type), + Some(LogicalType::Datetime), + ); + } + + #[test] + fn workbook_1904_epoch_is_plumbed_through_inspect_and_import() { + // Serial 43831 is 2020-01-01 under the default 1900 epoch. + let (dir, path) = book(|wb| { + let ws = wb.add_worksheet(); + ws.set_name("Sheet1")?; + ws.write_string(0, 0, "d")?; + date_cell(ws, 1, 0, 43831.0); + Ok(()) + }); + + // Baseline (default epoch): the flag is false and the date reads 2020. + let info0 = inspect(&path, None).unwrap(); + assert!(!info0.has_1904_epoch); + let (_c0, doc0) = import_grid(&path, &sheet_opts("Sheet1")); + assert_eq!(doc0.fetch_rows(&[0]).unwrap()[0][0], "2020-01-01"); + + // Inject the 1904 epoch flag exactly as a 1904 workbook stores it + // (``), leaving the serial untouched. + let patched = dir.path().join("book1904.xlsx"); + patch_entry(&path, &patched, "xl/workbook.xml", |xml| { + assert!(xml.contains(" (tempfile::TempDir, PathBuf) { + book(|wb| { + let ws = wb.add_worksheet(); + ws.set_name("Sheet1")?; + ws.write_string(0, 0, "a")?; + ws.write_string(0, 1, "b")?; + ws.write_string(0, 2, "c")?; + ws.merge_range(1, 0, 1, 2, "MERGED", &XFormat::new())?; + ws.write_string(2, 0, "x")?; + ws.write_string(2, 1, "y")?; + ws.write_string(2, 2, "z")?; + Ok(()) + }) + } + + #[test] + fn merged_top_left_only_blanks_the_rest() { + let (_dir, path) = merged_book(); + let opts = ExcelImportOptions { + merged: MergedPolicy::TopLeftOnly, + ..sheet_opts("Sheet1") + }; + let (_cache, doc) = import_grid(&path, &opts); + let rows = doc.fetch_rows(&[0, 1]).unwrap(); + assert_eq!( + rows[0], + vec!["MERGED".to_string(), String::new(), String::new()] + ); + assert_eq!( + rows[1], + vec!["x".to_string(), "y".to_string(), "z".to_string()] + ); + } + + #[test] + fn merged_repeat_fills_the_region() { + let (_dir, path) = merged_book(); + let opts = ExcelImportOptions { + merged: MergedPolicy::Repeat, + ..sheet_opts("Sheet1") + }; + let (_cache, doc) = import_grid(&path, &opts); + let rows = doc.fetch_rows(&[0]).unwrap(); + assert_eq!( + rows[0], + vec![ + "MERGED".to_string(), + "MERGED".to_string(), + "MERGED".to_string() + ] + ); + } + + // ----- formula policies --------------------------------------------------- + + fn formula_book() -> (tempfile::TempDir, PathBuf) { + book(|wb| { + let ws = wb.add_worksheet(); + ws.set_name("Sheet1")?; + ws.write_string(0, 0, "n")?; + ws.write_string(0, 1, "doubled")?; + ws.write_number(1, 0, 2)?; + ws.write_formula(1, 1, Formula::new("A2*10").set_result("424242"))?; + Ok(()) + }) + } + + #[test] + fn formula_cached_result_policy_uses_the_cached_value() { + let (_dir, path) = formula_book(); + let opts = ExcelImportOptions { + formula: FormulaPolicy::CachedResult, + ..sheet_opts("Sheet1") + }; + let (_cache, doc) = import_grid(&path, &opts); + assert_eq!(doc.fetch_rows(&[0]).unwrap()[0][1], "424242"); + } + + #[test] + fn formula_text_policy_emits_the_source() { + let (_dir, path) = formula_book(); + let opts = ExcelImportOptions { + formula: FormulaPolicy::FormulaText, + ..sheet_opts("Sheet1") + }; + let (_cache, doc) = import_grid(&path, &opts); + assert_eq!(doc.fetch_rows(&[0]).unwrap()[0][1], "=A2*10"); + } + + #[test] + fn formula_blank_policy_drops_the_formula() { + let (_dir, path) = formula_book(); + let opts = ExcelImportOptions { + formula: FormulaPolicy::Blank, + ..sheet_opts("Sheet1") + }; + let (_cache, doc) = import_grid(&path, &opts); + assert_eq!(doc.fetch_rows(&[0]).unwrap()[0][1], ""); + } + + #[test] + fn formula_without_cached_result_is_flagged_and_imports_blank() { + let (dir, path) = formula_book(); + // Strip the cached so the formula cell has no stored result, exactly + // as Excel/LibreOffice write it when the workbook was never recalculated. + let patched = dir.path().join("no-cache.xlsx"); + patch_entry(&path, &patched, "xl/worksheets/sheet1.xml", |xml| { + xml.replace("424242", "") + }); + + // The preview surfaces the no-cached-result warning flag. + let (_reg, c) = ctx(); + let preview = preview(&patched, &sheet_opts("Sheet1"), Some(&c)).unwrap(); + assert_eq!(preview.formulas_without_cached_results, 1); + assert!(preview + .warnings + .iter() + .any(|w| w.contains("no cached result"))); + + // And under the cached-result policy the cell imports blank. + let (_cache, doc) = import_grid(&patched, &sheet_opts("Sheet1")); + assert_eq!(doc.fetch_rows(&[0]).unwrap()[0][1], ""); + } + + // ----- range and table selection ------------------------------------------ + + #[test] + fn a1_range_selection_clips_to_the_requested_rectangle() { + let (_dir, path) = book(|wb| { + let ws = wb.add_worksheet(); + ws.set_name("Sheet1")?; + for r in 0..5u32 { + for col in 0..3u16 { + ws.write_string(r, col, format!("r{r}c{col}"))?; + } + } + Ok(()) + }); + let opts = ExcelImportOptions { + range: Some("A1:B3".to_string()), + ..sheet_opts("Sheet1") + }; + let (_cache, doc) = import_grid(&path, &opts); + assert_eq!(doc.headers(), &["r0c0", "r0c1"]); + assert_eq!(doc.n_rows(), 2, "rows 2-3 of the range are data"); + assert_eq!(doc.n_cols(), 2, "column C is outside the range"); + assert_eq!( + doc.fetch_rows(&[0]).unwrap()[0], + vec!["r1c0".to_string(), "r1c1".to_string()] + ); + } + + #[test] + fn named_table_selection_uses_the_table_columns_and_body() { + let (_dir, path) = book(|wb| { + let ws = wb.add_worksheet(); + ws.set_name("Sheet1")?; + ws.write_string(1, 0, "Apple")?; + ws.write_number(1, 1, 5)?; + ws.write_string(2, 0, "Pear")?; + ws.write_number(2, 1, 3)?; + let table = Table::new().set_name("Inventory").set_columns(&[ + TableColumn::new().set_header("Item"), + TableColumn::new().set_header("Qty"), + ]); + ws.add_table(0, 0, 2, 1, &table)?; + Ok(()) + }); + let opts = ExcelImportOptions { + table: Some("Inventory".to_string()), + ..Default::default() + }; + let (_cache, doc) = import_grid(&path, &opts); + assert_eq!(doc.headers(), &["Item", "Qty"]); + assert_eq!(doc.n_rows(), 2); + assert_eq!( + doc.fetch_rows(&[0]).unwrap()[0], + vec!["Apple".to_string(), "5".to_string()] + ); + } + + // ----- header modes + trimming -------------------------------------------- + + #[test] + fn chosen_header_row_drops_the_rows_above_it() { + let (_dir, path) = book(|wb| { + let ws = wb.add_worksheet(); + ws.set_name("Sheet1")?; + ws.write_string(0, 0, "Quarterly report")?; // title junk + ws.write_string(1, 0, "id")?; + ws.write_string(1, 1, "name")?; + ws.write_string(2, 0, "1")?; + ws.write_string(2, 1, "Ada")?; + ws.write_string(3, 0, "2")?; + ws.write_string(3, 1, "Bob")?; + Ok(()) + }); + let opts = ExcelImportOptions { + header: HeaderMode::Row { index: 1 }, + ..sheet_opts("Sheet1") + }; + let (_cache, doc) = import_grid(&path, &opts); + assert_eq!(doc.headers(), &["id", "name"]); + assert_eq!(doc.n_rows(), 2); + assert_eq!( + doc.fetch_rows(&[0]).unwrap()[0], + vec!["1".to_string(), "Ada".to_string()] + ); + } + + #[test] + fn blank_rows_and_columns_are_trimmed() { + let (_dir, path) = book(|wb| { + let ws = wb.add_worksheet(); + ws.set_name("Sheet1")?; + ws.write_string(0, 0, "a")?; // column B header + body all blank + ws.write_string(0, 2, "c")?; + ws.write_string(1, 0, "1")?; + ws.write_string(1, 2, "3")?; + // row 2 entirely blank + ws.write_string(3, 0, "4")?; + ws.write_string(3, 2, "6")?; + Ok(()) + }); + let opts = ExcelImportOptions { + trim_blank_rows: true, + trim_blank_columns: true, + ..sheet_opts("Sheet1") + }; + let (_cache, doc) = import_grid(&path, &opts); + assert_eq!(doc.headers(), &["a", "c"], "empty column B removed"); + assert_eq!(doc.n_rows(), 2, "the all-blank row removed"); + let rows = doc.fetch_rows(&[0, 1]).unwrap(); + assert_eq!(rows[0], vec!["1".to_string(), "3".to_string()]); + assert_eq!(rows[1], vec!["4".to_string(), "6".to_string()]); + } + + #[test] + fn forced_indexed_import_is_read_only_but_starts_unsaved() { + // `forceIndexed` spills to the read-only indexed backing. The result is + // still a brand-new document with no source CSV, so it must start dirty + // (closing warns, Save routes to Save As) rather than looking clean and + // silently dropping the just-imported data on close. + let (_dir, path) = book(|wb| { + let ws = wb.add_worksheet(); + ws.set_name("Sheet1")?; + ws.write_string(0, 0, "id")?; + ws.write_string(0, 1, "name")?; + ws.write_number(1, 0, 1)?; + ws.write_string(1, 1, "Ada")?; + Ok(()) + }); + let opts = ExcelImportOptions { + force_indexed: true, + ..sheet_opts("Sheet1") + }; + let (_cache, doc) = import_grid(&path, &opts); + assert!( + !doc.is_editable(), + "forceIndexed opens the read-only backing" + ); + assert!(doc.is_dirty(), "a fresh indexed import starts unsaved"); + assert!(doc.meta().path.is_none(), "no source path yet"); + assert_eq!(doc.n_rows(), 1); + } + + // ----- inspection --------------------------------------------------------- + + #[test] + fn inspect_reports_visibility_formulas_and_merges() { + let (_dir, path) = book(|wb| { + let ws1 = wb.add_worksheet(); + ws1.set_name("Data")?; + ws1.write_string(0, 0, "a")?; + ws1.write_number(1, 0, 1)?; + ws1.write_formula(1, 1, Formula::new("A2+1").set_result("2"))?; + ws1.merge_range(2, 0, 2, 1, "m", &XFormat::new())?; + let ws2 = wb.add_worksheet(); + ws2.set_name("Secret")?; + ws2.set_hidden(true); + ws2.write_string(0, 0, "x")?; + Ok(()) + }); + let info = inspect(&path, None).unwrap(); + assert!(!info.has_1904_epoch); + assert_eq!(info.sheets.len(), 2); + let data = info.sheets.iter().find(|s| s.name == "Data").unwrap(); + assert_eq!(data.visibility, "visible"); + assert_eq!(data.formula_count, 1); + assert_eq!(data.merged_count, 1); + let secret = info.sheets.iter().find(|s| s.name == "Secret").unwrap(); + assert_eq!(secret.visibility, "hidden"); + } + + #[test] + fn inspect_does_not_mark_an_empty_worksheet_as_having_data() { + // A blank first sheet must not look importable: calamine returns a + // `Some(range)` with empty extents for it, so `has_data` has to gate on + // the used dimensions, letting the chooser fall through to a real sheet. + let (_dir, path) = book(|wb| { + let blank = wb.add_worksheet(); + blank.set_name("Blank")?; + let data = wb.add_worksheet(); + data.set_name("Data")?; + data.write_string(0, 0, "id")?; + data.write_number(1, 0, 1)?; + Ok(()) + }); + let info = inspect(&path, None).unwrap(); + let blank = info.sheets.iter().find(|s| s.name == "Blank").unwrap(); + assert!(!blank.has_data, "an empty worksheet is not importable"); + assert_eq!(blank.used_rows, 0); + assert_eq!(blank.used_cols, 0); + let data = info.sheets.iter().find(|s| s.name == "Data").unwrap(); + assert!(data.has_data, "the populated worksheet has data"); + } + + // ----- export limits ------------------------------------------------------ + + #[test] + fn excel_limits_are_the_documented_maxima() { + assert_eq!(EXCEL_MAX_ROWS, 1_048_576); + assert_eq!(EXCEL_MAX_COLS, 16_384); + } + + #[test] + fn over_column_limit_export_is_refused_before_writing() { + let doc = Document::new_empty(1, (EXCEL_MAX_COLS + 1) as usize, 1); + let sources = [SheetSource { + doc: &doc, + name: "S".to_string(), + scope: ExportScope::All, + expected_revision: doc.revision(), + grid_widths_px: None, + }]; + let err = plan_export(&sources).unwrap_err(); + assert!(err.to_string().contains("columns"), "got: {err}"); + + // And the full export path refuses too, leaving no file behind. + let dir = tempfile::tempdir().unwrap(); + let dest = dir.path().join("out.xlsx"); + let (_reg, c) = ctx(); + assert!(export(&sources, &dest, &ExcelExportOptions::default(), &c).is_err()); + assert!(!dest.exists(), "refused export writes nothing"); + } + + // ----- multi-sheet export ------------------------------------------------- + + #[test] + fn multi_sheet_export_writes_one_workbook_per_tab() { + let d1 = doc_from("a,b\n1,2\n", true); + let d2 = doc_from("x\nhi\n", true); + let dir = tempfile::tempdir().unwrap(); + let dest = dir.path().join("multi.xlsx"); + let sources = [ + SheetSource { + doc: &d1, + name: "First".to_string(), + scope: ExportScope::All, + expected_revision: d1.revision(), + grid_widths_px: None, + }, + SheetSource { + doc: &d2, + name: "Second".to_string(), + scope: ExportScope::All, + expected_revision: d2.revision(), + grid_widths_px: None, + }, + ]; + let (_reg, c) = ctx(); + let bytes = export(&sources, &dest, &ExcelExportOptions::default(), &c).unwrap(); + assert!(bytes > 0); + assert_eq!(std::fs::metadata(&dest).unwrap().len(), bytes); + + let mut wb: Xlsx<_> = cal_open(&dest).unwrap(); + let names = wb.sheet_names(); + assert!(names.contains(&"First".to_string()) && names.contains(&"Second".to_string())); + let first = wb.worksheet_range("First").unwrap(); + assert_eq!(first.get_value((0, 0)), Some(&Data::String("a".into()))); + assert_eq!(first.get_value((1, 1)), Some(&Data::String("2".into()))); + let second = wb.worksheet_range("Second").unwrap(); + assert_eq!(second.get_value((1, 0)), Some(&Data::String("hi".into()))); + } + + #[test] + fn duplicate_sheet_names_are_rejected() { + let d = doc_from("a\n1\n", true); + let sources = [ + SheetSource { + doc: &d, + name: "Same".to_string(), + scope: ExportScope::All, + expected_revision: d.revision(), + grid_widths_px: None, + }, + SheetSource { + doc: &d, + name: "same".to_string(), + scope: ExportScope::All, + expected_revision: d.revision(), + grid_widths_px: None, + }, + ]; + assert!(plan_export(&sources) + .unwrap_err() + .to_string() + .contains("unique")); + } + + // ----- typed export ------------------------------------------------------- + + #[test] + fn typed_export_writes_numbers_booleans_and_dates() { + let mut d = doc_from("n,flag,when\n42,true,2020-01-15\n", true); + let ids = d.column_ids().to_vec(); + d.set_column_schema(ColumnSchema::new(ids[0].clone(), "n", LogicalType::Integer)); + d.set_column_schema(ColumnSchema::new( + ids[1].clone(), + "flag", + LogicalType::Boolean, + )); + d.set_column_schema(ColumnSchema::new(ids[2].clone(), "when", LogicalType::Date)); + let revision = d.revision(); + + let dir = tempfile::tempdir().unwrap(); + let dest = dir.path().join("typed.xlsx"); + let sources = [SheetSource { + doc: &d, + name: "Typed".to_string(), + scope: ExportScope::All, + expected_revision: revision, + grid_widths_px: None, + }]; + let opts = ExcelExportOptions { + typed: true, + ..Default::default() + }; + let (_reg, c) = ctx(); + export(&sources, &dest, &opts, &c).unwrap(); + + let mut wb: Xlsx<_> = cal_open(&dest).unwrap(); + let r = wb.worksheet_range("Typed").unwrap(); + assert_eq!(r.get_value((1, 0)).and_then(|d| d.as_f64()), Some(42.0)); + assert_eq!(r.get_value((1, 1)).and_then(|d| d.get_bool()), Some(true)); + assert_eq!( + r.get_value((1, 2)).and_then(|d| d.as_date()), + chrono::NaiveDate::from_ymd_opt(2020, 1, 15) + ); + } + + #[test] + fn typed_export_out_of_range_year_falls_back_to_text() { + // A custom input format can parse a signed year outside Excel's + // 1900..=9999 window. `67536 as u16 == 2000`, which would pass + // rust_xlsxwriter's own bounds check and write a bogus year-2000 date; + // the guard must instead fall back to text and preserve the source. + let mut d = doc_from("d\n+67536-01-01\n", true); + let ids = d.column_ids().to_vec(); + let mut sch = ColumnSchema::new(ids[0].clone(), "d", LogicalType::Date); + sch.input_formats = Some(vec!["%Y-%m-%d".to_string()]); + // Precondition: the value really is a Valid Date under the format (chrono + // accepts signed out-of-range years), so the text fallback below is the + // year-guard's doing, not a mere parse failure. + assert!( + matches!( + schema::classify(Some("+67536-01-01"), &sch), + CellState::Valid(TypedValue::Date(_)) + ), + "the out-of-range year must classify as a Valid Date" + ); + d.set_column_schema(sch); + let revision = d.revision(); + + let dir = tempfile::tempdir().unwrap(); + let dest = dir.path().join("oor.xlsx"); + let sources = [SheetSource { + doc: &d, + name: "S".to_string(), + scope: ExportScope::All, + expected_revision: revision, + grid_widths_px: None, + }]; + let opts = ExcelExportOptions { + typed: true, + ..Default::default() + }; + let (_reg, c) = ctx(); + export(&sources, &dest, &opts, &c).unwrap(); + + let mut wb: Xlsx<_> = cal_open(&dest).unwrap(); + let r = wb.worksheet_range("S").unwrap(); + // Preserved as text — NOT a truncated year-2000 date cell. + assert_eq!( + r.get_value((1, 0)), + Some(&Data::String("+67536-01-01".into())) + ); + } + + #[test] + fn untyped_export_keeps_leading_zero_codes_as_text() { + // Without a schema, a numeric-looking code stays a string cell so its + // leading zero survives the round-trip. + let d = doc_from("code\n00501\n", true); + let dir = tempfile::tempdir().unwrap(); + let dest = dir.path().join("codes.xlsx"); + let sources = [SheetSource { + doc: &d, + name: "Codes".to_string(), + scope: ExportScope::All, + expected_revision: d.revision(), + grid_widths_px: None, + }]; + let (_reg, c) = ctx(); + export(&sources, &dest, &ExcelExportOptions::default(), &c).unwrap(); + let mut wb: Xlsx<_> = cal_open(&dest).unwrap(); + let r = wb.worksheet_range("Codes").unwrap(); + assert_eq!(r.get_value((1, 0)), Some(&Data::String("00501".into()))); + } + + // ----- cancellation ------------------------------------------------------- + + #[test] + fn cancelled_export_leaves_no_output() { + let d = doc_from("a\n1\n2\n", true); + let dir = tempfile::tempdir().unwrap(); + let dest = dir.path().join("out.xlsx"); + std::fs::write(&dest, b"precious").unwrap(); + + let reg = JobRegistry::default(); + let c = reg.begin("export", None, |_| {}); + reg.cancel(c.id); + let sources = [SheetSource { + doc: &d, + name: "S".to_string(), + scope: ExportScope::All, + expected_revision: d.revision(), + grid_widths_px: None, + }]; + assert!(matches!( + export(&sources, &dest, &ExcelExportOptions::default(), &c), + Err(AppError::Cancelled) + )); + // The pre-existing destination is untouched (the atomic pipeline never + // ran) and no staging file is left behind. + assert_eq!(std::fs::read(&dest).unwrap(), b"precious"); + let stray = std::fs::read_dir(dir.path()) + .unwrap() + .filter_map(Result::ok) + .any(|e| e.file_name().to_string_lossy().contains(".ceesvee-save-")); + assert!(!stray); + } + + // ----- A1 helpers --------------------------------------------------------- + + #[test] + fn a1_parsing_round_trips() { + assert_eq!(parse_a1_cell("A1"), Some((0, 0))); + assert_eq!(parse_a1_cell("$B$2"), Some((1, 1))); + assert_eq!(parse_a1_cell("AA10"), Some((9, 26))); + assert_eq!(col_to_letters(0), "A"); + assert_eq!(col_to_letters(26), "AA"); + assert_eq!( + parse_a1_range("C3:A1").unwrap(), + ((0, 0), (2, 2)), + "normalised" + ); + assert!(parse_a1_range("nonsense").is_err()); + } + + #[test] + fn defined_name_parsing_extracts_sheet_and_range() { + assert_eq!( + parse_defined_name("Sheet1!$A$1:$C$9"), + Some(("Sheet1".to_string(), ((0, 0), (8, 2)))) + ); + assert_eq!( + parse_defined_name("'My Sheet'!$B$2"), + Some(("My Sheet".to_string(), ((1, 1), (1, 1)))) + ); + // A multi-area name is not a single contiguous range. + assert_eq!(parse_defined_name("Sheet1!$A$1,Sheet1!$C$3"), None); + } +} diff --git a/src-tauri/src/lib.rs b/src-tauri/src/lib.rs index f9ad838..49656fe 100644 --- a/src-tauri/src/lib.rs +++ b/src-tauri/src/lib.rs @@ -33,6 +33,11 @@ mod document; mod dto; mod encoding; mod error; +/// Public like [`job`]: the F34 Excel `.xlsx` interop engine (workbook +/// inspection, the import engine with merged/formula/date-system policies, and +/// the multi-sheet typed export) consumed by the F34 command surface and the +/// test harness. +pub mod excel; mod export; mod export_scope; mod filter; @@ -179,6 +184,8 @@ pub fn run() { .manage(crate::project::ProjectStore::default()) .manage(crate::annotations::AnnotationRegistry::default()) .manage(crate::highlight::HighlightStore::default()) + .manage(crate::excel::ExcelInspectCache::default()) + .manage(crate::excel::ExcelPreviewCache::default()) .setup(|app| { // Delete index caches orphaned by an abnormal termination. Live // instances hold their cache's lock file, so they are skipped. @@ -383,6 +390,12 @@ pub fn run() { commands::highlight_explain, commands::highlight_counts, commands::start_highlight_report, + commands::excel_inspect, + commands::get_excel_inspect, + commands::excel_import_preview, + commands::get_excel_import_preview, + commands::excel_import_apply, + commands::excel_export, ]) .build(tauri::generate_context!()) .expect("error while running tauri application") diff --git a/src/App.tsx b/src/App.tsx index 1804ef3..ab962a2 100644 --- a/src/App.tsx +++ b/src/App.tsx @@ -42,6 +42,8 @@ import { JsonExportDialog } from "./components/JsonExportDialog"; import { JsonImportDialog } from "./components/JsonImportDialog"; import { ColumnarExportDialog } from "./components/ColumnarExportDialog"; import { ParquetInspectDialog } from "./components/ParquetInspectDialog"; +import { ExcelExportDialog } from "./components/ExcelExportDialog"; +import { ExcelOpenDialog } from "./components/ExcelOpenDialog"; import { ProfilesDialog } from "./components/ProfilesDialog"; import { ProfileSuggestionBar } from "./components/ProfileSuggestionBar"; import { OpenModeDialog } from "./components/OpenModeDialog"; @@ -89,6 +91,7 @@ export default function App() { const setModal = useStore((s) => s.setModal); const jsonImportPath = useStore((s) => s.jsonImport?.path ?? null); const columnarOpenPath = useStore((s) => s.columnarOpen?.path ?? null); + const excelImportPath = useStore((s) => s.excelImport?.path ?? null); const diagnosticsOpen = useStore((s) => s.diagnosticsOpen); const changesOpen = useStore((s) => s.changesOpen); const annotationsPanelOpen = useStore((s) => s.annotationsPanelOpen); @@ -358,6 +361,8 @@ export default function App() { {activeModal === "columnarExport" && setModal(null)} />} {jsonImportPath && } {columnarOpenPath && } + {activeModal === "excelExport" && setModal(null)} />} + {excelImportPath && } diff --git a/src/components/ExcelExportDialog.tsx b/src/components/ExcelExportDialog.tsx new file mode 100644 index 0000000..d1baecf --- /dev/null +++ b/src/components/ExcelExportDialog.tsx @@ -0,0 +1,387 @@ +import { useMemo, useState } from "react"; + +import { + checkExportLimits, + columnWidthsLabel, + dedupeSheetNames, + defaultExportOptions, + gridWidthsPx, + outputColumnsForScope, + sanitizeSheetName, + sizingForScope, + suggestExcelFileName, + validateSheetNames, + type SheetSizing, +} from "../lib/excel"; +import { scopeChoices } from "../lib/export"; +import { useActiveMeta, useStore } from "../store/useStore"; +import type { + ExcelColumnWidths, + ExcelExportOptions, + ExcelSheetExport, + ExportScope, +} from "../types"; +import { Modal } from "./Modal"; + +type Mode = "active" | "tabs"; +const COLUMN_WIDTHS: ExcelColumnWidths[] = ["default", "autofit", "grid"]; + +/** + * Excel `.xlsx` export (F34). Two modes: one sheet from the active document + * (with the usual scope choices, and optional grid column widths), or one sheet + * per selected open tab into a single workbook. Header styling, a frozen header + * row, an autofilter, typed emission and column widths are all optional. Excel's + * row/column limits and sheet-name rules are checked BEFORE the write is offered + * (the backend re-checks and rejects too). Exports never touch a save point. + */ +export function ExcelExportDialog({ onClose }: { onClose: () => void }) { + const meta = useActiveMeta(); + const tabs = useStore((s) => s.tabs); + const activeId = useStore((s) => s.activeId); + const exportExcel = useStore((s) => s.exportExcel); + const columnWidths = useStore((s) => s.columnWidths); + const uiStates = useStore((s) => s.uiStates); + + const filtered = useStore((s) => s.tabs.find((t) => t.id === s.activeId)?.filtered ?? false); + const selectedRows = useStore((s) => s.selectedRows); + const selectedCols = useStore((s) => s.selectedCols); + const selectionRect = useStore((s) => s.selectionPhysicalRect)(); + const viewSorted = meta?.viewSorted ?? false; + + const [mode, setMode] = useState("active"); + const [opts, setOpts] = useState(() => defaultExportOptions()); + + // Active-document mode: scope + sheet name. + const choices = useMemo( + () => scopeChoices(filtered, selectionRect, selectedRows, selectedCols, viewSorted), + [filtered, selectionRect, selectedRows, selectedCols, viewSorted], + ); + const [scopeIdx, setScopeIdx] = useState(0); + const [activeName, setActiveName] = useState(() => + meta ? sanitizeSheetName(meta.fileName) : "Sheet", + ); + + // Multi-tab mode: which tabs are included, and each tab's (editable) sheet name. + const [selected, setSelected] = useState>( + () => new Set(activeId != null ? [activeId] : []), + ); + const [tabNames, setTabNames] = useState>(() => { + const deduped = dedupeSheetNames(tabs.map((t) => sanitizeSheetName(t.fileName))); + const out: Record = {}; + tabs.forEach((t, i) => (out[t.id] = deduped[i])); + return out; + }); + + const patch = (p: Partial) => setOpts((o) => ({ ...o, ...p })); + + const widthsForTab = (tabId: number): Record => + tabId === activeId ? columnWidths : (uiStates[tabId]?.columnWidths ?? {}); + + // Build the wire sheet set and the projected sizing for the limit pre-check. + const { sheets, sizings } = useMemo(() => { + if (mode === "active") { + if (!meta) return { sheets: [] as ExcelSheetExport[], sizings: [] as SheetSizing[] }; + const scope: ExportScope = (choices[scopeIdx] ?? choices[0]).scope; + const outCols = outputColumnsForScope(scope, meta.colCount); + const sheet: ExcelSheetExport = { + docId: meta.id, + name: activeName, + scope, + expectedRevision: meta.revision, + gridWidthsPx: + opts.columnWidths === "grid" ? gridWidthsPx(columnWidths, outCols) : undefined, + }; + const sizing = sizingForScope(activeName, scope, { + totalRows: meta.totalRowCount, + visibleRows: meta.rowCount, + columns: meta.colCount, + hasHeader: meta.hasHeaderRow, + }); + return { sheets: [sheet], sizings: [sizing] }; + } + // Multi-tab: one full sheet per selected tab, in tab order. + const included = tabs.filter((t) => selected.has(t.id)); + const allScope: ExportScope = { type: "all" }; + const sheetList: ExcelSheetExport[] = included.map((t) => { + const outCols = outputColumnsForScope(allScope, t.colCount); + return { + docId: t.id, + name: tabNames[t.id] ?? sanitizeSheetName(t.fileName), + scope: allScope, + expectedRevision: t.revision, + gridWidthsPx: + opts.columnWidths === "grid" ? gridWidthsPx(widthsForTab(t.id), outCols) : undefined, + }; + }); + const sizingList: SheetSizing[] = included.map((t) => ({ + name: tabNames[t.id] ?? sanitizeSheetName(t.fileName), + dataRows: t.totalRowCount, + columns: t.colCount, + hasHeader: t.hasHeaderRow, + })); + return { sheets: sheetList, sizings: sizingList }; + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [ + mode, + meta, + choices, + scopeIdx, + activeName, + tabs, + selected, + tabNames, + opts.columnWidths, + columnWidths, + uiStates, + activeId, + ]); + + const nameIssues = useMemo(() => validateSheetNames(sheets.map((s) => s.name)), [sheets]); + const limitViolations = useMemo(() => checkExportLimits(sizings), [sizings]); + + if (!meta) return null; + + const blocked = sheets.length === 0 || nameIssues.length > 0 || limitViolations.length > 0; + + const doExport = () => { + if (blocked) return; + const suggested = suggestExcelFileName( + mode === "active" + ? meta.fileName + : (tabs.find((t) => selected.has(t.id))?.fileName ?? meta.fileName), + ); + void exportExcel(sheets, opts, suggested); + onClose(); + }; + + const toggleTab = (id: number) => + setSelected((s) => { + const next = new Set(s); + if (next.has(id)) next.delete(id); + else next.add(id); + return next; + }); + + return ( + + + + + } + > +
+ {/* Mode */} +
+ setMode("active")}> + This document (one sheet) + + setMode("tabs")} + disabled={tabs.length < 2} + > + Open tabs (one sheet each) + +
+ + {mode === "active" ? ( + <> + + + + + setActiveName(e.target.value)} + className={`${inputCls} w-56`} + /> + + + ) : ( +
+

+ Tabs to include (a sheet per tab) +

+
+ {tabs.map((t) => ( +
+ toggleTab(t.id)} + className="accent-violet-600" + /> + + {t.fileName} + + setTabNames((n) => ({ ...n, [t.id]: e.target.value }))} + className={`${inputCls} flex-1 disabled:opacity-40`} + placeholder="sheet name" + /> + + {t.totalRowCount.toLocaleString()} × {t.colCount} + +
+ ))} +
+
+ )} + +
+ + {/* Options */} +
+ patch({ headerStyle: v })}> + Style the header (bold + fill) + + patch({ freezeHeader: v })}> + Freeze the header row + + patch({ autofilter: v })}> + Add an autofilter + + patch({ typed: v })}> + Typed numbers, dates & booleans (from the schema) + + patch({ backup: v ? "single" : "none" })} + > + Keep .bak of a replaced file + +
+ + + + + + {/* Sheet-name issues */} + {nameIssues.length > 0 && ( +
    + {nameIssues.map((issue, i) => ( +
  • • {issue.message}
    • + ))} +
+ )} + + {/* Limit pre-check */} + {limitViolations.length > 0 && ( +
    + {limitViolations.map((v, i) => ( +
  • • {v.message}
    • + ))} +
+ )} + +

+ Values only — formulas are never written. Cells invalid under their declared schema are + exported as text. Excel's 1,048,576-row × 16,384-column limits are enforced before + writing. +

+
+ + ); +} + +function ModeButton({ + active, + onClick, + disabled = false, + children, +}: { + active: boolean; + onClick: () => void; + disabled?: boolean; + children: React.ReactNode; +}) { + return ( + + ); +} + +function Row({ label, children }: { label: string; children: React.ReactNode }) { + return ( +
+ {label} + {children} +
+ ); +} + +function Check({ + checked, + onChange, + children, +}: { + checked: boolean; + onChange: (v: boolean) => void; + children: React.ReactNode; +}) { + return ( + + ); +} + +const btnGhost = + "rounded px-3 py-1.5 text-sm text-zinc-600 hover:bg-zinc-100 dark:text-zinc-300 dark:hover:bg-zinc-800"; +const selectCls = + "rounded border border-zinc-300 bg-transparent px-2 py-1 text-sm outline-none focus:border-violet-500 dark:border-zinc-700"; +const inputCls = + "rounded border border-zinc-300 bg-transparent px-2 py-1 text-sm outline-none focus:border-violet-500 dark:border-zinc-700"; diff --git a/src/components/ExcelOpenDialog.tsx b/src/components/ExcelOpenDialog.tsx new file mode 100644 index 0000000..429570b --- /dev/null +++ b/src/components/ExcelOpenDialog.tsx @@ -0,0 +1,667 @@ +import { useEffect, useMemo, useState } from "react"; + +import { + buildImportOptions, + defaultImportUi, + defaultSource, + headerCandidateChips, + isValidA1Range, + sheetKindLabel, + visibilityLabel, + type ExcelImportUi, + type ExcelSource, +} from "../lib/excel"; +import { useStore } from "../store/useStore"; +import type { ExcelFormulaPolicy, ExcelMergedPolicy, ExcelSheetInfo } from "../types"; +import { Modal } from "./Modal"; + +/** + * Upper bound on preview columns rendered at once. The import itself is + * unaffected; this only bounds the dialog's DOM, per the "bounded windows to + * React only" invariant. A wide sheet shows the first slice with a "+N more". + */ +const MAX_PREVIEW_COLUMNS = 60; + +/** + * Excel `.xlsx` open chooser (F34). Self-driven by the `excelImport` store + * slice, so opening a `.xlsx` file (or the "Open Excel…" command) shows it + * automatically. It renders the workbook inspection (sheets with visibility / + * dimensions / formula + merge counts, named tables, named ranges), lets the + * user pick a source and import options, previews the projected columns and + * sample rows through the job registry, and imports into a NEW document — the + * original workbook is never modified. + */ +export function ExcelOpenDialog() { + const st = useStore((s) => s.excelImport); + const derive = useStore((s) => s.derive); + const deriveError = useStore((s) => s.deriveError); + const runExcelPreview = useStore((s) => s.runExcelPreview); + const applyExcelImport = useStore((s) => s.applyExcelImport); + const cancelExcelPreview = useStore((s) => s.cancelExcelPreview); + const cancelDerive = useStore((s) => s.cancelDerive); + const dismiss = useStore((s) => s.dismissExcelImport); + + const workbook = st?.workbook ?? null; + + const [source, setSource] = useState(null); + const [ui, setUi] = useState(() => defaultImportUi()); + + // When the inspection lands (and nothing is chosen yet), select a default + // source so the first preview runs without a click. + useEffect(() => { + if (workbook && source === null) { + setSource(defaultSource(workbook)); + setUi(defaultImportUi()); + } + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [workbook]); + + const rangeValid = source?.kind === "sheet" ? isValidA1Range(ui.range) : true; + + const options = useMemo(() => (source ? buildImportOptions(source, ui) : null), [source, ui]); + const optionsKey = options ? JSON.stringify(options) : null; + const scannedKey = st?.options ? JSON.stringify(st.options) : null; + + // Re-run the preview whenever the built options diverge from the ones the + // current preview was scanned under (debounced, and never for an invalid range). + useEffect(() => { + if (optionsKey === null || optionsKey === scannedKey || !rangeValid) return; + const timer = setTimeout(() => void runExcelPreview(JSON.parse(optionsKey)), 300); + return () => clearTimeout(timer); + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [optionsKey, scannedKey, rangeValid]); + + if (!st) return null; + + const preview = st.preview; + const importing = derive?.kind === "excelImport"; + const scanning = st.previewJobId != null; + const inspecting = st.workbook === null && st.inspectJobId != null; + + const selectedSheet: ExcelSheetInfo | null = + source?.kind === "sheet" + ? (workbook?.sheets.find((s) => s.name === source.name) ?? null) + : null; + + const patchUi = (p: Partial) => setUi((u) => ({ ...u, ...p })); + + const shownColumns = preview ? preview.columns.slice(0, MAX_PREVIEW_COLUMNS) : []; + const hiddenColumns = (preview?.columnCount ?? 0) - shownColumns.length; + + const canImport = + !!source && + rangeValid && + !importing && + !scanning && + preview != null && + preview.columnCount > 0 && + st.inspectError === null; + + const noCached = preview?.formulasWithoutCachedResults ?? 0; + + return ( + + + + + } + > +
+ {inspecting && ( +

Inspecting the workbook…

+ )} + {st.inspectError && ( +

{st.inspectError}

+ )} + + {workbook && ( + <> +
+ {/* ----- Source picker ----- */} +
+

Source

+
+ + {workbook.sheets.map((sheet) => ( + setSource({ kind: "sheet", name: sheet.name })} + /> + ))} + + + {workbook.tables.length > 0 && ( + + {workbook.tables.map((table) => ( + setSource({ kind: "table", name: table.name })} + title={table.name} + subtitle={`${table.sheet} · ${table.columns.length} col${ + table.columns.length === 1 ? "" : "s" + } · ${table.rows.toLocaleString()} row${table.rows === 1 ? "" : "s"}${ + table.range ? ` · ${table.range}` : "" + }`} + /> + ))} + + )} + + {workbook.namedRanges.length > 0 && ( + + {workbook.namedRanges.map((nr) => { + const resolvable = nr.sheet != null && nr.range != null; + return ( + setSource({ kind: "namedRange", name: nr.name })} + title={nr.name} + subtitle={ + resolvable + ? `${nr.sheet}!${nr.range}` + : `${nr.formula} — not a single contiguous range` + } + /> + ); + })} + + )} +
+ {workbook.has1904Epoch && ( +

+ This workbook uses the 1904 date system — dates are read on that epoch and never + shifted. +

+ )} +
+ + {/* ----- Import options ----- */} +
+

+ Import options +

+ + {source?.kind === "table" && ( +

+ The table's own column names are the header row. +

+ )} + + {source?.kind === "sheet" && ( + <> + + patchUi({ range: e.target.value })} + className={`${inputCls} w-full ${ + rangeValid ? "" : "border-red-500 dark:border-red-500" + }`} + /> + + {!rangeValid && ( +

+ Enter a single cell or an A1 range like B2:F100, or leave it + blank for the whole used range. +

+ )} + + )} + + {(source?.kind === "sheet" || source?.kind === "namedRange") && ( + + )} + + + + + + + + + +
+ + + +
+
+
+ + {/* ----- No-cached-result warning banner ----- */} + {noCached > 0 && ui.formula === "cachedResult" && ( +
+ {noCached.toLocaleString()} formula cell{noCached === 1 ? "" : "s"} in this + selection have no cached result. CEESVEE does not evaluate formulas, so they import + blank under the cached-result policy — switch to “Keep the formula text” to keep the + source. +
+ )} + + {/* ----- Preview ----- */} + {preview && ( +
+
+ {preview.source} + + {preview.hasHeaderRow ? "with header row" : "no header row"} + + + {preview.rowCount.toLocaleString()} row{preview.rowCount === 1 ? "" : "s"} ×{" "} + {preview.columnCount} column{preview.columnCount === 1 ? "" : "s"} + +
+ + {/* Columns */} +
+ + + + + + + + + + + {shownColumns.map((c, i) => ( + + + + + + + ))} + +
ColumnTypeNon-emptyEmpty
+ {c.name || (unnamed)} + {c.inferredType} + {c.nonEmpty.toLocaleString()} + + {c.empty.toLocaleString()} +
+
+ {hiddenColumns > 0 && ( +

+ + {hiddenColumns.toLocaleString()} more column + {hiddenColumns === 1 ? "" : "s"} not shown (all {preview.columnCount} import). +

+ )} + + {/* Sample rows */} + {preview.sampleRows.length > 0 && ( +
+ + + + {shownColumns.map((c, i) => ( + + ))} + {hiddenColumns > 0 && ( + + )} + + + + {preview.sampleRows.map((row, ri) => ( + + {row.slice(0, MAX_PREVIEW_COLUMNS).map((cell, ci) => ( + + ))} + {hiddenColumns > 0 && } + + ))} + +
+ {c.name || "(unnamed)"} + + +{hiddenColumns.toLocaleString()}… +
+ {cell} + …
+
+ )} +
+ )} + + {/* Other warnings */} + {preview && preview.warnings.length > 0 && ( +
    + {preview.warnings.map((w, i) => ( +
  • • {w}
    • + ))} +
+ )} + +

+ Importing creates a new CEESVEE document — there is no in-place `.xlsx` save, and the + original workbook is never modified. +

+ + )} + + {/* Errors */} + {(st.previewError ?? deriveError) && ( +

{st.previewError ?? deriveError}

+ )} + + {/* Progress */} + {scanning && ( +
+ + Scanning… + {st.previewTotal != null && + st.previewTotal > 0 && + ` ${Math.min(100, Math.round((st.previewProcessed / st.previewTotal) * 100))}%`} + + +
+ )} + {importing && derive && ( +
+ + {derive.message ?? "importing"} — {derive.processed.toLocaleString()} + {derive.total != null && ` / ${derive.total.toLocaleString()}`} + + +
+ )} +
+ + ); +} + +function HeaderPicker({ + ui, + candidates, + onChange, +}: { + ui: ExcelImportUi; + candidates: number[]; + onChange: (p: Partial) => void; +}) { + const mode = ui.header.type; + const rowIndex = ui.header.type === "row" ? ui.header.index : 0; + return ( + +
+
+ + + +
+ {candidates.length > 0 && ( +
+ Detected: + {candidates.map((c) => ( + + ))} +
+ )} +
+ + ); +} + +function SheetRow({ + sheet, + selected, + onSelect, +}: { + sheet: ExcelSheetInfo; + selected: boolean; + onSelect: () => void; +}) { + const isWorksheet = sheet.kind === "worksheet"; + const selectable = isWorksheet && sheet.hasData; + const meta: string[] = []; + if (sheet.hasData) meta.push(`${sheet.usedRows.toLocaleString()} × ${sheet.usedCols}`); + else meta.push("empty"); + if (sheet.formulaCount > 0) meta.push(`${sheet.formulaCount.toLocaleString()} formula`); + if (sheet.mergedCount > 0) meta.push(`${sheet.mergedCount.toLocaleString()} merged`); + return ( + + {sheet.visibility !== "visible" && ( + {visibilityLabel(sheet.visibility)} + )} + {!isWorksheet && {sheetKindLabel(sheet.kind)}} + + } + subtitle={meta.join(" · ")} + /> + ); +} + +function SourceGroup({ label, children }: { label: string; children: React.ReactNode }) { + return ( +
+

+ {label} +

+ {children} +
+ ); +} + +function SourceRadio({ + name, + checked, + disabled = false, + onChange, + title, + subtitle, + badges, +}: { + name: string; + checked: boolean; + disabled?: boolean; + onChange: () => void; + title: string; + subtitle?: string; + badges?: React.ReactNode; +}) { + return ( + + ); +} + +function Badge({ children, tone }: { children: React.ReactNode; tone: "amber" | "zinc" }) { + const cls = + tone === "amber" + ? "bg-amber-100 text-amber-700 dark:bg-amber-950/60 dark:text-amber-300" + : "bg-zinc-100 text-zinc-500 dark:bg-zinc-800 dark:text-zinc-400"; + return ( + + {children} + + ); +} + +function Field({ label, children }: { label: string; children: React.ReactNode }) { + return ( + + ); +} + +const btnGhost = + "rounded px-3 py-1.5 text-sm text-zinc-600 hover:bg-zinc-100 dark:text-zinc-300 dark:hover:bg-zinc-800"; +const selectCls = + "rounded border border-zinc-300 bg-transparent px-2 py-1 text-sm outline-none focus:border-violet-500 dark:border-zinc-700"; +const inputCls = + "rounded border border-zinc-300 bg-transparent px-2 py-1 text-sm outline-none focus:border-violet-500 dark:border-zinc-700"; +const cancelBtn = + "rounded px-2 py-1 text-red-600 hover:bg-red-50 dark:text-red-400 dark:hover:bg-red-500/10"; diff --git a/src/lib/commandDefs.ts b/src/lib/commandDefs.ts index 8b52ef5..2240b63 100644 --- a/src/lib/commandDefs.ts +++ b/src/lib/commandDefs.ts @@ -80,6 +80,14 @@ function staticCommands(): AppCommand[] { allowInEditable: true, run: () => void state().openColumnarDialog(), }, + { + id: "file.openExcel", + title: "Open Excel…", + keywords: ["xlsx", "excel", "workbook", "spreadsheet", "sheet", "import", "table"], + category: "File", + allowInEditable: true, + run: () => void state().openExcelDialog(), + }, { id: "file.save", title: "Save", @@ -124,6 +132,14 @@ function staticCommands(): AppCommand[] { unavailableReason: needsDoc, run: () => openModal("columnarExport"), }, + { + id: "file.exportExcel", + title: "Export to Excel…", + keywords: ["xlsx", "excel", "workbook", "spreadsheet", "sheets", "tabs", "styling"], + category: "Export", + unavailableReason: needsDoc, + run: () => openModal("excelExport"), + }, { id: "file.closeTab", title: "Close tab", diff --git a/src/lib/excel.test.ts b/src/lib/excel.test.ts new file mode 100644 index 0000000..9fbf6e6 --- /dev/null +++ b/src/lib/excel.test.ts @@ -0,0 +1,370 @@ +import { describe, expect, it } from "vitest"; + +import { + buildImportOptions, + checkExportLimits, + dedupeSheetNames, + defaultExportOptions, + defaultImportUi, + defaultSource, + EXCEL_MAX_COLS, + EXCEL_MAX_ROWS, + gridWidthsPx, + headerCandidateChips, + isValidA1Range, + outputColumnsForScope, + sanitizeSheetName, + sizingForScope, + suggestExcelFileName, + validateSheetName, + validateSheetNames, + type ExcelSource, +} from "./excel"; +import type { ExcelWorkbookInfo } from "../types"; + +// ----- import option assembly ------------------------------------------------ + +describe("buildImportOptions", () => { + const ui = { ...defaultImportUi(), merged: "repeat" as const, formula: "formulaText" as const }; + + it("carries a sheet source's trimmed A1 range, dropping an empty one", () => { + const withRange = buildImportOptions( + { kind: "sheet", name: "Data" }, + { ...ui, range: " B2:F9 " }, + ); + expect(withRange.sheet).toBe("Data"); + expect(withRange.range).toBe("B2:F9"); + expect(withRange.table).toBeUndefined(); + expect(withRange.namedRange).toBeUndefined(); + + const noRange = buildImportOptions({ kind: "sheet", name: "Data" }, { ...ui, range: " " }); + expect(noRange.range).toBeUndefined(); + // The merged/formula policies still ride along. + expect(noRange.merged).toBe("repeat"); + expect(noRange.formula).toBe("formulaText"); + }); + + it("forces the intrinsic header for a table source and never sends a range", () => { + const opts = buildImportOptions( + { kind: "table", name: "Sales" }, + { ...ui, range: "B2:F9", header: { type: "none" } }, + ); + expect(opts.table).toBe("Sales"); + expect(opts.header).toEqual({ type: "firstRow" }); + expect(opts.range).toBeUndefined(); + expect(opts.sheet).toBeUndefined(); + }); + + it("passes a named-range source through with its chosen header mode", () => { + const opts = buildImportOptions( + { kind: "namedRange", name: "Region" }, + { ...ui, header: { type: "row", index: 2 } }, + ); + expect(opts.namedRange).toBe("Region"); + expect(opts.header).toEqual({ type: "row", index: 2 }); + expect(opts.sheet).toBeUndefined(); + }); +}); + +// ----- default source selection ---------------------------------------------- + +function workbook(partial: Partial): ExcelWorkbookInfo { + return { + has1904Epoch: false, + sheets: [], + tables: [], + namedRanges: [], + warnings: [], + ...partial, + }; +} + +function sheet(name: string, visibility: string, hasData: boolean, kind = "worksheet") { + return { + name, + visibility, + kind, + hasData, + startRow: 0, + startCol: 0, + usedRows: hasData ? 10 : 0, + usedCols: hasData ? 3 : 0, + formulaCount: 0, + mergedCount: 0, + formulasWithoutCachedResults: 0, + headerCandidates: [], + previewRows: [], + }; +} + +describe("defaultSource", () => { + it("prefers the first visible worksheet with data", () => { + const info = workbook({ + sheets: [ + sheet("Hidden", "hidden", true), + sheet("Empty", "visible", false), + sheet("Data", "visible", true), + ], + }); + expect(defaultSource(info)).toEqual({ kind: "sheet", name: "Data" }); + }); + + it("falls back to any worksheet with data, then a data table, then any worksheet", () => { + expect(defaultSource(workbook({ sheets: [sheet("H", "hidden", true)] }))).toEqual({ + kind: "sheet", + name: "H", + }); + // A data-bearing table beats an empty worksheet. + expect( + defaultSource( + workbook({ + sheets: [sheet("Blank", "visible", false)], + tables: [{ name: "T", sheet: "Blank", columns: ["a"], rows: 3, range: "A2:A4" }], + }), + ), + ).toEqual({ kind: "table", name: "T" }); + // With no data anywhere, an (empty) worksheet is still selectable last. + expect(defaultSource(workbook({ sheets: [sheet("Blank", "visible", false)] }))).toEqual({ + kind: "sheet", + name: "Blank", + }); + expect( + defaultSource( + workbook({ tables: [{ name: "T", sheet: "S", columns: ["a"], rows: 3, range: "A2:A4" }] }), + ), + ).toEqual({ kind: "table", name: "T" }); + }); + + it("returns null for an empty workbook", () => { + expect(defaultSource(workbook({}))).toBeNull(); + }); +}); + +// ----- detected header-row chips --------------------------------------------- + +describe("headerCandidateChips", () => { + it("surfaces the sheet's candidates for a whole-used-range sheet source", () => { + expect(headerCandidateChips({ kind: "sheet", name: "Data" }, [0, 2], "")).toEqual([0, 2]); + // Whitespace-only range still counts as "the whole used range". + expect(headerCandidateChips({ kind: "sheet", name: "Data" }, [1], " ")).toEqual([1]); + }); + + it("hides candidates once a custom A1 range is entered (offsets no longer align)", () => { + // The candidate offsets are sheet-used-range-relative; a sub-range shifts + // the import origin, so applying them would mis-select the header row. + expect(headerCandidateChips({ kind: "sheet", name: "Data" }, [0, 2], "B5:F99")).toEqual([]); + }); + + it("never surfaces candidates for a table or named-range source", () => { + expect(headerCandidateChips({ kind: "table", name: "T" }, [0, 2], "")).toEqual([]); + expect(headerCandidateChips({ kind: "namedRange", name: "R" }, [0, 2], "")).toEqual([]); + expect(headerCandidateChips(null, [0, 2], "")).toEqual([]); + }); + + it("tolerates an undefined candidate list", () => { + expect(headerCandidateChips({ kind: "sheet", name: "Data" }, undefined, "")).toEqual([]); + }); +}); + +// ----- A1 range validation --------------------------------------------------- + +describe("isValidA1Range", () => { + it("accepts an empty range (the whole used range)", () => { + expect(isValidA1Range("")).toBe(true); + expect(isValidA1Range(" ")).toBe(true); + }); + + it("accepts single cells and ranges with optional anchors", () => { + expect(isValidA1Range("A1")).toBe(true); + expect(isValidA1Range("B2:F100")).toBe(true); + expect(isValidA1Range("$B$2:$F$100")).toBe(true); + expect(isValidA1Range(" b2 : f9 ")).toBe(true); + }); + + it("rejects malformed ranges", () => { + expect(isValidA1Range("A")).toBe(false); + expect(isValidA1Range("1")).toBe(false); + expect(isValidA1Range("A1:B2:C3")).toBe(false); + expect(isValidA1Range("Sheet1!A1")).toBe(false); + }); +}); + +// ----- sheet-name validation ------------------------------------------------- + +describe("validateSheetName", () => { + it("accepts an ordinary name", () => { + expect(validateSheetName("Sales 2024")).toBeNull(); + }); + + it("rejects blank, over-long, forbidden-char and apostrophe-edge names", () => { + expect(validateSheetName(" ")).toMatch(/blank/); + expect(validateSheetName("x".repeat(32))).toMatch(/31-character/); + expect(validateSheetName("Q1/Q2")).toMatch(/forbids/); + expect(validateSheetName("Jan:Feb")).toMatch(/forbids/); + expect(validateSheetName("'quoted")).toMatch(/apostrophe/); + expect(validateSheetName("quoted'")).toMatch(/apostrophe/); + }); + + it("counts characters, not UTF-16 units, for the length limit", () => { + // 31 astral-plane emoji: 31 characters (62 UTF-16 units) — still valid. + expect(validateSheetName("😀".repeat(31))).toBeNull(); + expect(validateSheetName("😀".repeat(32))).toMatch(/31-character/); + }); +}); + +describe("validateSheetNames", () => { + it("flags case-insensitive duplicates", () => { + const issues = validateSheetNames(["Data", "Notes", "data"]); + expect(issues).toHaveLength(1); + expect(issues[0].index).toBe(2); + expect(issues[0].message).toMatch(/unique/); + }); + + it("returns no issues for a clean, distinct set", () => { + expect(validateSheetNames(["A", "B", "C"])).toEqual([]); + }); +}); + +describe("sanitizeSheetName / dedupeSheetNames", () => { + it("strips the extension, forbidden characters and apostrophe edges", () => { + expect(sanitizeSheetName("report/final.csv")).toBe("reportfinal"); + expect(sanitizeSheetName("'quoted'.xlsx")).toBe("quoted"); + }); + + it("clamps to 31 characters and never yields an empty name", () => { + expect([...sanitizeSheetName("a".repeat(50))].length).toBe(31); + expect(sanitizeSheetName("[]:*?/\\")).toBe("Sheet"); + expect(sanitizeSheetName("data.csv.xlsx")).toBe("data.csv"); + }); + + it("appends numeric suffixes to case-insensitive collisions", () => { + expect(dedupeSheetNames(["Data", "Data", "data"])).toEqual(["Data", "Data (2)", "data (3)"]); + }); +}); + +// ----- export limit checks (mirror plan_export) ------------------------------ + +describe("checkExportLimits", () => { + it("passes sheets within the limits", () => { + expect( + checkExportLimits([{ name: "S", dataRows: 1000, columns: 20, hasHeader: true }]), + ).toEqual([]); + }); + + it("counts the header toward the row limit", () => { + // Exactly at the row limit WITH a header: data rows = MAX - 1 is fine… + expect( + checkExportLimits([{ name: "S", dataRows: EXCEL_MAX_ROWS - 1, columns: 1, hasHeader: true }]), + ).toEqual([]); + // …but MAX data rows plus a header overflows by one. + const over = checkExportLimits([ + { name: "S", dataRows: EXCEL_MAX_ROWS, columns: 1, hasHeader: true }, + ]); + expect(over).toHaveLength(1); + expect(over[0].kind).toBe("rows"); + expect(over[0].actual).toBe(EXCEL_MAX_ROWS + 1); + }); + + it("flags over-wide sheets", () => { + const over = checkExportLimits([ + { name: "Wide", dataRows: 1, columns: EXCEL_MAX_COLS + 1, hasHeader: false }, + ]); + expect(over).toHaveLength(1); + expect(over[0].kind).toBe("columns"); + expect(over[0].message).toMatch(/16,384/); + }); + + it("reports each offending sheet independently", () => { + const over = checkExportLimits([ + { name: "OK", dataRows: 5, columns: 5, hasHeader: true }, + { name: "TallWide", dataRows: EXCEL_MAX_ROWS, columns: EXCEL_MAX_COLS + 1, hasHeader: true }, + ]); + expect(over).toHaveLength(2); + expect(over.every((v) => v.name === "TallWide")).toBe(true); + }); +}); + +// ----- scope → sizing / output columns / grid widths ------------------------- + +const dims = { totalRows: 100, visibleRows: 30, columns: 4, hasHeader: true }; + +describe("sizingForScope", () => { + it("maps each scope to its data-row and column extents", () => { + expect(sizingForScope("S", { type: "all" }, dims)).toMatchObject({ dataRows: 100, columns: 4 }); + expect(sizingForScope("S", { type: "visibleRows" }, dims)).toMatchObject({ dataRows: 30 }); + expect(sizingForScope("S", { type: "selectedRows", rows: [1, 2, 3] }, dims)).toMatchObject({ + dataRows: 3, + columns: 4, + }); + expect(sizingForScope("S", { type: "selectedColumns", columns: [0, 2] }, dims)).toMatchObject({ + dataRows: 30, + columns: 2, + }); + expect( + sizingForScope( + "S", + { type: "selectedRange", rect: { x: 1, y: 5, width: 2, height: 9 } }, + dims, + ), + ).toMatchObject({ dataRows: 9, columns: 2 }); + }); + + it("counts only the visible rows for selected columns under a filter", () => { + // A million-plus-row document filtered down to a subset within Excel's + // limit: exporting selected columns must size on the visible rows (what + // the backend writes), so the up-front limit check does not block it. + const filtered = { + totalRows: EXCEL_MAX_ROWS + 500, + visibleRows: 10, + columns: 4, + hasHeader: true, + }; + const sizing = sizingForScope("S", { type: "selectedColumns", columns: [0, 2] }, filtered); + expect(sizing).toMatchObject({ dataRows: 10, columns: 2 }); + expect(checkExportLimits([sizing])).toEqual([]); + }); +}); + +describe("outputColumnsForScope", () => { + it("returns every column for row scopes and the subset for column/range scopes", () => { + expect(outputColumnsForScope({ type: "all" }, 3)).toEqual([0, 1, 2]); + expect(outputColumnsForScope({ type: "selectedColumns", columns: [2, 0] }, 3)).toEqual([2, 0]); + expect( + outputColumnsForScope( + { type: "selectedRange", rect: { x: 1, y: 0, width: 2, height: 4 } }, + 5, + ), + ).toEqual([1, 2]); + }); +}); + +describe("gridWidthsPx", () => { + it("aligns widths to output columns, using 0 for unknown or non-positive widths", () => { + expect(gridWidthsPx({ 0: 80, 2: 120.6 }, [0, 1, 2])).toEqual([80, 0, 121]); + expect(gridWidthsPx({ 2: 100, 0: -5 }, [2, 0])).toEqual([100, 0]); + }); +}); + +// ----- misc ------------------------------------------------------------------ + +describe("defaults & file name", () => { + it("mirrors the Rust export defaults", () => { + expect(defaultExportOptions()).toEqual({ + headerStyle: true, + freezeHeader: true, + autofilter: false, + columnWidths: "default", + typed: true, + backup: "none", + }); + }); + + it("suggests an .xlsx name from a source name", () => { + expect(suggestExcelFileName("customers.csv")).toBe("customers.xlsx"); + expect(suggestExcelFileName("book")).toBe("book.xlsx"); + }); +}); + +// A tiny type-level anchor so an ExcelSource typo fails the build. +const _src: ExcelSource = { kind: "sheet", name: "x" }; +void _src; diff --git a/src/lib/excel.ts b/src/lib/excel.ts new file mode 100644 index 0000000..d09d407 --- /dev/null +++ b/src/lib/excel.ts @@ -0,0 +1,457 @@ +// Pure, framework-free helpers for the Excel `.xlsx` interop UI (F34): import +// option assembly, source selection, the Excel sheet-name and row/column limit +// checks (mirroring the Rust `excel` module EXACTLY so the dialog surfaces the +// same rejection the backend would, immediately and offline), and grid-width +// derivation for the export. Everything here is synchronous and side-effect +// free so it can be unit-tested without a backend. + +import type { + ExcelColumnWidths, + ExcelExportOptions, + ExcelHeaderMode, + ExcelImportOptions, + ExcelWorkbookInfo, + ExportScope, +} from "../types"; + +/** Excel's hard row limit (a worksheet has at most this many rows). */ +export const EXCEL_MAX_ROWS = 1_048_576; +/** Excel's hard column limit. */ +export const EXCEL_MAX_COLS = 16_384; +/** Longest sheet name Excel accepts. */ +export const MAX_SHEET_NAME_LEN = 31; +/** Characters Excel forbids in a sheet name (mirrors `INVALID_SHEET_CHARS`). */ +export const INVALID_SHEET_CHARS = ["[", "]", ":", "*", "?", "/", "\\"] as const; + +// --------------------------------------------------------------------------- +// Import options +// --------------------------------------------------------------------------- + +/** Which source an import reads from. */ +export type ExcelSourceKind = "sheet" | "table" | "namedRange"; + +/** A chosen import source in the open chooser. */ +export interface ExcelSource { + kind: ExcelSourceKind; + name: string; +} + +/** The per-source import UI state the dialog edits (before it becomes options). */ +export interface ExcelImportUi { + header: ExcelHeaderMode; + merged: ExcelImportOptions["merged"]; + formula: ExcelImportOptions["formula"]; + /** `A1` range text — sheet source only; empty means the whole used range. */ + range: string; + trimBlankRows: boolean; + trimBlankColumns: boolean; + forceIndexed: boolean; +} + +/** Sensible defaults for the per-source import controls (match the Rust defaults). */ +export function defaultImportUi(): ExcelImportUi { + return { + header: { type: "firstRow" }, + merged: "topLeftOnly", + formula: "cachedResult", + range: "", + trimBlankRows: false, + trimBlankColumns: false, + forceIndexed: false, + }; +} + +/** + * Assemble the wire `ExcelImportOptions` for a chosen source and UI state. A + * sheet source carries its optional `A1` range; a table's header is intrinsic + * (the backend ignores the header mode); a named range is a fixed region. + */ +export function buildImportOptions(source: ExcelSource, ui: ExcelImportUi): ExcelImportOptions { + const base: ExcelImportOptions = { + header: ui.header, + merged: ui.merged, + formula: ui.formula, + trimBlankRows: ui.trimBlankRows, + trimBlankColumns: ui.trimBlankColumns, + forceIndexed: ui.forceIndexed, + }; + if (source.kind === "table") { + // A table's own column names are the header; the header mode does not apply. + return { ...base, table: source.name, header: { type: "firstRow" } }; + } + if (source.kind === "namedRange") { + return { ...base, namedRange: source.name }; + } + const range = ui.range.trim(); + return { ...base, sheet: source.name, range: range === "" ? undefined : range }; +} + +/** + * The detected header-row candidate offsets to surface as one-click chips. + * + * The backend computes these offsets relative to the sheet's whole used-range + * start (they index the rows the import reads when the WHOLE used range is + * scanned). As soon as a custom A1 sub-range is entered, the import reads from + * that sub-range's origin instead, so a sheet-relative candidate offset no + * longer lines up — applying it would mis-select the header row (or error). + * We therefore surface candidates only for a sheet source with no custom range + * (and never for a table/named-range source, whose header is intrinsic/fixed). + * The manual "Row N" control, whose index is region-relative by construction, + * still works against any chosen range. + */ +export function headerCandidateChips( + source: ExcelSource | null, + sheetCandidates: number[] | undefined, + rangeText: string, +): number[] { + if (!source || source.kind !== "sheet") return []; + if (rangeText.trim() !== "") return []; + return sheetCandidates ?? []; +} + +/** + * Pick the source the chooser should select by default: the first visible + * worksheet that has data, else the first worksheet with data, else the first + * table (data beats an empty sheet), else the first worksheet, else null (an + * empty workbook). + */ +export function defaultSource(info: ExcelWorkbookInfo): ExcelSource | null { + const worksheets = info.sheets.filter((s) => s.kind === "worksheet"); + const visibleWithData = worksheets.find((s) => s.visibility === "visible" && s.hasData); + if (visibleWithData) return { kind: "sheet", name: visibleWithData.name }; + const withData = worksheets.find((s) => s.hasData); + if (withData) return { kind: "sheet", name: withData.name }; + if (info.tables.length > 0) return { kind: "table", name: info.tables[0].name }; + if (worksheets.length > 0) return { kind: "sheet", name: worksheets[0].name }; + return null; +} + +/** Human label for a sheet's visibility. */ +export function visibilityLabel(visibility: string): string { + switch (visibility) { + case "visible": + return "Visible"; + case "hidden": + return "Hidden"; + case "veryHidden": + return "Very hidden"; + default: + return visibility; + } +} + +/** Human label for a sheet's kind. */ +export function sheetKindLabel(kind: string): string { + switch (kind) { + case "worksheet": + return "Worksheet"; + case "dialog": + return "Dialog sheet"; + case "macro": + return "Macro sheet"; + case "chart": + return "Chart sheet"; + case "vba": + return "VBA"; + default: + return kind; + } +} + +/** + * Whether an `A1` range string is well-formed enough to send. An empty string + * is valid (it means "the whole used range"). Accepts a single cell (`B2`) or a + * `start:end` pair, each with optional `$` anchors — mirroring the shapes the + * backend's `parse_a1_range` accepts, so a doomed range disables the import + * before the invoke. + */ +export function isValidA1Range(text: string): boolean { + const trimmed = text.trim(); + if (trimmed === "") return true; + const parts = trimmed.split(":"); + if (parts.length > 2) return false; + return parts.every((p) => isA1Cell(p.trim())); +} + +function isA1Cell(cell: string): boolean { + return /^\$?[A-Za-z]{1,3}\$?[0-9]{1,7}$/.test(cell); +} + +// --------------------------------------------------------------------------- +// Export: sheet names +// --------------------------------------------------------------------------- + +/** + * Validate one Excel sheet name (mirrors the backend `validate_sheet_name`): + * non-blank, at most 31 characters, no forbidden characters, no leading/trailing + * apostrophe. Returns an error message, or null when the name is acceptable. + * Uniqueness is checked across the set by {@link validateSheetNames}. + */ +export function validateSheetName(name: string): string | null { + if (name.trim() === "") return "A sheet name must not be blank."; + if ([...name].length > MAX_SHEET_NAME_LEN) { + return `"${name}" is longer than Excel's ${MAX_SHEET_NAME_LEN}-character limit.`; + } + const bad = [...name].find((c) => (INVALID_SHEET_CHARS as readonly string[]).includes(c)); + if (bad !== undefined) { + return `"${name}" contains the character '${bad}', which Excel forbids in a sheet name.`; + } + if (name.startsWith("'") || name.endsWith("'")) { + return `"${name}" must not start or end with an apostrophe.`; + } + return null; +} + +/** One rejected sheet name in the export set. */ +export interface SheetNameIssue { + /** Index of the offending sheet in the input list. */ + index: number; + name: string; + message: string; +} + +/** + * Validate every export sheet name and reject case-insensitive duplicates + * (Excel sheet names are unique regardless of case). Returns one issue per + * offending sheet, in order. + */ +export function validateSheetNames(names: string[]): SheetNameIssue[] { + const issues: SheetNameIssue[] = []; + const seen = new Map(); + names.forEach((name, index) => { + const nameError = validateSheetName(name); + if (nameError) { + issues.push({ index, name, message: nameError }); + return; + } + const key = name.toLowerCase(); + const first = seen.get(key); + if (first !== undefined) { + issues.push({ + index, + name, + message: `Two sheets are both named "${name}" — Excel sheet names must be unique.`, + }); + return; + } + seen.set(key, index); + }); + return issues; +} + +/** + * Turn an arbitrary string (usually a file name) into a valid, non-empty Excel + * sheet name: strip the extension, drop forbidden characters, trim apostrophe + * edges and whitespace, clamp to 31 characters, and fall back to `"Sheet"`. + */ +export function sanitizeSheetName(raw: string): string { + const stem = raw.replace(/\.[^.\\/]+$/, ""); + let cleaned = [...stem] + .filter((c) => !(INVALID_SHEET_CHARS as readonly string[]).includes(c)) + .join(""); + cleaned = cleaned.replace(/^'+|'+$/g, "").trim(); + if ([...cleaned].length > MAX_SHEET_NAME_LEN) { + cleaned = [...cleaned].slice(0, MAX_SHEET_NAME_LEN).join("").trim(); + } + return cleaned === "" ? "Sheet" : cleaned; +} + +/** + * De-duplicate a list of sheet names case-insensitively, appending ` (2)`, + * ` (3)`, … to later collisions and re-clamping to Excel's length limit, so a + * multi-tab export starts from a writable set. + */ +export function dedupeSheetNames(names: string[]): string[] { + const seen = new Set(); + return names.map((name) => { + let candidate = name; + let n = 2; + while (seen.has(candidate.toLowerCase())) { + const suffix = ` (${n})`; + const room = MAX_SHEET_NAME_LEN - suffix.length; + const base = [...name].slice(0, Math.max(0, room)).join(""); + candidate = `${base}${suffix}`; + n++; + } + seen.add(candidate.toLowerCase()); + return candidate; + }); +} + +// --------------------------------------------------------------------------- +// Export: row/column limits +// --------------------------------------------------------------------------- + +/** The projected size of one export sheet, before any byte is written. */ +export interface SheetSizing { + name: string; + /** Data rows the scope resolves to (the header is added separately). */ + dataRows: number; + columns: number; + hasHeader: boolean; +} + +/** One sheet that would exceed an Excel dimension limit. */ +export interface LimitViolation { + name: string; + kind: "rows" | "columns"; + actual: number; + limit: number; + message: string; +} + +/** + * Check every export sheet against Excel's hard limits (1,048,576 rows × + * 16,384 columns), counting the header row toward the row total exactly as the + * backend's `plan_export` does. Returns one violation per offending sheet; an + * empty array means the export is within limits. The backend re-checks and is + * the authority, but this refuses an over-limit export up front. + */ +export function checkExportLimits(sheets: SheetSizing[]): LimitViolation[] { + const violations: LimitViolation[] = []; + for (const s of sheets) { + const outRows = s.dataRows + (s.hasHeader ? 1 : 0); + if (outRows > EXCEL_MAX_ROWS) { + violations.push({ + name: s.name, + kind: "rows", + actual: outRows, + limit: EXCEL_MAX_ROWS, + message: `Sheet "${s.name}" would have ${outRows.toLocaleString()} rows, over Excel's limit of ${EXCEL_MAX_ROWS.toLocaleString()}. Export fewer rows or split the document.`, + }); + } + if (s.columns > EXCEL_MAX_COLS) { + violations.push({ + name: s.name, + kind: "columns", + actual: s.columns, + limit: EXCEL_MAX_COLS, + message: `Sheet "${s.name}" would have ${s.columns.toLocaleString()} columns, over Excel's limit of ${EXCEL_MAX_COLS.toLocaleString()}.`, + }); + } + } + return violations; +} + +// --------------------------------------------------------------------------- +// Export: scope → sizing and grid widths +// --------------------------------------------------------------------------- + +/** Dimensions of a document, as the export sizing needs them. */ +export interface DocDimensions { + /** Total data rows (unfiltered). */ + totalRows: number; + /** Rows currently visible (after a filter / view). */ + visibleRows: number; + columns: number; + hasHeader: boolean; +} + +/** + * Project a scope onto a document to get the sheet's output size. Mirrors the + * shape of `export_scope::resolve_scope`: `all` writes every row, `visibleRows` + * and `selectedColumns` the filtered/view rows, and the row/range selections + * scope their own extents. + */ +export function sizingForScope(name: string, scope: ExportScope, dims: DocDimensions): SheetSizing { + switch (scope.type) { + case "all": + return { name, dataRows: dims.totalRows, columns: dims.columns, hasHeader: dims.hasHeader }; + case "visibleRows": + return { name, dataRows: dims.visibleRows, columns: dims.columns, hasHeader: dims.hasHeader }; + case "selectedRows": + return { + name, + dataRows: scope.rows.length, + columns: dims.columns, + hasHeader: dims.hasHeader, + }; + case "selectedColumns": + // The backend exports the VISIBLE rows of the selected columns, so a + // filtered view whose subset fits Excel's limit must not be blocked on + // the unfiltered total. + return { + name, + dataRows: dims.visibleRows, + columns: scope.columns.length, + hasHeader: dims.hasHeader, + }; + case "selectedRange": + return { + name, + dataRows: scope.rect.height, + columns: scope.rect.width, + hasHeader: dims.hasHeader, + }; + } +} + +/** + * The source column indices a scope writes, in output order. `all` / + * `visibleRows` / `selectedRows` write every column; the column and range + * scopes write their own subset. Used to align grid widths to output columns. + */ +export function outputColumnsForScope(scope: ExportScope, columnCount: number): number[] { + switch (scope.type) { + case "selectedColumns": + return [...scope.columns]; + case "selectedRange": { + const out: number[] = []; + for (let c = scope.rect.x; c < scope.rect.x + scope.rect.width; c++) out.push(c); + return out; + } + default: { + const out: number[] = []; + for (let c = 0; c < columnCount; c++) out.push(c); + return out; + } + } +} + +/** + * Per-output-column pixel widths for the `grid` column-width option, aligned to + * the sheet's output columns. A column with no recorded width contributes `0`, + * which the backend treats as "leave the default" (it only sets positive + * widths). + */ +export function gridWidthsPx( + columnWidths: Record, + outputColumns: number[], +): number[] { + return outputColumns.map((c) => { + const w = columnWidths[c]; + return typeof w === "number" && w > 0 ? Math.round(w) : 0; + }); +} + +// --------------------------------------------------------------------------- +// Export: options + labels +// --------------------------------------------------------------------------- + +/** Sensible defaults for a fresh Excel export (match the Rust `Default`). */ +export function defaultExportOptions(): ExcelExportOptions { + return { + headerStyle: true, + freezeHeader: true, + autofilter: false, + columnWidths: "default", + typed: true, + backup: "none", + }; +} + +const COLUMN_WIDTH_LABELS: Record = { + default: "Default width", + autofit: "Autofit to contents", + grid: "Match the current grid", +}; + +export function columnWidthsLabel(width: ExcelColumnWidths): string { + return COLUMN_WIDTH_LABELS[width]; +} + +/** Suggested `.xlsx` file name from a source name (extension replaced). */ +export function suggestExcelFileName(base: string): string { + const stem = base.replace(/\.[^.\\/]+$/, ""); + return `${stem}.xlsx`; +} diff --git a/src/lib/tauri.ts b/src/lib/tauri.ts index 3745167..1da8344 100644 --- a/src/lib/tauri.ts +++ b/src/lib/tauri.ts @@ -49,6 +49,11 @@ import type { DuplicateKeepStrategy, DuplicateReport, EncodingCompatibility, + ExcelExportOptions, + ExcelImportOptions, + ExcelImportPreview, + ExcelSheetExport, + ExcelWorkbookInfo, ExportOptions, ExportScope, ExternalChange, @@ -1210,6 +1215,57 @@ export const columnarExport = ( export const getColumnarExportReport = (jobId: number) => invoke("get_columnar_export_report", { jobId }); +// ----- Excel .xlsx interoperability (F34) ---------------------------------- + +/** + * Start a workbook inspection as a cancellable "scan" job (F34): sheets (with + * visibility), named tables, named ranges, used ranges + dimensions, formula + * and merged-cell counts, header candidates and bounded previews. Resolves + * with the job id; fetch the result with {@link getExcelInspect} once the + * `job-finished` event arrives. Nothing is opened. + */ +export const excelInspect = (path: string) => invoke("excel_inspect", { path }); + +/** The inspection of a finished Excel workbook scan, by its job id (F34). */ +export const getExcelInspect = (jobId: number) => + invoke("get_excel_inspect", { jobId }); + +/** + * Start an import preview of the chosen source/options as a cancellable "scan" + * job (F34): columns with inferred types, sample rows, projected dimensions and + * warnings (including formula cells with no cached result). Fetch the result + * with {@link getExcelImportPreview} after `job-finished`. + */ +export const excelImportPreview = (path: string, options?: ExcelImportOptions) => + invoke("excel_import_preview", { path, options }); + +/** The preview of a finished Excel import scan, by its job id (F34). */ +export const getExcelImportPreview = (jobId: number) => + invoke("get_excel_import_preview", { jobId }); + +/** + * Run an Excel import as a cancellable "derive" job (F34): the selected sheet / + * table / named range (optionally a cell sub-range) is read into a NEW CEESVEE + * document that registers under the returned docId when the job finishes. The + * original workbook is never modified; a failure leaves no document behind. + */ +export const excelImportApply = (path: string, options?: ExcelImportOptions) => + invoke("excel_import_apply", { path, options }); + +/** + * Start an Excel `.xlsx` export as a cancellable "export" job (F34): one sheet + * from one document, or several sheets (one per selected tab) into a single + * workbook. Revisions, scopes, sheet names and Excel's row/column limits are + * validated BEFORE the job spawns (the invoke rejects an over-limit request), + * then again inside it; the workbook commits through the atomic-save pipeline, + * so a failure or cancellation never touches an existing destination. + */ +export const excelExport = ( + sheets: ExcelSheetExport[], + path: string, + options: ExcelExportOptions, +) => invoke("excel_export", { sheets, path, options }); + // ----- project workspaces (F37) --------------------------------------------- // The ProjectStore is THE persistence boundary: typed, versioned sections are // written through `project_set_section` and flushed atomically by `project_save`. diff --git a/src/store/useStore.ts b/src/store/useStore.ts index b67a61a..892e318 100644 --- a/src/store/useStore.ts +++ b/src/store/useStore.ts @@ -135,6 +135,11 @@ import type { DraftField, HighlightRule, HighlightReportFormat, + ExcelImportOptions, + ExcelImportPreview, + ExcelExportOptions, + ExcelWorkbookInfo, + ExcelSheetExport, } from "../types"; import type { GatingWarning } from "../lib/project"; import { clampRecord, type RecordDraft } from "../lib/recordForm"; @@ -183,6 +188,7 @@ export type ModalName = | "dictionary" | "jsonExport" | "columnarExport" + | "excelExport" | "sampling" | "tagToColumn" | "annotationExport" @@ -193,6 +199,7 @@ const FILE_FILTERS = [ { name: "Delimited text", extensions: ["csv", "tsv", "tab", "txt", "psv", "dat"] }, { name: "JSON (F33)", extensions: ["json", "jsonl", "ndjson"] }, { name: "Columnar (F32)", extensions: ["parquet", "arrow", "feather", "ipc", "arrows"] }, + { name: "Excel (F34)", extensions: ["xlsx"] }, { name: "Compressed (F17)", extensions: ["gz", "zip"] }, { name: "All files", extensions: ["*"] }, ]; @@ -205,6 +212,9 @@ const COLUMNAR_FILE_FILTERS = [ { name: "All files", extensions: ["*"] }, ]; +/** File filter for the Excel `.xlsx` open/export dialogs (F34). */ +const EXCEL_FILE_FILTERS = [{ name: "Excel workbook", extensions: ["xlsx"] }]; + /** File filters for a JSON / JSON Lines export target (F33). */ const JSON_FILE_FILTERS = [ { name: "JSON", extensions: ["json"] }, @@ -595,7 +605,7 @@ export interface DeriveState { jobId: number; /** The id the NEW document will register under. */ docId: number; - kind: "append" | "join" | "groupBy" | "reshape" | "jsonImport"; + kind: "append" | "join" | "groupBy" | "reshape" | "jsonImport" | "excelImport"; processed: number; total: number | null; message: string | null; @@ -646,6 +656,32 @@ export interface ColumnarExportState { error: string | null; } +/** + * Excel `.xlsx` import flow state (F34). Non-null while the open chooser is on + * screen. Two scans back it: a one-shot workbook inspection (the chooser tree) + * and a per-source import preview that re-runs as the selected source / options + * change. Both run through the job registry (kind "scan"); the apply step reuses + * the shared `derive` slot (kind "excelImport"), so the finished document lands + * via the same pipeline as every other producer. + */ +export interface ExcelImportState { + path: string; + fileName: string; + /** Workbook inspection (the chooser); null until the inspect job finishes. */ + workbook: ExcelWorkbookInfo | null; + /** In-flight inspection job id; null when idle. */ + inspectJobId: number | null; + inspectError: string | null; + /** The options the current `preview` was scanned under. */ + options: ExcelImportOptions | null; + preview: ExcelImportPreview | null; + /** In-flight preview scan job id; null when idle. */ + previewJobId: number | null; + previewProcessed: number; + previewTotal: number | null; + previewError: string | null; +} + /** A running sampling/partitioning job (F48). Unlike a derive job it can emit * MANY new documents (one per partition) or none at all (a direct export). */ export interface SampleState { @@ -952,6 +988,8 @@ interface Store { columnarOpen: ColumnarOpenState | null; /** Result of the most recent Parquet / Arrow export (F32). */ columnarExportResult: ColumnarExportState; + /** Excel `.xlsx` import flow (F34); non-null while the open chooser is open. */ + excelImport: ExcelImportState | null; /** Running sampling/partitioning job (F48), if any. */ sample: SampleState | null; /** Error from the last sampling job, for the dialog that started it. */ @@ -1426,6 +1464,26 @@ interface Store { /** Reset the columnar export result (dialog closed / reopened). */ clearColumnarExport: () => void; + // Excel .xlsx interoperability (F34) + /** Prompt for an `.xlsx` file and route it into the open chooser. */ + openExcelDialog: () => Promise; + /** Open the Excel chooser for a file and run the workbook inspection. */ + openExcelImport: (path: string) => Promise; + /** (Re)run the import preview for a chosen source/options (supersedes any in-flight). */ + runExcelPreview: (options: ExcelImportOptions) => Promise; + /** Cancel the running import preview, if any. */ + cancelExcelPreview: () => Promise; + /** Import the chosen source into a NEW document (reuses the shared derive slot). */ + applyExcelImport: (options: ExcelImportOptions) => Promise; + /** Close the Excel chooser, cancelling any in-flight inspection/preview. */ + dismissExcelImport: () => void; + /** Prompt for a path and export the given sheet set to an `.xlsx` workbook (F34). */ + exportExcel: ( + sheets: ExcelSheetExport[], + options: ExcelExportOptions, + suggestedName: string, + ) => Promise; + // data-cleaning transforms (F06) /** * Apply a previewed transform (one undo step). Returns whether it was @@ -2378,6 +2436,7 @@ export const useStore = create((set, get) => { jsonImport: null, columnarOpen: null, columnarExportResult: { running: false, report: null, error: null }, + excelImport: null, sample: null, sampleError: null, samplingInitialMode: "sampling", @@ -2814,6 +2873,11 @@ export const useStore = create((set, get) => { if (typeof selected === "string") await get().openPath(selected); }, + openExcelDialog: async () => { + const selected = await openFileDialog({ multiple: false, filters: EXCEL_FILE_FILTERS }); + if (typeof selected === "string") await get().openPath(selected); + }, + openPath: async (path) => { const existing = get().tabs.find((t) => t.path === path); if (existing) { @@ -2854,6 +2918,12 @@ export const useStore = create((set, get) => { await get().openColumnarInspect(path); return; } + // F34: Excel workbooks route through the open chooser (sheet/table/range + // selection + import options), never the CSV open path. + if (lower.endsWith(".xlsx")) { + await get().openExcelImport(path); + return; + } set({ busy: true, error: null }); try { // F10: estimate the in-memory cost first. Large files pause here and @@ -4776,14 +4846,27 @@ export const useStore = create((set, get) => { return; } - // F33: JSON import preview scan progress (guarded by our own job id so a - // stray "scan"-kind job from elsewhere never touches this state). + // F33/F34: JSON and Excel import scans share the "scan" job kind, so match + // on our own in-flight job ids (a stray "scan" job never touches state). if (progress.kind === "scan") { - const st = get().jsonImport; - if (!st || st.scanJobId !== progress.jobId) return; - set({ - jsonImport: { ...st, scanProcessed: progress.processed, scanTotal: progress.total }, - }); + const js = get().jsonImport; + if (js && js.scanJobId === progress.jobId) { + set({ + jsonImport: { ...js, scanProcessed: progress.processed, scanTotal: progress.total }, + }); + return; + } + const xl = get().excelImport; + if (xl && (xl.previewJobId === progress.jobId || xl.inspectJobId === progress.jobId)) { + set({ + excelImport: { + ...xl, + previewProcessed: progress.processed, + previewTotal: progress.total, + }, + }); + return; + } return; } @@ -5069,9 +5152,10 @@ export const useStore = create((set, get) => { // The job registered the NEW document; add its tab and focus it. const meta = await api.getMeta(derive.docId); set((s) => ({ ...switchPatch(s, meta.id), tabs: [...s.tabs, meta] })); - // F33: a successful JSON import closes its dialog (the document is - // now open); a failure above leaves it open to show the error. + // F33/F34: a successful JSON/Excel import closes its dialog (the + // document is now open); a failure above leaves it open to show why. if (derive.kind === "jsonImport") set({ jsonImport: null }); + if (derive.kind === "excelImport") set({ excelImport: null }); } catch (e) { set({ deriveError: String(e) }); } @@ -5822,6 +5906,69 @@ export const useStore = create((set, get) => { } }, + // ----- Excel .xlsx interoperability (F34) -------------------------------------- + + openExcelImport: async (path) => { + const fileName = path.split(/[\\/]/).pop() ?? path; + set({ + excelImport: { + path, + fileName, + workbook: null, + inspectJobId: null, + inspectError: null, + options: null, + preview: null, + previewJobId: null, + previewProcessed: 0, + previewTotal: null, + previewError: null, + }, + }); + // Inspect the workbook (the chooser tree). The dialog drives the per-source + // preview once this lands and it has picked a default source. + try { + const jobId = await api.excelInspect(path); + if (get().excelImport?.path !== path) return; // dialog closed meanwhile + set((s) => + s.excelImport ? { excelImport: { ...s.excelImport, inspectJobId: jobId } } : {}, + ); + const finished = await awaitJob(jobId); + if (get().excelImport?.path !== path) return; + if (finished.status === "done") { + const workbook = await api.getExcelInspect(jobId); + set((s) => + s.excelImport + ? { excelImport: { ...s.excelImport, workbook, inspectJobId: null } } + : {}, + ); + } else if (finished.status === "failed") { + set((s) => + s.excelImport + ? { + excelImport: { + ...s.excelImport, + inspectJobId: null, + inspectError: finished.error ?? "could not read the workbook", + }, + } + : {}, + ); + } else { + set((s) => + s.excelImport ? { excelImport: { ...s.excelImport, inspectJobId: null } } : {}, + ); + } + } catch (e) { + if (get().excelImport?.path !== path) return; + set((s) => + s.excelImport + ? { excelImport: { ...s.excelImport, inspectJobId: null, inspectError: String(e) } } + : {}, + ); + } + }, + openColumnarRestore: async (path) => { // Non-interactive columnar restore (F37 project open). Reuse an already // open tab, otherwise reopen the source as an indexed READ-ONLY document @@ -5957,6 +6104,122 @@ export const useStore = create((set, get) => { clearColumnarExport: () => set({ columnarExportResult: { running: false, report: null, error: null } }), + runExcelPreview: async (options) => { + const st = get().excelImport; + if (!st) return; + // Supersede any in-flight preview: cancel it and reset progress. + if (st.previewJobId != null) void api.cancelJob(st.previewJobId).catch(() => undefined); + set((s) => + s.excelImport + ? { + excelImport: { + ...s.excelImport, + previewJobId: null, + previewProcessed: 0, + previewTotal: null, + previewError: null, + }, + } + : {}, + ); + try { + const jobId = await api.excelImportPreview(st.path, options); + if (get().excelImport?.path !== st.path) return; + set((s) => + s.excelImport ? { excelImport: { ...s.excelImport, previewJobId: jobId } } : {}, + ); + const finished = await awaitJob(jobId); + // A newer preview may have superseded this one. + if (get().excelImport?.previewJobId !== jobId) return; + if (finished.status === "done") { + const preview = await api.getExcelImportPreview(jobId); + set((s) => + s.excelImport + ? { + excelImport: { + ...s.excelImport, + previewJobId: null, + preview, + options, + previewError: null, + }, + } + : {}, + ); + } else if (finished.status === "failed") { + set((s) => + s.excelImport + ? { + excelImport: { + ...s.excelImport, + previewJobId: null, + previewError: finished.error ?? "preview failed", + }, + } + : {}, + ); + } else { + set((s) => + s.excelImport ? { excelImport: { ...s.excelImport, previewJobId: null } } : {}, + ); + } + } catch (e) { + if (get().excelImport?.path !== st.path) return; + set((s) => + s.excelImport + ? { excelImport: { ...s.excelImport, previewJobId: null, previewError: String(e) } } + : {}, + ); + } + }, + + cancelExcelPreview: async () => { + const jobId = get().excelImport?.previewJobId; + if (jobId != null) await api.cancelJob(jobId).catch(() => undefined); + }, + + applyExcelImport: async (options) => { + const st = get().excelImport; + // One derive slot at a time (shared with append/join/group/reshape/json). + if (!st || get().derive) return; + set({ deriveError: null }); + try { + const started = await api.excelImportApply(st.path, options); + get().trackDerive(started.jobId, started.docId, "excelImport"); + pushRecent(st.path); + } catch (e) { + set({ deriveError: String(e) }); + } + }, + + dismissExcelImport: () => { + const st = get().excelImport; + if (st?.inspectJobId != null) void api.cancelJob(st.inspectJobId).catch(() => undefined); + if (st?.previewJobId != null) void api.cancelJob(st.previewJobId).catch(() => undefined); + set({ excelImport: null }); + }, + + exportExcel: async (sheets, options, suggestedName) => { + if (sheets.length === 0) return; + const chosen = await saveFileDialog({ + defaultPath: suggestedName, + filters: EXCEL_FILE_FILTERS, + }); + if (!chosen) return; + try { + // plan_export() runs BEFORE the job spawns — revisions, sheet names and + // Excel's row/column limits reject the invoke synchronously, so an + // over-limit export never begins writing. We surface that rejection here. + const jobId = await api.excelExport(sheets, chosen, options); + const finished = await awaitJob(jobId); + if (finished.status === "failed") { + set({ error: finished.error ?? "Excel export failed" }); + } + } catch (e) { + set({ error: String(e) }); + } + }, + // ----- compare (F09) ----------------------------------------------------------- runCompare: async (rightDocId, spec) => { diff --git a/src/types.ts b/src/types.ts index d5344d8..d186957 100644 --- a/src/types.ts +++ b/src/types.ts @@ -1953,6 +1953,147 @@ export interface ColumnarExportOptions { backup: BackupPolicy; } +// ----- Excel .xlsx interoperability (F34) ----------------------------------- + +/** + * Which row of the selected region is the header (mirrors the Rust + * `HeaderMode`, a `#[serde(tag = "type")]` enum). `row` drops every row above + * the chosen index (title/notes rows); `none` synthesises `Column N` names. + */ +export type ExcelHeaderMode = + | { type: "firstRow" } + | { type: "row"; index: number } + | { type: "none" }; + +/** How merged cells are imported (mirrors Rust `MergedPolicy`). */ +export type ExcelMergedPolicy = "topLeftOnly" | "repeat"; + +/** How formula cells are imported (mirrors Rust `FormulaPolicy`). */ +export type ExcelFormulaPolicy = "cachedResult" | "formulaText" | "blank"; + +/** + * Everything an Excel import (preview or apply) needs (mirrors the Rust + * `ExcelImportOptions`). Exactly one of `sheet` / `table` / `namedRange` + * identifies the source; `range` narrows a sheet source to an `A1` rectangle. + */ +export interface ExcelImportOptions { + /** Worksheet to import (required unless `table` or `namedRange` is set). */ + sheet?: string; + /** Named table to import (its own column names become the header). */ + table?: string; + /** Named range to import (resolved to a sheet + cell range). */ + namedRange?: string; + /** `A1` sub-range within `sheet` (e.g. `"B2:F100"`); ignored otherwise. */ + range?: string; + header: ExcelHeaderMode; + merged: ExcelMergedPolicy; + formula: ExcelFormulaPolicy; + /** Drop rows entirely empty within the selection. */ + trimBlankRows: boolean; + /** Drop columns entirely empty within the selection. */ + trimBlankColumns: boolean; + /** Spill straight to the indexed read-only backing. */ + forceIndexed: boolean; +} + +/** One sheet in the Excel open chooser (mirrors Rust `SheetInfo`). */ +export interface ExcelSheetInfo { + name: string; + /** `"visible"`, `"hidden"` or `"veryHidden"`. */ + visibility: string; + /** `"worksheet"`, `"dialog"`, `"macro"`, `"chart"` or `"vba"`. */ + kind: string; + hasData: boolean; + startRow: number; + startCol: number; + usedRows: number; + usedCols: number; + formulaCount: number; + mergedCount: number; + /** Formula cells for which Excel stored no cached result. */ + formulasWithoutCachedResults: number; + /** Header-row candidates, as 0-based offsets from the used-range start. */ + headerCandidates: number[]; + /** Bounded preview of the cached cell values (top-left corner). */ + previewRows: string[][]; +} + +/** One named table in the Excel open chooser (mirrors Rust `TableInfo`). */ +export interface ExcelTableInfo { + name: string; + sheet: string; + columns: string[]; + rows: number; + /** `A1` range of the table body (headers excluded), when it has rows. */ + range?: string | null; +} + +/** One named range in the Excel open chooser (mirrors Rust `NamedRangeInfo`). */ +export interface ExcelNamedRangeInfo { + name: string; + /** The raw defined-name formula (e.g. `Sheet1!$A$1:$C$9`). */ + formula: string; + /** The sheet the range resolves to, when it is a simple single area. */ + sheet?: string | null; + range?: string | null; +} + +/** Everything the Excel open chooser renders (mirrors Rust `WorkbookInfo`). */ +export interface ExcelWorkbookInfo { + has1904Epoch: boolean; + sheets: ExcelSheetInfo[]; + tables: ExcelTableInfo[]; + namedRanges: ExcelNamedRangeInfo[]; + warnings: string[]; +} + +/** One column of an Excel import preview (mirrors Rust `PreviewColumn`). */ +export interface ExcelPreviewColumn { + name: string; + inferredType: LogicalType; + nonEmpty: number; + empty: number; +} + +/** + * The preview of importing the selected source under the chosen options + * (mirrors the Rust `ExcelImportPreview`). + */ +export interface ExcelImportPreview { + has1904Epoch: boolean; + /** Human description of what was scanned (sheet + range / table). */ + source: string; + hasHeaderRow: boolean; + columns: ExcelPreviewColumn[]; + rowCount: number; + columnCount: number; + sampleRows: string[][]; + /** Formula cells with no cached result that landed in the selection. */ + formulasWithoutCachedResults: number; + warnings: string[]; +} + +/** How an Excel export sizes its columns (mirrors Rust `ExcelColumnWidths`). */ +export type ExcelColumnWidths = "default" | "autofit" | "grid"; + +/** + * Options controlling an Excel `.xlsx` export (mirrors the Rust + * `ExcelExportOptions`). Values only — never formulas. + */ +export interface ExcelExportOptions { + /** Bold + filled styling on the header row. */ + headerStyle: boolean; + /** Freeze the header row so it stays visible while scrolling. */ + freezeHeader: boolean; + /** Add an autofilter over the used range (header row required). */ + autofilter: boolean; + columnWidths: ExcelColumnWidths; + /** Emit typed numbers/dates/booleans for schema-carrying columns. */ + typed: boolean; + /** Backup policy for the previous destination file. */ + backup: BackupPolicy; +} + /** Per-column invalid-cell total on a finished export (mirrors Rust * `ColumnWarning`). */ export interface ColumnWarning { @@ -1975,6 +2116,19 @@ export interface ColumnarExportReport { columnWarnings: ColumnWarning[]; } +/** + * One sheet of a (possibly multi-sheet) Excel export (mirrors the Rust + * `ExcelSheetExport`). + */ +export interface ExcelSheetExport { + docId: number; + name: string; + scope: ExportScope; + expectedRevision: number; + /** Per-output-column pixel widths (used only with `grid` column widths). */ + gridWidthsPx?: number[] | null; +} + // ----- project workspaces (F37) --------------------------------------------- /** Header state of the open project, for the project bar (F37). */