-
Notifications
You must be signed in to change notification settings - Fork 565
Docs: Add WOA documentation #2264
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,241 @@ | ||
| # Configure ModelOpt and build PyArrow on N1x | ||
|
|
||
| N1x uses Python 3.13 on Windows ARM64. Install ModelOpt and all dependencies | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I suggest we dont tie documentation with 1 SKU unless absolutely needed. We can focus on ModelOpt for WoA in general. |
||
| that publish compatible wheels with `pip`; build only PyArrow from source. | ||
|
Comment on lines
+3
to
+4
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift 🔎 Supported by static analysis🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- repository knowledge files ---'
find /tmp/coderabbit-repo-knowledge/nvidia-model-optimizer-9541ba76 -maxdepth 2 -type f -name '*.md' -print
printf '%s\n' '--- target files ---'
git ls-files | grep -E '(^|/)(README\.md|pyproject\.toml)$|examples/windows/python_313_arm64'
printf '%s\n' '--- target README sections ---'
sed -n '1,90p' examples/windows/python_313_arm64/README.md
sed -n '185,215p' examples/windows/python_313_arm64/README.md
printf '%s\n' '--- dependency declarations ---'
rg -n -C 8 'onnxruntime|onnxruntime_ep_nv_tensorrt_rtx|cupy-cuda13x|pyarrow|platform_machine|sys_platform|aarch64|arm64' pyproject.toml examples/windows/python_313_arm64
printf '%s\n' '--- scoped conventions and learnings ---'
for f in /tmp/coderabbit-repo-knowledge/nvidia-model-optimizer-9541ba76/*/*.md; do
printf '\n--- %s ---\n' "$f"
cat "$f"
doneRepository: NVIDIA/Model-Optimizer Length of output: 50380 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- README setup and validation ---'
sed -n '45,65p' examples/windows/python_313_arm64/README.md
sed -n '196,235p' examples/windows/python_313_arm64/README.md
printf '%s\n' '--- exact dependency references ---'
rg -n -C 3 'onnxruntime_ep_nv_tensorrt_rtx|onnxruntime-ep-nv-tensorrt-rtx|cupy-cuda12x|cupy-cuda13x|onnxruntime-gpu|onnxruntime~=|onnxruntime==' --glob '!*.lock' --glob '!*.ipynb' .Repository: NVIDIA/Model-Optimizer Length of output: 23171 🌐 Web query:
💡 Result: For the requested packages and libraries, here is the current status as of August 27, 2026: CuPy Windows ARM64 Support CuPy version 14.2.0 is available on PyPI [1][2]. Official documentation and package listings specify that CuPy wheels (precompiled binaries) for Windows are currently tested and provided primarily for x86_64 architectures [3][4]. While CuPy provides broader support for aarch64 on Linux, there is no official support or precompiled Citations:
Align the installation block with the smoke-test dependencies.
🤖 Prompt for AI Agents |
||
|
|
||
| Only environment setup is platform-specific. After it is complete, use the | ||
| [standard ONNX PTQ examples](../../onnx_ptq/README.md); quantization APIs, | ||
| formats, and generated models are the same as on regular Windows. | ||
|
|
||
| The PyArrow steps follow Apache Arrow's official | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I suggest reorganizing this content. As an alterntive, PyArrow steps can go in a new doc-source file in https://github.com/NVIDIA/Model-Optimizer/tree/main/docs/source/getting_started/windows. The main install page can have a install for WoA section / bullet - which should mention something of the following kind: ModelOpt's torch module (or xyz part) has dependencies on PyArrow package among others. PR-xyz is needed for PyArrow's WoA support. Until PyArrow is released with WoA support (PR-xyz), it needs to be built and installed locally. Refer instructions <here: hyperlink to pyarrow page>. I just gave one way to keep this content neat and clearer. You can organize it any other suitable way too. |
||
| [Windows ARM64 C++ build](https://arrow.apache.org/docs/dev/developers/cpp/windows.html#building-on-windows-arm64-using-ninja-and-clang) | ||
| and [self-contained wheel](https://arrow.apache.org/docs/dev/developers/python/building.html#self-contained-wheel) | ||
| instructions, with two fixes needed by the tested toolchain. | ||
|
|
||
| > [!WARNING] | ||
| > Apache Arrow considers Windows ARM64 support experimental. | ||
|
|
||
| The following configuration has been tested; these versions are not minimum | ||
| requirements. | ||
|
|
||
| | Component | Tested configuration | | ||
| |---|---| | ||
| | Platform | N1x, Windows ARM64 | | ||
| | Python | 3.13.14 ARM64 | | ||
| | Visual Studio | 2026 Community, ARM64 C++ tools | | ||
| | LLVM | 22.1.8 Windows ARM64 | | ||
| | CUDA and CuPy | CUDA 13.4, `cupy-cuda13x` 14.2.0 | | ||
| | PyArrow | 26.0.0 development source build | | ||
| | ONNX Runtime and TensorRT RTX ABI EP | 1.24.4 and 0.4.0 | | ||
| | ModelOpt | 0.47.0 development tree | | ||
|
|
||
| ## Configure the N1x environment | ||
|
|
||
| Create one virtual environment for both ModelOpt and the local PyArrow build: | ||
|
|
||
| ```powershell | ||
| $Python = "$env:LOCALAPPDATA\Programs\Python\Python313-arm64\python.exe" | ||
| $Venv = "$env:USERPROFILE\Desktop\py313-onnx-modelopt" | ||
| $ModelOptSource = "$env:USERPROFILE\Desktop\Model-Optimizer" | ||
|
Comment on lines
+37
to
+39
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win Make The guide sets Also applies to: 51-53 🤖 Prompt for AI Agents |
||
|
|
||
| if (-not (Test-Path "$Venv\Scripts\python.exe")) { | ||
| & $Python -m venv $Venv | ||
| } | ||
| $PythonExe = "$Venv\Scripts\python.exe" | ||
| & $PythonExe -m pip install --upgrade pip | ||
| ``` | ||
|
|
||
| Use `pip` for ModelOpt and every dependency that has an ARM64 wheel: | ||
|
|
||
| ```powershell | ||
| Push-Location $ModelOptSource | ||
| try { | ||
| & $PythonExe -m pip install -e ".[onnx]" | ||
| } finally { | ||
| Pop-Location | ||
| } | ||
| ``` | ||
|
|
||
| If another ModelOpt workflow, extra, or requirements file requests PyArrow, | ||
| build and install the missing wheel below, then rerun that original `pip install` | ||
| command. Do not build the other dependencies from source. | ||
|
|
||
| ## Prepare the PyArrow build | ||
|
|
||
| Install Visual Studio ARM64 C++ tools, the Windows SDK, Git, and LLVM for | ||
| Windows ARM64. Preserve LF endings when creating a new Arrow checkout: | ||
|
Comment on lines
+65
to
+66
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- repository knowledge files ---'
find /tmp/coderabbit-repo-knowledge/nvidia-model-optimizer-9541ba76 \
-maxdepth 2 -type f -name '*.md' -print | sort
printf '%s\n' '--- README outline and relevant ranges ---'
wc -l examples/windows/python_313_arm64/README.md
sed -n '1,145p' examples/windows/python_313_arm64/README.md
printf '%s\n' '--- direct LLVM/source-path references ---'
rg -n -C 3 '\$LlvmRoot|\$ModelOptSource|llvm-arm64|LLVM|Arrow|onnxruntime|cupy' \
examples/windows/python_313_arm64/README.mdRepository: NVIDIA/Model-Optimizer Length of output: 16972 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- examples convention ---'
cat /tmp/coderabbit-repo-knowledge/nvidia-model-optimizer-9541ba76/conventions/examples.md
printf '%s\n' '--- repository-wide convention ---'
cat /tmp/coderabbit-repo-knowledge/nvidia-model-optimizer-9541ba76/conventions/repo-wide.mdRepository: NVIDIA/Model-Optimizer Length of output: 6097 Point The setup defines 🤖 Prompt for AI Agents |
||
|
|
||
| ```powershell | ||
| git -c core.autocrlf=false clone https://github.com/apache/arrow.git ` | ||
| "$env:USERPROFILE\Desktop\arrow" | ||
| ``` | ||
|
|
||
| Set the build paths and load the Visual Studio ARM64 environment: | ||
|
|
||
| ```powershell | ||
| $ArrowSource = "$env:USERPROFILE\Desktop\arrow" | ||
| $ArrowBuild = "$ArrowSource\cpp\build-py313-arm64" | ||
| $ArrowHome = "$Venv\arrow-dist" | ||
| $LlvmRoot = "$Venv\llvm-arm64" | ||
| $VsRoot = "C:\Program Files\Microsoft Visual Studio\18\Community" | ||
|
|
||
| & "$VsRoot\Common7\Tools\Launch-VsDevShell.ps1" ` | ||
| -Arch arm64 -HostArch arm64 -SkipAutomaticLocation | ||
| $env:PATH = "$LlvmRoot\bin;$Venv\Scripts;$env:PATH" | ||
|
|
||
| & $PythonExe -m pip install --upgrade ` | ||
| build cmake ninja "cython>=3.1" "numpy>=2.0" ` | ||
| "scikit-build-core>=1.0" "setuptools_scm[toml]>=8" "libcst>=1.8.6" | ||
| ``` | ||
|
|
||
| Confirm that Python reports `ARM64` and LLVM reports an ARM64 target: | ||
|
|
||
| ```powershell | ||
| & $PythonExe -c "import platform; print(platform.machine())" | ||
| & "$LlvmRoot\bin\clang-cl.exe" --version | ||
| ``` | ||
|
|
||
| ## Build Arrow C++ | ||
|
|
||
| Create the small Arrow Dataset compatibility header required by the tested | ||
| Visual Studio and LLVM combination: | ||
|
|
||
| ```powershell | ||
| $DatasetShim = "$Venv\arrow-dataset-fileinfo.h" | ||
| @' | ||
| #pragma once | ||
| #ifdef ARROW_DS_EXPORTING | ||
| #include "arrow/filesystem/filesystem.h" | ||
| #endif | ||
| '@ | Set-Content -LiteralPath $DatasetShim -Encoding ascii | ||
| $DatasetShim = $DatasetShim.Replace('\', '/') | ||
| ``` | ||
|
|
||
| Configure the local-file features commonly used by ModelOpt. bzip2 is disabled | ||
| to avoid Arrow's Makefile-based bzip2 recipe on `clang-cl`. | ||
|
|
||
| ```powershell | ||
| & "$Venv\Scripts\cmake.exe" ` | ||
| -S "$ArrowSource\cpp" -B $ArrowBuild -G Ninja ` | ||
| "-DCMAKE_MAKE_PROGRAM=$Venv\Scripts\ninja.exe" ` | ||
| "-DCMAKE_C_COMPILER=$LlvmRoot\bin\clang-cl.exe" ` | ||
| "-DCMAKE_CXX_COMPILER=$LlvmRoot\bin\clang-cl.exe" ` | ||
| "-DCMAKE_INSTALL_PREFIX=$ArrowHome" ` | ||
| -DCMAKE_BUILD_TYPE=Release ` | ||
| -DARROW_DEPENDENCY_SOURCE=BUNDLED ` | ||
| -DARROW_BUILD_SHARED=ON -DARROW_BUILD_STATIC=OFF ` | ||
| -DARROW_BUILD_TESTS=OFF -DARROW_BUILD_EXAMPLES=OFF ` | ||
| -DARROW_ACERO=ON -DARROW_COMPUTE=ON ` | ||
| -DARROW_CSV=ON -DARROW_DATASET=ON -DARROW_FILESYSTEM=ON ` | ||
| -DARROW_JSON=ON -DARROW_PARQUET=ON ` | ||
| -DARROW_WITH_BROTLI=ON -DARROW_WITH_BZ2=OFF ` | ||
| -DARROW_WITH_LZ4=ON -DARROW_WITH_SNAPPY=ON ` | ||
| -DARROW_WITH_ZLIB=ON -DARROW_WITH_ZSTD=ON ` | ||
| -DARROW_SIMD_LEVEL=NONE -DARROW_RUNTIME_SIMD_LEVEL=NONE ` | ||
| -DPARQUET_REQUIRE_ENCRYPTION=OFF ` | ||
| "-DARROW_CXXFLAGS=/FI$DatasetShim" | ||
| if ($LASTEXITCODE) { throw "Arrow configuration failed" } | ||
| ``` | ||
|
|
||
| Make generated xsimd use LLVM's standard `arm_neon.h` definitions: | ||
|
|
||
| ```powershell | ||
| $XsimdHeader = "$ArrowBuild\_deps\xsimd-src\include\xsimd\types\xsimd_neon_register.hpp" | ||
| $XsimdText = [IO.File]::ReadAllText($XsimdHeader) | ||
| $Old = "#if defined(_WIN32) && XSIMD_WITH_NEON64" | ||
| $New = "#if defined(_WIN32) && XSIMD_WITH_NEON64 && !defined(__clang__)" | ||
| if ($XsimdText.Contains($Old)) { | ||
| [IO.File]::WriteAllText($XsimdHeader, $XsimdText.Replace($Old, $New)) | ||
| } | ||
| ``` | ||
|
|
||
| Build and install Arrow C++: | ||
|
|
||
| ```powershell | ||
| & "$Venv\Scripts\cmake.exe" --build $ArrowBuild --parallel 4 | ||
| if ($LASTEXITCODE) { throw "Arrow build failed" } | ||
| & "$Venv\Scripts\cmake.exe" --install $ArrowBuild | ||
| if ($LASTEXITCODE) { throw "Arrow install failed" } | ||
| ``` | ||
|
|
||
| ## Build and install the PyArrow wheel | ||
|
|
||
| ```powershell | ||
| $env:ARROW_HOME = $ArrowHome | ||
| $env:CMAKE_PREFIX_PATH = $ArrowHome | ||
| $env:CC = "$LlvmRoot\bin\clang-cl.exe" | ||
| $env:CXX = "$LlvmRoot\bin\clang-cl.exe" | ||
| $env:CMAKE_GENERATOR = "Ninja" | ||
| $env:CMAKE_BUILD_PARALLEL_LEVEL = "4" | ||
| $env:PYARROW_BUNDLE_ARROW_CPP = "ON" | ||
|
|
||
| Push-Location "$ArrowSource\python" | ||
| try { | ||
| & $PythonExe -m build --wheel --no-isolation . | ||
| if ($LASTEXITCODE) { throw "PyArrow wheel build failed" } | ||
| } finally { | ||
| Pop-Location | ||
| } | ||
|
|
||
| $Wheel = Get-ChildItem "$ArrowSource\python\dist\pyarrow-*-cp313-cp313-win_arm64.whl" | | ||
| Sort-Object LastWriteTime -Descending | Select-Object -First 1 | ||
| & $PythonExe -m pip install --force-reinstall $Wheel.FullName | ||
| ``` | ||
|
|
||
| If the original N1x setup command stopped because PyArrow had no compatible | ||
| wheel, rerun it now. `pip` will use the installed local PyArrow wheel and resolve | ||
| the remaining published dependencies normally. | ||
|
|
||
| The wheel can be reused in other N1x environments running CPython 3.13 ARM64. | ||
| Rebuild it when the Python ABI, Arrow source version, or native toolchain changes. | ||
|
|
||
| ## Validate the environment | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There are "Verify the installation" points in https://github.com/NVIDIA/Model-Optimizer/blob/main/docs/source/getting_started/windows/_installation_standalone.rst (point 6). We should update it further as required. |
||
|
|
||
| This smoke test checks the WoA environment, not quantization behavior: | ||
|
|
||
| ```powershell | ||
| @' | ||
| import platform | ||
|
|
||
| import cupy as cp | ||
| import onnxruntime as ort | ||
| import onnxruntime_ep_nv_tensorrt_rtx as trt_rtx_ep | ||
| import pyarrow as pa | ||
| import pyarrow.compute | ||
| import pyarrow.dataset | ||
| import pyarrow.parquet | ||
|
|
||
| from modelopt.onnx.quantization import int4 | ||
|
|
||
| assert platform.machine().lower() in {"arm64", "aarch64"} | ||
| assert int4.has_cupy | ||
| assert cp.arange(4).sum().item() == 6 | ||
|
|
||
| ep_name = trt_rtx_ep.get_ep_name() | ||
| devices = [device for device in ort.get_ep_devices() if device.ep_name == ep_name] | ||
| if not devices: | ||
| ort.register_execution_provider_library(ep_name, trt_rtx_ep.get_library_path()) | ||
| devices = [device for device in ort.get_ep_devices() if device.ep_name == ep_name] | ||
| assert devices, f"No compatible {ep_name} device found" | ||
|
|
||
| print("Python:", platform.python_version(), platform.machine()) | ||
| print("CuPy/CUDA:", cp.__version__, cp.cuda.runtime.runtimeGetVersion()) | ||
| print("PyArrow:", pa.__version__, pa.runtime_info()) | ||
| print("TensorRT RTX ABI EP:", ep_name) | ||
| '@ | & $PythonExe - | ||
|
|
||
| & $PythonExe -m pip check | ||
| ``` | ||
|
|
||
| ## Known limitations | ||
|
|
||
| - The built wheel is specific to CPython 3.13 and Windows ARM64. | ||
| - Arrow SIMD is disabled in this configuration. | ||
| - bzip2 and Parquet encryption are not included. | ||
| - Quantization APIs, formats, and examples are unchanged from standard Windows. | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| - If bundled Thrift reports a corrupt patch, use an Arrow checkout with LF line | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should suitably update ModelOpt-FAQ documentation: https://github.com/NVIDIA/Model-Optimizer/blob/main/docs/source/support/2_faqs.rst |
||
| endings as shown above. | ||
| - If CMake or PyArrow finds stale files, use a new Arrow build directory. | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have detailed install instructions at https://github.com/NVIDIA/Model-Optimizer/tree/main/docs/source/getting_started/windows (github io documentation), and the example readmes here (e.g. main examples/windows/README.md) would usually refer to it.
We should update install/build documentation for Modelopt + WoA in similar way.
We also need to update https://nvidia.github.io/Model-Optimizer/getting_started/windows/_installation_for_Windows.html and https://github.com/NVIDIA/Model-Optimizer/blob/main/docs/source/guides/0_support_matrix.rst