Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,8 @@ See [Partitioner API](arm-ethos-u-partitioner.md) for more information of the Pa
## Quantization

Since the Ethos-U backend is integer-only, all operators intended be executed on the NPU needs to be quantized. The Ethos-U quantizer supports
[Post Training Quantization (PT2E)](https://docs.pytorch.org/ao/main/tutorials_source/pt2e_quant_ptq.html) and
[Quantization-Aware Training (QAT)](https://docs.pytorch.org/ao/main/tutorials_source/pt2e_quant_qat.html) quantization.
[Post Training Quantization (PT2E)](https://docs.pytorch.org/tutorials/prototype/pt2e_quant_ptq.html) and

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These links don't seem to work either.

[Quantization-Aware Training (QAT)](https://docs.pytorch.org/tutorials/prototype/pt2e_quant_qat.html) quantization.

For more information on quantization, see [Quantization](arm-ethos-u-quantization.md) <!-- @lint-ignore -->

Expand Down
4 changes: 2 additions & 2 deletions docs/source/backends/arm-ethos-u/arm-ethos-u-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,8 @@ See [Partitioner API](arm-ethos-u-partitioner.md) for more information of the Pa
## Quantization

Since the Ethos-U backend is integer-only, all operators intended be executed on the NPU needs to be quantized. The Ethos-U quantizer supports
[Post Training Quantization (PT2E)](https://docs.pytorch.org/ao/main/tutorials_source/pt2e_quant_ptq.html) and
[Quantization-Aware Training (QAT)](https://docs.pytorch.org/ao/main/tutorials_source/pt2e_quant_qat.html) quantization.
[Post Training Quantization (PT2E)](https://docs.pytorch.org/tutorials/prototype/pt2e_quant_ptq.html) and
[Quantization-Aware Training (QAT)](https://docs.pytorch.org/tutorials/prototype/pt2e_quant_qat.html) quantization.

For more information on quantization, see [Quantization](arm-ethos-u-quantization.md) <!-- @lint-ignore -->

Expand Down
2 changes: 1 addition & 1 deletion docs/source/backends/coreml/coreml-quantization.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ quantizer = CoreMLQuantizer(weight_only_8bit_config)

Quantizing activations requires calibrating the model on representative data. Also note that PT2E currently requires passing at least 1 calibration sample before calling `convert_pt2e`, even for data-free weight-only quantization.

See [PyTorch 2 Export Post Training Quantization](https://docs.pytorch.org/ao/main/tutorials_source/pt2e_quant_ptq.html) for more information.
See [PyTorch 2 Export Post Training Quantization](https://docs.pytorch.org/tutorials/prototype/pt2e_quant_ptq.html) for more information.

### LLM quantization with quantize_

Expand Down
2 changes: 1 addition & 1 deletion docs/source/backends/samsung/samsung-quantization.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,5 +56,5 @@ et_program = to_edge_transform_and_lower( # (6)
).to_executorch()
```

See [PyTorch 2 Export Post Training Quantization](https://docs.pytorch.org/ao/main/tutorials_source/pt2e_quant_ptq.html)
See [PyTorch 2 Export Post Training Quantization](https://docs.pytorch.org/tutorials/prototype/pt2e_quant_ptq.html)
for more information.
4 changes: 2 additions & 2 deletions docs/source/backends/xnnpack/xnnpack-quantization.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ et_program = to_edge_transform_and_lower( # (6)
).to_executorch()
```

See [PyTorch 2 Export Post Training Quantization](https://docs.pytorch.org/ao/main/tutorials_source/pt2e_quant_ptq.html) for more information.
See [PyTorch 2 Export Post Training Quantization](https://docs.pytorch.org/tutorials/prototype/pt2e_quant_ptq.html) for more information.

### LLM quantization with quantize_

Expand All @@ -64,7 +64,7 @@ The XNNPACK backend also supports quantizing models with the [torchao](https://g
* Quantize embeddings with `IntxWeightOnlyConfig` (with weight_dtype torch.int2, torch.int4, or torch.int8, using PerGroup or PerAxis granularity)
* Quantize linear layers with 4 bit weight and 8bit dynamic activation, use `Int8DynamicActivationIntxWeightConfig` (with weight_dtype=torch.int4, using PerGroup or PerAxis granularity)

Below is a simple example, but a more detailed tutorial including accuracy evaluation on popular LLM benchmarks can be found in the [torchao documentation](https://docs.pytorch.org/ao/main/serving.html#mobile-deployment-with-executorch).
Below is a simple example, but a more detailed tutorial including accuracy evaluation on popular LLM benchmarks can be found in the [torchao documentation](https://docs.pytorch.org/ao/main/eager_tutorials/serving.html#mobile-deployment-with-executorch).

```python
from torchao.quantization.granularity import PerGroup, PerAxis
Expand Down
6 changes: 3 additions & 3 deletions docs/source/llm/export-llm.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,8 @@ python -m extension.llm.export.export_llm \

## Quantization
Quantization options are defined by [`QuantizationConfig`](https://github.com/pytorch/executorch/blob/main/extension/llm/export/config/llm_config.py#L283). ExecuTorch does quantization in two ways:
1. TorchAO [`quantize_`](https://docs.pytorch.org/ao/stable/generated/torchao.quantization.quantize_.html) API
2. [pt2e quantization](https://docs.pytorch.org/ao/main/tutorials_source/pt2e_quant_ptq.html)
1. TorchAO [`quantize_`](https://docs.pytorch.org/ao/stable/api_reference/api_ref_quantization.html) API
2. [pt2e quantization](https://docs.pytorch.org/tutorials/prototype/pt2e_quant_ptq.html)

### TorchAO (XNNPACK)
TorchAO quantizes at the source code level, swapping out Linear modules for QuantizedLinear modules.
Expand Down Expand Up @@ -127,7 +127,7 @@ python -m extension.llm.export.export_llm \
### pt2e (QNN, CoreML, and Vulkan)
pt2e quantizes at the post-export graph level, swapping nodes and injecting quant/dequant nodes.
**To quantize on non-CPU backends (QNN, CoreML, Vulkan), this is the quantization path to follow.**
Read more about pt2e [here](https://docs.pytorch.org/ao/main/tutorials_source/pt2e_quant_ptq.html), and how ExecuTorch uses pt2e [here](https://github.com/pytorch/executorch/blob/main/docs/source/quantization-overview.md).
Read more about pt2e [here](https://docs.pytorch.org/tutorials/prototype/pt2e_quant_ptq.html), and how ExecuTorch uses pt2e [here](https://github.com/pytorch/executorch/blob/main/docs/source/quantization-overview.md).

*CoreML and Vulkan support for export_llm is currently experimental and limited. To read more about QNN export, please read [Running on Android (Qualcomm)](build-run-llama3-qualcomm-ai-engine-direct-backend.md).*

Expand Down
Loading