Vitis Unified Backend

MANIFEST.in

@@ -7,3 +7,4 @@ recursive-include hls4ml *.py
 recursive-include hls4ml/contrib *
 global-exclude .git .gitmodules .gitlab-ci.yml *.pyc
 include hls4ml/backends/vivado_accelerator/supported_boards.json
+include hls4ml/backends/vitis_unified/supported_boards.json

docs/backend/vitis_unified.rst

@@ -0,0 +1,167 @@
+============
+VitisUnified
+============
+
+The **VitisUnified** backend provides an end-to-end workflow for AMD SoC boards, from an ML model to a design that is ready to deploy on `PYNQ <http://pynq.io/>`_. It is inherited from the :doc:`Vitis <vitis>` backend. We use the new Vitis Unified software, which can automatically link the HLS kernel to the system hardware. The current version supports only SoC boards with a PYNQ Python driver.
+
+Currently ``hls4ml`` officially supports the following boards and tool versions:
+
+* `zcu102 <https://www.xilinx.com/products/boards-and-kits/ek-u1-zcu102-g.html>`_ (Vitis and Vivado 2023.2)
+* `kv260 <https://www.xilinx.com/products/som/kria/kv260-vision-starter-kit.html>`_ (Vitis and Vivado 2023.2 and 2025.2)
+
+If you use another board, another Vivado version, or want to optimize the system design for your own workload, you can build your own platform. The steps are covered in the platform setup tutorial in the accelerator backend section of the `hls4ml-tutorial <https://github.com/fastmachinelearning/hls4ml-tutorial>`_ repository.
+
+
+System Flow
+===========
+
+The figure below shows the flow of the backend, from the generated HLS files to the files that are ready to ship to the board.
+
+.. image:: ../img/vitis_unified_flow.png
+  :width: 450px
+  :align: center
+  :alt: Vitis Unified backend system flow
+
+
+.. _vitis_unified_axi_modes:
+
+AXI interface modes
+===================
+
+The backend supports two ways to move data between the PS and the kernel, selected with ``axi_mode``.
+In both modes the CPU controls the kernel through AXI-Lite and receives an interrupt when the kernel is done.
+
+``axi_master`` (default)
+    The kernel reads its input and writes its output in DDR by itself through an AXI master port.
+    The driver allocates one DDR buffer per model input and output, so models with several inputs or outputs are supported.
+
+    .. image:: ../img/vitis_unified_axi_master.png
+      :width: 450px
+      :align: center
+      :alt: axi_master mode
+
+``axi_stream``
+    The kernel has one AXI-Stream input and one AXI-Stream output. An AXI DMA in the platform moves the data between DDR and the kernel.
+    Only models with one input and one output are supported.
+
+    .. image:: ../img/vitis_unified_axi_stream.png
+      :width: 450px
+      :align: center
+      :alt: axi_stream mode
+
+Configuration options
+=====================
+
+The options below are passed as keyword arguments to the converter (for example ``convert_from_keras_model``).
+They are stored under ``VitisUnifiedConfig`` in the model configuration.
+
+.. list-table::
+   :header-rows: 1
+   :widths: 30 18 52
+
+   * - Option
+     - Default
+     - Description
+   * - ``board``
+     - ``zcu102``
+     - | Target board.
+       | It selects the FPGA part, the platform, and the Python driver template.
+       | The current version only supports the boards in ``supported_boards.json`` (``zcu102`` and ``kv260``).
+   * - ``part``
+     - from board
+     - | FPGA part name.
+       | If not given, it is taken from the board entry in ``supported_boards.json``.
+       | If the board is not in the list and no part is given, the zcu102 part is used.
+   * - ``clock_period``
+     - ``5``
+     - | Kernel clock period in ns.
+       | The same clock is used when the kernel is linked to the platform.
+   * - ``clock_uncertainty``
+     - ``12.5%``
+     - | Clock uncertainty passed to Vitis HLS.
+   * - ``io_type``
+     - ``io_stream``
+     - | hls4ml I/O type of the model.
+       | The current version only supports ``io_stream``.
+   * - ``axi_mode``
+     - ``axi_master``
+     - | Interface between the PS and the kernel: ``axi_master`` or ``axi_stream``.
+       | See :ref:`AXI interface modes <vitis_unified_axi_modes>`.
+   * - ``driver``
+     - ``python``
+     - | Type of driver generated for the board.
+       | The current version only supports ``python`` (PYNQ).
+   * - ``input_type``
+     - ``float``
+     - | Data type of the model input on the AXI interface.
+       | The current version only supports ``float`` and ``double``.
+       | The PYNQ driver uses the matching NumPy type.
+   * - ``output_type``
+     - ``float``
+     - | Data type of the model output on the AXI interface.
+       | The current version only supports ``float`` and ``double``.
+       | It must be the same as ``input_type``.
+   * - ``in_stream_buf_size``
+     - ``128``
+     - | Depth of the FIFO between the wrapper input (AXI master read or AXI-Stream) and the HLS model.
+       | Used in both AXI modes.
+   * - ``out_stream_buf_size``
+     - ``128``
+     - | Depth of the FIFO between the HLS model and the wrapper output (AXI master write or AXI-Stream).
+       | Used in both AXI modes.
+
+Example:
+
+.. code-block:: Python
+
+    hls_model = hls4ml.converters.convert_from_keras_model(model,
+                                                           hls_config=config,
+                                                           output_dir='hls4ml_prj_unified',
+                                                           backend='VitisUnified',
+                                                           board='kv260',
+                                                           axi_mode='axi_stream',
+                                                           clock_period=10,
+                                                           in_stream_buf_size=256,
+                                                           out_stream_buf_size=256)
+
+
+Output directory layout
+-----------------------
+
+.. code-block:: text
+
+    <output_dir>/
+    ├── firmware/                          HLS sources: model, AXI wrapper, weights
+    ├── tb_data/                           testbench input and reference output
+    ├── myproject_test.cpp                 C testbench
+    ├── <project_name>_bridge.cpp          bridge used by hls_model.predict()
+    ├── build_lib.sh                       builds the shared library for predict()
+    ├── hls4ml_config.yml
+    ├── hls_kernel_config_csim.cfg         Vitis HLS config for csynth, package, and csim
+    ├── hls_kernel_config_cosim.cfg        Vitis HLS config for cosim
+    ├── hls_kernel_config.cfg              copy of the config used by the last step
+    ├── fifo_depths.json                   with FIFO depth optimization only
+    ├── <step>_stdout.log, <step>_stderr.log   with log_to_stdout=False only
+    ├── vitis_workspace/
+    │   ├── <project_name>/
+    │   │   ├── vitis-comp.json            Vitis Unified component
+    │   │   └── vitis_unified_project/     hls/, logs/, reports/, <project_name>_axi_*.xo
+    │   ├── system_link/
+    │   │   ├── link_system.cfg, link_system.sh
+    │   │   ├── <project_name>.xclbin      link output (bitfile=True)
+    │   │   └── _x/                        Vivado project of the system link
+    │   └── <board>/
+    │       ├── tcl_scripts/               create_xsa.tcl, platform tcl, output/<board>_*.xsa
+    │       └── python_drivers/            driver templates
+    ├── export/
+    │   ├── system.bit                     bitstream (bitfile=True)
+    │   ├── system.hwh                     hardware handoff (bitfile=True)
+    │   └── axi_master_driver.py or axi_stream_driver.py
+    └── final_reports/                     timing, utilization, power, link summary, hls_compile.rpt
+
+Tutorial
+========
+
+A step-by-step tutorial with notebooks is available in the accelerator backend section of the
+`hls4ml-tutorial <https://github.com/fastmachinelearning/hls4ml-tutorial>`_ repository.
+It covers prediction, C simulation, co-simulation, FIFO depth optimization, bitstream generation, and how to build your own platform.

docs/index.rst

@@ -35,6 +35,7 @@
 
     backend/vitis
     backend/accelerator
+    backend/vitis_unified
     backend/oneapi
     backend/catapult
     backend/quartus

hls4ml/backends/__init__.py

@@ -15,11 +15,14 @@
 from hls4ml.backends.coyote.coyote_backend import CoyoteBackend
 from hls4ml.backends.xls.xls_backend import XLSBackend
 
+from hls4ml.backends.vitis_unified.vitis_unified_backend import VitisUnifiedBackend  # isort: skip
+
 
 def _register_builtin_backends():
     register_backend('Vivado', VivadoBackend)
     register_backend('VivadoAccelerator', VivadoAcceleratorBackend)
     register_backend('Vitis', VitisBackend)
+    register_backend('VitisUnified', VitisUnifiedBackend)
     register_backend('Quartus', QuartusBackend)
     register_backend('Catapult', CatapultBackend)
     register_backend('SymbolicExpression', SymbolicExpressionBackend)

hls4ml/backends/vitis_unified/passes/fifo_depth_optimization.py

@@ -0,0 +1,114 @@
+# we inherit it from vitis
+import re
+import zipfile
+
+from hls4ml.backends.vitis.passes.fifo_depth_optimization import (
+    generate_depths_file,
+    initialize_large_fifos,
+    set_optimized_fifo_depths,
+)
+from hls4ml.model.optimizer.optimizer import ConfigurableOptimizerPass, ModelOptimizerPass
+
+
+def get_vitis_optimized_fifo_depths(model, cus_hls_prj_path=None):
+    """Parse the files generated by the co-simulation to retrieve the optimized depths for the FIFOs.
+    Attention, only the FIFOs between the layers are profiled!
+
+    Args:
+        model (ModelGraph): The model to which FIFO depth optimization is applied.
+
+    Returns:
+        Dict[str, int]: A dictionary that contains the FIFO names as keys and the optimized depths as values.
+    """
+    # channel.zip is generated after the co-simulation and contains the chan_status*.csv files
+    # in the chan_status*.csv files the max depth achieved during co-simulation can be found at the last (4th) line
+
+    if cus_hls_prj_path is None:
+        cus_hls_prj_path = model.config.get_output_dir() + '/' + model.config.get_project_name() + '/_prj/solution1'
+
+    path_to_zip_file = cus_hls_prj_path + '/.autopilot/db/channel_depth_info/'
+
+    with zipfile.ZipFile(f'{path_to_zip_file}channel.zip', 'r') as zip_ref:
+        zip_ref.extractall(path_to_zip_file)
+
+    # the channel_info.csv file contains the mapping of each fifo name (i.e layer4_out_U) to the respective
+    # chan_status*.csv file
+    names_file_path = cus_hls_prj_path + '/.autopilot/db/channel_info.csv'
+
+    csv_fifo_depth_files = {}
+    with open(names_file_path) as names_file:
+        for line in names_file:
+            slitted_line = line.split(',')
+            if len(slitted_line[0]) == 0:
+                continue
+            layer_name = slitted_line[1]
+            csv_file_name = slitted_line[3][:-1]
+            csv_fifo_depth_files[layer_name] = csv_file_name
+
+    optmized_fifo_depths = {}
+    for layer_name, file_name in csv_fifo_depth_files.items():
+        with open(path_to_zip_file + file_name) as chan_status_file:
+            lines = chan_status_file.readlines()
+            cleaned_layer_name = re.sub(r'(_i)?_U$', '', layer_name)
+            optmized_fifo_depths[cleaned_layer_name] = int(
+                lines[-1]
+            )  # remove "_i_U" (axis) or "_U" (axim) from the layer name
+            # string and keep the last line of the file that contains the max depth
+
+    return optmized_fifo_depths
+
+
+def execute_cosim_to_profile_fifos(model):
+    model.write()
+    model.build(
+        reset=False,
+        csim=False,
+        synth=True,
+        cosim=True,
+        vsynth=False,
+        fifo_opt=True,
+        bitfile=False,
+        log_to_stdout=False,
+    )
+
+
+class FifoDepthOptimization(ConfigurableOptimizerPass, ModelOptimizerPass):
+    def __init__(self):
+        self.profiling_fifo_depth = 100_000
+
+    def transform(self, model):
+        """Perform FIFO depth optimization between the FIFOs of all layers to reduce resource utilization as the
+        initial FIFOs set by hls4ml might be larger than required. At the end of the optimization the FIFOs will
+        have the largest depths achieved during co-simulation without causing any deadlocks between the layers
+        (producer-consumer), thus no additional delays between the layers. In some cases, this optimization
+        might lead to bigger FIFOs than initially set by the hls4ml tool in order to prevent deadlocks.
+
+        Args:
+            model (ModelGraph): The model to which FIFO depth optimization is applied.
+
+        Raises:
+            ValueError: If the FIFO depth for profiling provided by the user is not a non-negative integer.
+            RuntimeError: If the IO type is not set to "io_stream".
+
+        Returns:
+            bool: The execution state of the Optimizer Pass
+        """
+
+        if not isinstance(self.profiling_fifo_depth, int) or self.profiling_fifo_depth <= 0:
+            raise ValueError('The FIFO depth for profiling (profiling_fifo_depth variable) must be a non-negative integer.')
+
+        # check axi-stream or io-stream
+        if not (model.config.get_config_value('IOType') == 'io_stream'):
+            raise RuntimeError('To use this optimization you have to set `IOType` field to `io_stream` in the HLS config.')
+
+        hlsPrjPath = model.config.backend.writer.get_vitis_hls_exec_dir(model)
+
+        initial_fifo_depths = initialize_large_fifos(model, self.profiling_fifo_depth)
+        execute_cosim_to_profile_fifos(model)
+        optimized_fifo_depths = get_vitis_optimized_fifo_depths(model, cus_hls_prj_path=hlsPrjPath + '/hls')
+        generate_depths_file(model, initial_fifo_depths, optimized_fifo_depths)
+        set_optimized_fifo_depths(model, optimized_fifo_depths)
+
+        print('FIFO optimization completed')
+
+        return False

hls4ml/backends/vitis_unified/supported_boards.json

@@ -0,0 +1,31 @@
+{
+  "zcu102": {
+    "part": "xczu9eg-ffvb1156-2-e",
+    "axi_master": {
+      "platform_file": "base_platforms/xilinx_zcu102_base_202320_1/xilinx_zcu102_base_202320_1.xpfm",
+      "python_driver": "axi_master_driver.py",
+      "c_drivers": ""
+    },
+    "axi_stream": {
+      "platform_generator_tcl": "zcu102/tcl_scripts/create_xsa.tcl",
+      "platform_output": "zcu102/tcl_scripts/output/zcu102_axi_stream_platform.xsa",
+      "python_driver": "axi_stream_driver.py",
+      "c_drivers": ""
+    }
+  },
+  "kv260": {
+    "part": "xck26-sfvc784-2LV-c",
+    "axi_master": {
+      "platform_generator_tcl": "kv260/tcl_scripts/create_xsa.tcl",
+      "platform_output": "kv260/tcl_scripts/output/kv260_axi_all_platform.xsa",
+      "python_driver": "axi_master_driver.py",
+      "c_drivers": ""
+    },
+    "axi_stream": {
+      "platform_generator_tcl": "kv260/tcl_scripts/create_xsa.tcl",
+      "platform_output": "kv260/tcl_scripts/output/kv260_axi_all_platform.xsa",
+      "python_driver": "axi_stream_driver.py",
+      "c_drivers": ""
+    }
+  }
+}