Vitis Unified Backend

[Tanawin1701d]

Description

VitisUnified backend

Motivation

• The current Vitis backend does not support a complete flow from HLS4ML to bitstream generation and driver deployment (zcu102 for my case)

Summarized features

• Automatic creation of AXI master reader/writer interfaces for HLS4ML kernels.
• Based on the v++ compiler and packaging flow.
• Configuration aligned with the Vitis Unified IDE project structure.
• Seamless integration with Xilinx hardware platforms:
  • the platform is the Xilinx package that contains the hardware structure such as axi interconnect, PS configuration, interrupt(except HLS4ML kernel)
  • Platforms encapsulate hardware structures such as AXI interconnects, PS configuration, and interrupts (excluding the HLS4ML kernel).
  • Xilinx provides platform for some boards integrated in Vitis /tools/Xilinx/Vitis/2023.2/base_platforms
  • Developers can also create custom platforms by following the official tutorial: https://github.com/Xilinx/Vitis-Tutorials/tree/2025.1/Vitis_Platform_Creation/Design_Tutorials/01-Edge-KV260
• Automatic PYNQ driver generation for streamlined deployment.

Type of change

For a new feature or function, please create an issue first to discuss it
with us before submitting a pull request.

Note: Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)

Tests

• we test with the tiny keras unet model @ test/pytest/test_backend/vitis_unified.py with 4 main aspect

bridge test

• Compare VitisUnified with Vitis
• we check predict file with 100% match

cosimulation

• we use two VitisUnified:
  • first one is used to generate bridge simulation
  • second one is used to generate start cosimulation and get the simulation result from cosimulator
• Compare with 1e-4 acceptable torelant (it comes from dat file rounding a bit)

fifo test optimization

• Procedure is similar to Co-simulation and inspect that there is fifo_depth.json exist

hardware test

• Stress test with 10,000 queries but have only 128 (input) + 128 (output) buffer size to make sure there is no deadlock from autogenerated xilinx platform axi-connection
• the tested was in function test_gen_unified in
  • the test was conducted in zcu102 with pynq framework

test reproduce

• Run pytest on test/pytest/test_backend/vitis_unified.py file
• for hardware test (test_gen_unified), you should specify XPFM_PATH(path to xpfm file) to the correct place.
• if LOG_STD == True, HLS4ML will give the HLS+linker compiling message @ console.
• if not, HLS4ML will give the messages @ <output_project_dir>/<prefix>_err.log or <output_project_dir>/<prefix>_out.log

Test Configuration:

Checklist

• I have read the guidelines for contributing.
• I have commented my code, particularly in hard-to-understand areas.
• I have made corresponding changes to the documentation.
• My changes generate no new warnings. (see the section below)
• I have installed and run pre-commit on the files I edited or added.
• I have added tests that prove my fix is effective or that my feature works.

implementation detail

• This flow of this backend to build the ready to ship file, we should do three flow things
  • file generation(HLS4ML generated file) prepare file for system Generation and pynq driver
  • synthesis Kernel (Synthesis Kernel (v++)) do c-synthesis for HLS4ML model
  • linker (Linker+vivado+Bitfile+hwh)

File structure

template structure

• the tree below is the template file allocate at hls4ml/templates/vitis_unified

├── build_lib_multigraph.sh
├── build_lib.sh
├── driver
│   └── pynq
│       └── pynq_driver.py.hls4ml (template for pynq driver)
├── hls_kernel_config.cfg         (config for HLS4ML model Synthesis)
├── myproject_bridge.cpp          (wrapper for C++ simulation using python .predict())
├── myproject_dm.cpp              (wrapper for HLS4ML model convert axi to axi stream)
├── myproject_dm.h   
├── myproject_test.cpp            (for cosimulation and fifo-optimization)
└── workspace
    ├── projectName
    │   └── vitis-comp.json       (project meta-data used for opening using vitis unified IDE)
    └── sysProj
        ├── buildAcc.sh           (script for linking the kernel with platform)
        └── buildConfig.cfg       (config file for linking progress)

output file structure

├ export                        (ready to ship file placed here!)
│   ├ pynq_driver.py
│   ├ system.bit
│   └ system.hwh
├ firmware
│   ├ <project_name>_dm.cpp     (wrapper for HLS4ML model convert axi to axi stream)
│   ├ <project_name>_dm.h       (the syntesizer not use this but required by cosim and bridge sim)
│   ├ <other files>             (other HLS src file generated from Vitis and vivado backend)
├ unifiedWorkspace              (folder for kernel synthesis and linking progress)
│   ├ linker                    (folder for platform linking project)
│   │   ├ buildAcc.sh           (build script for platform link)
│   │   ├ buildConfig.cfg       (config script for platform link)
│   │   └ <other files>         (file that v++ generated during link the platform)
│   └ <project_name>            (folder for HLS project from HLS4ML model)
│       ├ unifiedPrj            (folder for Vitis HLS internal file)
│       └ vitis-comp.json       (project meta-data used for opening using vitis unified IDE)
├ build_lib.sh                  (build script for bridge simulation)
├ hls_kernel_config_cosim.cfg   (config file for cosim and fifo depth optimization)
├ hls_kernel_config_csim.cfg    (config file for csim )
├ myproject_bridge.cpp          (wrapper for C++ simulation using python .predict())
└ myproject_test.cpp          (for cosimulation and fifo-optimization)

configuration

board='zcu102',
        part=None,
        clock_period=5,
        clock_uncertainty='12.5%',
        io_type='io_stream',
        driver='python',
        input_type='float',
        output_type='float',
        in_stream_buf_size=128,
        out_stream_buf_size=128,
        xpfmPath='/opt/Xilinx/Vitis/2023.2/base_platforms/' 'xilinx_zcu102_base_202320_1/xilinx_zcu102_base_202320_1.xpfm',
        **_,

• input_type and output_type are support only float and double. And it must be match
• {in/out}_stream_buf_size unit is in amount elements of the nnet::array
  xpfmPath

note to developer

• In case, you want to debug the generated HLS project using Vitis unified IDE, you can select the workspace folder at the program at unifiedWorkspace. The IDE will automatically detect your project
• For bridge simulation, if the configuration input_type/output_type was set to type x (double or float), you cannot predict with numpy array with different input/output type
• the depth argument @ axi_master write @ <project_name>_dm.cpp must be match of the array size generated the output array@ ````myproject_test.cpp``` for cosim and csim.
  • if the array allocation is larger than depth, the result will not correct
  • if the array allocation is lower than depth, the result is correct, but the system will throw segment falut error
  • the depth size will not impact the resource usage in hls generation
• The linked Vivado project are at <project_folder>/unifiedWorkspace/linker/_x/link/vivado/vpl/prj
• This backend will reject multigraph feature

note to tutorial

• we provide the tutorial at this repository
  https://github.com/Tanawin1701d/vitisUnifiedTutorial

generated warning

• warning in HLS4ML is only about the unet model that we use in pytest, I think it is not warning in the new backend

WARNING:absl:Skipping variable loading for optimizer 'Adam', because it has 17 variables whereas the saved optimizer has 1 variables. 
WARNING: Config parameter "algorithm" overwrites an existing attribute in layer "up_sampling2d" (Resize)

• for kernel synthesis with Vitis, I think it is general warning such as unused parameter, deprecated pragma, dataflow conflict

[JanFSchulte]

Thanks @Tanawin1701d The main outstanding issues are the documentation and the CI tests. The documentation could be added as a new section in this file https://github.com/fastmachinelearning/hls4ml/blob/main/docs/backend/accelerator.rst and can follow along the lines of the other accelerator backends.

For the CI, having something that runs synthesis as it is done now requires some modifications away from the hard-coded Vitisi HLS locations that are currently used and that result in those tests failing when run as part of the hls4ml CI. Synthesis as part of pytests was developed by @marco66colombo, so maybe he can help with suggesting the necessary changes.

[gflengas]

I have prepared some initial fixes that work localy but also plan to run them through the gitlab pipeline at CERN. I didnt have enough time to do the last yet.

They fixes remove the hardcoded synthesis requirements and add a v++ command that required to run the Vitis Unified workflow. But i kept the full bitstream generation out of the CI for now.

I will try to have them run today or tomorrow and prepare a PR. But let me know if you have any thoughts about the "full bitstream generation out of the CI" part.

@Tanawin1701d could you take a look at the documentation ? I think you have a lot of material for it and you are the best to write the final version.

[JanFSchulte]

@gflengas Thanks, that sounds very good! I agree that we probably don't need to exercise the full bitstream generation in the CI, AFAIK we don't do it for the other accelerator backends.

[gflengas]

PR with the tests fix is up. CERN GitLab CI could not start the jobs because of runner-side registry and disk-space issues, rather than a test failure. I therefore reproduced the CI setup locally using the exact CERN CI image.

The Vitis Unified suite passed with 10 passed and 2 intentionally skipped bitstream tests, covering both AXI-stream and AXI-master paths. Final step should be triggering the pipeline from here once merged

[Tanawin1701d]

Thanks @Tanawin1701d The main outstanding issues are the documentation and the CI tests. The documentation could be added as a new section in this file https://github.com/fastmachinelearning/hls4ml/blob/main/docs/backend/accelerator.rst and can follow along the lines of the other accelerator backends.

For the CI, having something that runs synthesis as it is done now requires some modifications away from the hard-coded Vitisi HLS locations that are currently used and that result in those tests failing when run as part of the hls4ml CI. Synthesis as part of pytests was developed by @marco66colombo, so maybe he can help with suggesting the necessary changes.

Thanks, I will look into the document.

[JanFSchulte]

Thanks! @Tanawin1701d! if you could also merge Tanawin1701d#5 quickly that would be great so we can exercise the fixed tests here.

[Tanawin1701d]

@gflengas Thank you very much for the CI fix. I am going to look into the document.

[JanFSchulte]

Looks like the synthesis tests for VitisUnified are failing with the current configuration:

[gflengas]

New PR that fixes this Tanawin1701d#6 .

The way I was calling v++ and vitis-run was incomplete, and the tools could not access the correct project folder structure from inside the Apptainer container.

Also, Tanawin added another test that generates a bitstream, so I will apply the same conditional skip that I previously added to the other bitstream-generation tests.

[Tanawin1701d]

@JanFSchulte I’ve added the documentation for the VitisUnified backend. What do you think?

Currently, the links for the tutorial and custom platform setup point to my repository. I’m not sure whether they should be moved somewhere more official.

Please let me know if there are any fixes we still need to make at this point.

[JanFSchulte]

Hi @Tanawin1701d Docs look good to me. Tutorials can be migrated to https://github.com/fastmachinelearning/hls4ml-tutorial, I think it would be better to have them maintained under the official FastML umbrella.

[Tanawin1701d]

Hi @JanFSchulte, I’ve updated the path in the documentation to point to the tutorial repository, and I’ve also created a pull request for the tutorial .

I would appreciate it if you could take a look at PR #130 when you have a chance. Thanks