Timeline for GPU support

[varun-saran-ai]

What's the timeline for iceoryx2 to support IPC with GPU memory? I see on the roadmap there are plans for it, but roughly when can this be expected? And is there any guidance on what platforms will be supported / what underlying API will be used? ex: cudaIpcGetMemHandle is not supported on Tegra platforms.

[elfenpiff]

@varun-saran-ai

What's the timeline for iceoryx2 to support IPC with GPU memory?

The implementation time would be around 2 months for CUDA based GPU memory and 4 months for NVSci-based GPU memory, since it is a much more complex API. Currently, this feature requires funding or a sponsor, and starts as soon as this is secured.

And is there any guidance on what platforms will be supported / what underlying API will be used? ex: cudaIpcGetMemHandle is not supported on Tegra platforms.

We are aware that not all CUDA calls are available on embedded mission-critical platforms, in those cases we would use NvSciIpc https://developer.nvidia.com/docs/drive/drive-os/6.0.9/public/drive-os-linux-sdk/common/topics/nvsci_nvsciipc/Inter-ProcessCommunication1.html

[ElliotHYLee]

I wanted to share some thoughts from an end-user perspective, though I’ll leave the business feasibility to your expertise.

Regarding the GPU memory sharing feature: have you considered a crowdfunding, donation, or paywalled model? Seeing PyTorch results shared directly would be an incredible addition. While I’m a huge supporter of open-source, I think a paid tier or crowdfunding for specific high-value features like this would make a lot of sense to ensure the dev team is supported. I’d love to see this happen!

Currently, dora-rs seems to have the feature - at least in my usecase, not the technical detail-wise ;). Anyway, if iox2 can have a similar feature, it would be much lighter than Dora-RS, as Dora-RS has more packages than the shared memory itself. iox2 can have much more flexibility for production-grade applications.

[elfenpiff]

@ElliotHYLee

Regarding the GPU memory sharing feature: have you considered a crowdfunding, donation, or paywalled model?

What we usually consider is that a company that is interested in the feature pays for the implementation for the community. We already have a proof of concept for this as a commercial iceoryx2 extensions available through https://ekxide.io . To make it available as a community feature, we need to invest some more time in the implementation, testing, and documentation, until we can contribute it to iceoryx2.

Seeing PyTorch results shared directly would be an incredible addition.

I agree, and hopefully we can realize CUDA GPU memory this year.