However, while most required functionality is available (async, specifying stream, etc), pycuda does not have Graph support yet.
This PR adds some initial support to launch a kernel pipeline using a CUgraph.

I'd love your comments and feedback, most likely I am not freeing memory correctly etc, let me know!
All in all everything seems to be working enough to be useful already :)

Nice bonus is CUDA Graph API offers a function to output dot files, see picture below and the demo inexamples/demo_graph.py.
Note that the demo launches the kernel only once.
Due to overhead, benefits of the Graph API should only really start showing when launching kernels repeatedly.

gfokkema added3 commits

January 15, 2022 13:04

Add support for CUDA Graphs.

57ca7e8

Add example using new pycuda graph api.

0d9515a

Catch exceptions by const ref per the cpp guidelines.

6f9297d

Reduces the amount of compiler warnings significantly.

Copy link

Owner

inducer commentedJan 16, 2022

This looks great, thanks for working on this! To be merged, it'd of course need docs and tests. For lack of GPUs, I don't have usable CI for PyCUDA on Github, but I do have that on a Gitlab instance I run. Mind if I create a user account for you there?

cc@kaushikcfd

Copy link

Author

gfokkema commentedJan 17, 2022

Hi, thanks for the feedback! Yes, this PR was meant primarily to pitch the idea and get some early feedback :)

And access to already usable CI would be great!

Copy link

Owner

inducer commentedJan 17, 2022

Made an account for you, you should have that info in your email. The site is athttps://gitlab.tiker.net/inducer/pycuda.

Copy link

mgaedtke commentedJun 15, 2022

I did some experiments and tests with this and it seems to work without any errors so far. What would be the next steps to bring this to a future release?

Copy link

Owner

inducer commentedJun 15, 2022

It's clear that this should happen, ideally soon. As it happens, there are now two (draft) versions of this, one here:

https://gitlab.tiker.net/kaushikcfd/pycuda/-/merge_requests/2/diffs

and the other one in this PR. (They got started independently.)@mitkotak, could you comment on your plans with respect to upstreaming your work?

Copy link

Contributor

mitkotak commentedJun 15, 2022

Thanks for your interest in this PR. Right now my estimate is to merge this feature intomain in about a month. Most of the wrapper building is done. The purpose of my PR is to broaden the graph creation routes i.e exposing the finer-grained graph building routines in CUDAGraph API alongside the(begin|end)_capture approach. Right now I am handling regression failures, adding more tests and working on docs. Thanks !

Copy link

YanBC commentedOct 20, 2022

Hi there, any updates on the cuda graph feature?

Copy link

Contributor

mitkotak commentedOct 21, 2022•
edited
Loading

Hi there, any updates on the cuda graph feature?

Thank you very much for the interest ! We are still testing the PR to make sure that we don't break any existing functionality but if you are curious to learn more then you can try it out usinggit clone https://gitlab.tiker.net/kaushikcfd/pycuda.git --branch cudagraph and then install it usingpip install -e .. You can get comfortable with the syntax throughexamples/cudagraph_kernel.py andexamples/cudagraph_streamcapture.py, and for the docs you can look forCUDAGraphs indoc/driver.rst. Thanks again for the interest and apologies for the delay !