- Notifications
You must be signed in to change notification settings - Fork352
Adding a way to clear GPU memory#722
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
montanalow left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This looks good to me. 2 nits.
- We should start including all this wonderful PR example/documentation as crate docs over API functions. i.e. /// blah blah blah
- I think we should use a f32 for the memory limit argument since a fraction between [0:1]
jsaied99 commentedJun 10, 2023
|
montanalow commentedJun 10, 2023
I appreciate you beginning the great documentation crusade with this function. For the argument type, all of our other APIs follow more scikit-esque floating point numbers. Torch is the most common backend, maybe the only one that matters these days, but I think floating point is still the better API for relative percentages, and I think consistency is generally important. |
jsaied99 commentedJun 10, 2023
Got it. I agree with consistency. No worries. I will change it to |
Adding a way of clearing GPU memory by doing
SELECT pgml.clear_gpu_cache();. One can also pass amemory_usageparameter which will add a filter to whether or not to clear GPU memory if utilization is greater than the passed value. For instance, if you want to clear only if GPU memory utilization is > 50%, you can do the following:select pgml.clear_gpu_cache(memory_usage => 50);I also wanted to implement the general clear_cache, but it requires a bit more, and I think clearing GPU should be okay for the first pass.