Why are these changes needed?

This is a follow up for#51734

The reason#51305 caused illegal GPU memory access in vLLM is that it assumesCUDA_VISIBLE_DEVICES for an actor is limited to a single GPU ID, and the worker can always accesses the GPU withcuda:0. While this is a reasonable assumption for Ray backed applications in general, it is not the case for vLLM. Historically vLLM altersCUDA_VISIBLE_DEVICES and sets it to the list of local GPUs (on the same node) for maintaining distributed groups for both the Ray and MP backends. While this could be cleaned up, it will be a longer-term process.

This PR refactors and cleans up the utils.get_devices() API.

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e.,git commit -s) in this PR.
• I've runscripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed forhttps://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it indoc/source/tune/api/ under the
    corresponding.rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures athttps://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(