@vmoens This is one part where I'm trying to call a method on the LLM object to init a process group with the vllm workers, the second part is below on L293

In this case
SyncDataCollector is remote and has an attribute.policy

.policy is the ModuleDict object returned byfrom_vllm, and the actual llm instance is in the generate key (the LLM instance is local to the SyncDataCollector)

How can I have a handle to the LLM instance within the SyncDataCollector to call remote methods on it without the hackycall_policy_method implementation below?

@vmoens I keep finding that I want to get info off vllm directly :/, what would you do here?

Should this vLLMRemoteWeightUpdaterBase be aware of all the vllm engines and tp size of each owned by its parent RayCollector in its __init__, I already needed to pass separate master_address and master_port I guess

I think you're right, there's no way around it

We need for the main worker to know that stuff about the remote ones

importfind_spec might be preferred:https://docs.python.org/3/library/importlib.html#importlib.util.find_spec as it doesn't do the actual import until needed.

I agree, that's the proper way to do it
All third party imports should be done locally even if not optional (otherwise that slows down multiproc / distributed start time and can cause bugs that are hard to debug)