- Notifications
You must be signed in to change notification settings - Fork749
Closed
Description
Environment
- Pythonnet version:
- Python version:
- Operating System:
- .NET Runtime:
- GeForce RTX 3080 (computeCapability: 8.6)
- Windows 10 20H2
- Net Framework 4.8 (Compiled with Visual Studio 2019)
- Python 3.7 (64-bit)
- Nvidia CUDA 11.
- Tensforflow-gpu 2.4.1
- Tf-Agents 0.7.1
- pythonnet 3.0.0-preview2021-04-03
Details
I am current working on usingPython.NET to build C# environments for interactionTensorFlow Agents and am receiving a TensorFlow error attempting to load Cuda DLLs.
When I run pure python examples Tensor flow loads the CUDA DLLs without issue:
2021-04-19 03:22:41.062449: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll2021-04-19 03:22:41.062943: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublas64_11.dll2021-04-19 03:22:41.063347: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cublasLt64_11.dll2021-04-19 03:22:41.063709: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cufft64_10.dll2021-04-19 03:22:41.064088: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library curand64_10.dll2021-04-19 03:22:41.064455: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusolver64_10.dll2021-04-19 03:22:41.064832: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cusparse64_11.dll2021-04-19 03:22:41.065202: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudnn64_8.dllHowever, when I run an environment that uses a Python environment that is essentially a wrapper for an environment written in C# using Python.Net is recieve errors the Cuda DLLs were not found:
2021-04-19 03:15:14.884746: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'cublas64_11.dll'; dlerror: cublas64_11.dll not found2021-04-19 03:15:14.885031: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'cublasLt64_11.dll'; dlerror: cublasLt64_11.dll not found2021-04-19 03:15:14.885281: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'cufft64_10.dll'; dlerror: cufft64_10.dll not found2021-04-19 03:15:14.885586: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'curand64_10.dll'; dlerror: curand64_10.dll not found2021-04-19 03:15:14.885851: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'cusolver64_10.dll'; dlerror: cusolver64_10.dll not found2021-04-19 03:15:14.886174: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'cusparse64_11.dll'; dlerror: cusparse64_11.dll not found2021-04-19 03:15:14.886454: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'cudnn64_8.dll'; dlerror: cudnn64_8.dll not foundMinimal code to reproduce the issue:
importtensorflowastffromTicTacToeSharpEnvironmentWrapperimportTicTacToeEnvironmentenv=TicTacToeEnvironment()physical_devices=tf.config.list_physical_devices('GPU')
With TicTacToeSharpEnvironmentWrapper.py
importtensorflowastffromtf_agents.environmentsimportpy_environmentfromtf_agents.specsimportBoundedArraySpecfromtf_agents.trajectories.time_stepimportStepTypefromtf_agents.trajectories.time_stepimportTimeStepimportnumpyasnpassembly_path1=r"C:\DesktopGym\bin\Debug"importsyssys.path.append(assembly_path1)importclrclr.AddReference("GymSharp")fromGymSharpimportTicTacToeSharpEnvironment"""A CSharp environment for Tic-Tac-Toe game."""classTicTacToeEnvironment(py_environment.PyEnvironment):"""A state-settable environment for Tic-Tac-Toe game. """def__init__(self):super(TicTacToeEnvironment,self).__init__()self.sharp_env=TicTacToeSharpEnvironment()
TicTacToeSharpEnvironment is a c# class libary compiled as 64bit dll
publicclassTicTacToeSharpEnvironment{staticTicTacToeSharpEnvironment(){PythonInitiliazer.InitializePython();}}
And PythonInitiliazer is used to initalize Python.Net
publicclassPythonInitiliazer{staticPythonInitiliazer(){InitializePython();}staticboolinitialized;publicstaticvoidInitializePython(){if(!initialized){initPython();initialized=true;}}privatestaticvoidinitPython(){stringpathToVirtualEnv=@"C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\";Environment.SetEnvironmentVariable("PATH",pathToVirtualEnv,EnvironmentVariableTarget.Process);Environment.SetEnvironmentVariable("PYTHONHOME",pathToVirtualEnv,EnvironmentVariableTarget.Process);Environment.SetEnvironmentVariable("PYTHONPATH",$"{pathToVirtualEnv}\\Lib\\site-packages;{pathToVirtualEnv}\\Lib;{pathToVirtualEnv}\\scripts",EnvironmentVariableTarget.Process);Runtime.PythonDLL="C:\\Program Files (x86)\\Microsoft Visual Studio\\Shared\\Python37_64\\python37.dll";PythonEngine.PythonHome=pathToVirtualEnv;PythonEngine.PythonPath=Environment.GetEnvironmentVariable("PYTHONPATH",EnvironmentVariableTarget.Process);PythonEngine.Initialize();}}
The full code works. The Python Wrapper of the C# environment passes theTensorflow Agents unit tests for theTic Tac Toe environment. The C# environment can be wrapped as Python or a Tensor flow environment and the various agents can train against the environment.
I don't think this is a compatibility issue using the x64 .Net DLL because I am using 64 bit python, but I a can't be certain.
Metadata
Metadata
Assignees
Labels
No labels