Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Build single Python.Runtime.dll for all platforms#1365

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes fromall commits
Commits
Show all changes
21 commits
Select commitHold shift + click to select a range
909ed1f
dropped net40 target from modern projects
lostmsuNov 30, 2020
47e926e
use .NET Standard 2.0 platform detection features
lostmsuDec 2, 2020
21683b3
drop NativeCodePage alltogether
lostmsuDec 2, 2020
972c41d
WIP: use C# 9 function pointers for PInvoke
lostmsuDec 4, 2020
51e5184
allow setting PythonDLL
lostmsuDec 10, 2020
2498d47
always explicitly specify the way strings are marshaled
lostmsuJan 22, 2021
70fc803
CI: figure out DLL name from environment
lostmsuJan 22, 2021
28a5dab
use Roslyn preview in CI
lostmsuJan 22, 2021
c75229a
fixed Linux and Mac DLL loaders breaking dll path
lostmsuJan 22, 2021
a0a1dc1
correctly detect DLL on *nix when running from Python
lostmsuJan 22, 2021
1b88783
Windows library loader: add support for hModule == 0
lostmsuJan 22, 2021
2c1aaef
fix dll loading in tests
lostmsuJan 22, 2021
39e41d0
mentiond PythonDLL in changelog
lostmsuJan 22, 2021
17040fe
set PYDLL in AppVeyor
lostmsuJan 22, 2021
b7410b6
revert automatically added 'm' suffix for *nix default dll name
lostmsuJan 22, 2021
275cae9
specify full DLL name instead of PYVER in GH Actions
lostmsuJan 22, 2021
b4cb37e
use Microsoft.Net.Compilers.Toolset instead of Microsoft.Net.Compilers
lostmsuJan 23, 2021
f68e581
in CI MacOS python DLL has 'm' suffix
lostmsuJan 23, 2021
cda604a
only set PYTHONNET_PYDLL for test runs from .NET
lostmsuJan 23, 2021
3982892
workaround for pytest/clr module not preloading python C API library
lostmsuJan 23, 2021
a6cbe20
addressed a few code comments
lostmsuJan 26, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 39 additions & 3 deletions.github/workflows/main.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -12,9 +12,33 @@ jobs:
fail-fast: false
matrix:
os: [windows, ubuntu, macos]
python: [3.6,3.7,3.8,3.9]
pyver_minor: [6, 7, 8, 9]
platform: [x64]
shutdown_mode: [Normal, Soft]
include:
- os: ubuntu
pyver_minor: 6
dll_suffix: m
- os: ubuntu
pyver_minor: 7
dll_suffix: m

- os: macos
dll_prefix: lib
dll_pyver_major: '3.'
dll_suffix: m
- os: ubuntu
dll_prefix: lib
dll_pyver_major: '3.'
- os: windows
dll_pyver_major: '3'

- os: ubuntu
dll_ext: .so
- os: windows
dll_ext: .dll
- os: macos
dll_ext: .dylib

env:
PYTHONNET_SHUTDOWN_MODE: ${{ matrix.SHUTDOWN_MODE }}
Expand All@@ -32,10 +56,10 @@ jobs:
- name: Setup .NET
uses: actions/setup-dotnet@v1

- name: Set up Python ${{ matrix.python }}
- name: Set up Python3.${{ matrix.pyver_minor }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python }}
python-version:3.${{ matrix.pyver_minor }}
architecture: ${{ matrix.platform }}

- name: Install dependencies
Expand All@@ -47,16 +71,28 @@ jobs:
python setup.py configure
pip install -v .

# TODO this should be gone once clr module sets PythonDLL or preloads it
- name: Python Tests
run: pytest
if: ${{ matrix.os != 'macos' }}
env:
PYTHONNET_PYDLL: ${{ matrix.DLL_PREFIX }}python${{matrix.DLL_PYVER_MAJOR}}${{matrix.PYVER_MINOR}}${{matrix.DLL_SUFFIX}}${{matrix.DLL_EXT}}

- name: Python Tests
run: pytest
if: ${{ matrix.os == 'macos' }}

- name: Embedding tests
run: dotnet test --runtime any-${{ matrix.platform }} src/embed_tests/
if: ${{ matrix.os != 'macos' }} # Not working right now, doesn't find libpython
env:
PYTHONNET_PYDLL: ${{ matrix.DLL_PREFIX }}python${{matrix.DLL_PYVER_MAJOR}}${{matrix.PYVER_MINOR}}${{matrix.DLL_SUFFIX}}${{matrix.DLL_EXT}}

- name: Python tests run from .NET
run: dotnet test --runtime any-${{ matrix.platform }} src/python_tests_runner/
if: ${{ matrix.os == 'windows' }} # Not working for others right now
env:
PYTHONNET_PYDLL: ${{ matrix.DLL_PREFIX }}python${{matrix.DLL_PYVER_MAJOR}}${{matrix.PYVER_MINOR}}${{matrix.DLL_SUFFIX}}${{matrix.DLL_EXT}}

# TODO: Run perf tests
# TODO: Run mono tests on Windows?
4 changes: 4 additions & 0 deletionsCHANGELOG.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -34,6 +34,8 @@ details about the cause of the failure
when .NET expects an integer [#1342][i1342]
- More specific error messages for method argument mismatch
- BREAKING: Methods with `ref` or `out` parameters and void return type return a tuple of only the `ref` and `out` parameters.
- BREAKING: to call Python from .NET `Runtime.PythonDLL` property must be set to Python DLL name
or the DLL must be loaded in advance. This must be done before calling any other Python.NET functions.

### Fixed

Expand All@@ -57,6 +59,8 @@ when .NET expects an integer [#1342][i1342]
### Removed

- implicit assembly loading (you have to explicitly `clr.AddReference` before doing import)
- support for .NET Framework 4.0-4.6; Mono before 5.4. Python.NET now requires .NET Standard 2.0
(see [the matrix](https://docs.microsoft.com/en-us/dotnet/standard/net-standard#net-implementation-support))

## [2.5.0][] - 2020-06-14

Expand Down
6 changes: 5 additions & 1 deletionDirectory.Build.props
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -8,7 +8,11 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.CSharp" Version="4.7.0" />
<PackageReference Include="NonCopyableAnalyzer" Version="0.6.0">
<PackageReference Include="Microsoft.Net.Compilers.Toolset" Version="3.9.0-3.final">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
</PackageReference>
<PackageReference Include="Lost.NonCopyableAnalyzer" Version="0.7.0-m04">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
Expand Down
1 change: 1 addition & 0 deletionsappveyor.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -30,6 +30,7 @@ environment:
init:
# Update Environment Variables based on matrix/platform
- set PY_VER=%PYTHON_VERSION:.=%
- set PYTHONNET_PYDLL=python%PY_VER%.dll
- set PYTHON=C:\PYTHON%PY_VER%
- if %PLATFORM%==x64 (set PYTHON=%PYTHON%-x64)

Expand Down
2 changes: 1 addition & 1 deletionsrc/console/pythonconsole.cs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -26,7 +26,7 @@ private PythonConsole()
[STAThread]
public static int Main(string[] args)
{
// Onlynet40 is capable to safely inject python.runtime.dll into resources.
// Only.NET Framework is capable to safely inject python.runtime.dll into resources.
#if NET40
// reference the static assemblyLoader to stop it being optimized away
AssemblyLoader a = assemblyLoader;
Expand Down
4 changes: 2 additions & 2 deletionssrc/embed_tests/References.cs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,7 +23,7 @@ public void Dispose()
public void MoveToPyObject_SetsNull()
{
var dict = new PyDict();
NewReference reference = Runtime.PyDict_Items(dict.Handle);
NewReference reference = Runtime.PyDict_Items(dict.Reference);
try
{
Assert.IsFalse(reference.IsNull());
Expand All@@ -41,7 +41,7 @@ public void MoveToPyObject_SetsNull()
public void CanBorrowFromNewReference()
{
var dict = new PyDict();
NewReference reference = Runtime.PyDict_Items(dict.Handle);
NewReference reference = Runtime.PyDict_Items(dict.Reference);
try
{
PythonException.ThrowIfIsNotZero(Runtime.PyList_Reverse(reference));
Expand Down
9 changes: 5 additions & 4 deletionssrc/embed_tests/TestDomainReload.cs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -332,7 +332,7 @@ static void RunAssemblyAndUnload(string domainName)
// assembly (and Python .NET) to reside
var theProxy = CreateInstanceInstanceAndUnwrap<Proxy>(domain);

theProxy.Call("InitPython", ShutdownMode.Soft);
theProxy.Call(nameof(PythonRunner.InitPython), ShutdownMode.Soft, PyRuntime.PythonDLL);
// From now on use the Proxy to call into the new assembly
theProxy.RunPython();

Expand DownExpand Up@@ -400,7 +400,7 @@ static void RunDomainReloadSteps<T1, T2>() where T1 : CrossCaller where T2 : Cro
try
{
var theProxy = CreateInstanceInstanceAndUnwrap<Proxy>(domain);
theProxy.Call("InitPython", ShutdownMode.Reload);
theProxy.Call(nameof(PythonRunner.InitPython), ShutdownMode.Reload, PyRuntime.PythonDLL);

var caller = CreateInstanceInstanceAndUnwrap<T1>(domain);
arg = caller.Execute(arg);
Expand All@@ -418,7 +418,7 @@ static void RunDomainReloadSteps<T1, T2>() where T1 : CrossCaller where T2 : Cro
try
{
var theProxy = CreateInstanceInstanceAndUnwrap<Proxy>(domain);
theProxy.Call("InitPython", ShutdownMode.Reload);
theProxy.Call(nameof(PythonRunner.InitPython), ShutdownMode.Reload, PyRuntime.PythonDLL);

var caller = CreateInstanceInstanceAndUnwrap<T2>(domain);
caller.Execute(arg);
Expand DownExpand Up@@ -478,8 +478,9 @@ public static void RunPython()

private static IntPtr _state;

public static void InitPython(ShutdownMode mode)
public static void InitPython(ShutdownMode mode, string dllName)
{
PyRuntime.PythonDLL = dllName;
PythonEngine.Initialize(mode: mode);
_state = PythonEngine.BeginAllowThreads();
}
Expand Down
8 changes: 5 additions & 3 deletionssrc/embed_tests/TestRuntime.cs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -96,13 +96,15 @@ public static void PyCheck_Iter_PyObject_IsIterable_ThreadingLock_Test()
// TypeFlags.HaveIter set in Python 2. This tests a different code path in PyObject_IsIterable and PyIter_Check.
var threading = Runtime.Runtime.PyImport_ImportModule("threading");
Exceptions.ErrorCheck(threading);
var threadingDict = Runtime.Runtime.PyModule_GetDict(threading);
var threadingDict = Runtime.Runtime.PyModule_GetDict(new BorrowedReference(threading));
Exceptions.ErrorCheck(threadingDict);
var lockType = Runtime.Runtime.PyDict_GetItemString(threadingDict, "Lock");
if (lockType == IntPtr.Zero)
if (lockType.IsNull)
throw new KeyNotFoundException("class 'Lock' was not found in 'threading'");

var lockInstance = Runtime.Runtime.PyObject_CallObject(lockType, Runtime.Runtime.PyTuple_New(0));
var args = Runtime.Runtime.PyTuple_New(0);
var lockInstance = Runtime.Runtime.PyObject_CallObject(lockType.DangerousGetAddress(), args);
Runtime.Runtime.XDecref(args);
Exceptions.ErrorCheck(lockInstance);

Assert.IsFalse(Runtime.Runtime.PyObject_IsIterable(lockInstance));
Expand Down
65 changes: 0 additions & 65 deletionssrc/embed_tests/TestTypeManager.cs
View file
Open in desktop

This file was deleted.

3 changes: 2 additions & 1 deletionsrc/runtime/BorrowedReference.cs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -10,10 +10,11 @@ readonly ref struct BorrowedReference
readonly IntPtr pointer;
public bool IsNull => this.pointer == IntPtr.Zero;


/// <summary>Gets a raw pointer to the Python object</summary>
public IntPtr DangerousGetAddress()
=> this.IsNull ? throw new NullReferenceException() : this.pointer;
/// <summary>Gets a raw pointer to the Python object</summary>
public IntPtr DangerousGetAddressOrNull() => this.pointer;

/// <summary>
/// Creates new instance of <see cref="BorrowedReference"/> from raw pointer. Unsafe.
Expand Down
11 changes: 6 additions & 5 deletionssrc/runtime/CustomMarshaler.cs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -41,8 +41,9 @@ public int GetNativeDataSize()
/// </summary>
internal class UcsMarshaler : MarshalerBase
{
internal static readonly int _UCS = Runtime.PyUnicode_GetMax() <= 0xFFFF ? 2 : 4;
internal static readonly Encoding PyEncoding = _UCS == 2 ? Encoding.Unicode : Encoding.UTF32;
private static readonly MarshalerBase Instance = new UcsMarshaler();
private static readonly Encoding PyEncoding = Runtime.PyEncoding;

public override IntPtr MarshalManagedToNative(object managedObj)
{
Expand DownExpand Up@@ -91,13 +92,13 @@ public static int GetUnicodeByteLength(IntPtr p)
var len = 0;
while (true)
{
int c =Runtime._UCS == 2
int c = _UCS == 2
? Marshal.ReadInt16(p, len * 2)
: Marshal.ReadInt32(p, len * 4);

if (c == 0)
{
return len *Runtime._UCS;
return len * _UCS;
}
checked
{
Expand DownExpand Up@@ -147,7 +148,7 @@ public static string PtrToPy3UnicodePy2String(IntPtr p)
internal class StrArrayMarshaler : MarshalerBase
{
private static readonly MarshalerBase Instance = new StrArrayMarshaler();
private static readonly Encoding PyEncoding =Runtime.PyEncoding;
private static readonly Encoding PyEncoding =UcsMarshaler.PyEncoding;

public override IntPtr MarshalManagedToNative(object managedObj)
{
Expand All@@ -159,7 +160,7 @@ public override IntPtr MarshalManagedToNative(object managedObj)
}

int totalStrLength = argv.Sum(arg => arg.Length + 1);
int memSize = argv.Length * IntPtr.Size + totalStrLength *Runtime._UCS;
int memSize = argv.Length * IntPtr.Size + totalStrLength *UcsMarshaler._UCS;

IntPtr mem = Marshal.AllocHGlobal(memSize);
try
Expand Down
20 changes: 20 additions & 0 deletionssrc/runtime/NewReference.cs
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -11,6 +11,16 @@ ref struct NewReference
{
IntPtr pointer;

/// <summary>Creates a <see cref="NewReference"/> pointing to the same object</summary>
public NewReference(BorrowedReference reference, bool canBeNull = false)
{
var address = canBeNull
? reference.DangerousGetAddressOrNull()
: reference.DangerousGetAddress();
Runtime.XIncref(address);
this.pointer = address;
}

[Pure]
public static implicit operator BorrowedReference(in NewReference reference)
=> new BorrowedReference(reference.pointer);
Expand All@@ -28,6 +38,16 @@ public PyObject MoveToPyObject()
return result;
}

/// <summary>Moves ownership of this instance to unmanged pointer</summary>
public IntPtr DangerousMoveToPointer()
{
if (this.IsNull()) throw new NullReferenceException();

var result = this.pointer;
this.pointer = IntPtr.Zero;
return result;
}

/// <summary>Moves ownership of this instance to unmanged pointer</summary>
public IntPtr DangerousMoveToPointerOrNull()
{
Expand Down
2 changes: 1 addition & 1 deletionsrc/runtime/Python.Runtime.csproj
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2,9 +2,9 @@
<PropertyGroup>
<TargetFrameworks>netstandard2.0</TargetFrameworks>
<Platforms>AnyCPU</Platforms>
<LangVersion>9.0</LangVersion>
<RootNamespace>Python.Runtime</RootNamespace>
<AssemblyName>Python.Runtime</AssemblyName>
<LangVersion>9.0</LangVersion>
<PackageId>pythonnet</PackageId>
<PackageLicenseUrl>https://github.com/pythonnet/pythonnet/blob/master/LICENSE</PackageLicenseUrl>
<RepositoryUrl>https://github.com/pythonnet/pythonnet</RepositoryUrl>
Expand Down
Loading
[8]ページ先頭
©2009-2025 Movatter.jp