NotificationsYou must be signed in to change notification settings
Fork750
Star5.1k

WIP: Combine #546 and #531#557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Closed

fractus wants to merge6 commits intopythonnet:masterfromfractus:pr546+531

Closed

WIP: Combine #546 and #531#557

fractus wants to merge6 commits intopythonnet:masterfromfractus:pr546+531

Conversation

$fractus$

Copy link

Contributor

$@fractus$ fractus commentedOct 15, 2017

What does this implement/fix? Explain your changes.

Supersedes two PRs by mergingBuild system improvements pack - addressing comments to PR and some issues. #546 andInterop methods with Py_ssize_t works differently in NetCoreApp 2.0 #531 as suggested by@dmitriyse in the discussion ofInterop methods with Py_ssize_t works differently in NetCoreApp 2.0 #531.
Fixed a typo inruntime.cs
Commented a case-sensitive copy that break the build on macOS inPython.Runtime.15.csproj

Does this close any currently open issues?

No, it just now builds without problems on macOS.

Any other comments?

Versions used: python=3.6.3, mono=5.2.0.224, dotnet=2.0.0 on High Sierra (10.13).
Build with--xplat and tested on Mono/DotNetCore projects.
@dmitriyse if this not appropriate; I can submit the minor fixes in a new PR

Checklist

Check all those that are applicable and complete.

Make sure to include one or more tests for your change
If an enhancement PR, please create docs and at best an example
Add yourself toAUTHORS
Updated theCHANGELOG

Copy link

Contributor

dmitriyse commentedOct 15, 2017

Hi! Thank you for testing under Mac and your fixes.
I reviewed your changes and will approve them once CI finish builds.

I don't know how but ms fixed their Case issue with System.XML.dll(that should be System.Xml.dll) (probably with help of CoreCLR 2.0.0 improvement or silent myget package update).
Otherwise travis CI builds should fails.

Copy link

Contributor

dmitriyse commentedOct 15, 2017•
edited
Loading

PRs merging process currently is too slow.
We needs to do something with this.

dmitriyse self-assigned this

Oct 15, 2017

dmitriyse self-requested a review

October 15, 2017 21:12

dmitriyse removed their assignment

Oct 15, 2017

dmitriyse approved these changes

Oct 15, 2017

View reviewed changes

Copy link

codecovbot commentedOct 15, 2017

Codecov Report

Merging#557 intomaster willdecrease coverage by0.08%.
The diff coverage is69.76%.

@@            Coverage Diff             @@##           master     #557      +/-   ##==========================================- Coverage   77.16%   77.08%   -0.09%==========================================  Files          65       65                Lines        5610     5651      +41       Branches      888      888              ==========================================+ Hits         4329     4356      +27- Misses        993     1007      +14  Partials      288      288

Flag	Coverage Δ
#setup_linux	`71.83% <0%> (-1.82%)`	⬇️
#setup_windows	`76.23% <69.76%> (-0.08%)`	⬇️

Impacted Files	Coverage Δ
src/runtime/exceptions.cs	`80.35% <100%> (ø)`	⬆️
src/runtime/runtime.cs	`88.23% <68.57%> (-2.75%)`	⬇️
setup.py	`88.97% <71.42%> (-0.85%)`	⬇️

Continue to review full report at Codecov.

Legend -Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing data
Powered byCodecov. Last updatedf1c224...4cb92d4. Read thecomment docs.

Copy link

codecovbot commentedOct 15, 2017•
edited
Loading

Codecov Report

Merging#557 intomaster willdecrease coverage by0.05%.
The diff coverage is69.44%.

@@            Coverage Diff             @@##           master     #557      +/-   ##==========================================- Coverage   77.14%   77.08%   -0.06%==========================================  Files          65       65                Lines        5617     5651      +34       Branches      888      888              ==========================================+ Hits         4333     4356      +23- Misses        996     1007      +11  Partials      288      288

Flag	Coverage Δ
#setup_linux	`71.83% <ø> (ø)`	⬆️
#setup_windows	`76.23% <69.44%> (-0.06%)`	⬇️

Impacted Files	Coverage Δ
src/runtime/exceptions.cs	`80.35% <100%> (ø)`	⬆️
src/runtime/runtime.cs	`88.23% <68.57%> (-2.75%)`	⬇️
setup.py	`88.97% <0%> (ø)`	⬆️

Continue to review full report at Codecov.

Legend -Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing data
Powered byCodecov. Last updatef5a002f...e09749c. Read thecomment docs.

Copy link

Contributor

den-run-ai commentedOct 15, 2017

@fractus why do you have WIP in your PR title? Let me also check tomorrow morning on my Mac.

$@fractus$

Copy link

ContributorAuthor

fractus commentedOct 15, 2017

@denfromufa Thanks -- the reason for the WIP is the potentially breaking change in thecsproj; I also want to check on Win10 and Linux.

$@fractus$

Copy link

ContributorAuthor

fractus commentedOct 17, 2017

OnMSBuild 14 (VS2015) (Win7), the branch builds (python setup.py install) without any problems.

OnMSBuild 15 and dotnet2.0 (Win10), the branch fails; standardpip install works for 2.3.0. In particular:

Noxplat:

> python setup.py install --dry-runrunning installrunning buildrunning build_extChecking for updates from https://www.nuget.org/api/v2/.Currently running NuGet.exe 4.3.0.NuGet.exe is up to date.MSBuild auto-detection: using msbuild version '15.3.409.57025' from 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\bin'.All packages listed in packages.config are already installed.Microsoft (R) Manifest ToolCopyright (c) Microsoft Corporation.All rights reserved.Microsoft (R) Build Engine version 15.3.409.57025 for .NET FrameworkCopyright (C) Microsoft Corporation. All rights reserved.Microsoft (R) Build Engine version 15.3.409.57025 for .NET FrameworkCopyright (C) Microsoft Corporation. All rights reserved.C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\Microsoft\NuGet\15.0\Microsoft.NuGet.targets(178,5): error : Your  project.json doesn't have a runtimes section. You should add '"runtimes": { "win": { } }' to your project.json and then re-run NuGet restore. [c:\dev\pythonnet\src\runtime\Python.Runtime.csproj]C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\Microsoft\NuGet\15.0\Microsoft.NuGet.targets(178,5): error : Your  project.json doesn't have a runtimes section. You should add '"runtimes": { "win-x64": { } }' to your project.json and then re-run NuG et restore. [c:\dev\pythonnet\src\clrmodule\clrmodule.csproj]Traceback (most recent call last):  File "setup.py", line 516, in <module>    zip_safe=False,  ...  File "setup.py", line 259, in build_extension    subprocess.check_call(" ".join(cmd + ["/t:Build"]), shell=use_shell)  File "c:\tools\Anaconda3\envs\c36\lib\subprocess.py", line 291, in check_call    raise CalledProcessError(retcode, cmd)subprocess.CalledProcessError: Command '"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\MSBuild.exe" pythonnet.sln /p:Configuration=ReleaseWin /p:Platform=x64 /p:DefineConstants="PYTHON36%3BPYTHON3%3BUCS2" /p:PythonBuildDir="c:\dev\pythonnet\build\lib.win-amd64-3.6" /p:PythonInteropFile="interop36.cs" /verbosity:minimal /p:PythonManifest="c:\dev\pythonnet\build\lib.win-amd64-3.6\app.manifest" /t:Build' returned non-zero exit status 1.

Withxplat:

> python setup.py install --xplat --dry-runrunning installrunning buildrunning build_extMicrosoft (R) Build Engine version 15.3.409.57025 for .NET CoreCopyright (C) Microsoft Corporation. All rights reserved.  Restoring packages for c:\dev\pythonnet\src\embed_tests\Python.EmbeddingTest.15.csproj...  Restoring packages for c:\dev\pythonnet\src\runtime\Python.Runtime.15.csproj...  Restoring packages for c:\dev\pythonnet\src\console\Console.15.csproj...  Restore completed in 18.12 ms for c:\dev\pythonnet\src\clrmodule\clrmodule.15.csproj.  Restoring packages for c:\dev\pythonnet\src\testing\Python.Test.15.csproj...  Restore completed in 181.26 ms for c:\dev\pythonnet\src\testing\Python.Test.15.csproj.  Restore completed in 203.27 ms for c:\dev\pythonnet\src\runtime\Python.Runtime.15.csproj.  Restore completed in 253.6 ms for c:\dev\pythonnet\src\console\Console.15.csproj.  Restore completed in 590.73 ms for c:\dev\pythonnet\src\embed_tests\Python.EmbeddingTest.15.csproj.Microsoft (R) Manifest ToolCopyright (c) Microsoft Corporation.All rights reserved.Microsoft (R) Build Engine version 15.3.409.57025 for .NET FrameworkCopyright (C) Microsoft Corporation. All rights reserved.c:\dev\pythonnet\src\testing\Python.Test.15.csproj(2,31): error MSB4236: The SDK 'Microsoft.NET.Sdk' specified could not be found.c:\dev\pythonnet\src\console\Console.15.csproj(2,31): error MSB4236: The SDK 'Microsoft.NET.Sdk' specified could not be found.c:\dev\pythonnet\src\clrmodule\clrmodule.15.csproj(3,31): error MSB4236: The SDK 'Microsoft.NET.Sdk' specified could not be found.c:\dev\pythonnet\src\embed_tests\Python.EmbeddingTest.15.csproj(2,31): error MSB4236: The SDK 'Microsoft.NET.Sdk' specified could not b e found.c:\dev\pythonnet\src\runtime\Python.Runtime.15.csproj(2,31): error MSB4236: The SDK 'Microsoft.NET.Sdk' specified could not be found.Traceback (most recent call last):  File "setup.py", line 516, in <module>    zip_safe=False,  ...  File "setup.py", line 258, in build_extension    subprocess.check_call(" ".join(cmd + ["/t:Clean"]), shell=use_shell)  File "c:\tools\Anaconda3\envs\c36\lib\subprocess.py", line 291, in check_call    raise CalledProcessError(retcode, cmd)subprocess.CalledProcessError: Command '"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin\MSBuild.exe" pythonnet.15.sln /p:Configuration=ReleaseWin /p:Platform=x64 /p:CustomDefineConstants="PYTHON36%3BPYTHON3%3BUCS2" /p:PythonBuildDir="c:\dev\pythonnet\build\lib.win-amd64-3.6" /p:PythonInteropFile="interop36.cs" /verbosity:minimal /p:PythonManifest="c:\dev\pythonnet\build\lib.win-amd64-3.6\app.manifest" /t:Clean' returned non-zero exit status 1.

The following "manual" builds work:

> dotnet build src\runtime\Python.Runtime.15.csproj /p:Configuration=ReleaseWinPY3    8 Warning(s)    0 Error(s)Time Elapsed 00:00:04.50

but this one fails:

> dotnet build src\clrmodule\clrmodule.15.csproj /p:Configuration=DebugWinPY3Microsoft (R) Build Engine version 15.3.409.57025 for .NET CoreCopyright (C) Microsoft Corporation. All rights reserved.  clrmodule.15 -> c:\dev\pythonnet\src\clrmodule\bin\clrmodule.dllC:\Users\vpapakos\.nuget\packages\unmanagedexports\1.2.7\tools\RGiesecke.DllExport.targets(58,3): error MSB4062: The "RGiesecke.DllExport.MSBuild.DllExportAppDomainIsolatedTask" task could not be loaded from the assembly C:\Users\vpapakos\.nuget\packages\unmanagedexports\1.2.7\tools\RGiesecke.DllExport.MSBuild.dll. Could not load file or assembly 'Microsoft.Build.Utilities, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified. Confirm that the <UsingTask> declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [c:\dev\pythonnet\src\clrmodule\clrmodule.15.csproj]Build FAILED.

Copy link

Contributor

dmitriyse commentedOct 17, 2017•
edited
Loading

I will try to debug this problem. --dry-run switch was never debugged with --xplat

$@fractus$

Copy link

ContributorAuthor

fractus commentedOct 26, 2017

Something that may be helpful: on Win10 + MSBuild15, installing the wheel file from the PRartifacts worked:

C:\temp(c36) λ pip install pythonnet-2.4.0.dev0-cp36-cp36m-win_amd64.whlProcessing c:\temp\pythonnet-2.4.0.dev0-cp36-cp36m-win_amd64.whlInstalling collected packages: pythonnetSuccessfully installed pythonnet-2.4.0.dev0C:\temp(c36) λ pythonPython 3.6.3 |Anaconda, Inc.| (default, Oct 15 2017, 03:27:45) [MSC v.1900 64 bit (AMD64)] on win32Type "help", "copyright", "credits" or "license" for more information.>>> import clr>>> clr<module 'clr' from 'C:\\tools\\anaconda3\\envs\\c36xplat\\lib\\site-packages\\clr.pyd'>>>> clr.__version__'2.4.0.dev0'

Copy link

Contributor

dmitriyse commentedOct 26, 2017

> dotnet build src\clrmodule\clrmodule.15.csproj /p:Configuration=DebugWinPY3Microsoft (R) Build Engine version 15.3.409.57025for .NET CoreCopyright (C) Microsoft Corporation. All rights reserved.  clrmodule.15 -> c:\dev\pythonnet\src\clrmodule\bin\clrmodule.dllC:\Users\vpapakos\.nuget\packages\unmanagedexports\1.2.7\tools\RGiesecke.DllExport.targets(58,3): error MSB4062: The"RGiesecke.DllExport.MSBuild.DllExportAppDomainIsolatedTask" task couldnot be loaded from the assembly C:\Users\vpapakos\.nuget\packages\unmanagedexports\1.2.7\tools\RGiesecke.DllExport.MSBuild.dll. Couldnot load file or assembly 'Microsoft.Build.Utilities, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannotfind the file specified. Confirm that the<UsingTask> declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask. [c:\dev\pythonnet\src\clrmodule\clrmodule.15.csproj]Build FAILED.

I figured out the problem

dotnet msbuild (it's worked on dotnet core)
and
VS 2017 Dev Env -> msbuild (it's worked on .Net Framework 4.x)
Is not the same.

So you should execute (under VS 2017 DevEnvironment)

msbuild src\clrmodule\clrmodule.15.csproj /p:Configuration=DebugWinPY3

Copy link

Contributor

den-run-ai commentedNov 4, 2017

@fractus @dmitriyse so is there still any open issue on mac or windows. did you try to build this under linux?

den-run-ai requested review fromfilmor,tonyroberts,dmitriyse andvmuriart

November 4, 2017 04:03

den-run-ai added the requires review label

Nov 4, 2017

Copy link

Contributor

dmitriyse commentedNov 4, 2017

Some work still needed to handle --dry-run correctly.

Copy link

Contributor

dmitriyse commentedNov 4, 2017

dotnet msbuild .... will never work on windows, so@fractus should change his test scheme.

$@fractus$

Copy link

ContributorAuthor

fractus commentedNov 4, 2017

Win

Agreed with@dmitriyse ondotnet msbuild -- we should drop it from Windows
[PENDING] Still failing with MSBuild15

Linux

Build without problems with Mono 5.4 and Python 3.6; also with--xplat (with dotnet)

$@fractus$ fractus closed this

Nov 4, 2017

$@fractus$

Copy link

ContributorAuthor

fractus commentedNov 4, 2017

(Sorry, for the message pollution -- closed it by mistake)

$@fractus$ fractus reopened this

Nov 4, 2017

Copy link

Contributor

dmitriyse commentedNov 4, 2017

@fractus , sorry but problem is solely in your environment.
I tested the same commands on a super fresh windows 10, VS 2017, python 3.6, and problem was not reproduced.

Please read some related post.
https://andrewlock.net/the-sdk-microsoft-net-sdk-web-specified-could-not-be-found/

Ensure that you are able to build any .net core 2.0 project with your VS 2017 msbuild.