dotnet/android-librariesPublic

NotificationsYou must be signed in to change notification settings
Fork63
Star290

Fix extraction of Xamarin.Build.Download#1048

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Open

maonaoda wants to merge2 commits intodotnet:main

base:main

Choose a base branch

frommaonaoda:bugfix/1047-fix-tgz-extraction

Open

Fix extraction of Xamarin.Build.Download#1048

maonaoda wants to merge2 commits intodotnet:mainfrommaonaoda:bugfix/1047-fix-tgz-extraction

Conversation

Copy link

maonaoda commentedNov 16, 2024•
edited
Loading

Fix#1047

Before fix:

2024-11-16.23.39.10.mov

After fix:

maonaoda added2 commits

November 16, 2024 23:22

fix: fix extraction

e7d07f2

test: add test

6b6ccd4

maonaoda changed the title~~fix: fix extraction~~Fix extraction of Xamarin.Build.Download

Nov 16, 2024

dellis1972 requested a review fromjpobst

November 16, 2024 16:48

moljac reviewed

Nov 17, 2024

View reviewed changes

util/Xamarin.Build.Download/source/Xamarin.Build.Download.Tests/Test.cs


		prel.AddItem(
		"XamarinBuildDownload","GAppM-10.28.0",newDictionary<string,string>{
		{"Url","https://dl.google.com/firebase/ios/analytics/4d5ec9a36b6d4fd4/GoogleAppMeasurement-10.28.0.tar.gz"},

Copy link

Contributor

moljacNov 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Maybe we'll need to move this test data somewhere else.

moljac approved these changes

Nov 17, 2024

View reviewed changes

Copy link

Contributor

moljac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

LGTM with comment to move test data somewhere else

jpobst reviewed

Nov 20, 2024

View reviewed changes

Copy link

Contributor

jpobst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I feel like we're missing the root cause of the issue here, and this is just a workaround.

The reported issue is that theC:\Users\wangyk\AppData\Local\XamarinBuildDownloadCache\GAppM-10.28.0\GAppM-10.28.0.tar file already exists.

If the file already exists then we should not be downloading it again in the first place, so the question is "why are we trying to download this?".

My guess is that maybe a previous extraction failed before theGAppM-10.28.0.unpacked file got written.

If you delete yourC:\Users\wangyk\AppData\Local\XamarinBuildDownloadCache\GAppM-10.28.0 directory and then try again, do you still see this error? If so, did theGAppM-10.28.0.unpacked file get written?

We need to figure out if:

This was a one time bad state and clearing the directory fixes it.
There is an issue writing the.tar, extracting the.tar file, or writing the.unpacked file.
Everything is extracted correctly and we aren't detecting the existing files and are trying to run the process a second time when it isn't needed.

I tried running the unit test locally without the overwrite change and it succeeded. I'm not sure how the test could fail because it always tries writing to a freshPath.GetTempFileName (), so there should never be an existing file that needs to be overwritten. If the test is failing for you locally, hopefully debugging it will help determine which of the above scenarios is occuring.

Copy link

Author

maonaoda commentedNov 21, 2024

Yes, a new folder will be created every time, but I will definitely reproduce this

Copy link

Author

maonaoda commentedNov 21, 2024

It has been staying in Extracting

テストプロジェクトのビルド
========== テストの実行を開始しています ==========
[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.5+1caef2f33e (64-bit .NET 9.0.0)
[xUnit.net 00:00:00.46] Starting: Xamarin.Build.Download.Tests
Build started.
Project "project.csproj" (_XamarinBuildDownload target(s)):
Target "_XamarinBuildAddDownloadedItems" in file "D:\practice\android-libraries\util\Xamarin.Build.Download\source\Xamarin.Build.Download.Tests\bin\Release\net9.0\Xamarin.Build.Download.targets" from project "C:\Users\wangyk\AppData\Local\Temp\tmpavnnef.tmp\project.csproj" (target "_XamarinBuildDownload" depends on it):
Done building target "_XamarinBuildAddDownloadedItems" in project "project.csproj".
Target "_XamarinBuildDownloadCore" in file "D:\practice\android-libraries\util\Xamarin.Build.Download\source\Xamarin.Build.Download.Tests\bin\Release\net9.0\Xamarin.Build.Download.targets" from project "C:\Users\wangyk\AppData\Local\Temp\tmpavnnef.tmp\project.csproj" (target "_XamarinBuildDownload" depends on it):
The ".overridetasks" files could not be successfully loaded from their expected location "D:\practice\android-libraries\util\Xamarin.Build.Download\source\Xamarin.Build.Download.Tests\bin\Release\net9.0". Default tasks will not be overridden.
The ".tasks" files could not be successfully loaded from their expected location "D:\practice\android-libraries\util\Xamarin.Build.Download\source\Xamarin.Build.Download.Tests\bin\Release\net9.0". Default tasks will not be available.
Using "XamarinDownloadArchives" task from assembly "D:\practice\android-libraries\util\Xamarin.Build.Download\source\Xamarin.Build.Download.Tests\bin\Release\net9.0\Xamarin.Build.Download.dll".
Task "XamarinDownloadArchives"
D:\Program Files\7-Zip
C:\Program Files\Microsoft Visual Studio\2022\Enterprise
Downloadinghttps://dl.google.com/firebase/ios/analytics/4d5ec9a36b6d4fd4/GoogleAppMeasurement-10.28.0.tar.gz to C:\Users\wangyk\AppData\Local\Temp\tmpavnnef.tmp\unpacked\GAppM-10.28.0.tgz
(2271/10038515b), total 0.0%
(1004512/10038515b), total 10.0%
(2015232/10038515b), total 20.0%
(3014640/10038515b), total 30.0%
(4030448/10038515b), total 40.0%
(5029856/10038515b), total 50.0%
(6029280/10038515b), total 60.0%
(7028688/10038515b), total 70.0%
(8044496/10038515b), total 80.0%
(9043904/10038515b), total 90.0%
(10038515/10038515b), total 100.0%
Downloading Complete
Extracting C:\Users\wangyk\AppData\Local\Temp\tmpavnnef.tmp\unpacked\GAppM-10.28.0.tgz to C:\Users\wangyk\AppData\Local\Temp\tmpavnnef.tmp\unpacked\GAppM-10.28.0

The only difference is that I set the compilation environment to net9

Copy link

Author

maonaoda commentedNov 21, 2024

Copy link

Author

maonaoda commentedNov 21, 2024

It is not a download, but a decompression problem of GAppM-10.28.0

Copy link

Contributor

jpobst commentedNov 21, 2024

It is not a download, but a decompression problem of GAppM-10.28.0

Sure, but I still don't understand what the root issue is.

When we decompressGAppM-10.28.0,why do the file(s) already exist?

If this is the first time it is running, then the folder should be empty and there won't be any existing files.
If this is the second time it is running, it should have detected that the file(s) already exist and it shouldn't be downloading or extracting them again.

Perhaps another way to explore this:

Does theTestGMapsDownload test fail for you as well without this change?
If not, why does it succeed butTestGoogleAppMeasurementDownload hangs? They are exactly the same test but with a different.tar.gz file, so what makesGoogleAppMeasurement-10.28.0.tar.gz hang whilec0e534927c0c955e-GoogleMaps-1.11.1.tar.gz succeeds?

Copy link

Author

maonaoda commentedNov 22, 2024

In fact, tar.gz performs two extractions. The first extraction is successful, but during the second extraction, a confirmation about the existence of the file will appear.

Copy link

Author

maonaoda commentedNov 22, 2024

The first extraction:

psi

	名前	値	種類
◢	psi	FileName = "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\Extensions\Xamarin.VisualStudio\7-Zip\7z.exe",Arguments = "e "-oC:\Users\wangyk\AppData\Local\Temp\tmpp0glvd.tmp\unpacked\GAppM-10.28.0" "C:\Users\wangyk\AppData\Local\Temp\tmpp0glvd.tmp\unpacked\GAppM-10.28.0.tgz"", WorkingDirectory = ""	System.Diagnostics.ProcessStartInfo

The next extraction:

psi

	名前	値	種類
◢	psi	FileName = "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\Extensions\Xamarin.VisualStudio\7-Zip\7z.exe", Arguments = "x -snl- "-oC:\Users\wangyk\AppData\Local\Temp\tmpp0glvd.tmp\unpacked\GAppM-10.28.0" "C:\Users\wangyk\AppData\Local\Temp\tmpp0glvd.tmp\unpacked\GAppM-10.28.0\GAppM-10.28.0.tar"", WorkingDirectory = ""	System.Diagnostics.ProcessStartInfo

Copy link

Author

maonaoda commentedNov 22, 2024•
edited
Loading

Now that we have obtained the execution parameters for the two decompressions, try to execute them in PowerShell

& "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\Extensions\Xamarin.VisualStudio\7-Zip\7z.exe" e "-oD:\Test\GAppM-10.28.0" "D:\Test\GAppM-10.28.0.tgz"

PS D:\Test> & "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\Extensions\Xamarin.VisualStudio\7-Zip\7z.exe" e "-oD:\Test\GAppM-10.28.0" "D:\Test\GAppM-10.28.0.tgz"7-Zip 18.05 (x86) : Copyright (c) 1999-2018 Igor Pavlov : 2018-04-30Scanning the drive for archives:1 file, 10038515 bytes (9804 KiB)Extracting archive: D:\Test\GAppM-10.28.0.tgz--Path = D:\Test\GAppM-10.28.0.tgzType = gzipHeaders Size = 10Everything is OkSize:       40711680Compressed: 10038515

& "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\Extensions\Xamarin.VisualStudio\7-Zip\7z.exe" x -snl- "-oD:\Test\GAppM-10.28.0" "D:\Test\GAppM-10.28.0\GAppM-10.28.0.tar"

PS D:\Test> & "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\Extensions\Xamarin.VisualStudio\7-Zip\7z.exe" x -snl- "-oD:\Test\GAppM-10.28.0" "D:\Test\GAppM-10.28.0\GAppM-10.28.0.tar"7-Zip 18.05 (x86) : Copyright (c) 1999-2018 Igor Pavlov : 2018-04-30Scanning the drive for archives:1 file, 40711680 bytes (39 MiB)Extracting archive: D:\Test\GAppM-10.28.0\GAppM-10.28.0.tarWARNINGS:Headers Error--Path = D:\Test\GAppM-10.28.0\GAppM-10.28.0.tarType = tarWARNINGS:Headers ErrorPhysical Size = 40711680Headers Size = 124416Code Page = UTF-8Would you like to replace the existing file:  Path:     D:\Test\GAppM-10.28.0\GoogleAppMeasurement-10.28.0\Frameworks\GoogleAppMeasurementIdentitySupport.xcframework\ios-arm64_x86_64-maccatalyst\GoogleAppMeasurementIdentitySupport.framework\PaxHeader\GoogleAppMeasurementIdentitySupport  Size:     44 bytes (1 KiB)  Modified: 2024-06-11 03:14:30with the file from archive:  Path:     .\GoogleAppMeasurement-10.28.0\Frameworks\GoogleAppMeasurementIdentitySupport.xcframework\ios-arm64_x86_64-maccatalyst\GoogleAppMeasurementIdentitySupport.framework\PaxHeader\GoogleAppMeasurementIdentitySupport  Size:     14 bytes (1 KiB)  Modified: 2024-06-11 03:11:10? (Y)es / (N)o / (A)lways / (S)kip all / A(u)to rename all / (Q)uit?