Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

ImplementIUtf8SpanParsable onIPAddress andIPNetwork#102144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged

Conversation

@edwardneal
Copy link
Contributor

@edwardnealedwardneal commentedMay 13, 2024
edited by MihaZupan
Loading

Closes#103111

Relates to#81500. cc@tannergooding - hopefully this doesn't tread on any prior work.

This implementsIUtf8SpanParsable onIPAddress andIPNetwork. It does so by makingIPAddressParser generic betweenbyte andchar. Doing so needed deeper changes than I'd have liked (the separators aren't considered constants, which means that I had to swap a switch statement and its "goto case" statements for a set of if statements) As I did that, I replaced most of the unsafe code and pointers withReadOnlySpans (which eliminates a pattern where downstream code was pinning an ROS and passing its pointer.)

IPAddressParser,IPv4AddressHelper andIPv6AddressHelper are shared with System.Net.Quic, System.Net.Security and System.Private.Uri. The changes to the first two projects are pretty minor, consisting of one change toTargetHostNameHelper. System.Private.Uri is similar, but it makes more extensive use of pointers.

There are two differences between this PR and the underlying issue:

  • I've added the public methodsParse(ReadOnlySpan<byte>) andTryParse(ReadOnlySpan<byte>, out IPNetwork) toIPNetwork.
  • IPNetwork implementsIUtf8SpanParsable explicitly rather than implicitly.

In both cases, this is done to maintain consistency with theISpanParsable members which were already implemented. I've taken this to be covered by the original API review.

It's worth noting thatIPv6AddressHelper<TChar>.IsValid in System.Private.Uri continues to referencechar* rather thanReadOnlySpan<TChar>. This is inconsistent, but I've left it alone for now. I'd prefer to address that in a follow-up PR which replaces the unsafe components fromUri.PrivateParseMinimal and the methods it calls with a ROS.

neon-sunset reacted with heart emoji
This is preparation for the work required to enable IUtf8SpanParsable support on IPAddress and IPNetwork.Also replaced a few instances of unsafe code with spans.This uses the same style of generic constraints as the numeric parsing in corelib (although it can't access IUtfChar<TSelf>.)All tests pass, although the UTF8 code paths haven't yet been tested.Some restructuring is pending as I plumb IPAddressParser to the updated generic methods.Future commits will handle the downstream dependencies of IPvXAddressHelper - at present, this looks like Uri and TargetHostNameHelper.
This allows me to eliminate the nested class and now-redundant generic type constraints on each method
These are just the changes required to make System.Private.Uri compile.* Split IPAddressParser and IPv6AddressHelper into multiple files and shared common components between projects.* Updated some of the Uri-specific additions to IPvXAddressHelper files to use spans.* Minor adjustments to project files to support the above.
Removed a using statement in the ref project, small reduction in the csproj diff for System.Net.Primitives
@ghost
Copy link

Note regarding thenew-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@tannergooding
Copy link
Member

Hi@edwardneal, sorry for the delay in getting to this, I had missed the tag initially since this is under the networking area.

It's worth noting thatIPAddress andIPNetwork are not part of APIs that#81500 was approved for, and so before this can move forward there needs to be a new API proposal that specifically asks for implementingIUtf8SpanParsable on these types.

That proposal should be straightforward and I expect have no issues going through API review, at which point this can likely get merged. If you could assist in opening such a proposal, that should help the process go along faster.

@edwardneal
Copy link
ContributorAuthor

Thanks@tannergooding. Sorry for the confusion here - I read the original issue as approving the addition ofIUtf8SpanParsable andIUtf8SpanFormattable to anything already implementingISpanFormattable, andIPAddress andIPNetwork both do so. Should I just raise the API proposal for the additionalIPNetwork.Parse(ReadOnlySpan<byte>) andIPNetwork.TryParse(ReadOnlySpan<byte>, out IPNetwork) methods andIPNetwork's explicit interface implementation, or should I also includeIPAddress in that?

@tannergooding
Copy link
Member

should I also include IPAddress in that?

We want to include all new public APIs and interfaces in the proposal whether that is forIUtf8SpanParsable,IUtf8SpanFormattable, or both; as well as any overloads beyond this.

I read the original issue as approving the addition of IUtf8SpanParsable and IUtf8SpanFormattable to anything already implementing ISpanFormattable

The general spirit behind the feature fits that, but we still do explicit review and sign-off as there are often edge cases that we may warrant different consideration.

edwardneal reacted with thumbs up emoji

@karelz
Copy link
Member

Given that we are blocked on API review, I would recommend to either change the PR to Draft or close it, until the API is approved.@edwardneal what do you prefer and can you please do it? Thanks!

edwardneal reacted with thumbs up emoji

@karelzkarelz added the blockedIssue/PR is blocked on something - see comments labelJun 6, 2024
@edwardnealedwardneal marked this pull request as draftJune 6, 2024 16:42
@MihaZupanMihaZupan marked this pull request as ready for reviewJune 14, 2024 20:33
}
buffer[buffer.Length-1]='\0';

if(Interop.IpHlpApi.ConvertInterfaceNameToLuid(buffer,refinterfaceLuid)!=0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

If this produces an error, but in the finally we then Free the native memory, will that end up corrupting whatever error code the caller might need to retrieve?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It won't on Windows: the documentation doesn't say that any of the methods changes the result of GetLastError, and testing confirms this - the return value is the only success/failure response, so there's no error code to corrupt.SetLastError was set to true on the interop definitions though, I've corrected this.

It will on Unix, so I've wrapped the Free in GetLastPInvokeError/SetLastPInvokeError as per the pattern inCriticalHandle.Cleanup.

Copy link
Member

@stephentoubstephentoub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks! Left a few comments/questions, but otherwise LGTM.

* SetLastError has the correct value in the Windows interop layer.* Preserve InterfaceNameToIndex's errno in the Unix InterfaceInfoPal.
@MihaZupan
Copy link
Member

@EgorBot

usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Net;publicclassBench{[Benchmark][Arguments("192.168.0.1")][Arguments("4294967295")][Arguments("037777777777")][Arguments("0xff.0x7f.0x20.0x01")][Arguments("192.168.0.0/16")][Arguments("::192.168.0.1")][Arguments("100:0:1:2:0:0:000:abcd")][Arguments("Fe08::1%13542")][Arguments("1:2:3:4:5:6:7:8::")]publicboolTryParse(strings)=>IPAddress.TryParse(s,out_);[Benchmark][Arguments("http://192.168.0.1:123/foo")][Arguments("http://[100:0:1:2:0:0:000:abcd]:123/foo")]publicstringUriHost(strings)=>newUri(s).Host;}
EgorBot reacted with thumbs up emoji

@edwardneal
Copy link
ContributorAuthor

edwardneal commentedOct 6, 2024
edited
Loading

Thanks MihaZupan. That's quite a wide regression in::192.168.0.1, and I don't see a clear reason why. Some performance is left on the table inHexConverter.FromChar and similar (it doesn't cast its input to a uint, so the JIT doesn't elide the bounds check and we end up performing it twice) but we were using that method anyway - just indirectly, as a result of calling Uri.FromHex. In any event though, this method isn't called in the slower case - that test case's code path hasn't actually changed.

I've reverted the only other difference so that there's a common baseline, and I'll keep looking at this. I'm faintly suspicious that passing around an int rather than a char is causing the JIT to implicitly reintroduce bounds checks, but I wouldn't expect them to have such a large impact in this case.

@EgorBo
Copy link
Member

EgorBo commentedOct 6, 2024
edited
Loading

Some performance is left on the table in HexConverter.FromChar and similar (it doesn't cast its input to a uint, so the JIT doesn't elide the bounds check and we end up performing it twice)

Can you point me exactly to the non-elided bound check? Last I checked, all users ofFromChar give it something JIT can see it's never negative. E.g. Uri.FromHex passeschar toHexConverter.FromChar, hence, never negative.

@edwardneal
Copy link
ContributorAuthor

edwardneal commentedOct 7, 2024
edited
Loading

I agree.FromChar is defined as

[MethodImpl(MethodImplOptions.AggressiveInlining)]publicstaticintFromChar(intc){returnc>=CharToHexLookup.Length?0xFF:CharToHexLookup[c];}

c isn't cast to an unsigned type before the comparison toCharToHexLookup.Length, so if the caller passes it an int then a bounds check is performed.

That's not normally a problem - like you said, Uri.FromHex passes it a char, so when it's inlined it makes perfect sense that the JIT can see which datatype is really being passed around. I'm passing it an integer though, so the bounds check would likely remain.

I'm hesitant to say that it's the only cause of the performance regression because when I benchmarked it, the bounds checks only introduced a 0.2ns/call performance gap, even on a slower laptop. I'm going to test the idea over the next few days (edit: this'll likely be the weekend now, sorry.)


Updating on this: investigating the performance regressions led to a few places whereCreateTruncating of a non-constant value performed a few nanoseconds slower than I'd expected, in the middle of a number of hot loops. I've replaced this with a separateToUShort method, which performs a direct cast.

The UriHost benchmarks were stranger. As best I can tell,IPv4AddressHelper.Parse's resultant assembly was 200 bytes smaller when JITed under the PR environment than when JITed under .NET 9.0 RC2. This led to a number of different inlining decisions. I've brought the PR up to date with the main branch and re-run the benchmarks... they look roughly acceptable to me.

Prior to code review
BenchmarkDotNet v0.14.0, Windows 10 (10.0.20348.2582) (Hyper-V)Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 8 logical and 4 physical cores.NET SDK 9.0.100-rc.2.24474.11  [Host]   : .NET 9.0.0 (9.0.24.47305), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI  Baseline : .NET 9.0.0 (9.0.24.47305), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI  CoreRun  : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodJobRuntimesMeanErrorStdDevRatioGen0Code SizeAllocatedAlloc Ratio
TryParseBaseline.NET 9.0::192.168.0.183.40 ns0.421 ns0.393 ns1.000.00313,363 B80 B1.00
TryParseCoreRun.NET 10.0::192.168.0.182.96 ns0.256 ns0.227 ns0.990.003171 B80 B1.00
TryParseBaseline.NET 9.003777777777729.75 ns0.194 ns0.182 ns1.000.00151,730 B40 B1.00
TryParseCoreRun.NET 10.003777777777726.48 ns0.111 ns0.104 ns0.890.001671 B40 B1.00
TryParseBaseline.NET 9.00xff.0x7f.0x20.0x0136.47 ns0.122 ns0.108 ns1.000.00151,725 B40 B1.00
TryParseCoreRun.NET 10.00xff.0x7f.0x20.0x0131.04 ns0.179 ns0.167 ns0.850.001571 B40 B1.00
TryParseBaseline.NET 9.01:2:3:4:5:6:7:8::46.71 ns0.041 ns0.037 ns1.00-1,996 B-NA
TryParseCoreRun.NET 10.01:2:3:4:5:6:7:8::44.48 ns0.074 ns0.066 ns0.95-71 B-NA
TryParseBaseline.NET 9.0100:0(...):abcd [22]116.31 ns0.318 ns0.282 ns1.000.00313,013 B80 B1.00
TryParseCoreRun.NET 10.0100:0(...):abcd [22]109.84 ns0.174 ns0.163 ns0.940.003171 B80 B1.00
TryParseBaseline.NET 9.0192.168.0.0/1622.89 ns0.052 ns0.049 ns1.00-1,708 B-NA
TryParseCoreRun.NET 10.0192.168.0.0/1622.29 ns0.147 ns0.130 ns0.97-71 B-NA
TryParseBaseline.NET 9.0192.168.0.129.99 ns0.226 ns0.211 ns1.000.00151,750 B40 B1.00
TryParseCoreRun.NET 10.0192.168.0.128.78 ns0.202 ns0.179 ns0.960.001571 B40 B1.00
TryParseBaseline.NET 9.0429496729524.58 ns0.104 ns0.097 ns1.000.00161,766 B40 B1.00
TryParseCoreRun.NET 10.0429496729524.98 ns0.189 ns0.177 ns1.020.001671 B40 B1.00
TryParseBaseline.NET 9.0Fe08::1%1354278.23 ns0.186 ns0.174 ns1.000.00483,413 B120 B1.00
TryParseCoreRun.NET 10.0Fe08::1%1354265.86 ns0.224 ns0.210 ns0.840.003171 B80 B0.67
UriHostBaseline.NET 9.0http:(...)3/foo [39]415.16 ns0.485 ns0.454 ns1.000.007617,319 B192 B1.00
UriHostCoreRun.NET 10.0http:(...)3/foo [39]373.66 ns1.215 ns1.077 ns0.900.007617,769 B192 B1.00
UriHostBaseline.NET 9.0http:(...)3/foo [26]227.89 ns0.270 ns0.252 ns1.000.007215,314 B184 B1.00
UriHostCoreRun.NET 10.0http:(...)3/foo [26]228.84 ns0.269 ns0.224 ns1.000.007216,081 B184 B1.00
Post code review
BenchmarkDotNet v0.14.0, Windows 10 (10.0.20348.2582) (Hyper-V)Intel Xeon Platinum 8370C CPU 2.80GHz, 1 CPU, 8 logical and 4 physical cores.NET SDK 9.0.100-rc.2.24474.11  [Host]   : .NET 9.0.0 (9.0.24.47305), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI  Baseline : .NET 9.0.0 (9.0.24.47305), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI  CoreRun  : .NET 10.0.0 (42.42.42.42424), X64 RyuJIT AVX-512F+CD+BW+DQ+VL+VBMI
MethodJobRuntimesMeanErrorStdDevRatioRatioSDGen0Code SizeAllocatedAlloc Ratio
TryParseBaseline.NET 9.0::192.168.0.181.18 ns0.306 ns0.271 ns1.000.000.00313,341 B80 B1.00
TryParseCoreRun.NET 10.0::192.168.0.174.32 ns0.815 ns0.763 ns0.920.010.003171 B80 B1.00
TryParseBaseline.NET 9.003777777777728.85 ns0.157 ns0.146 ns1.000.010.00151,730 B40 B1.00
TryParseCoreRun.NET 10.003777777777725.44 ns0.266 ns0.249 ns0.880.010.001671 B40 B1.00
TryParseBaseline.NET 9.00xff.0x7f.0x20.0x0135.73 ns0.309 ns0.289 ns1.000.010.00151,725 B40 B1.00
TryParseCoreRun.NET 10.00xff.0x7f.0x20.0x0129.78 ns0.244 ns0.216 ns0.830.010.001571 B40 B1.00
TryParseBaseline.NET 9.01:2:3:4:5:6:7:8::46.69 ns0.042 ns0.033 ns1.000.00-1,966 B-NA
TryParseCoreRun.NET 10.01:2:3:4:5:6:7:8::44.08 ns0.108 ns0.090 ns0.940.00-71 B-NA
TryParseBaseline.NET 9.0100:0(...):abcd [22]114.79 ns0.407 ns0.381 ns1.000.000.00313,013 B80 B1.00
TryParseCoreRun.NET 10.0100:0(...):abcd [22]106.68 ns0.706 ns0.660 ns0.930.010.003171 B80 B1.00
TryParseBaseline.NET 9.0192.168.0.0/1622.88 ns0.066 ns0.062 ns1.000.00-1,708 B-NA
TryParseCoreRun.NET 10.0192.168.0.0/1622.01 ns0.165 ns0.155 ns0.960.01-71 B-NA
TryParseBaseline.NET 9.0192.168.0.128.77 ns0.251 ns0.235 ns1.000.010.00151,750 B40 B1.00
TryParseCoreRun.NET 10.0192.168.0.127.31 ns0.282 ns0.264 ns0.950.010.001671 B40 B1.00
TryParseBaseline.NET 9.0429496729523.37 ns0.315 ns0.295 ns1.000.020.00161,766 B40 B1.00
TryParseCoreRun.NET 10.0429496729523.70 ns0.166 ns0.155 ns1.010.010.001671 B40 B1.00
TryParseBaseline.NET 9.0Fe08::1%1354275.05 ns0.848 ns0.793 ns1.000.010.00483,415 B120 B1.00
TryParseCoreRun.NET 10.0Fe08::1%1354263.02 ns0.704 ns0.658 ns0.840.010.003171 B80 B0.67
UriHostBaseline.NET 9.0http:(...)3/foo [39]410.91 ns0.532 ns0.444 ns1.000.000.007617,316 B192 B1.00
UriHostCoreRun.NET 10.0http:(...)3/foo [39]364.69 ns0.556 ns0.520 ns0.890.000.007618,560 B192 B1.00
UriHostBaseline.NET 9.0http:(...)3/foo [26]229.11 ns0.663 ns0.588 ns1.000.000.007215,314 B184 B1.00
UriHostCoreRun.NET 10.0http:(...)3/foo [26]229.23 ns0.923 ns0.863 ns1.000.000.007216,081 B184 B1.00

Pending any surprises from CI, I'll re-request the benchmark from EgorBot.

* Replaced int.CreateTruncating with a JIT pattern which converts from TChar to a ushort.* Removed various TChar.CreateTruncating constants, reinstating direct character constants.* Replaced branch statements when parsing IPv4 numerics with a HexConverter lookup.* IPv4 parts stored in an array of longs to improve register usage.* Small improvement in reassembly to eliminate extra array access.* Reverted changes to Uri parsing layer, removing the translation logic between a Span and a pointer for IPv4 addresses.
Copy link
Member

@MihaZupanMihaZupan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thank you

* Replaced ushorts with integers to remove some zero-extensions.* Replaced NativeMemory with an ArrayPool<byte>.* Removed unnecessary assignment to ch.
@MihaZupan
Copy link
Member

@EgorBot

usingBenchmarkDotNet.Attributes;usingBenchmarkDotNet.Running;usingSystem;usingSystem.Net;publicclassBench{[Benchmark][Arguments("192.168.0.1")][Arguments("4294967295")][Arguments("037777777777")][Arguments("0xff.0x7f.0x20.0x01")][Arguments("192.168.0.0/16")][Arguments("::192.168.0.1")][Arguments("100:0:1:2:0:0:000:abcd")][Arguments("Fe08::1%13542")][Arguments("1:2:3:4:5:6:7:8::")]publicboolTryParse(strings)=>IPAddress.TryParse(s,out_);[Benchmark][Arguments("http://192.168.0.1:123/foo")][Arguments("http://[100:0:1:2:0:0:000:abcd]:123/foo")]publicstringUriHost(strings)=>newUri(s).Host;}
EgorBot reacted with thumbs up emoji

Copy link
Member

@MihaZupanMihaZupan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks for sticking through all the reviews, change & perf LGTM

edwardneal reacted with hooray emoji
@MihaZupanMihaZupan merged commitf199591 intodotnet:mainOct 24, 2024
@MihaZupanMihaZupan added this to the10.0.0 milestoneOct 24, 2024
@edwardneal
Copy link
ContributorAuthor

That's great, thanks for your patience@MihaZupan and everyone who reviewed. There are a few optimisations which I want to pick up from the earlier revisions, but I'll definitely re-read the earlier conversations on this PR first - I'm fairly sure I dragged the review process out by missing the point of some of them!

@edwardnealedwardneal deleted the issue-81500-ipaddress-ipnetwork branchOctober 24, 2024 16:55
@github-actionsgithub-actionsbot locked and limited conversation to collaboratorsNov 24, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account?Sign in.

Reviewers

@stephentoubstephentoubstephentoub left review comments

@jkotasjkotasjkotas left review comments

@tannergoodingtannergoodingtannergooding left review comments

@MihaZupanMihaZupanMihaZupan approved these changes

Labels

area-System.Netcommunity-contributionIndicates that the PR has been added by a community membernew-api-needs-documentation

Projects

None yet

Milestone

10.0.0

Development

Successfully merging this pull request may close these issues.

[API Proposal]: Implement IUtf8SpanParsable on IPAddress & IPNetwork

11 participants

@edwardneal@tannergooding@karelz@neon-sunset@MihaZupan@EgorBot@EgorBo@ManickaP@stephentoub@liveans@jkotas

[8]ページ先頭

©2009-2025 Movatter.jp