- Notifications
You must be signed in to change notification settings - Fork5.2k
CryptoStream Memory-based ReadAsync/WriteAsync overrides#47207
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
ghost commentedJan 20, 2021
Note regarding the This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, to please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change. |
ghost commentedJan 20, 2021
Tagging subscribers to this area:@bartonjs,@vcsjones,@krwq Issue DetailsWhile working onOverride Stream ReadAsync/WriteAsync, I found thatCryptoStream is missing the memory-based async overrides. I was able to implement the memory-based I went ahead and implemented I don't suppose it's feasible to add span-based overloads to
|
vcsjones commentedJan 20, 2021
This was discussed in#38764 and was ultimately determined DIMs on this interface were too risky. |
| varrentedBuffer=ArrayPool<byte>.Shared.Rent(inputBuffer.Length); | ||
| inputBuffer.CopyTo(rentedBuffer); | ||
| intresult=transform.TransformBlock(rentedBuffer,0,inputBuffer.Length,outputBuffer,outputOffset); | ||
| ArrayPool<byte>.Shared.Return(rentedBuffer); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
TherentedBuffer contains sensitive data and is being returned to a shared pool. We should clear the array before returning it to the pool so that when some other API rents it, it isn’t populated with the sensitive data. See other usages ofCryptographicOperations.ZeroMemory in this class for an example.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Fixed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
If this is a real concern, CryptoStream should probably override CopyTo/CopyToAsync as well, to clear the temporary buffer that might be used as part of the copy.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I can do that. I'll implement all suggested changes once I figure out why it's failing CI. It passed when I ran the tests on my machine. I must be doing something wrong.
| { | ||
| CheckReadArguments(buffer,offset,count); | ||
| returnReadAsyncInternal(buffer,offset,count,cancellationToken); | ||
| returnReadAsync(buffer.AsMemory(offset,count),cancellationToken).AsTask(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Possible breaking change: if somebody subclassesCryptoStream and overridesReadAsync(Memory<byte>, ...) as a wrapper aroundReadAsync(byte[], ...), the code will now stack overflow. I don't know if anybody is likely to have done this in practice. But it's a potential hazard of having one virtual method begin to dispatch to a different already-existing virtual method.
@bartonjs is there a pattern for handling this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
If we're concerned about that, both overloads of ReadAsync can delegate to a non-virtual ReadAsyncCore.
I've never seen a type derived from CryptoStream, though. I'm sure someone somewhere has done it, but I don't think we need to be too concerned (famous last words). If we were actually concerned, we'd potentially want to take it a step further and have the ReadAsync memory overload check whether the type was derived or not, delegating to the base implementation if it was, in case someone had overridden ReadAsync(byte[], ...) to do something special, in which case it would be a breaking change for ReadAsync(Memory, ...) to not use it.
NewellClarkJan 20, 2021 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Would we want to do reflection to see if the derived type has actually overridden ReadAsync(byte[]...) so we can possibly skip the memory copy?
stephentoubJan 20, 2021 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Would we want to do reflection
The answer to that question is pretty much always "no" 😄
| returnresult; | ||
| } | ||
| catch |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The catch block here is unnecessary. Instead, consider restructuring the outer try block as the following pseudocode:
byte[]rented=ArrayPool.Rent();try{input.CopyTo(rented);Transform(from:rented,to:output);}finally{ZeroMem(rented);}ArrayPool.Return(rented);
By putting a finally block around the code where the rented buffer contains potentially sensitive data, we can ensure that the whole thing is zeroed out whether the operation completes successfully or fails.
We could also consider pinning the temporary buffer as part of this operation to provide further protection against copies of the data being made.
Note to other reviewers: Should we skip the array pool entirely and instead use pre-pinned arrays? If an adversary can force the crypto transform to fail, they can force the application to abandon a bunch of Gen2 arrays, which could lead to perf degradation.
...ies/System.Security.Cryptography.Primitives/src/System/Security/Cryptography/CryptoStream.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
| { | ||
| CheckReadArguments(buffer,offset,count); | ||
| returnReadAsyncInternal(buffer,offset,count,cancellationToken); | ||
| returnReadAsync(buffer.AsMemory(offset,count),cancellationToken).AsTask(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
If we're concerned about that, both overloads of ReadAsync can delegate to a non-virtual ReadAsyncCore.
I've never seen a type derived from CryptoStream, though. I'm sure someone somewhere has done it, but I don't think we need to be too concerned (famous last words). If we were actually concerned, we'd potentially want to take it a step further and have the ReadAsync memory overload check whether the type was derived or not, delegating to the base implementation if it was, in case someone had overridden ReadAsync(byte[], ...) to do something special, in which case it would be a breaking change for ReadAsync(Memory, ...) to not use it.
| varrentedBuffer=ArrayPool<byte>.Shared.Rent(inputBuffer.Length); | ||
| inputBuffer.CopyTo(rentedBuffer); | ||
| intresult=transform.TransformBlock(rentedBuffer,0,inputBuffer.Length,outputBuffer,outputOffset); | ||
| ArrayPool<byte>.Shared.Return(rentedBuffer); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
If this is a real concern, CryptoStream should probably override CopyTo/CopyToAsync as well, to clear the temporary buffer that might be used as part of the copy.
…em/Security/Cryptography/CryptoStream.csUse explicit typeCo-authored-by: Stephen Toub <stoub@microsoft.com>
Unintentional breaking change with exceptions that was blocking CI. Note to self: remember to rebuild before running unit tests.
Still need to override CopyTo/CopyToAsync. I just want to make sure that CI will pass with changes so far.
| // and return | ||
| Buffer.BlockCopy(buffer,offset,_inputBuffer,_inputBufferIndex,count); | ||
| _inputBufferIndex+=count; | ||
| buffer.Slice(0,buffer.Length).CopyTo(_inputBuffer.AsMemory(_inputBufferIndex)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
| buffer.Slice(0,buffer.Length).CopyTo(_inputBuffer.AsMemory(_inputBufferIndex)); | |
| buffer.CopyTo(_inputBuffer.AsMemory(_inputBufferIndex)); |
| } | ||
| else | ||
| { | ||
| varrentedBuffer=ArrayPool<byte>.Shared.Rent(inputBuffer.Length); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
var shouldn't be used here, and please add the comment explaining why this isn't using CryptoPool:
| varrentedBuffer=ArrayPool<byte>.Shared.Rent(inputBuffer.Length); | |
| // UseArrayPool.Shared instead of CryptoPool because the array is passed out. | |
| byte[]rentedBuffer=ArrayPool<byte>.Shared.Rent(inputBuffer.Length); |
| } | ||
| finally | ||
| { | ||
| CryptographicOperations.ZeroMemory(rentedBuffer.AsSpan(0,inputBuffer.Length)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
If the array is being pinned for the operation that's presumably so that it can be populated and cleared without GC compaction applying, so I'd expect the clear to be inside the pin.
- Applied fixes suggested by@bartonjs- Added overrides for CopyTo/CopyToAsync to ensure any temporary buffers are properly cleared.
NewellClark commentedJan 21, 2021 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
@bartonjs I've applied the changes you've suggested Quick question: when I apply a change that a reviewer suggested, should I click "resolve conversation", or wait for them to do it? |
NewellClark commentedJan 21, 2021
Just hit issue#32805. Rerunning. |
...ies/System.Security.Cryptography.Primitives/src/System/Security/Cryptography/CryptoStream.csShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
…em/Security/Cryptography/CryptoStream.cs
...ies/System.Security.Cryptography.Primitives/src/System/Security/Cryptography/CryptoStream.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
...aries/System.Security.Cryptography.Primitives/ref/System.Security.Cryptography.Primitives.cs OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
…em.Security.Cryptography.Primitives.cs
While working onOverride Stream ReadAsync/WriteAsync, I found thatCryptoStream is missing the memory-based async overrides.
I was able to implement the memory-based
ReadAsyncwithout issue. However,WriteAsynchas a problem because it calls intoTransformBlockonICryptoTransform, which is a public interface. Ouch.I went ahead and implemented
WriteAsync, and where calls toICryptoTransform.TransformBlockwere made, I check if the underlyingReadOnlyMemoryis actually an array skip the copy when it is.I don't suppose it's feasible to add span-based overloads to
ICryptoTransformas default interface implementations? I'd be willing to write up the API proposal in the case that it is.