Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

CryptoStream Memory-based ReadAsync/WriteAsync overrides#47207

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
bartonjs merged 14 commits intodotnet:masterfromNewellClark:memorify-CryptoStream
Feb 26, 2021

Conversation

@NewellClark
Copy link
Contributor

While working onOverride Stream ReadAsync/WriteAsync, I found thatCryptoStream is missing the memory-based async overrides.

I was able to implement the memory-basedReadAsync without issue. However,WriteAsync has a problem because it calls intoTransformBlock onICryptoTransform, which is a public interface. Ouch.

I went ahead and implementedWriteAsync, and where calls toICryptoTransform.TransformBlock were made, I check if the underlyingReadOnlyMemory is actually an array skip the copy when it is.

I don't suppose it's feasible to add span-based overloads toICryptoTransform as default interface implementations? I'd be willing to write up the API proposal in the case that it is.

@ghost
Copy link

Note regarding thenew-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, to please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@ghost
Copy link

Tagging subscribers to this area:@bartonjs,@vcsjones,@krwq
See info in area-owners.md if you want to be subscribed.

Issue Details

While working onOverride Stream ReadAsync/WriteAsync, I found thatCryptoStream is missing the memory-based async overrides.

I was able to implement the memory-basedReadAsync without issue. However,WriteAsync has a problem because it calls intoTransformBlock onICryptoTransform, which is a public interface. Ouch.

I went ahead and implementedWriteAsync, and where calls toICryptoTransform.TransformBlock were made, I check if the underlyingReadOnlyMemory is actually an array skip the copy when it is.

I don't suppose it's feasible to add span-based overloads toICryptoTransform as default interface implementations? I'd be willing to write up the API proposal in the case that it is.

Author:NewellClark
Assignees:-
Labels:

area-System.Security,new-api-needs-documentation

Milestone:-

@vcsjones
Copy link
Member

I don't suppose it's feasible to add span-based overloads to ICryptoTransform as default interface implementations?

This was discussed in#38764 and was ultimately determined DIMs on this interface were too risky.

varrentedBuffer=ArrayPool<byte>.Shared.Rent(inputBuffer.Length);
inputBuffer.CopyTo(rentedBuffer);
intresult=transform.TransformBlock(rentedBuffer,0,inputBuffer.Length,outputBuffer,outputOffset);
ArrayPool<byte>.Shared.Return(rentedBuffer);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

TherentedBuffer contains sensitive data and is being returned to a shared pool. We should clear the array before returning it to the pool so that when some other API rents it, it isn’t populated with the sensitive data. See other usages ofCryptographicOperations.ZeroMemory in this class for an example.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Fixed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

If this is a real concern, CryptoStream should probably override CopyTo/CopyToAsync as well, to clear the temporary buffer that might be used as part of the copy.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I can do that. I'll implement all suggested changes once I figure out why it's failing CI. It passed when I ran the tests on my machine. I must be doing something wrong.

{
CheckReadArguments(buffer,offset,count);
returnReadAsyncInternal(buffer,offset,count,cancellationToken);
returnReadAsync(buffer.AsMemory(offset,count),cancellationToken).AsTask();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Possible breaking change: if somebody subclassesCryptoStream and overridesReadAsync(Memory<byte>, ...) as a wrapper aroundReadAsync(byte[], ...), the code will now stack overflow. I don't know if anybody is likely to have done this in practice. But it's a potential hazard of having one virtual method begin to dispatch to a different already-existing virtual method.

@bartonjs is there a pattern for handling this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

If we're concerned about that, both overloads of ReadAsync can delegate to a non-virtual ReadAsyncCore.

I've never seen a type derived from CryptoStream, though. I'm sure someone somewhere has done it, but I don't think we need to be too concerned (famous last words). If we were actually concerned, we'd potentially want to take it a step further and have the ReadAsync memory overload check whether the type was derived or not, delegating to the base implementation if it was, in case someone had overridden ReadAsync(byte[], ...) to do something special, in which case it would be a breaking change for ReadAsync(Memory, ...) to not use it.

Copy link
ContributorAuthor

@NewellClarkNewellClarkJan 20, 2021
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Would we want to do reflection to see if the derived type has actually overridden ReadAsync(byte[]...) so we can possibly skip the memory copy?

Copy link
Member

@stephentoubstephentoubJan 20, 2021
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Would we want to do reflection

The answer to that question is pretty much always "no" 😄

NewellClark reacted with laugh emoji

returnresult;
}
catch

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The catch block here is unnecessary. Instead, consider restructuring the outer try block as the following pseudocode:

byte[]rented=ArrayPool.Rent();try{input.CopyTo(rented);Transform(from:rented,to:output);}finally{ZeroMem(rented);}ArrayPool.Return(rented);

By putting a finally block around the code where the rented buffer contains potentially sensitive data, we can ensure that the whole thing is zeroed out whether the operation completes successfully or fails.

We could also consider pinning the temporary buffer as part of this operation to provide further protection against copies of the data being made.

Note to other reviewers: Should we skip the array pool entirely and instead use pre-pinned arrays? If an adversary can force the crypto transform to fail, they can force the application to abandon a bunch of Gen2 arrays, which could lead to perf degradation.

{
CheckReadArguments(buffer,offset,count);
returnReadAsyncInternal(buffer,offset,count,cancellationToken);
returnReadAsync(buffer.AsMemory(offset,count),cancellationToken).AsTask();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

If we're concerned about that, both overloads of ReadAsync can delegate to a non-virtual ReadAsyncCore.

I've never seen a type derived from CryptoStream, though. I'm sure someone somewhere has done it, but I don't think we need to be too concerned (famous last words). If we were actually concerned, we'd potentially want to take it a step further and have the ReadAsync memory overload check whether the type was derived or not, delegating to the base implementation if it was, in case someone had overridden ReadAsync(byte[], ...) to do something special, in which case it would be a breaking change for ReadAsync(Memory, ...) to not use it.

varrentedBuffer=ArrayPool<byte>.Shared.Rent(inputBuffer.Length);
inputBuffer.CopyTo(rentedBuffer);
intresult=transform.TransformBlock(rentedBuffer,0,inputBuffer.Length,outputBuffer,outputOffset);
ArrayPool<byte>.Shared.Return(rentedBuffer);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

If this is a real concern, CryptoStream should probably override CopyTo/CopyToAsync as well, to clear the temporary buffer that might be used as part of the copy.

NewellClarkand others added3 commitsJanuary 20, 2021 10:28
…em/Security/Cryptography/CryptoStream.csUse explicit typeCo-authored-by: Stephen Toub <stoub@microsoft.com>
Unintentional breaking change with exceptions that was blocking CI. Note to self: remember to rebuild before running unit tests.
Still need to override CopyTo/CopyToAsync. I just want to make sure that CI will pass with changes so far.
// and return
Buffer.BlockCopy(buffer,offset,_inputBuffer,_inputBufferIndex,count);
_inputBufferIndex+=count;
buffer.Slice(0,buffer.Length).CopyTo(_inputBuffer.AsMemory(_inputBufferIndex));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Suggested change
buffer.Slice(0,buffer.Length).CopyTo(_inputBuffer.AsMemory(_inputBufferIndex));
buffer.CopyTo(_inputBuffer.AsMemory(_inputBufferIndex));

}
else
{
varrentedBuffer=ArrayPool<byte>.Shared.Rent(inputBuffer.Length);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

var shouldn't be used here, and please add the comment explaining why this isn't using CryptoPool:

Suggested change
varrentedBuffer=ArrayPool<byte>.Shared.Rent(inputBuffer.Length);
// UseArrayPool.Shared instead of CryptoPool because the array is passed out.
byte[]rentedBuffer=ArrayPool<byte>.Shared.Rent(inputBuffer.Length);

}
finally
{
CryptographicOperations.ZeroMemory(rentedBuffer.AsSpan(0,inputBuffer.Length));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

If the array is being pinned for the operation that's presumably so that it can be populated and cleared without GC compaction applying, so I'd expect the clear to be inside the pin.

- Applied fixes suggested by@bartonjs- Added overrides for CopyTo/CopyToAsync to ensure any temporary buffers are properly cleared.
@NewellClark
Copy link
ContributorAuthor

NewellClark commentedJan 21, 2021
edited
Loading

@bartonjs I've applied the changes you've suggested
@stephentoub I've implemented overrides ofCopyTo andCopyToAsync to ensure temporary buffers are zeroed out.

Quick question: when I apply a change that a reviewer suggested, should I click "resolve conversation", or wait for them to do it?

@NewellClark
Copy link
ContributorAuthor

Just hit issue#32805. Rerunning.

@bartonjsbartonjs merged commita7669a7 intodotnet:masterFeb 26, 2021
@ghostghost locked asresolvedand limited conversation to collaboratorsMar 28, 2021
Sign up for freeto subscribe to this conversation on GitHub. Already have an account?Sign in.

Reviewers

@vcsjonesvcsjonesvcsjones left review comments

@GrabYourPitchforksGrabYourPitchforksGrabYourPitchforks left review comments

@stephentoubstephentoubstephentoub left review comments

@bartonjsbartonjsbartonjs approved these changes

Assignees

No one assigned

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@NewellClark@vcsjones@GrabYourPitchforks@stephentoub@bartonjs

[8]ページ先頭

©2009-2025 Movatter.jp