Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

implement list codec#1084

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
lostmsu merged 25 commits intopythonnet:masterfromkoubaa:list-codec
Feb 18, 2021
Merged

implement list codec#1084

lostmsu merged 25 commits intopythonnet:masterfromkoubaa:list-codec
Feb 18, 2021

Conversation

koubaa
Copy link
Contributor

@koubaakoubaa commentedMar 10, 2020
edited
Loading

What does this implement/fix? Explain your changes.

Implement conversions from python lists, iterables, and sequences to appropriate C# interfaces
...

Does this close any currently open issues?

Probably a few of them. I'll look for it later.

Any other comments?

I've done the codec for three levels of collections. roughly I map these:

pythonC#
iterableIEnumerable
sequenceICollection
listIList

The mapping isn't exactly perfect, naturally. But this is the best I think I can do right now.

I need to add some more codec unit tests and functional embedding tests.

Checklist

Check all those that are applicable and complete.

  • Make sure to include one or more tests for your change
  • If an enhancement PR, please create docs and at best an example
  • Add yourself toAUTHORS
  • Updated theCHANGELOG

lostmsu and jaqueamo reacted with thumbs up emoji
@koubaakoubaa changed the titlebegin to implement list codecimplement list codecMar 11, 2020
@koubaa
Copy link
ContributorAuthor

@amos402@filmor@lostmsu please review. I think this PR may help with our implicit conversions of lists.

@koubaakoubaa mentioned this pull requestMar 11, 2020
4 tasks
@lostmsu
Copy link
Member

lostmsu commentedMar 11, 2020
edited
Loading

Haven't done a full pass yet, but instead of doing "levels" you can just have 3 decoders, and register them in order from highest priority to lowest priority. E.g.

Register(ListEncoder.Instance);Register(CollectionDecoder.Instance);Register(IterableDecoder.Instance);

Also, please, do not dump all classes in the encoder file.

@koubaa
Copy link
ContributorAuthor

@lostmsu I can change the physical layout but wanted to make sure I was on the right track first. I like the idea of having multiple codecs with priority, that way clients can choose which encoder they need. I'll work on that.


//we'd have to copy into a list to do this. not the best idea to support it.
//maybe there can be a flag on listcodec to allow it.
Assert.IsFalse(codec.CanDecode(x, typeof(List<int>)));
Copy link
ContributorAuthor

@koubaakoubaaMar 11, 2020
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

@lostmsu With your idea of multiple codecs I can have a DeepCopyListCodec that takes any ienumerable and converts it by copying to a list. This is what we do right now in master and if we get rid of that implicit conversion the user can just register this codec if that causes an undesirable behavior change for them.@filmor what do you think?

Copy link
Member

@lostmsulostmsuMar 11, 2020
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I am thinking, thatPython.Runtime.dll should only contain codecs, that perform lossless conversions, perhaps even enabled by default. E.g. if a type is exactlybuiltins.list it is safe to convert it toIList<T> via some safe wrapper typePythonList<T>, that would contain a reference to the original object. Preferably, when this wrapper is passed back to Python it should either just pass the wrapped object, or behave exactly like a normallist would.

The codecs, that are not lossless, like converting abuiltins.list toT[], and codecs, that act on relatively uncommon types likeSystem.Uri should go to a separate extra package (created yesterday after seeing your PR):https://github.com/pythonnet/codecs

Copy link
ContributorAuthor

@koubaakoubaaMar 22, 2020
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

@lostmsu done. I didn't go as far as to enable by default but that can be done as a followup PR. Please look closely at the test cases as I tried to capture the behavior of each of the three codecs to make sure it meets your expectation.

@koubaa
Copy link
ContributorAuthor

This is related to#912

@koubaa
Copy link
ContributorAuthor

@lostmsu While the unit tests work when I tried it from python I realized (again) that CanDecode is based on the type rather than the instance, so I changed that. Also I added some python tests

@amos402
Copy link
Member

If you're considering the performance improvement, this branch may help you.
https://github.com/joonhwan/pythonnet/tree/fast-call
It's an ancient branch I deleted a long time ago, my current internal branch has far more than that.
You can do some performance tests, like method calls, properties get-set, etc, you will see that the branch above has a lot faster than the master now. The simple points are avoiding anyMethodInfo.Invoke, use pre-created delegates instead, prevent any boxing or unboxing if you can. After that, if you want a higher than it, code generation is an option.

Thecodec has some design points can improve:

  • Decode,Encode usedobject, which means it may occur boxing.
  • When itdecode/encode an object, it has iteration operations which can be avoided, or use some hash table for getting the coder directly.
  • For achieving no reflection calling, no boxing or unboxing, you may need to redesign the method calling mechanism just like I did, you can check themethodobject.cs on the above branch.

Copy link
ContributorAuthor

@koubaakoubaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

@lostmsu could you please take another look at this?

@koubaa
Copy link
ContributorAuthor

If you're considering the performance improvement, this branch may help you.
https://github.com/joonhwan/pythonnet/tree/fast-call
It's an ancient branch I deleted a long time ago, my current internal branch has far more than that.
You can do some performance tests, like method calls, properties get-set, etc, you will see that the branch above has a lot faster than the master now. The simple points are avoiding anyMethodInfo.Invoke, use pre-created delegates instead, prevent any boxing or unboxing if you can. After that, if you want a higher than it, code generation is an option.

Thecodec has some design points can improve:

  • Decode,Encode usedobject, which means it may occur boxing.
  • When itdecode/encode an object, it has iteration operations which can be avoided, or use some hash table for getting the coder directly.
  • For achieving no reflection calling, no boxing or unboxing, you may need to redesign the method calling mechanism just like I did, you can check themethodobject.cs on the above branch.

Thank you for sharing! I haven't measured performance issues but now I know what to try if I do.

@koubaa
Copy link
ContributorAuthor

@lostmsu would you mind reviewing this again? I resolved all of your previous comments

@lostmsu
Copy link
Member

@koubaa sure, but can't do it until Saturday.

@koubaa
Copy link
ContributorAuthor

@lostmsu I'll look to enable these by default (and handle issues associated w/that) in a later PR

@koubaa
Copy link
ContributorAuthor

@lostmsu I rebased to fix the merge conflict. Is there anything else needed before this can be merged?

@christabella
Copy link
Contributor

@koubaa do you plan to finish this PR? The open issues that this would close include
#909#601#962

@koubaa
Copy link
ContributorAuthor

@christabella I have been waiting for@lostmsu to merge it

@koubaa
Copy link
ContributorAuthor

@lostmsu I rebased the changes to resolve the conflicts. Is there anything needed before this can be merged?

@lostmsu
Copy link
Member

@koubaa@christabella once the CI tests pass, I can take this. Note, however, that I have a different approach in mind, based onmixins and base class faking (a feature I have in my fork, that is yet to be upstreamed, its implementation is relatively involved). So this will probably be replaced before 3.0 is released.

@koubaa
Copy link
ContributorAuthor

@lostmsu the CI is fixed

@lostmsulostmsu merged commit707ef36 intopythonnet:masterFeb 18, 2021
@koubaakoubaa deleted the list-codec branchFebruary 19, 2021 02:10
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@lostmsulostmsulostmsu requested changes

@amos402amos402amos402 left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

4 participants
@koubaa@lostmsu@amos402@christabella

[8]ページ先頭

©2009-2025 Movatter.jp