Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Best effort serialization#1892

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Conversation

@BadSingleton
Copy link
Contributor

@BadSingletonBadSingleton commentedJul 29, 2022•
edited
Loading

What does this implement/fix? Explain your changes.

This PR adds a "last line of defense, best effort serialization" to serialize types not marked as Serializable. Such objects, when derivable, are deserialized as derived classes with all (overridable) methods and properties overriden to throw a "Not Serialized" Exception. Fields are not initialized and may be null. Instances of classes that are not derivable are null.

Does this close any currently open issues?

No, but it is an issue we've encountered with Unity.

Any other comments?

This is a "best effort" to ensure no SerializationExceptions are thrown on domain unloads/reloads. This should help users deal with serialization issues as sometimes unserializable classes may be in compiled libraries and therefore unmodifiable.

Checklist

Check all those that are applicable and complete.

  • Make sure to include one or more tests for your change
  • If an enhancement PR, please create docs and at best an example
  • Ensure you have signed the.NET Foundation CLA
  • Add yourself toAUTHORS
  • Updated theCHANGELOG

This commit adds a "last line of defense, best effort serialization"to serialize types not marked as Serializable. Such objects aredeserialized as derived classes with all methods and propertiesoverriden to throw a "Not Serialized" Exception.Fields are not initialized and may be null.Sealed classes and implemented interface methods are still a problem tobe solved.
plus guard rails for private classesand review fixes
don't try to derive from sealed types
@BadSingleton
Copy link
ContributorAuthor

BadSingleton commentedJul 29, 2022•
edited
Loading

Also just a note that I'll soon be away for vacations until october, I just want to get that ball rolling and gather feedback in the meantime.

@lostmsu
Copy link
Member

lostmsu commentedJul 29, 2022•
edited
Loading

@BadSingleton is there a way to provide that functionality without bringing theNonSerializedTypeBuilder into Python.NET? E.g. can we allow user to override the serializer somehow, and you guys would be able to keep this code inside Unity or as a new subproject like pythonnet/unity-serializer?

@lostmsu
Copy link
Member

Also, can you describe the approach in more details?

@BadSingleton
Copy link
ContributorAuthor

@lostmsu I think the serializer was meant to be overridable in the first implementations of the serialization. I think I recall@amos402 saying something about the need for users to implement their own. If it becomes overridable we probably can keep it outside of this repo (just need to make sure with some order of initialization things)

The idea was to ensure that the presence of a non-serializable object in the graph wouldn't throw an exception, therefore corrupting the whole shutdown process and causing crashes on domain (un)load. As sometimes those objects are outside of our control, we can't modify them and/or avoid easily to serialize them (and/or the users of our APIs may be passing a Unity object that is not serializable). After a few google searches I found out that serialization surrogates could be used to serialize objects not marked[Serializable]. Since these objects are not meant to be serialized, but they're in the object graph that is being serialized, we serialize the type, not the object. When deserializing, there are two possibilities:

  1. The object can be derived from, and we add_NotSerialized suffix to the class name. If it is, every method and property (get and set) are overridden by a simple function that throws a NotSerializedException (the inspiration for the exception message comes from PyQt. Those who knows, know the pain.). Fields are left null.
  2. The object cannot be derived from (private, internal, ...). Not much to do here, except returning a null reference.

When serializing a derived_NotSerialized class, we serialize the base class to avoid having_NotSerialized_NotSerialized_... looping.

@BadSingleton
Copy link
ContributorAuthor

Ignore the part of the previous comment about the serializer not being overridable, I misremembered the code, itis overridable.

@lostmsu
Copy link
Member

lostmsu commentedJul 29, 2022•
edited
Loading

It seems that this code can be abstracted out of this repo with:

  1. Having a public post stash and pre restore hooks
  2. Exposing a wrapper for PyCapsule and maybe PyMem (though I think it can be replaced withMarshal.AllocHGlobal)
  3. ReplacingFormatterType property withFormatterFactory
  4. ExposingMaybeType

@BadSingleton
Copy link
ContributorAuthor

Worst case,MaybeType is easily re-implementable. I'm not sure about exposing it.

@larryPlayabl
Copy link

I am also experiencing this issue when trying to use this library with Unity3D. Here is the specific serialization error I get on shutdown.@BadSingleton, does your fix prevent this and allow you to initialize and shutdown the python engine multiple times without error?

Got exception during shutdown: Message:Type '__System_Action`1\[\[System_String\, mscorlib\, Version=4_0_0_0\, Culture=neutral\, PublicKeyToken=b77a5c561934e089\]\]Dispatcher' in Assembly '__Python_Runtime_Generated_Assembly0, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' is not marked as serializable.  at System.Runtime.Serialization.FormatterServices.InternalGetSerializableMembers (System.RuntimeType type) [0x00045] in <4c36328e5e3c40ecaf35733ee9461b26>:0   at System.Runtime.Serialization.FormatterServices+<>c__DisplayClass9_0.<GetSerializableMembers>b__0 (System.Runtime.Serialization.MemberHolder _) [0x00000] in <4c36328e5e3c40ecaf35733ee9461b26>:0   at System.Collections.Concurrent.ConcurrentDictionary`2[TKey,TValue].GetOrAdd (TKey key, System.Func`2[T,TResult] valueFactory) [0x00034] in <4c36328e5e3c40ecaf35733ee9461b26>:0   at System.Runtime.Serialization.FormatterServices.GetSerializableMembers (System.Type type, System.Runtime.Serialization.StreamingContext context) [0x0005e] in <4c36328e5e3c40ecaf35733ee9461b26>:0   at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.InitMemberInfo () [0x0003d] in <4c36328e5e3c40ecaf35733ee9461b26>:0   at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.InitSerialize (System.Object obj, System.Runtime.Serialization.ISurrogateSelector surrogateSelector, System.Runtime.Serialization.StreamingContext context, System.Runtime.Serialization.Formatters.Binary.SerObjectInfoInit serObjectInfoInit, System.Runtime.Serialization.IFormatterConverter converter, System.Runtime.Serialization.Formatters.Binary.ObjectWriter objectWriter, System.Runtime.Serialization.SerializationBinder binder) [0x00158] in <4c36328e5e3c40ecaf35733ee9461b26>:0   at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.Serialize (System.Object obj, System.Runtime.Serialization.ISurrogateSelector surrogateSelector, System.Runtime.Serialization.StreamingContext context, System.Runtime.Serialization.Formatters.Binary.SerObjectInfoInit serObjectInfoInit, System.Runtime.Serialization.IFormatterConverter converter, System.Runtime.Serialization.Formatters.Binary.ObjectWriter objectWriter, System.Runtime.Serialization.SerializationBinder binder) [0x00006] in <4c36328e5e3c40ecaf35733ee9461b26>:0   at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Write (System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo objectInfo, System.Runtime.Serialization.Formatters.Binary.NameInfo memberNameInfo, System.Runtime.Serialization.Formatters.Binary.NameInfo typeNameInfo) [0x00101] in <4c36328e5e3c40ecaf35733ee9461b26>:0   at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Serialize (System.Object graph, System.Runtime.Remoting.Messaging.Header[] inHeaders, System.Runtime.Serialization.Formatters.Binary.__BinaryWriter serWriter, System.Boolean fCheck) [0x001d3] in <4c36328e5e3c40ecaf35733ee9461b26>:0   at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize (System.IO.Stream serializationStream, System.Object graph, System.Runtime.Remoting.Messaging.Header[] headers, System.Boolean fCheck) [0x0006e] in <4c36328e5e3c40ecaf35733ee9461b26>:0   at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize (System.IO.Stream serializationStream, System.Object graph, System.Runtime.Remoting.Messaging.Header[] headers) [0x00000] in <4c36328e5e3c40ecaf35733ee9461b26>:0   at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize (System.IO.Stream serializationStream, System.Object graph) [0x00000] in <4c36328e5e3c40ecaf35733ee9461b26>:0   at Python.Runtime.RuntimeData.Stash () [0x00049] in <17d3e183cc45426f82a1a9207d7443a1>:0   at Python.Runtime.Runtime.Shutdown () [0x00030] in <17d3e183cc45426f82a1a9207d7443a1>:0   at Python.Runtime.PythonEngine.Shutdown () [0x00064] in <17d3e183cc45426f82a1a9207d7443a1>:0

@BadSingleton
Copy link
ContributorAuthor

Hello,@larryPlayabl. It has been a while, but if my memory serves right, this looks like the exact error this fix was made for. For added convenience, Unity's Python Scripting package has already that fix since version 6.0.0

larryPlayabl reacted with thumbs up emoji

@BadSingletonBadSingleton mentioned this pull requestMar 7, 2024
5 tasks
@BadSingleton
Copy link
ContributorAuthor

Closing, superseded by#2336

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@BadSingleton@lostmsu@larryPlayabl
[8]ページ先頭
©2009-2026 Movatter.jp