IMO, this needs a NEWS blurb. I'd even add a What's New entry --_pointer_type_cache is nominally internal API, but it's been there for a long time and people are using it.

Thanks! I'll add both entries later today.

I have added news and whatsnew entries, but I believe we should update them after we decided what to do with _ctypes_ptrtype_cache behaviour.

As far as I can see, this numeric ID key is only used for a check, and I think that it's OK to remove a check for obsolete functionality from a deprecated function.

The feature is "incomplete pointers" -- ones that were created with a name, but weren't assigned an underlying type yet. Seetest_ctypes.test_incomplete.py for the intended usage. That test references "the tutorial", but the current tutorial doesn't include this -- instead, you can createincompletestructs for this use case.
Back when ctypes was an independent project, thisstarted working in ctypes 0.6.3 (2003) andwas made obsolete in 0.9.5 (2005).ctypes was added to Python 2.5 (2006), withoutSetPointerType mentioned in docs.

What I'm a bit worried about is the duplication between_pointer_type_cache andinfo->pointer_type. Which one should have priority? When one is updated, should the other one be updated too?
What kind of relationship isexpected and/orguaranteed are betweeninfo->proto (ptr_type._type_) andinfo->pointer_type?

Should we expose the pointer type as a Python attribute?
If we do, with a setter to allow registration, could_pointer_type_cache be turned into a deprecated proxy for that? Something like:

classPointerTypeCache:def__setitem__(self,cls,pointer_type):cls.pointer_type=pointer_typedef__getitem__(self,cls):try:returncls.pointer_typeexceptAttributeError:raiseKeyError(cls)def__contains__(self,cls):returnhasattr(cls,'pointer_type')_pointer_type_cache=PointerTypeCache()

Would having all the info in one place be worth that?

Or am I overthinking it, and we should fix the immediate issue and merge this PR as is?
Entirely possible, but I'd like to have solid understanding first.

What I'm a bit worried about is the duplication between_pointer_type_cache andinfo->pointer_type. Which one should have priority? When one is updated, should the other one be updated too? What kind of relationship isexpected and/orguaranteed are betweeninfo->proto (ptr_type._type_) andinfo->pointer_type?

AFAIU, if we introduceinfo->pointer_type then the only use for_pointer_type_cache is a manual registration like forcomtypes, wherePOINTER for non-ctypes type created and registered in the cache. And for incomplete types as well. So, since we createPOINTER type from our factory, it doesn't matter how_pointer_type_cache andinfo->pointer_type are related. They storedifferent data. Also, I expect that ctypesPOINTER types will not be registered in_pointer_type_cache and I see the only way to control this is to deprecate_pointer_type_cache and add a registration function, that should check where to store thePOINTER type (likeregister_pointer_type(type, pointer_type) and iftype is a non-ctypes then register their pointer in_pointer_type_cache, also same for incomplete types). So, I expect no regular ctypes pointer types will be stored in_pointer_type_cache.

I'm not that familiar withinfo->proto andptr_type._type_. I need to dig deeper to answer this.

Should we expose the pointer type as a Python attribute? If we do, with a setter to allow registration, could_pointer_type_cache be turned into a deprecated proxy for that? Something like:

If we allow to poison regular Python classes with ourpointer_type attribute, then it is worth idea. But this is will not work for incomplete types (causecls will bestr for all of them), we should strictly check thatcls is not a builtin type and so on. I'm not sure it will be an easy way to go :)

Just in my gut feeling, but would the following make sense?

• EnhancePyCPointerType.set_type() (currently used in the deprecatedSetPointerType()) to storepointer_type in stginfo.

• Make_pointer_type_cache a dict subclass, where the__setitem__() uses theset_type(). The dict would be like{None: c_void_p}.

Just in my gut feeling, but would the following make sense?

• EnhancePyCPointerType.set_type() (currently used in the deprecatedSetPointerType()) to storepointer_type in stginfo.
• Make_pointer_type_cache a dict subclass, where the__setitem__() uses theset_type(). The dict would be like{None: c_void_p}.

Yeah, this is like@encukou proposed for PointerTypeCache with addition of mapping interface that would be useful for introspection for example. But please see my reply about incomplete types and non-ctypes types above.

As far as I can see, only three kinds of keys are acceptable in_pointer_type_cache:

• a ctypes type, which do have stginfo. Note that this includes comtypes (those inherit from ctypes)
• aninteger -- theid of an incomplete pointer. AFAICS, such an entry in_pointer_type_cache is only used as a “flag” to allowSetPointerType to work; this info might as well be stored in stginfo instead. (I imagine it's equivalent toinfo->proto == NULL, but I haven't checked for further complications.)
• None, a special case that can be handled elsewhere

Is there a use case for putting something else in_pointer_type_cache? If we can't find one, let's not support it. We don't want to break comtypes and such (i.e. users who don't have another way of doing what they need), but on the other hand,_pointer_type_cacheis an internal implementation detail that's subject to change.

Make _pointer_type_cache a dict subclass

A dict-like object, please. Subclassingdict is too fragile, plus this proxy this can't support iteration.

There's one more aspect to think about: the registration that comtypes does isn't really the right thing for comtypes to do.

comtypes needs to add some functionality to all of its types, so it defines subclasses of the ctypes (meta)classes. But, it also wants to add some functionality topointers to its types, and pointers are normally created on demand.
So, for every type ctypes defines, it also defines a pointer type, and registers that. It only does that for one “level” though: a pointer to apointer to a comtype is a plain old ctypes pointer. comtypes could register another level, but not infintely many of them.
IMO, the proper solution here would be to add amake_pointer_type method to the metaclasses, which comtypes could override, so that its pointer types are created on demand using the regular mechanism.
This would make registration unnecessary for comtypes.

I can't rule out another use case for registration, but I don't see one. Perhaps we can deprecate_pointer_type_cache and plan to remove it entirely unless someone speaks up.
(Deprecation/removal would also be easier if it becomes a simple proxy forset_type & co., rather than a source of truth.)

I'm very new to the internals of ctypes, but I agree with three of the use cases of_pointer_type_cache that you mentioned.
One comment: I may not have checked correctly. I remember that when I analyzedPyCSimpleTypeAsMetaclassTest, I saw that a non-ctypescls was passed toPOINTER.
As far as I can tell, this is because the codeptr_bases = (self, POINTER(bases[0]) called with CtBase that hasn't StgInfo. I'm not sure that I understand how we can replace this with regular pointer type creation and avoid registration.

Ah! Looks like you're correct, thanks!

I'm not on a Windows box now to check.
I believeCtBase mirrors comtypes'IUnknown, which hasan interesting comment:

        `IUnknown` behaves as a ctypes type, and `POINTER` can take it.        This behavior is defined by some metaclasses in runtime.

So, IUnknown is a ctypes type in spirit, but not in reality :)

We could exposeinfo->pointer_type as a reserved Python attribute__pointer_type__, and for classes withoutstginfo, look up the attribute. Do you think that's worth pursuing?
I don't think we need to worry builtin types (and others that disable setting attributes).

Some concerns about having_pointer_type_cache store only the manually registered types:

• lookups in_pointer_type_cache, as a way to check if a pointer type is assigned (without creating the type if it's not), stop working.
• iteration over_pointer_type_cache will give fewer results.

But, given that it is an internal implementation detail, perhaps that's acceptable.

I sent a few more suggestions as a PR

For me,test_pointers.py on the PR sometimes fails.PyCPointerType.set_type() can update a pointer type'sattrdict through the_type_ key, which is not always reflected when accessing the attribute.

Looks like that's a 3.13 regression. I hope to fix it later today.

As I wrote yesterday this should fix itsergey-miryanov@aa44997

PyType_Modified works now, but setattr is safer; see#133292 (separate PR since it should be backported)

PyType_Modified works now, but setattr is safer; see#133292 (separate PR since it should be backported)

Thanks! I'll keep it in mind.

🤖 New build scheduled with the buildbot fleet by@encukou for commit3129ef5 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F131282%2Fmerge

If you want to schedule another build, you need to add the🔨 test-with-buildbots label again.

Thanks! It wasn't straightforward, but I think everyone learned a bunch, and the issue is fixed :)
Let's merge if the buildbots don't find anything.

@encukou,@neonene,@junkmd Thanks a lot for review, help, support and commenting. A really appreciate it. And I fully agree that I learned a lot working on this issue.

@kumaraditya303 Could you check the__pointer_type__ getset as well for free-threaded builds?

I'll look into this by next week.

Created thread safety fix here#133843