I just pushed a commit to get stuff compiling. Excited to review this!

One thing to note is that some things are directly translation from PyPy, others from CPython's. Also there's that difference I said on Gitter, regarding declaring a class like:

classCustom(_SimpleCData):pass...Traceback (mostrecentcalllast):File"<stdin>",line1,in<module>AttributeError:classmustdefinea'_type_'attribute

which here will succeed.
Besides this, I've tried to stay as close as CPython's behavior as possible.

What are the major blockers for this PR? I think I want to give this a try :)

What are the major blockers for this PR? I think I want to give this a try :)

Unfortunately there are several problems with the state of things and I haven't been able to continue working on this (and will not be able to do so for a while). The main problem is how it behaves differently from CPython's and PyPy's, a meta class should be added for Struct, CSimpleType, Pointer, ... (all but CFuntion), so that you can doc_int * 10 and return a proper array. In the current impltype(CSimpleType) == type, which does not has a__mul__ as required for ac_int, for instance.

@youknowone I tried to sync with master a while ago, but there was a problem and I got too busy to look at that again, so thanks for that! Are you planning into work on this? If so, I think there are some tests I ported from CPython that I altered just to see if the implementation was working-ish.

To be fully compatible to CPython and PyPy, the types must have a metaclass that's notPyType. I tried to work something there, but I couldn't find a solution. This happens because a base type has to implement some ops at type level, eg,c_int * 8 should call__mul__ from its meta (a yet non-existing classPrimitiveTypeMeta which is a subclass ofPyType) and return ac_array.

I am happy to help with discussion and pointers, but I'm afraid I don't have much time to code :/

@darleybarreto Hi, thank you for the comment. I am not planning for working on ctypes at the moment, but looking for a way to merge it as it is for future contributors - because you already worked really a lot of it - if it doesn't break things too much. How do you think about it?

You mean merging as is?

yes, if we can manage it not to break CI

I think that we need to do a few things to help future contributions in this code before merging:

1. Refactor types structure.
2. Documentation.
3. A thorough review to improve code quality.

Undoubtedly1 is the major blocker here. Since I don't know much of the internals of RustPython, I wasn't able to solve this problem. Perhaps you can give me pointers and I can work it out. PyPy does is basically this way:

class_CDataMeta(type):    ...def__mul__(self,length):returncreate_array_type(self,length)    ...classSimpleType(_CDataMeta):    ...classSimpleCData(...,metaclass=SimpleType):    ...

I couldn't find a way to maketype(SimpleCData) beSimpleType instead oftype.

Point2 is extremely important since this code is basically too experimental and not that much well written. I could gladly do it based on PyPy, CPython and documentations.
Point3 is somewhat important because I am not a very experienced Rust programmer, so if anyone could help me writing idiomatic andperformant Rust, I would appreciate!

Thank you for detailed explanation. The metaclass issue is recently solved. Here is the test:

$ cat t.py class _CDataMeta(type):    def __mul__(self, length):return create_array_type(self, length)class SimpleType(_CDataMeta):    passclass SimpleCData(object, metaclass=SimpleType):    passprint(type(SimpleCData))assert type(SimpleCData) == SimpleType$ cargo run t.py     Finished dev [unoptimized + debuginfo] target(s)in 0.06s     Running`target/debug/rustpython t.py`<class'__main__.SimpleType'>

I will look in the code for 3

Oh, I'm sorry, I meant to do the same thing in the Rust side.CPython does it in the C side of things by manually filling the type:

#defineMOD_ADD_TYPE(TYPE_EXPR,TP_TYPE,TP_BASE) \    do { \        PyTypeObject *type = (TYPE_EXPR); \        Py_SET_TYPE(type, TP_TYPE); \        type->tp_base = TP_BASE; \        if (PyModule_AddType(mod, type) < 0) { \            return -1; \        } \    } while (0)MOD_ADD_TYPE(&Simple_Type,&PyCSimpleType_Type,&PyCData_Type);

HerePyCSimpleType_Type is the meta andPyCData_Type is the base class.

by watchingrustpython_vm::builtins::pytype::new, I feel like that'd be simply done by replacingtyp field inPyObject, but not sure it actually will be that easy.
Because I don't understand this patch that much, would you make specific place in this PR that you need those type stuff? Then I will fill the fitting code there.

would you make specific place in this PR that you need those type stuff?

Let me get the CPython definitions as example

MOD_ADD_TYPE(&Struct_Type,&PyCStructType_Type,&PyCData_Type);MOD_ADD_TYPE(&Union_Type,&UnionType_Type,&PyCData_Type);MOD_ADD_TYPE(&PyCPointer_Type,&PyCPointerType_Type,&PyCData_Type);MOD_ADD_TYPE(&PyCArray_Type,&PyCArrayType_Type,&PyCData_Type);MOD_ADD_TYPE(&Simple_Type,&PyCSimpleType_Type,&PyCData_Type);MOD_ADD_TYPE(&PyCFuncPtr_Type,&PyCFuncPtrType_Type,&PyCData_Type);

Here we haveMOD_ADD_TYPE(&Class, &MetaClass, &BaseClass) right? Looking atSimple_Type (in RustPython this isPySimpleType inprimitive.rs), we see that its metaclass implements the__mul__, so we can havec_int * 8 just fine, wherec_int is subclass ofSimple_Type. I used a traitPyCDataSequenceMethods inbasics.rs to implement the__mul__, but doingimpl PyCDataSequenceMethods for PySimpleTypeMeta {} inprimitive.rs doesn't work becausePySimpleTypeMeta is not the metaclass ofPySimpleType.

I'm not sure why, but I cannot comment on the following reply

The safe expression of this type in Rust is Box<[u8]> which is exactly equivalent to this struct. Is this intended work to avoid somethiing like ownership check? I think there must be a better way.

The idea ofRawBuffer is to be a "view" of the external buffer, so when needed, we copy the bytes from that buffer. I think there's one case we need to copy data and another where we need a view.

What happens when the buffer is destroyed? Does it deletes only the pointer(as a reference) or also deletes the data it contains (as an owner)? If it does both, then does it need a flag to distinguish them?

I forgot where these two functions are used, but based on the docs:
from_buffer_copy:C.from_buffer_copy(object, offset=0) -> C instance\ncreate a C instance from a readable buffer
from_buffer:C.from_buffer(object, offset=0) -> C instance\ncreate a C instance from a writeable buffer

Yes, that's possible to adapt to all files. It is up to you for your convenience.dll module looked good to use it. And I am pretty sure every file can use it. But you don't need to use it if you don't feel like to do.

@youknowone I basically patched all your comments, you wrote:

no, it isn't. slice is a sort of view

For this code

fnobj_bytes(&self) ->BorrowedValue<[u8]>{PyRwLockReadGuard::map(self.data.borrow_value(), |x|unsafe{        slice::from_raw_parts(x.inner, x.size)}).into()}

What did you meant there?

There was a question asking if it does copy. The comment is an answer about it. Creating an slice object doesn't copy anything.

I added a C file that should be compiled as a shared lib and bundled together with the interpreter. Its_ctypes_test.c. How would one make this happen? Would it be some sort of build script usingcc? And we should be able to do this:

import_ctypes_test

build.rs file can contain any build script. Usingcc crate will be helpful.

@youknowone I haven't figured out a nice way to implement thefrom_buffer.from_buffer_copy is straight forward, you simply create a new instance (default args for both tp_new and init) and copy the bytes from the value to create an instance. For example:

data=b'data'ubyte=c_ubyte*len(data)byteslike=ubyte.from_buffer_copy(data)# an instance of ubyte made from data by copying things

from_buffer does a similar thing, but it doesn't copy things, it points to the source. So if you changebyteslike, you changedata.

My implementation offrom_buffer_copy is something in these lines

#[pyclassmethod]fnfrom_buffer_copy(cls:PyRef<Self>,obj:PyObjectRef,offset:OptionalArg,vm:&VirtualMachine,) ->PyResult{let buffer =try_buffer_from_object(vm,&obj)?;let opts = buffer.get_options().clone();let(size, offset) =Self::buffer_check(cls, opts, offset, vm);//ignore this for nowlet src_buffer = buffer.obj_bytes();let empty_instance = ...//creates a new empty instance    letdst_buffer = empty_instance.obj_bytes_mut();    dst_buffer.copy_from_slice(&src_buffer[offset..offset+size]);Ok(empty_instance.as_object().clone())}

So the idea is simply get the bytes fromobj (it should implement the buffer protocol) and point to them. How would I point to the data inobj instead of copying it? Should I use some refcounted type to avoid pointing to dangling things?obj should always be a buffer with write access.

I don't know well about your requirements, but I guess you can do same way as howPyMemoryView does. Or even just use it. it just holds aPyBufferRef and copy the data only when it needs to be. It also locksPyBufferRef when the lock is required.

I think this PR is (practically) not possible to rebase anymore in this state. Do you mind if we squash the commits into single commit for rebase?

I think this PR is (practically) not possible to rebase anymore in this state. Do you mind if we squash the commits into single commit for rebase?

Not at all :)

I'm afraid I can't contribute in the following months. I can help others eventually, tho.

Is this effort still ongoing / has this ctypes implementation been abandoned for another?

Hi folks, I think we should close this, it's too old for any merging attempt.
Happy to see@arihant2math tackling ctypes. I was actually look at this again (starting from scratch).
I'd like to help on designing the new implementation, but haven't got much spare time to code.

This is still a best trial of ctypes now. I wouldn't close it unless we actually merge any ctypes implementation.

For anyone who will try another ctypes implementation, I hope it can be gradually mergable, not to be suffered by huge size of rebase.

Agreed, the rebase effort was slowed by refactors that have happened since this pr, I'm planning to start incremental work on this soon.

Note that#5572 is now the tracking issue for further ctypes implementation.