Movatterモバイル変換

Boost.Python C++ & Python Integration

Motivation Leveraging of strong sides in both languages: 1. C++: a. performance, b. effective resource management 2. Python: a. high coding performance, b. variety of available solutions c. easy to start

Why Boost.Python? Python C-API: requires too much manual work SWIG (aka generators): usually requires manual adjusting - take time to understand “someones” code. Boost.Python (middleware lib): most flexible - expose what you need in the way you like.

Getting Boost ready Getting Boost ready: get your copy of boost and build it: ./bootstrap.sh --prefix=../../ --with-python-version=3.4 --with-python-version=X.Y --with-python=/path/to/bin/python --with-python-root=/path/to ./b2 -j4 address-model=64 --with-python stage builds python 3.4 lib (only) for 64 bit os, all binaries are in stage folder

… pick what to wrap Simple C API to wrap into Python int LZ4_compress(const char* source, char* dest, int sourceSize); int LZ4_decompress_safe ( const char* source, char* dest, int compressedSize, int maxDecompressedSize); basic API of lz4 compression library

Step #1: declare Python module #include <boost/python.hpp> #include <lz4.h> using namespace boost::python; BOOST_PYTHON_MODULE(lz4py) // make sure that your output binary will have same name // lz4py.so (not liblz4py.so ) or lz4py.dll { // place for our wrappers }

basic wrappers... def("dump", // name of function on Python side LZ4_compress /*, return_value_policy<>() - optional */ ); def("load", LZ4_decompress_safe);

… but def() will wrap functions as they are … and call on Python side will not be so comfortable: lz4py.dump( input, # data to compress output, # ALLOCATED OUTPUT BUFFER (!?) size) # size of input data (!?) and function will return size of used bytes in output buffer it is too much C style - this is not Python...

Step #2: make it more Python add tiny wrapper in C++ code: const std::string compress(const std::string& d); const std::string decompress(const std::string& d); and wrap it for Python: def("dump", compress ); def("load", decompress); and … new trouble

… string and bytes In Python 3 all string are UNICODE: 1. can’t return binary data as string - will fail a. could work for python 2 - but this is hidden problem in future 2. need a way to return binary data a. convert output to bytes()

Step 4: type conversion Declare own type: class buffer { public: typedef boost::shared_ptr<char> data_ptr_t; public: buffer(data_ptr_t& data, int size): data_(data), size_(size){} int len() const { return size_; } char* const get_data() const { return data_.get(); } private: data_ptr_t data_; int size_; };

Converter to Python Converter interface declaration: template< typename T > struct type_into_python { static PyObject* convert( T const &); }; Converter implementation: template<> PyObject* type_into_python<buffer>::convert( buffer const& data){ return PyBytes_FromStringAndSize(data.get_data(), data.len()); }

Converter from Python: declaration template< typename T > struct type_from_python { type_from_python(){ converter::registry::push_back( convertible, construct, type_id<T>() ); } static void* convertible( PyObject* ); static void construct( PyObject*, converter::rvalue_from_python_stage1_data* ); };

Convert from Python: implement template<> void* type_from_python<buffer>::convertible( PyObject* obj){ return PyBytes_Check(obj) ? obj : nullptr; } template<> void type_from_python<buffer>::construct( PyObject* obj, converter::rvalue_from_python_stage1_data* data){ auto storage = reinterpret_cast< converter::rvalue_from_python_storage<buffer>* >( data )->storage.bytes; int size = PyBytes_Size(obj); buffer::data_ptr_t buffer_data = buffer::data_ptr_t(new char[size]); memcpy(buffer_data.get(), PyBytes_AsString(obj), size); new(storage) buffer(buffer_data, size); data->convertible = storage; }

New API New C++ functions: const buffer compress(buffer const& src); const buffer compress(std::string const& src); const buffer decompress(buffer const& src); Python exports: def("dump", static_cast<const buffer (*)(buffer const&)>( compress )); def("dump", static_cast<const buffer (*)(std::string const&)>( compress )); def("load", decompress);

Step #5: class Wrapp interface: class lz4_c { class bytes_ref{ PyObject* bytes_; public: int len() const { return PyBytes_Size(bytes_);} char const* data() const { return PyBytes_AsString(bytes_); } }; class buffer{ private: lz4_c& owner_; }; public: // initialize & use pool of preallocated blocks lz4_c(int block_max_size, int count); const buffer compress(bytes_ref const& src); const buffer compress(std::string const& src); const buffer decompress(bytes_ref const& src); void release(data_block_t&); };

Boost.Python class wrapper Declare Python class for lz4_c: class_<lz4_c>("lz4", init<int, int>( args("block_max_size", "count"))) //.def(init<int, int>( args("block_max_size", "count"))) ● “lz4” - class name on Python side; ● second argument - constructor, can be skipped if class have default one; ● other constructors could be added using .def() like functions; ● args - you can give names to your arguments and use named args in python;

Class methods extend our class by methods: .def("dump", static_cast<const lz4_c::buffer (lz4_c::*)( lz4_c::bytes_ref const&)>(&lz4_c::compress)) .def("dump", static_cast<const lz4_c::buffer (lz4_c::*)(std::string const&)>( &lz4_c::compress)) .def("load", &lz4_c::decompress) similar to regular functions.

Class properties can be exposed to python code: // read only .add_property("ratio", &lz4_c::ratio) // read / write .add_property("digit", make_getter(&lz4_c::digit), make_setter(&lz4_c::digit))

Exceptions class my_ex: public std::exception {}; ... void translate_error( error const& my_ex); ... //boost::python::register_exception_translator<T,F>(F) boost::python::register_exception_translator<my_ex>( translate_error)

And more STL type conversion: ● std::map <-> dict(); ● std::vector <-> list(), tuple() Define Python operators ● __str__, __repr__, __ call__ ● __add__, __lt__: o .def(self + self) o .def(self < self) Inheritance: ● make available in Python ● declare C++ relation Default args: ● wrapper to allow default args: Python function pointers: ● typedef boost::function<void (string s) > funcptr;

Return Value Policy ● with_custodian_and_ward: Ties lifetimes of the arguments ● with_custodian_and_ward_postcall: Ties lifetimes of the arguments and results ● return_internal_reference: Ties lifetime of one argument to that of result ● return_value_policy<T> with T one of: o reference_existing_object: naive (dangerous) approach o copy_const_reference: Boost.Python v1 approach o copy_non_const_reference: o manage_new_object: Adopt a pointer and hold the instance

Thank You ! Andriy.Ohorodnyk@globallogic.com

Movatterモバイル変換

Change Language

Boost.Python: C++ and Python Integration

Embed presentation

Recommended

More Related Content

What's hot

Viewers also liked

Similar to Boost.Python: C++ and Python Integration

More from GlobalLogic Ukraine

Boost.Python: C++ and Python Integration