- Notifications
You must be signed in to change notification settings - Fork10
FEAT: Optimize executemany() performance#138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Pull Request Overview
This pull request introduces column-wise parameter binding optimization for theexecutemany()
method in the mssql_python library to improve performance for batched SQL execution. The changes enable binding multiple parameter sets at once rather than executing each parameter set individually.
- Refactored Python
executemany()
method to use column-wise parameter binding instead of iterative execution - Added C++ binding functions to support array parameter binding for various SQL data types
- Introduced helper functions for efficient parameter buffer allocation and memory management
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.
File | Description |
---|---|
mssql_python/cursor.py | Completely rewroteexecutemany() to transpose row-wise parameters to column-wise format and use new batched execution method |
mssql_python/pybind/ddbc_bindings.cpp | AddedBindParameterArray() ,SQLExecuteMany_wrap() , andAllocateParamBufferArray() functions to support column-wise parameter binding |
a0aadfc
to16a2e58
Compare16a2e58
to4ea32c7
CompareUh oh!
There was an error while loading.Please reload this page.
This reverts commit4ea32c7.
bf67d3a
to42824f1
CompareUh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
fd6e4ee
toc860e5d
CompareUh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
03f8ee9
intomainUh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
ADO Work Item Reference
Summary
This pull request introduces significant updates to the
executemany
method in themssql_python
library to enable column-wise parameter binding for batched SQL execution. It also includes enhancements to the underlying C++ bindings to support this functionality. The most important changes involve implementing column-wise binding logic, adding helper functions for parameter buffer allocation, and exposing a newSQLExecuteMany
method for executing batched statements.Enhancements to Python
executemany
method:mssql_python/cursor.py
: Refactored theexecutemany
method to use column-wise parameter binding for improved performance and scalability. Added_transpose_rowwise_to_columnwise
helper function to convert row-wise parameters into column-wise format.Updates to C++ bindings:
mssql_python/pybind/ddbc_bindings.cpp
: AddedBindParameterArray
function to handle column-wise parameter binding, supporting multiple data types such asSQL_C_LONG
,SQL_C_DOUBLE
, andSQL_C_WCHAR
.mssql_python/pybind/ddbc_bindings.cpp
: IntroducedSQLExecuteMany_wrap
function to enable batched execution of SQL statements with multiple parameter sets.mssql_python/pybind/ddbc_bindings.cpp
: Exposed the newSQLExecuteMany
method to Python viaPYBIND11_MODULE
.Helper functions for buffer allocation:
mssql_python/pybind/ddbc_bindings.cpp
: AddedAllocateParamBufferArray
template function to allocate buffers for column-wise parameter binding.