- Notifications
You must be signed in to change notification settings - Fork27
FEAT: Logging Framework#312
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Conversation
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
| logger.setLevel(FINEST) | ||
| # Log a message with a password | ||
| test_message="Connection string: Server=localhost;PWD=secret123;Database=test" |
Check notice
Code scanning / devskim
Accessing localhost could indicate debug code, or could hinder scaling.
github-actionsbot commentedOct 31, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
📊 Code Coverage Report
Diff CoverageDiff: main...HEAD, staged and unstaged changes
Summary
mssql_python/connection.pyLines 731-739 731escape_char="\\"732self._searchescape=escape_char733exceptExceptionase:734# Log the exception for debugging, but do not expose sensitive info!735logger.debug(736"warning",737"Failed to retrieve search escape character, using default '\\'. "738"Exception: %s",739type(e).__name__, Lines 1025-1033 1025"debug",1026"Automatically closed cursor after batch execution error",1027 )1028exceptExceptionasclose_err:!1029logger.debug(1030"warning",1031f"Error closing cursor after execution failure:{close_err}",1032 )1033# Re-raise the original exception Lines 1173-1181 1173exceptUnicodeDecodeError:1174try:1175returnactual_data.decode("latin1").rstrip("\0")1176exceptExceptionase:!1177logger.debug(1178"error",1179"Failed to decode string in getinfo: %s. "1180"Returning None to avoid silent corruption.",1181e, Lines 1363-1375 1363cursor.close()1364exceptExceptionase:# pylint: disable=broad-exception-caught1365# Collect errors but continue closing other cursors1366close_errors.append(f"Error closing cursor:{e}")!1367logger.warning(f"Error closing cursor:{e}")13681369# If there were errors closing cursors, log them but continue1370ifclose_errors:!1371logger.debug(1372"warning",1373"Encountered %d errors while closing cursors",1374len(close_errors),1375 ) mssql_python/cursor.pyLines 390-398 390exponent=decimal_as_tuple.exponent391392# Handle special values (NaN, Infinity, etc.)393ifisinstance(exponent,str):!394logger.finer('_map_sql_type: DECIMAL special value - index=%d, exponent=%s',i,exponent)395# For special values like 'n' (NaN), 'N' (sNaN), 'F' (Infinity)396# Return default precision and scale397precision=38# SQL Server default max precision398else: Lines 465-473 465param.startswith("POINT")466orparam.startswith("LINESTRING")467orparam.startswith("POLYGON")468 ):!469logger.finest('_map_sql_type: STR is geometry type - index=%d',i)470return (471ddbc_sql_const.SQL_WVARCHAR.value,472ddbc_sql_const.SQL_C_WCHAR.value,473len(param), Lines 1068-1076 1068ddbc_sql_const.SQL_ATTR_QUERY_TIMEOUT.value,1069timeout_value,1070 )1071check_error(ddbc_sql_const.SQL_HANDLE_STMT.value,self.hstmt,ret)!1072logger.debug("Set query timeout to %d seconds",timeout_value)1073exceptExceptionase:# pylint: disable=broad-exception-caught1074logger.warning("Failed to set query timeout: %s",str(e))10751076logger.finest('execute: Creating parameter type list') Lines 1180-1188 1180ifdescanddesc[1]==uuid.UUID:# Column type code at index 11181self._uuid_indices.append(i)1182# Verify we have complete description tuples (7 items per PEP-249)1183elifdescandlen(desc)!=7:!1184logger.debug(1185"warning",1186f"Column description at index{i} has incorrect tuple length:{len(desc)}",1187 )1188self.rowcount=-1 Lines 1220-1231 1220column_metadata= []1221try:1222ddbc_bindings.DDBCSQLDescribeCol(self.hstmt,column_metadata)1223exceptInterfaceErrorase:!1224logger.error(f"Driver interface error during metadata retrieval:{e}")1225exceptExceptionase:# pylint: disable=broad-exception-caught1226# Log the exception with appropriate context!1227logger.debug(1228"error",1229f"Failed to retrieve column metadata:{e}. "1230f"Using standard ODBC column definitions instead.",1231 ) Lines 1750-1760 1750ddbc_sql_const.SQL_ATTR_QUERY_TIMEOUT.value,1751timeout_value,1752 )1753check_error(ddbc_sql_const.SQL_HANDLE_STMT.value,self.hstmt,ret)!1754logger.debug(f"Set query timeout to{self._timeout} seconds")1755exceptExceptionase:# pylint: disable=broad-exception-caught!1756logger.warning(f"Failed to set query timeout:{e}")17571758# Get sample row for parameter type detection and validation1759sample_row= (1760seq_of_parameters[0] Lines 2257-2265 22572258ifsysandsys._is_finalizing():2259# Suppress logging during interpreter shutdown2260return!2261logger.debug("Exception during cursor cleanup in __del__: %s",e)22622263defscroll(self,value:int,mode:str="relative")->None:# pylint: disable=too-many-branches2264 """ 2265 Scroll using SQLFetchScroll only, matching test semantics: Lines 2466-2474 2466 )24672468exceptExceptionase:# pylint: disable=broad-exception-caught2469# Log the error and re-raise!2470logger.error(f"Error executing tables query:{e}")2471raise24722473defcallproc(2474self,procname:str,parameters:Optional[Sequence[Any]]=None mssql_python/exceptions.pyLines 523-531 523string_second=error_message[error_message.index("]")+1 :]524string_third=string_second[string_second.index("]")+1 :]525returnstring_first+string_third526exceptExceptionase:!527logger.error("Error while truncating error message: %s",e)528returnerror_message529530531defraise_exception(sqlstate:str,ddbc_error:str)->None: mssql_python/helpers.pyLines 57-65 57logger.finest('add_driver_to_connection_str: Driver added (had_existing=%s, attr_count=%d)',58str(driver_found),len(final_connection_attributes))5960exceptExceptionase:!61logger.finer('add_driver_to_connection_str: Failed to process connection string - %s',str(e))62raiseValueError(63"Invalid connection string, Please follow the format: "64"Server=server_name;Database=database_name;UID=user_name;PWD=password"65 )frome Lines 111-119 111# Overwrite the value with 'MSSQL-Python'112app_found=True113key,_=param.split("=",1)114modified_parameters.append(f"{key}=MSSQL-Python")!115logger.finest('add_driver_name_to_app_parameter: Existing APP parameter overwritten')116else:117# Keep other parameters as is118modified_parameters.append(param) Lines 156-164 156 """ 157 logger.finest('sanitize_user_input: Sanitizing input (type=%s, length=%d)', 158 type(user_input).__name__, len(user_input) if isinstance(user_input, str) else 0) 159 if not isinstance(user_input, str):! 160 logger.finest('sanitize_user_input: Non-string input detected') 161 return "<non-string>"162163# Remove control characters and non-printable characters164# Allow alphanumeric, dash, underscore, and dot (common in encoding names) mssql_python/logging.pyLines 55-63 55def__init__(self):56"""Initialize the logger (only once)"""57# Skip if already initialized58ifhasattr(self,'_initialized'):!59return6061self._initialized=True6263# Create the underlying Python logger Lines 80-88 80str:Pathtothelogfile81 """ 82 # Clear any existing handlers 83 if self._logger.handlers:! 84 self._logger.handlers.clear() 85 86 # Create log file in current working directory (not package directory) 87 timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")88pid=os.getpid() Lines 215-223 215self._log(logging.ERROR,f"[Python]{msg}",*args,**kwargs)216217defcritical(self,msg:str,*args,**kwargs):218"""Log at CRITICAL level"""!219self._log(logging.CRITICAL,f"[Python]{msg}",*args,**kwargs)220221deflog(self,level:int,msg:str,*args,**kwargs):222"""Log a message at the specified level"""223self._log(level,f"[Python]{msg}",*args,**kwargs) Lines 270-278 270271 @property272defhandlers(self)->list:273"""Get list of handlers attached to the logger"""!274returnself._logger.handlers275276defreset_handlers(self):277 """278Reset/recreatefilehandler. Lines 298-308 298# Import here to avoid circular dependency299from .importddbc_bindings300ifhasattr(ddbc_bindings,'update_log_level'):301ddbc_bindings.update_log_level(level)!302except (ImportError,AttributeError):303# C++ bindings not available or not yet initialized!304pass305306# Properties307308 @property Lines 333-347 333log_level:Logginglevel (mapstoclosestFINE/FINER/FINEST)334 """335# Map old levels to new levels336iflog_level<=FINEST:!337logger.setLevel(FINEST)338eliflog_level<=FINER:339logger.setLevel(FINER)!340eliflog_level<=FINE:!341logger.setLevel(FINE)342else:!343logger.setLevel(log_level)344345returnlogger346 mssql_python/pybind/connection/connection.cppLines 20-28 20static SqlHandlePtrgetEnvHandle() {21static SqlHandlePtr envHandle = []() -> SqlHandlePtr {22LOG_FINER("Allocating ODBC environment handle");23if (!SQLAllocHandle_ptr) {!24LOG_FINER("Function pointers not initialized, loading driver");25DriverLoader::getInstance().loadDriver();26 }27 SQLHANDLE env =nullptr;28 SQLRETURN ret =SQLAllocHandle_ptr(SQL_HANDLE_ENV, SQL_NULL_HANDLE, Lines 217-225 217218// Convert to wide string219 std::wstring wstr = Utf8ToWString(utf8_str);220if (wstr.empty() && !utf8_str.empty()) {!221LOG_FINER("Failed to convert string value to wide string for attribute=%d", attribute);222return SQL_ERROR;223 }224225// Limit static buffer growth for memory safety Lines 238-246 238 #ifdefined(__APPLE__) || defined(__linux__) 239// For macOS/Linux, convert wstring to SQLWCHAR buffer 240 std::vector<SQLWCHAR> sqlwcharBuffer = WStringToSQLWCHAR(wstr);241if (sqlwcharBuffer.empty() && !wstr.empty()) {!242LOG_FINER("Failed to convert wide string to SQLWCHAR buffer for attribute=%d", attribute);243return SQL_ERROR;244 }245246 ptr = sqlwcharBuffer.data(); Lines 261-269 261LOG_FINER("Set string attribute=%d successfully", attribute);262 }263return ret;264 }catch (const std::exception& e) {!265LOG_FINER("Exception during string attribute=%d setting: %s", attribute, e.what());266return SQL_ERROR;267 }268 }elseif (py::isinstance<py::bytes>(value) ||269 py::isinstance<py::bytearray>(value)) { Lines 288-304 288 attribute, ptr, length);289if (!SQL_SUCCEEDED(ret)) {290LOG_FINER("Failed to set binary attribute=%d, ret=%d", attribute, ret);291 }else {!292LOG_FINER("Set binary attribute=%d successfully (length=%d)", attribute, length);293 }294return ret;295 }catch (const std::exception& e) {!296LOG_FINER("Exception during binary attribute=%d setting: %s", attribute, e.what());297return SQL_ERROR;298 }299 }else {!300LOG_FINER("Unsupported attribute value type for attribute=%d", attribute);301return SQL_ERROR;302 }303 } Lines 344-352 344 SQL_ATTR_RESET_CONNECTION,345 (SQLPOINTER)SQL_RESET_CONNECTION_YES,346 SQL_IS_INTEGER);347if (!SQL_SUCCEEDED(ret)) {!348LOG_FINER("Failed to reset connection (ret=%d). Marking as dead.", ret);349disconnect();350returnfalse;351 }352updateLastUsed(); mssql_python/pybind/connection/connection_pool.cppLines 71-79 71for (auto& conn : to_disconnect) {72try {73 conn->disconnect();74 }catch (const std::exception& ex) {!75LOG_FINER("Disconnect bad/expired connections failed: %s", ex.what());76 }77 }78return valid_conn;79 } Lines 102-110 102for (auto& conn : to_close) {103try {104 conn->disconnect();105 }catch (const std::exception& ex) {!106LOG_FINER("ConnectionPool::close: disconnect failed: %s", ex.what());107 }108 }109 } mssql_python/pybind/ddbc_bindings.cppLines 278-286 278 !py::isinstance<py::bytes>(param)) {279ThrowStdException(MakeParamMismatchErrorStr(paramInfo.paramCType, paramIndex));280 }281if (paramInfo.isDAE) {!282LOG_FINER("BindParameters: param[%d] SQL_C_CHAR - Using DAE (Data-At-Execution) for large string streaming", paramIndex);283 dataPtr =const_cast<void*>(reinterpret_cast<constvoid*>(¶mInfos[paramIndex]));284 strLenOrIndPtr = AllocateParamBuffer<SQLLEN>(paramBuffers);285 *strLenOrIndPtr =SQL_LEN_DATA_AT_EXEC(0);286 bufferLength =0; Lines 380-388 380 &describedDigits,381 &nullable382 );383if (!SQL_SUCCEEDED(rc)) {!384LOG_FINER("BindParameters: SQLDescribeParam failed for param[%d] (NULL parameter) - SQLRETURN=%d", paramIndex, rc);385return rc;386 }387 sqlType = describedType;388 columnSize = describedSize; Lines 591-600 591 }592 py::bytes uuid_bytes = param.cast<py::bytes>();593constunsignedchar* uuid_data =reinterpret_cast<constunsignedchar*>(PyBytes_AS_STRING(uuid_bytes.ptr()));594if (PyBytes_GET_SIZE(uuid_bytes.ptr()) !=16) {!595LOG_FINER("BindParameters: param[%d] SQL_C_GUID - Invalid UUID length: expected 16 bytes, got %ld bytes", !596 paramIndex,PyBytes_GET_SIZE(uuid_bytes.ptr()));597ThrowStdException("UUID binary data must be exactly 16 bytes long.");598 }599 SQLGUID* guid_data_ptr = AllocateParamBuffer<SQLGUID>(paramBuffers);600 guid_data_ptr->Data1 = Lines 640-653 640if (paramInfo.paramCType == SQL_C_NUMERIC) {641 SQLHDESC hDesc =nullptr;642 rc =SQLGetStmtAttr_ptr(hStmt, SQL_ATTR_APP_PARAM_DESC, &hDesc,0,NULL);643if(!SQL_SUCCEEDED(rc)) {!644LOG_FINER("BindParameters: SQLGetStmtAttr(SQL_ATTR_APP_PARAM_DESC) failed for param[%d] - SQLRETURN=%d", paramIndex, rc);645return rc;646 }647 rc =SQLSetDescField_ptr(hDesc,1, SQL_DESC_TYPE, (SQLPOINTER) SQL_C_NUMERIC,0);648if(!SQL_SUCCEEDED(rc)) {!649LOG_FINER("BindParameters: SQLSetDescField(SQL_DESC_TYPE) failed for param[%d] - SQLRETURN=%d", paramIndex, rc);650return rc;651 }652 SQL_NUMERIC_STRUCT* numericPtr =reinterpret_cast<SQL_NUMERIC_STRUCT*>(dataPtr);653 rc =SQLSetDescField_ptr(hDesc,1, SQL_DESC_PRECISION, Lines 652-660 652 SQL_NUMERIC_STRUCT* numericPtr =reinterpret_cast<SQL_NUMERIC_STRUCT*>(dataPtr);653 rc = SQLSetDescField_ptr(hDesc,1, SQL_DESC_PRECISION,654 (SQLPOINTER) numericPtr->precision,0);655if(!SQL_SUCCEEDED(rc)) {!656LOG_FINER("BindParameters: SQLSetDescField(SQL_DESC_PRECISION) failed for param[%d] - SQLRETURN=%d", paramIndex, rc);657return rc;658 }659660 rc = SQLSetDescField_ptr(hDesc,1, SQL_DESC_SCALE, Lines 659-667 659660 rc = SQLSetDescField_ptr(hDesc,1, SQL_DESC_SCALE,661 (SQLPOINTER) numericPtr->scale,0);662if(!SQL_SUCCEEDED(rc)) {!663LOG_FINER("BindParameters: SQLSetDescField(SQL_DESC_SCALE) failed for param[%d] - SQLRETURN=%d", paramIndex, rc);664return rc;665 }666667 rc = SQLSetDescField_ptr(hDesc,1, SQL_DESC_DATA_PTR, (SQLPOINTER) numericPtr,0); Lines 665-673 665 }666667 rc = SQLSetDescField_ptr(hDesc,1, SQL_DESC_DATA_PTR, (SQLPOINTER) numericPtr,0);668if(!SQL_SUCCEEDED(rc)) {!669LOG_FINER("BindParameters: SQLSetDescField(SQL_DESC_DATA_PTR) failed for param[%d] - SQLRETURN=%d", paramIndex, rc);670return rc;671 }672 }673 } Lines 745-753 745if (pos != std::string::npos) {746 std::string dir = module_file.substr(0, pos);747return dir;748 }!749LOG_FINEST("GetModuleDirectory: Could not extract directory from module path - path='%s'", module_file.c_str());750return module_file;751 #endif752 } Lines 768-777 768 #else769// macOS/Unix: Use dlopen770void* handle = dlopen(driverPath.c_str(), RTLD_LAZY);771if (!handle) {!772LOG_FINER("LoadDriverLibrary: dlopen failed for path='%s' - %s", !773 driverPath.c_str(),dlerror() ?dlerror() :"unknown error");774 }775return handle;776 #endif777 } Lines 913-922 913 }914915 DriverHandle handle = LoadDriverLibrary(driverPath.string());916if (!handle) {!917LOG_FINER("LoadDriverOrThrowException: Failed to load ODBC driver - path='%s', error='%s'", !918 driverPath.string().c_str(),GetLastErrorMessage().c_str());919ThrowStdException("Failed to load the driver. Please read the documentation (https://github.com/microsoft/mssql-python#installation) to install the required dependencies.");920 }921LOG_FINER("LoadDriverOrThrowException: ODBC driver library loaded successfully from '%s'", driverPath.string().c_str()); Lines 1296-1304 1296 ErrorInfoSQLCheckError_Wrap(SQLSMALLINT handleType, SqlHandlePtr handle, SQLRETURN retcode) {1297LOG_FINER("SQLCheckError: Checking ODBC errors - handleType=%d, retcode=%d", handleType, retcode);1298 ErrorInfo errorInfo;1299if (retcode == SQL_INVALID_HANDLE) {!1300LOG_FINER("SQLCheckError: SQL_INVALID_HANDLE detected - handle is invalid");1301 errorInfo.ddbcErrorMsg =std::wstring(L"Invalid handle!");1302return errorInfo;1303 }1304assert(handle !=0); Lines 1304-1312 1304assert(handle !=0);1305 SQLHANDLE rawHandle = handle->get();1306if (!SQL_SUCCEEDED(retcode)) {1307if (!SQLGetDiagRec_ptr) {!1308LOG_FINER("SQLCheckError: SQLGetDiagRec function pointer not initialized, loading driver");1309DriverLoader::getInstance().loadDriver();// Load the driver1310 }13111312 SQLWCHAR sqlState[6], message[SQL_MAX_MESSAGE_LENGTH]; Lines 1335-1343 1335 py::listSQLGetAllDiagRecords(SqlHandlePtr handle) {1336LOG_FINER("SQLGetAllDiagRecords: Retrieving all diagnostic records for handle %p, handleType=%d",1337 (void*)handle->get(), handle->type());1338if (!SQLGetDiagRec_ptr) {!1339LOG_FINER("SQLGetAllDiagRecords: SQLGetDiagRec function pointer not initialized, loading driver");1340DriverLoader::getInstance().loadDriver();1341 }13421343 py::list records; Lines 1399-1411 1399 }14001401// Wrap SQLExecDirect1402 SQLRETURNSQLExecDirect_wrap(SqlHandlePtr StatementHandle,const std::wstring& Query) {!1403 std::string queryUtf8 =WideToUTF8(Query);!1404LOG_FINE("SQLExecDirect: Executing query directly - statement_handle=%p, query_length=%zu chars", !1405 (void*)StatementHandle->get(), Query.length());1406if (!SQLExecDirect_ptr) {!1407LOG_FINER("SQLExecDirect: Function pointer not initialized, loading driver");1408DriverLoader::getInstance().loadDriver();// Load the driver1409 }14101411// Ensure statement is scrollable BEFORE executing Lines 1428-1436 1428 queryPtr =const_cast<SQLWCHAR*>(Query.c_str());1429 #endif1430 SQLRETURN ret = SQLExecDirect_ptr(StatementHandle->get(), queryPtr, SQL_NTS);1431if (!SQL_SUCCEEDED(ret)) {!1432LOG_FINER("SQLExecDirect: Query execution failed - SQLRETURN=%d", ret);1433 }1434return ret;1435 } Lines 1441-1449 1441const std::wstring& table,1442const std::wstring& tableType) {14431444if (!SQLTables_ptr) {!1445LOG_FINER("SQLTables: Function pointer not initialized, loading driver");1446DriverLoader::getInstance().loadDriver();1447 }14481449 SQLWCHAR* catalogPtr =nullptr; Lines 1526-1534 1526 py::list& isStmtPrepared,constbool usePrepare =true) {1527LOG_FINE("SQLExecute: Executing %s query - statement_handle=%p, param_count=%zu, query_length=%zu chars",1528 (params.size() >0 ?"parameterized" :"direct"), (void*)statementHandle->get(), params.size(), query.length());1529if (!SQLPrepare_ptr) {!1530LOG_FINER("SQLExecute: Function pointer not initialized, loading driver");1531DriverLoader::getInstance().loadDriver();// Load the driver1532 }1533assert(SQLPrepare_ptr && SQLBindParameter_ptr && SQLExecute_ptr && SQLExecDirect_ptr); Lines 1539-1547 15391540 RETCODE rc;1541 SQLHANDLE hStmt = statementHandle->get();1542if (!statementHandle || !statementHandle->get()) {!1543LOG_FINER("SQLExecute: Statement handle is null or invalid");1544 }15451546// Ensure statement is scrollable BEFORE executing1547if (SQLSetStmtAttr_ptr && hStmt) { Lines 1579-1587 1579assert(isStmtPrepared.size() == 1);1580if (usePrepare) {1581 rc =SQLPrepare_ptr(hStmt, queryPtr, SQL_NTS);1582if (!SQL_SUCCEEDED(rc)) {!1583LOG_FINER("SQLExecute: SQLPrepare failed - SQLRETURN=%d, statement_handle=%p", rc, (void*)hStmt);1584return rc;1585 }1586 isStmtPrepared[0] =py::cast(true);1587 }else { Lines 1644-1653 1644ThrowStdException("Chunk size exceeds maximum allowed by SQLLEN");1645 }1646 rc = SQLPutData_ptr(hStmt, (SQLPOINTER)(dataPtr + offset),static_cast<SQLLEN>(lenBytes));1647if (!SQL_SUCCEEDED(rc)) {!1648LOG_FINEST("SQLExecute: SQLPutData failed for SQL_C_WCHAR chunk - offset=%zu, total_chars=%zu, chunk_bytes=%zu, SQLRETURN=%d", !1649 offset, totalChars, lenBytes, rc);1650return rc;1651 }1652 offset += len;1653 } Lines 1661-1670 1661size_t len = std::min(chunkBytes, totalBytes - offset);16621663 rc = SQLPutData_ptr(hStmt, (SQLPOINTER)(dataPtr + offset),static_cast<SQLLEN>(len));1664if (!SQL_SUCCEEDED(rc)) {!1665LOG_FINEST("SQLExecute: SQLPutData failed for SQL_C_CHAR chunk - offset=%zu, total_bytes=%zu, chunk_bytes=%zu, SQLRETURN=%d", !1666 offset, totalBytes, len, rc);1667return rc;1668 }1669 offset += len;1670 } Lines 1680-1689 1680for (size_t offset =0; offset < totalBytes; offset += chunkSize) {1681size_t len =std::min(chunkSize, totalBytes - offset);1682 rc =SQLPutData_ptr(hStmt, (SQLPOINTER)(dataPtr + offset),static_cast<SQLLEN>(len));1683if (!SQL_SUCCEEDED(rc)) {!1684LOG_FINEST("SQLExecute: SQLPutData failed for binary/bytes chunk - offset=%zu, total_bytes=%zu, chunk_bytes=%zu, SQLRETURN=%d", !1685 offset, totalBytes, len, rc);1686return rc;1687 }1688 }1689 }else { Lines 1690-1699 1690ThrowStdException("DAE only supported for str or bytes");1691 }1692 }1693if (!SQL_SUCCEEDED(rc)) {!1694LOG_FINER("SQLExecute: SQLParamData final call %s - SQLRETURN=%d", !1695 (rc == SQL_NO_DATA ?"completed with no data" :"failed"), rc);1696return rc;1697 }1698LOG_FINER("SQLExecute: DAE streaming completed successfully, SQLExecute resumed");1699 } Lines 1725-1734 1725const ParamInfo& info = paramInfos[paramIndex];1726LOG_FINEST("BindParameterArray: Processing param_index=%d, C_type=%d, SQL_type=%d, column_size=%zu, decimal_digits=%d",1727 paramIndex, info.paramCType, info.paramSQLType, info.columnSize, info.decimalDigits);1728if (columnValues.size() != paramSetSize) {!1729LOG_FINER("BindParameterArray: Size mismatch - param_index=%d, expected=%zu, actual=%zu", !1730 paramIndex, paramSetSize, columnValues.size());1731ThrowStdException("Column" +std::to_string(paramIndex) +" has mismatched size.");1732 }1733void* dataPtr =nullptr;1734 SQLLEN* strLenOrIndArray =nullptr; Lines 1743-1751 1743if (!strLenOrIndArray)1744 strLenOrIndArray = AllocateParamBufferArray<SQLLEN>(tempBuffers, paramSetSize);1745 dataArray[i] =0;1746 strLenOrIndArray[i] = SQL_NULL_DATA;!1747 null_count++;1748 }else {1749 dataArray[i] = columnValues[i].cast<int>();1750if (strLenOrIndArray) strLenOrIndArray[i] =0;1751 } Lines 1754-1764 1754 dataPtr = dataArray;1755break;1756 }1757case SQL_C_DOUBLE: {!1758LOG_FINEST("BindParameterArray: Binding SQL_C_DOUBLE array - param_index=%d, count=%zu", paramIndex, paramSetSize);1759double* dataArray = AllocateParamBufferArray<double>(tempBuffers, paramSetSize);!1760size_t null_count =0;1761for (size_t i =0; i < paramSetSize; ++i) {1762if (columnValues[i].is_none()) {1763if (!strLenOrIndArray)1764 strLenOrIndArray = AllocateParamBufferArray<SQLLEN>(tempBuffers, paramSetSize); Lines 1763-1771 1763if (!strLenOrIndArray)1764 strLenOrIndArray = AllocateParamBufferArray<SQLLEN>(tempBuffers, paramSetSize);1765 dataArray[i] =0;1766 strLenOrIndArray[i] = SQL_NULL_DATA;!1767 null_count++;1768 }else {1769 dataArray[i] = columnValues[i].cast<double>();1770if (strLenOrIndArray) strLenOrIndArray[i] =0;1771 } Lines 1769-1777 1769 dataArray[i] = columnValues[i].cast<double>();1770if (strLenOrIndArray) strLenOrIndArray[i] =0;1771 }1772 }!1773LOG_FINEST("BindParameterArray: SQL_C_DOUBLE bound - param_index=%d, null_values=%zu", paramIndex, null_count);1774 dataPtr = dataArray;1775break;1776 }1777case SQL_C_WCHAR: { Lines 1794-1805 1794 total_chars += utf16_len;1795// Check UTF-16 length (excluding null terminator) against column size1796if (utf16Buf.size() >0 && utf16_len > info.columnSize) {1797 std::string offending =WideToUTF8(wstr);!1798LOG_FINER("BindParameterArray: SQL_C_WCHAR string too long - param_index=%d, row=%zu, utf16_length=%zu, max=%zu",!1799 paramIndex, i, utf16_len, info.columnSize);1800ThrowStdException("Input string UTF-16 length exceeds allowed column size at parameter index" +std::to_string(paramIndex) + !1801". UTF-16 length:" +std::to_string(utf16_len) +", Column size:" +std::to_string(info.columnSize));1802 }1803// If we reach here, the UTF-16 string fits - copy it completely1804std::memcpy(wcharArray + i * (info.columnSize +1), utf16Buf.data(), utf16Buf.size() * sizeof(SQLWCHAR));1805 #else Lines 1833-1842 1833 null_count++;1834 }else {1835int intVal = columnValues[i].cast<int>();1836if (intVal <0 || intVal >255) {!1837LOG_FINER("BindParameterArray: TINYINT value out of range - param_index=%d, row=%zu, value=%d",!1838 paramIndex, i, intVal);1839ThrowStdException("UTINYINT value out of range at rowIndex" +std::to_string(i));1840 }1841 dataArray[i] =static_cast<unsignedchar>(intVal);1842if (strLenOrIndArray) strLenOrIndArray[i] =0; Lines 1856-1870 1856if (!strLenOrIndArray)1857 strLenOrIndArray = AllocateParamBufferArray<SQLLEN>(tempBuffers, paramSetSize);1858 dataArray[i] =0;1859 strLenOrIndArray[i] = SQL_NULL_DATA;!1860 null_count++;1861 }else {1862int intVal = columnValues[i].cast<int>();1863if (intVal < std::numeric_limits<short>::min() ||1864 intVal > std::numeric_limits<short>::max()) {!1865LOG_FINER("BindParameterArray: SHORT value out of range - param_index=%d, row=%zu, value=%d",!1866 paramIndex, i, intVal);1867ThrowStdException("SHORT value out of range at rowIndex" +std::to_string(i));1868 }1869 dataArray[i] =static_cast<short>(intVal);1870if (strLenOrIndArray) strLenOrIndArray[i] =0; Lines 1890-1901 1890 }else {1891 std::string str = columnValues[i].cast<std::string>();1892 total_bytes += str.size();1893if (str.size() > info.columnSize) {!1894LOG_FINER("BindParameterArray: String/binary too long - param_index=%d, row=%zu, size=%zu, max=%zu",!1895 paramIndex, i, str.size(), info.columnSize);1896ThrowStdException("Input exceeds column size at index" +std::to_string(i));!1897 }1898std::memcpy(charArray + i * (info.columnSize +1), str.c_str(), str.size());1899 strLenOrIndArray[i] =static_cast<SQLLEN>(str.size());1900 }1901 } Lines 1905-1930 1905 bufferLength = info.columnSize +1;1906break;1907 }1908case SQL_C_BIT: {!1909LOG_FINEST("BindParameterArray: Binding SQL_C_BIT array - param_index=%d, count=%zu", paramIndex, paramSetSize);1910char* boolArray = AllocateParamBufferArray<char>(tempBuffers, paramSetSize);1911 strLenOrIndArray = AllocateParamBufferArray<SQLLEN>(tempBuffers, paramSetSize);!1912size_t null_count =0, true_count =0;1913for (size_t i =0; i < paramSetSize; ++i) {1914if (columnValues[i].is_none()) {1915 boolArray[i] =0;1916 strLenOrIndArray[i] = SQL_NULL_DATA;!1917 null_count++;1918 }else {!1919bool val = columnValues[i].cast<bool>();!1920 boolArray[i] = val ?1 :0;!1921if (val) true_count++;1922 strLenOrIndArray[i] =0;1923 }1924 }!1925LOG_FINEST("BindParameterArray: SQL_C_BIT bound - param_index=%d, null_values=%zu, true_values=%zu",!1926 paramIndex, null_count, true_count);1927 dataPtr = boolArray;1928 bufferLength =sizeof(char);1929break;1930 } Lines 1929-1945 1929break;1930 }1931case SQL_C_STINYINT:1932case SQL_C_USHORT: {!1933LOG_FINEST("BindParameterArray: Binding SQL_C_USHORT/STINYINT array - param_index=%d, count=%zu", paramIndex, paramSetSize);1934unsignedshort* dataArray = AllocateParamBufferArray<unsignedshort>(tempBuffers, paramSetSize);1935 strLenOrIndArray = AllocateParamBufferArray<SQLLEN>(tempBuffers, paramSetSize);!1936size_t null_count =0;1937for (size_t i =0; i < paramSetSize; ++i) {1938if (columnValues[i].is_none()) {1939 strLenOrIndArray[i] = SQL_NULL_DATA;1940 dataArray[i] =0;!1941 null_count++;1942 }else {1943 dataArray[i] = columnValues[i].cast<unsignedshort>();1944 strLenOrIndArray[i] =0;1945 } Lines 1943-1951 1943 dataArray[i] = columnValues[i].cast<unsignedshort>();1944 strLenOrIndArray[i] =0;1945 }1946 }!1947LOG_FINEST("BindParameterArray: SQL_C_USHORT bound - param_index=%d, null_values=%zu", paramIndex, null_count);1948 dataPtr = dataArray;1949 bufferLength =sizeof(unsignedshort);1950break;1951 } Lines 1960-1968 1960for (size_t i =0; i < paramSetSize; ++i) {1961if (columnValues[i].is_none()) {1962 strLenOrIndArray[i] = SQL_NULL_DATA;1963 dataArray[i] =0;!1964 null_count++;1965 }else {1966 dataArray[i] = columnValues[i].cast<int64_t>();1967 strLenOrIndArray[i] =0;1968 } Lines 1980-1988 1980for (size_t i =0; i < paramSetSize; ++i) {1981if (columnValues[i].is_none()) {1982 strLenOrIndArray[i] = SQL_NULL_DATA;1983 dataArray[i] =0.0f;!1984 null_count++;1985 }else {1986 dataArray[i] = columnValues[i].cast<float>();1987 strLenOrIndArray[i] =0;1988 } Lines 1992-2008 1992 bufferLength =sizeof(float);1993break;1994 }1995case SQL_C_TYPE_DATE: {!1996LOG_FINEST("BindParameterArray: Binding SQL_C_TYPE_DATE array - param_index=%d, count=%zu", paramIndex, paramSetSize);1997 SQL_DATE_STRUCT* dateArray = AllocateParamBufferArray<SQL_DATE_STRUCT>(tempBuffers, paramSetSize);1998 strLenOrIndArray = AllocateParamBufferArray<SQLLEN>(tempBuffers, paramSetSize);!1999size_t null_count =0;2000for (size_t i =0; i < paramSetSize; ++i) {2001if (columnValues[i].is_none()) {2002 strLenOrIndArray[i] = SQL_NULL_DATA;2003std::memset(&dateArray[i],0,sizeof(SQL_DATE_STRUCT));!2004 null_count++;2005 }else {2006 py::object dateObj = columnValues[i];2007 dateArray[i].year = dateObj.attr("year").cast<SQLSMALLINT>();2008 dateArray[i].month = dateObj.attr("month").cast<SQLUSMALLINT>(); Lines 2009-2017 2009 dateArray[i].day = dateObj.attr("day").cast<SQLUSMALLINT>();2010 strLenOrIndArray[i] =0;2011 }2012 }!2013LOG_FINEST("BindParameterArray: SQL_C_TYPE_DATE bound - param_index=%d, null_values=%zu", paramIndex, null_count);2014 dataPtr = dateArray;2015 bufferLength =sizeof(SQL_DATE_STRUCT);2016break;2017 } Lines 2015-2031 2015 bufferLength =sizeof(SQL_DATE_STRUCT);2016break;2017 }2018case SQL_C_TYPE_TIME: {!2019LOG_FINEST("BindParameterArray: Binding SQL_C_TYPE_TIME array - param_index=%d, count=%zu", paramIndex, paramSetSize);2020 SQL_TIME_STRUCT* timeArray = AllocateParamBufferArray<SQL_TIME_STRUCT>(tempBuffers, paramSetSize);2021 strLenOrIndArray = AllocateParamBufferArray<SQLLEN>(tempBuffers, paramSetSize);!2022size_t null_count =0;2023for (size_t i =0; i < paramSetSize; ++i) {2024if (columnValues[i].is_none()) {2025 strLenOrIndArray[i] = SQL_NULL_DATA;2026std::memset(&timeArray[i],0,sizeof(SQL_TIME_STRUCT));!2027 null_count++;2028 }else {2029 py::object timeObj = columnValues[i];2030 timeArray[i].hour = timeObj.attr("hour").cast<SQLUSMALLINT>();2031 timeArray[i].minute = timeObj.attr("minute").cast<SQLUSMALLINT>(); Lines 2032-2040 2032 timeArray[i].second = timeObj.attr("second").cast<SQLUSMALLINT>();2033 strLenOrIndArray[i] =0;2034 }2035 }!2036LOG_FINEST("BindParameterArray: SQL_C_TYPE_TIME bound - param_index=%d, null_values=%zu", paramIndex, null_count);2037 dataPtr = timeArray;2038 bufferLength =sizeof(SQL_TIME_STRUCT);2039break;2040 } Lines 2038-2054 2038 bufferLength =sizeof(SQL_TIME_STRUCT);2039break;2040 }2041case SQL_C_TYPE_TIMESTAMP: {!2042LOG_FINEST("BindParameterArray: Binding SQL_C_TYPE_TIMESTAMP array - param_index=%d, count=%zu", paramIndex, paramSetSize);2043 SQL_TIMESTAMP_STRUCT* tsArray = AllocateParamBufferArray<SQL_TIMESTAMP_STRUCT>(tempBuffers, paramSetSize);2044 strLenOrIndArray = AllocateParamBufferArray<SQLLEN>(tempBuffers, paramSetSize);!2045size_t null_count =0;2046for (size_t i =0; i < paramSetSize; ++i) {2047if (columnValues[i].is_none()) {2048 strLenOrIndArray[i] = SQL_NULL_DATA;2049std::memset(&tsArray[i],0,sizeof(SQL_TIMESTAMP_STRUCT));!2050 null_count++;2051 }else {2052 py::object dtObj = columnValues[i];2053 tsArray[i].year = dtObj.attr("year").cast<SQLSMALLINT>();2054 tsArray[i].month = dtObj.attr("month").cast<SQLUSMALLINT>(); Lines 2059-2067 2059 tsArray[i].fraction =static_cast<SQLUINTEGER>(dtObj.attr("microsecond").cast<int>() *1000);// µs to ns2060 strLenOrIndArray[i] =0;2061 }2062 }!2063LOG_FINEST("BindParameterArray: SQL_C_TYPE_TIMESTAMP bound - param_index=%d, null_values=%zu", paramIndex, null_count);2064 dataPtr = tsArray;2065 bufferLength =sizeof(SQL_TIMESTAMP_STRUCT);2066break;2067 } Lines 2078-2086 20782079if (param.is_none()) {2080std::memset(&dtoArray[i],0,sizeof(DateTimeOffset));2081 strLenOrIndArray[i] = SQL_NULL_DATA;!2082 null_count++;2083 }else {2084if (!py::isinstance(param, datetimeType)) {2085ThrowStdException(MakeParamMismatchErrorStr(info.paramCType, paramIndex));2086 } Lines 2116-2127 2116 bufferLength =sizeof(DateTimeOffset);2117break;2118 }2119case SQL_C_NUMERIC: {!2120LOG_FINEST("BindParameterArray: Binding SQL_C_NUMERIC array - param_index=%d, count=%zu", paramIndex, paramSetSize);2121 SQL_NUMERIC_STRUCT* numericArray = AllocateParamBufferArray<SQL_NUMERIC_STRUCT>(tempBuffers, paramSetSize);2122 strLenOrIndArray = AllocateParamBufferArray<SQLLEN>(tempBuffers, paramSetSize);!2123size_t null_count =0;2124for (size_t i =0; i < paramSetSize; ++i) {2125const py::handle& element = columnValues[i];2126if (element.is_none()) {2127 strLenOrIndArray[i] = SQL_NULL_DATA; Lines 2125-2142 2125const py::handle& element = columnValues[i];2126if (element.is_none()) {2127 strLenOrIndArray[i] = SQL_NULL_DATA;2128std::memset(&numericArray[i],0,sizeof(SQL_NUMERIC_STRUCT));!2129 null_count++;2130continue;2131 }2132if (!py::isinstance<NumericData>(element)) {!2133LOG_FINER("BindParameterArray: NUMERIC type mismatch - param_index=%d, row=%zu", paramIndex, i);2134throwstd::runtime_error(MakeParamMismatchErrorStr(info.paramCType, paramIndex));2135 }2136 NumericData decimalParam = element.cast<NumericData>();!2137LOG_FINEST("BindParameterArray: NUMERIC value - param_index=%d, row=%zu, precision=%d, scale=%d, sign=%d",!2138 paramIndex, i, decimalParam.precision, decimalParam.scale, decimalParam.sign);2139 SQL_NUMERIC_STRUCT& target = numericArray[i];2140std::memset(&target,0,sizeof(SQL_NUMERIC_STRUCT));2141 target.precision = decimalParam.precision;2142 target.scale = decimalParam.scale; Lines 2146-2154 2146std::memcpy(target.val, decimalParam.val.data(), copyLen);2147 }2148 strLenOrIndArray[i] =sizeof(SQL_NUMERIC_STRUCT);2149 }!2150LOG_FINEST("BindParameterArray: SQL_C_NUMERIC bound - param_index=%d, null_values=%zu", paramIndex, null_count);2151 dataPtr = numericArray;2152 bufferLength =sizeof(SQL_NUMERIC_STRUCT);2153break;2154 } Lines 2173-2186 2173 }2174elseif (py::isinstance<py::bytes>(element)) {2175 py::bytes b = element.cast<py::bytes>();2176if (PyBytes_GET_SIZE(b.ptr()) !=16) {!2177LOG_FINER("BindParameterArray: GUID bytes wrong length - param_index=%d, row=%zu, length=%d",!2178 paramIndex, i,PyBytes_GET_SIZE(b.ptr()));2179ThrowStdException("UUID binary data must be exactly 16 bytes long.");2180 }2181std::memcpy(uuid_bytes.data(),PyBytes_AS_STRING(b.ptr()),16);!2182 bytes_count++;2183 }2184elseif (py::isinstance(element, uuid_class)) {2185 py::bytes b = element.attr("bytes_le").cast<py::bytes>();2186std::memcpy(uuid_bytes.data(),PyBytes_AS_STRING(b.ptr()),16); Lines 2186-2194 2186std::memcpy(uuid_bytes.data(), PyBytes_AS_STRING(b.ptr()), 16);2187 uuid_count++;2188 }2189else {!2190LOG_FINER("BindParameterArray: GUID type mismatch - param_index=%d, row=%zu", paramIndex, i);2191ThrowStdException(MakeParamMismatchErrorStr(info.paramCType, paramIndex));2192 }2193 guidArray[i].Data1 = (static_cast<uint32_t>(uuid_bytes[3]) <<24) |2194 (static_cast<uint32_t>(uuid_bytes[2]) <<16) | Lines 2207-2215 2207 bufferLength =sizeof(SQLGUID);2208break;2209 }2210default: {!2211LOG_FINER("BindParameterArray: Unsupported C type - param_index=%d, C_type=%d", paramIndex, info.paramCType);2212ThrowStdException("BindParameterArray: Unsupported C type:" +std::to_string(info.paramCType));2213 }2214 }2215LOG_FINEST("BindParameterArray: Calling SQLBindParameter - param_index=%d, buffer_length=%lld", Lines 2226-2239 2226 bufferLength,2227 strLenOrIndArray2228 );2229if (!SQL_SUCCEEDED(rc)) {!2230LOG_FINER("BindParameterArray: SQLBindParameter failed - param_index=%d, SQLRETURN=%d", paramIndex, rc);2231return rc;2232 }2233 }2234 }catch (...) {!2235LOG_FINER("BindParameterArray: Exception during binding, cleaning up buffers");2236throw;2237 }2238 paramBuffers.insert(paramBuffers.end(), tempBuffers.begin(), tempBuffers.end());2239LOG_FINER("BindParameterArray: Successfully bound all parameters - total_params=%zu, buffer_count=%zu", Lines 2260-2270 2260LOG_FINEST("SQLExecuteMany: Using wide string query directly");2261 #endif2262 RETCODE rc = SQLPrepare_ptr(hStmt, queryPtr, SQL_NTS);2263if (!SQL_SUCCEEDED(rc)) {!2264LOG_FINER("SQLExecuteMany: SQLPrepare failed - rc=%d", rc);!2265return rc;!2266 }2267LOG_FINEST("SQLExecuteMany: Query prepared successfully");22682269bool hasDAE =false;2270for (constauto& p : paramInfos) { Lines 2278-2294 2278LOG_FINER("SQLExecuteMany: Using array binding (non-DAE) - calling BindParameterArray");2279 std::vector<std::shared_ptr<void>> paramBuffers;2280 rc = BindParameterArray(hStmt, columnwise_params, paramInfos, paramSetSize, paramBuffers);2281if (!SQL_SUCCEEDED(rc)) {!2282LOG_FINER("SQLExecuteMany: BindParameterArray failed - rc=%d", rc);!2283return rc;!2284 }22852286 rc = SQLSetStmtAttr_ptr(hStmt, SQL_ATTR_PARAMSET_SIZE, (SQLPOINTER)paramSetSize,0);2287if (!SQL_SUCCEEDED(rc)) {!2288LOG_FINER("SQLExecuteMany: SQLSetStmtAttr(PARAMSET_SIZE) failed - rc=%d", rc);!2289return rc;!2290 }2291LOG_FINEST("SQLExecuteMany: PARAMSET_SIZE set to %zu", paramSetSize);22922293 rc = SQLExecute_ptr(hStmt);2294LOG_FINER("SQLExecuteMany: SQLExecute completed - rc=%d", rc); Lines 2293-2366 2293 rc = SQLExecute_ptr(hStmt);2294LOG_FINER("SQLExecuteMany: SQLExecute completed - rc=%d", rc);2295return rc;2296 }else {!2297LOG_FINER("SQLExecuteMany: Using DAE (data-at-execution) - row_count=%zu", columnwise_params.size());2298size_t rowCount = columnwise_params.size();2299for (size_t rowIndex =0; rowIndex < rowCount; ++rowIndex) {!2300LOG_FINEST("SQLExecuteMany: Processing DAE row %zu of %zu", rowIndex +1, rowCount);2301 py::list rowParams = columnwise_params[rowIndex];23022303 std::vector<std::shared_ptr<void>> paramBuffers;2304 rc =BindParameters(hStmt, rowParams,const_cast<std::vector<ParamInfo>&>(paramInfos), paramBuffers);!2305if (!SQL_SUCCEEDED(rc)) {!2306LOG_FINER("SQLExecuteMany: BindParameters failed for row %zu - rc=%d", rowIndex, rc);!2307return rc;!2308 }!2309LOG_FINEST("SQLExecuteMany: Parameters bound for row %zu", rowIndex);23102311 rc =SQLExecute_ptr(hStmt);!2312LOG_FINEST("SQLExecuteMany: SQLExecute for row %zu - initial_rc=%d", rowIndex, rc);!2313size_t dae_chunk_count =0;2314while (rc == SQL_NEED_DATA) {2315 SQLPOINTER token;2316 rc =SQLParamData_ptr(hStmt, &token);!2317LOG_FINEST("SQLExecuteMany: SQLParamData called - chunk=%zu, rc=%d, token=%p", !2318 dae_chunk_count, rc, token);!2319if (!SQL_SUCCEEDED(rc) && rc != SQL_NEED_DATA) {!2320LOG_FINER("SQLExecuteMany: SQLParamData failed - chunk=%zu, rc=%d", dae_chunk_count, rc);!2321return rc;!2322 }23232324 py::object* py_obj_ptr =reinterpret_cast<py::object*>(token);!2325if (!py_obj_ptr) {!2326LOG_FINER("SQLExecuteMany: NULL token pointer in DAE - chunk=%zu", dae_chunk_count);!2327return SQL_ERROR;!2328 }23292330if (py::isinstance<py::str>(*py_obj_ptr)) {2331 std::string data = py_obj_ptr->cast<std::string>();2332 SQLLEN data_len =static_cast<SQLLEN>(data.size());!2333LOG_FINEST("SQLExecuteMany: Sending string DAE data - chunk=%zu, length=%lld", !2334 dae_chunk_count,static_cast<longlong>(data_len));2335 rc =SQLPutData_ptr(hStmt, (SQLPOINTER)data.c_str(), data_len);!2336if (!SQL_SUCCEEDED(rc) && rc != SQL_NEED_DATA) {!2337LOG_FINER("SQLExecuteMany: SQLPutData(string) failed - chunk=%zu, rc=%d", dae_chunk_count, rc);!2338 }2339 }elseif (py::isinstance<py::bytes>(*py_obj_ptr) || py::isinstance<py::bytearray>(*py_obj_ptr)) {2340 std::string data = py_obj_ptr->cast<std::string>();2341 SQLLEN data_len =static_cast<SQLLEN>(data.size());!2342LOG_FINEST("SQLExecuteMany: Sending bytes/bytearray DAE data - chunk=%zu, length=%lld", !2343 dae_chunk_count,static_cast<longlong>(data_len));2344 rc =SQLPutData_ptr(hStmt, (SQLPOINTER)data.c_str(), data_len);!2345if (!SQL_SUCCEEDED(rc) && rc != SQL_NEED_DATA) {!2346LOG_FINER("SQLExecuteMany: SQLPutData(bytes) failed - chunk=%zu, rc=%d", dae_chunk_count, rc);!2347 }2348 }else {!2349LOG_FINER("SQLExecuteMany: Unsupported DAE data type - chunk=%zu", dae_chunk_count);2350return SQL_ERROR;2351 }!2352 dae_chunk_count++;2353 }!2354LOG_FINEST("SQLExecuteMany: DAE completed for row %zu - total_chunks=%zu, final_rc=%d", !2355 rowIndex, dae_chunk_count, rc);2356 !2357if (!SQL_SUCCEEDED(rc)) {!2358LOG_FINER("SQLExecuteMany: DAE row %zu failed - rc=%d", rowIndex, rc);!2359return rc;!2360 }2361 }!2362LOG_FINER("SQLExecuteMany: All DAE rows processed successfully - total_rows=%zu", rowCount);2363return SQL_SUCCESS;2364 }2365 } Lines 2368-2376 2368// Wrap SQLNumResultCols2369 SQLSMALLINTSQLNumResultCols_wrap(SqlHandlePtr statementHandle) {2370LOG_FINER("SQLNumResultCols: Getting number of columns in result set for statement_handle=%p", (void*)statementHandle->get());2371if (!SQLNumResultCols_ptr) {!2372LOG_FINER("SQLNumResultCols: Function pointer not initialized, loading driver");2373DriverLoader::getInstance().loadDriver();// Load the driver2374 }23752376 SQLSMALLINT columnCount; Lines 2382-2390 2382// Wrap SQLDescribeCol2383 SQLRETURNSQLDescribeCol_wrap(SqlHandlePtr StatementHandle, py::list& ColumnMetadata) {2384LOG_FINER("SQLDescribeCol: Getting column descriptions for statement_handle=%p", (void*)StatementHandle->get());2385if (!SQLDescribeCol_ptr) {!2386LOG_FINER("SQLDescribeCol: Function pointer not initialized, loading driver");2387DriverLoader::getInstance().loadDriver();// Load the driver2388 }23892390 SQLSMALLINT ColumnCount; Lines 2390-2398 2390 SQLSMALLINT ColumnCount;2391 SQLRETURN retcode =2392SQLNumResultCols_ptr(StatementHandle->get(), &ColumnCount);2393if (!SQL_SUCCEEDED(retcode)) {!2394LOG_FINER("SQLDescribeCol: Failed to get number of columns - SQLRETURN=%d", retcode);2395return retcode;2396 }23972398for (SQLUSMALLINT i =1; i <= ColumnCount; ++i) { Lines 2474-2484 2474 }24752476// Wrap SQLFetch to retrieve rows2477 SQLRETURNSQLFetch_wrap(SqlHandlePtr StatementHandle) {!2478LOG_FINER("SQLFetch: Fetching next row for statement_handle=%p", (void*)StatementHandle->get());2479if (!SQLFetch_ptr) {!2480LOG_FINER("SQLFetch: Function pointer not initialized, loading driver");2481DriverLoader::getInstance().loadDriver();// Load the driver2482 }24832484returnSQLFetch_ptr(StatementHandle->get()); Lines 2510-2518 2510 oss <<"Error fetching LOB for column" << colIndex2511 <<", cType=" << cType2512 <<", loop=" << loopCount2513 <<", SQLGetData return=" << ret;!2514LOG_FINER("FetchLobColumnData: %s", oss.str().c_str());2515ThrowStdException(oss.str());2516 }2517if (actualRead == SQL_NULL_DATA) {2518LOG_FINEST("FetchLobColumnData: Column %d is NULL at loop %d", colIndex, loopCount); Lines 2599-2607 2599// Helper function to retrieve column data2600 SQLRETURNSQLGetData_wrap(SqlHandlePtr StatementHandle, SQLUSMALLINT colCount, py::list& row) {2601LOG_FINER("SQLGetData: Getting data from %d columns for statement_handle=%p", colCount, (void*)StatementHandle->get());2602if (!SQLGetData_ptr) {!2603LOG_FINER("SQLGetData: Function pointer not initialized, loading driver");2604DriverLoader::getInstance().loadDriver();// Load the driver2605 }26062607 SQLRETURN ret = SQL_SUCCESS; Lines 2616-2624 26162617 ret = SQLDescribeCol_ptr(hStmt, i, columnName,sizeof(columnName) /sizeof(SQLWCHAR),2618 &columnNameLen, &dataType, &columnSize, &decimalDigits, &nullable);2619if (!SQL_SUCCEEDED(ret)) {!2620LOG_FINER("SQLGetData: Error retrieving metadata for column %d - SQLDescribeCol SQLRETURN=%d", i, ret);2621 row.append(py::none());2622continue;2623 } Lines 2648-2656 2648 row.append(std::string(reinterpret_cast<char*>(dataBuffer.data())));2649 #endif2650 }else {2651// Buffer too small, fallback to streaming!2652LOG_FINER("SQLGetData: CHAR column %d data truncated (buffer_size=%zu), using streaming LOB", i, dataBuffer.size());2653 row.append(FetchLobColumnData(hStmt, i, SQL_C_CHAR,false,false));2654 }2655 }elseif (dataLen == SQL_NULL_DATA) {2656LOG_FINEST("SQLGetData: Column %d is NULL (CHAR)", i); Lines 2657-2672 2657 row.append(py::none());2658 }elseif (dataLen ==0) {2659 row.append(py::str(""));2660 }elseif (dataLen == SQL_NO_TOTAL) {!2661LOG_FINER("SQLGetData: Cannot determine data length (SQL_NO_TOTAL) for column %d (SQL_CHAR), returning NULL", i);2662 row.append(py::none());2663 }elseif (dataLen <0) {!2664LOG_FINER("SQLGetData: Unexpected negative data length for column %d - dataType=%d, dataLen=%ld", i, dataType, (long)dataLen);2665ThrowStdException("SQLGetData returned an unexpected negative data length");2666 }2667 }else {!2668LOG_FINER("SQLGetData: Error retrieving data for column %d (SQL_CHAR) - SQLRETURN=%d, returning NULL", i, ret);2669 row.append(py::none());2670 }2671 }2672break; Lines 2712-2727 2712 row.append(py::none());2713 }elseif (dataLen ==0) {2714 row.append(py::str(""));2715 }elseif (dataLen == SQL_NO_TOTAL) {!2716LOG_FINER("SQLGetData: Cannot determine NVARCHAR data length (SQL_NO_TOTAL) for column %d, returning NULL", i);2717 row.append(py::none());2718 }elseif (dataLen <0) {!2719LOG_FINER("SQLGetData: Unexpected negative data length for column %d (NVARCHAR) - dataLen=%ld", i, (long)dataLen);2720ThrowStdException("SQLGetData returned an unexpected negative data length");2721 }2722 }else {!2723LOG_FINER("SQLGetData: Error retrieving data for column %d (NVARCHAR) - SQLRETURN=%d", i, ret);2724 row.append(py::none());2725 }2726 }2727break; Lines 2741-2749 2741 ret = SQLGetData_ptr(hStmt, i, SQL_C_SHORT, &smallIntValue,0,NULL);2742if (SQL_SUCCEEDED(ret)) {2743 row.append(static_cast<int>(smallIntValue));2744 }else {!2745LOG_FINER("SQLGetData: Error retrieving SQL_SMALLINT for column %d - SQLRETURN=%d", i, ret);2746 row.append(py::none());2747 }2748break;2749 } Lines 2752-2760 2752 ret = SQLGetData_ptr(hStmt, i, SQL_C_FLOAT, &realValue,0,NULL);2753if (SQL_SUCCEEDED(ret)) {2754 row.append(realValue);2755 }else {!2756LOG_FINER("SQLGetData: Error retrieving SQL_REAL for column %d - SQLRETURN=%d", i, ret);2757 row.append(py::none());2758 }2759break;2760 } Lines 2800-2813 2800// Add to row2801 row.append(decimalObj);2802 }catch (const py::error_already_set& e) {2803// If conversion fails, append None!2804LOG_FINER("SQLGetData: Error converting to decimal for column %d - %s", i, e.what());2805 row.append(py::none());2806 }2807 }2808else {!2809LOG_FINER("SQLGetData: Error retrieving SQL_NUMERIC/DECIMAL for column %d - SQLRETURN=%d", i, ret);2810 row.append(py::none());2811 }2812break;2813 } Lines 2818-2826 2818 ret = SQLGetData_ptr(hStmt, i, SQL_C_DOUBLE, &doubleValue,0,NULL);2819if (SQL_SUCCEEDED(ret)) {2820 row.append(doubleValue);2821 }else {!2822LOG_FINER("SQLGetData: Error retrieving SQL_DOUBLE/FLOAT for column %d - SQLRETURN=%d", i, ret);2823 row.append(py::none());2824 }2825break;2826 } Lines 2829-2837 2829 ret = SQLGetData_ptr(hStmt, i, SQL_C_SBIGINT, &bigintValue,0,NULL);2830if (SQL_SUCCEEDED(ret)) {2831 row.append(static_cast<longlong>(bigintValue));2832 }else {!2833LOG_FINER("SQLGetData: Error retrieving SQL_BIGINT for column %d - SQLRETURN=%d", i, ret);2834 row.append(py::none());2835 }2836break;2837 } Lines 2866-2874 2866 timeValue.second2867 )2868 );2869 }else {!2870LOG_FINER("SQLGetData: Error retrieving SQL_TYPE_TIME for column %d - SQLRETURN=%d", i, ret);2871 row.append(py::none());2872 }2873break;2874 } Lines 2890-2898 2890 timestampValue.fraction /1000// Convert back ns to µs2891 )2892 );2893 }else {!2894LOG_FINER("SQLGetData: Error retrieving SQL_TYPE_TIMESTAMP for column %d - SQLRETURN=%d", i, ret);2895 row.append(py::none());2896 }2897break;2898 } Lines 2939-2947 2939 tzinfo2940 );2941 row.append(py_dt);2942 }else {!2943LOG_FINER("SQLGetData: Error fetching DATETIMEOFFSET for column %d - SQLRETURN=%d, indicator=%ld", i, ret, (long)indicator);2944 row.append(py::none());2945 }2946break;2947 } Lines 2972-2984 2972 }else {2973 std::ostringstream oss;2974 oss <<"Unexpected negative length (" << dataLen <<") returned by SQLGetData. ColumnID="2975 << i <<", dataType=" << dataType <<", bufferSize=" << columnSize;!2976LOG_FINER("SQLGetData: %s", oss.str().c_str());2977ThrowStdException(oss.str());2978 }2979 }else {!2980LOG_FINER("SQLGetData: Error retrieving VARBINARY data for column %d - SQLRETURN=%d", i, ret);2981 row.append(py::none());2982 }2983 }2984break; Lines 2988-2996 2988 ret = SQLGetData_ptr(hStmt, i, SQL_C_TINYINT, &tinyIntValue,0,NULL);2989if (SQL_SUCCEEDED(ret)) {2990 row.append(static_cast<int>(tinyIntValue));2991 }else {!2992LOG_FINER("SQLGetData: Error retrieving SQL_TINYINT for column %d - SQLRETURN=%d", i, ret);2993 row.append(py::none());2994 }2995break;2996 } Lines 2999-3007 2999 ret = SQLGetData_ptr(hStmt, i, SQL_C_BIT, &bitValue,0,NULL);3000if (SQL_SUCCEEDED(ret)) {3001 row.append(static_cast<bool>(bitValue));3002 }else {!3003LOG_FINER("SQLGetData: Error retrieving SQL_BIT for column %d - SQLRETURN=%d", i, ret);3004 row.append(py::none());3005 }3006break;3007 } Lines 3029-3037 3029 row.append(uuid_obj);3030 }elseif (indicator == SQL_NULL_DATA) {3031 row.append(py::none());3032 }else {!3033LOG_FINER("SQLGetData: Error retrieving SQL_GUID for column %d - SQLRETURN=%d, indicator=%ld", i, ret, (long)indicator);3034 row.append(py::none());3035 }3036break;3037 } Lines 3039-3047 3039default:3040 std::ostringstream errorString;3041 errorString <<"Unsupported data type for column -" << columnName <<", Type -"3042 << dataType <<", column ID -" << i;!3043LOG_FINER("SQLGetData: %s", errorString.str().c_str());3044ThrowStdException(errorString.str());3045break;3046 }3047 } Lines 3050-3058 30503051 SQLRETURNSQLFetchScroll_wrap(SqlHandlePtr StatementHandle, SQLSMALLINT FetchOrientation, SQLLEN FetchOffset, py::list& row_data) {3052LOG_FINE("SQLFetchScroll_wrap: Fetching with scroll orientation=%d, offset=%ld", FetchOrientation, (long)FetchOffset);3053if (!SQLFetchScroll_ptr) {!3054LOG_FINER("SQLFetchScroll_wrap: Function pointer not initialized. Loading the driver.");3055DriverLoader::getInstance().loadDriver();// Load the driver3056 }30573058// Unbind any columns from previous fetch operations to avoid memory corruption Lines 3212-3220 3212 std::wstring columnName = columnMeta["ColumnName"].cast<std::wstring>();3213 std::ostringstream errorString;3214 errorString <<"Unsupported data type for column -" << columnName.c_str()3215 <<", Type -" << dataType <<", column ID -" << col;!3216LOG_FINER("SQLBindColums: %s", errorString.str().c_str());3217ThrowStdException(errorString.str());3218break;3219 }3220if (!SQL_SUCCEEDED(ret)) { Lines 3221-3229 3221 std::wstring columnName = columnMeta["ColumnName"].cast<std::wstring>();3222 std::ostringstream errorString;3223 errorString <<"Failed to bind column -" << columnName.c_str() <<", Type -"3224 << dataType <<", column ID -" << col;!3225LOG_FINER("SQLBindColums: %s", errorString.str().c_str());3226ThrowStdException(errorString.str());3227return ret;3228 }3229 } Lines 3259-3271 3259 }3260// TODO: variable length data needs special handling, this logic wont suffice3261// This value indicates that the driver cannot determine the length of the data3262if (dataLen == SQL_NO_TOTAL) {!3263LOG_FINER("FetchBatchData: Cannot determine data length for column %d - returning NULL", col);3264 row.append(py::none());3265continue;3266 }elseif (dataLen == SQL_NULL_DATA) {!3267LOG_FINEST("FetchBatchData: Column %d data is NULL", col);3268 row.append(py::none());3269continue;3270 }elseif (dataLen ==0) {3271// Handle zero-length (non-NULL) data Lines 3276-3284 3276 }elseif (dataType == SQL_BINARY || dataType == SQL_VARBINARY || dataType == SQL_LONGVARBINARY) {3277 row.append(py::bytes(""));3278 }else {3279// For other datatypes, 0 length is unexpected. Log & append None!3280LOG_FINER("FetchBatchData: Unexpected 0-length data for column %d (type=%d) - returning NULL", col, dataType);3281 row.append(py::none());3282 }3283continue;3284 }elseif (dataLen <0) { Lines 3282-3290 3282 }3283continue;3284 }elseif (dataLen <0) {3285// Negative value is unexpected, log column index, SQL type & raise exception!3286LOG_FINER("FetchBatchData: Unexpected negative data length - column=%d, SQL_type=%d, dataLen=%ld", col, dataType, (long)dataLen);3287ThrowStdException("Unexpected negative data length, check logs for details");3288 }3289assert(dataLen >0 &&"Data length must be > 0"); Lines 3492-3500 3492 std::wstring columnName = columnMeta["ColumnName"].cast<std::wstring>();3493 std::ostringstream errorString;3494 errorString <<"Unsupported data type for column -" << columnName.c_str()3495 <<", Type -" << dataType <<", column ID -" << col;!3496LOG_FINER("FetchBatchData: %s", errorString.str().c_str());3497ThrowStdException(errorString.str());3498break;3499 }3500 } Lines 3580-3588 3580 std::wstring columnName = columnMeta["ColumnName"].cast<std::wstring>();3581 std::ostringstream errorString;3582 errorString <<"Unsupported data type for column -" << columnName.c_str()3583 <<", Type -" << dataType <<", column ID -" << col;!3584LOG_FINER("calculateRowSize: %s", errorString.str().c_str());3585ThrowStdException(errorString.str());3586break;3587 }3588 } Lines 3612-3620 3612// Retrieve column metadata3613 py::list columnNames;3614 ret = SQLDescribeCol_wrap(StatementHandle, columnNames);3615if (!SQL_SUCCEEDED(ret)) {!3616LOG_FINER("FetchMany_wrap: Failed to get column descriptions - SQLRETURN=%d", ret);3617return ret;3618 }36193620 std::vector<SQLUSMALLINT> lobColumns; Lines 3651-3659 36513652// Bind columns3653 ret = SQLBindColums(hStmt, buffers, columnNames, numCols, fetchSize);3654if (!SQL_SUCCEEDED(ret)) {!3655LOG_FINER("FetchMany_wrap: Error when binding columns - SQLRETURN=%d", ret);3656return ret;3657 }36583659 SQLULEN numRowsFetched; Lines 3661-3669 3661SQLSetStmtAttr_ptr(hStmt, SQL_ATTR_ROWS_FETCHED_PTR, &numRowsFetched,0);36623663 ret = FetchBatchData(hStmt, buffers, columnNames, rows, numCols, numRowsFetched, lobColumns);3664if (!SQL_SUCCEEDED(ret) && ret != SQL_NO_DATA) {!3665LOG_FINER("FetchMany_wrap: Error when fetching data - SQLRETURN=%d", ret);3666return ret;3667 }36683669// Reset attributes before returning to avoid using stack pointers later Lines 3695-3703 3695// Retrieve column metadata3696 py::list columnNames;3697 ret = SQLDescribeCol_wrap(StatementHandle, columnNames);3698if (!SQL_SUCCEEDED(ret)) {!3699LOG_FINER("FetchAll_wrap: Failed to get column descriptions - SQLRETURN=%d", ret);3700return ret;3701 }37023703// Define a memory limit (1 GB) Lines 3772-3780 37723773// Bind columns3774 ret = SQLBindColums(hStmt, buffers, columnNames, numCols, fetchSize);3775if (!SQL_SUCCEEDED(ret)) {!3776LOG_FINER("FetchAll_wrap: Error when binding columns - SQLRETURN=%d", ret);3777return ret;3778 }37793780 SQLULEN numRowsFetched; Lines 3828-3836 3828// Wrap SQLMoreResults3829 SQLRETURNSQLMoreResults_wrap(SqlHandlePtr StatementHandle) {3830LOG_FINE("SQLMoreResults_wrap: Check for more results");3831if (!SQLMoreResults_ptr) {!3832LOG_FINER("SQLMoreResults_wrap: Function pointer not initialized. Loading the driver.");3833DriverLoader::getInstance().loadDriver();// Load the driver3834 }38353836returnSQLMoreResults_ptr(StatementHandle->get()); Lines 3837-3847 3837 }38383839// Wrap SQLFreeHandle3840 SQLRETURNSQLFreeHandle_wrap(SQLSMALLINT HandleType, SqlHandlePtr Handle) {!3841LOG_FINE("SQLFreeHandle_wrap: Free SQL handle type=%d", HandleType);3842if (!SQLAllocHandle_ptr) {!3843LOG_FINER("SQLFreeHandle_wrap: Function pointer not initialized. Loading the driver.");3844DriverLoader::getInstance().loadDriver();// Load the driver3845 }38463847 SQLRETURN ret =SQLFreeHandle_ptr(HandleType, Handle->get()); Lines 3845-3853 3845 }38463847 SQLRETURN ret = SQLFreeHandle_ptr(HandleType, Handle->get());3848if (!SQL_SUCCEEDED(ret)) {!3849LOG_FINER("SQLFreeHandle_wrap: SQLFreeHandle failed with error code - %d", ret);3850return ret;3851 }3852return ret;3853 } Lines 3855-3863 3855// Wrap SQLRowCount3856 SQLLENSQLRowCount_wrap(SqlHandlePtr StatementHandle) {3857LOG_FINE("SQLRowCount_wrap: Get number of rows affected by last execute");3858if (!SQLRowCount_ptr) {!3859LOG_FINER("SQLRowCount_wrap: Function pointer not initialized. Loading the driver.");3860DriverLoader::getInstance().loadDriver();// Load the driver3861 }38623863 SQLLEN rowCount; Lines 3862-3870 38623863 SQLLEN rowCount;3864 SQLRETURN ret = SQLRowCount_ptr(StatementHandle->get(), &rowCount);3865if (!SQL_SUCCEEDED(ret)) {!3866LOG_FINER("SQLRowCount_wrap: SQLRowCount failed with error code - %d", ret);3867return ret;3868 }3869LOG_FINER("SQLRowCount_wrap: SQLRowCount returned %ld", (long)rowCount);3870return rowCount; Lines 4049-4058 4049try {4050mssql_python::logging::LoggerBridge::initialize();4051 }catch (const std::exception& e) {4052// Log initialization failure but don't throw!4053fprintf(stderr,"Logger bridge initialization failed: %s\n", e.what());!4054 }40554056try {4057// Try loading the ODBC driver when the module is imported4058LOG_FINE("Module initialization: Loading ODBC driver"); Lines 4058-4065 4058LOG_FINE("Module initialization: Loading ODBC driver");4059DriverLoader::getInstance().loadDriver();// Load the driver4060 }catch (const std::exception& e) {4061// Log the error but don't throw - let the error happen when functions are called!4062LOG_FINER("Module initialization: Failed to load ODBC driver - %s", e.what());4063 }4064 } mssql_python/pybind/logger_bridge.cppLines 25-34 25 std::lock_guard<std::mutex>lock(mutex_);2627// Skip if already initialized28if (initialized_) {!29return;!30 }3132try {33// Acquire GIL for Python API calls34 py::gil_scoped_acquire gil; Lines 52-65 5253 }catch (const py::error_already_set& e) {54// Failed to initialize - log to stderr and continue55// (logging will be disabled but won't crash)!56 std::cerr <<"LoggerBridge initialization failed:" << e.what() << std::endl;!57 initialized_ =false;!58 }catch (const std::exception& e) {!59 std::cerr <<"LoggerBridge initialization failed:" << e.what() << std::endl;!60 initialized_ =false;!61 }62 }6364voidLoggerBridge::updateLevel(int level) {65// Update the cached level atomically Lines 66-192 66// This is lock-free and can be called from any thread67 cached_level_.store(level, std::memory_order_relaxed);68 }69 !70intLoggerBridge::getLevel() {!71return cached_level_.load(std::memory_order_relaxed);!72 }73 !74boolLoggerBridge::isInitialized() {!75return initialized_;!76 }77 !78 std::stringLoggerBridge::formatMessage(constchar* format, va_list args) {79// Use a stack buffer for most messages (4KB should be enough)!80char buffer[4096];8182// Format the message using safe std::vsnprintf (C++11 standard)83// std::vsnprintf is safe: always null-terminates, never overflows buffer84// DevSkim warning is false positive - this is the recommended safe alternative!85 va_list args_copy;!86va_copy(args_copy, args);!87int result =std::vsnprintf(buffer,sizeof(buffer), format, args_copy);!88va_end(args_copy);89 !90if (result <0) {91// Error during formatting!92return"[Formatting error]";!93 }94 !95if (result <static_cast<int>(sizeof(buffer))) {96// Message fit in buffer (vsnprintf guarantees null-termination)!97returnstd::string(buffer,std::min(static_cast<size_t>(result),sizeof(buffer) -1));!98 }99100// Message was truncated - allocate larger buffer101// (This should be rare for typical log messages)!102 std::vector<char>large_buffer(result +1);!103va_copy(args_copy, args);104// std::vsnprintf is safe here too - proper bounds checking with buffer size!105std::vsnprintf(large_buffer.data(), large_buffer.size(), format, args_copy);!106va_end(args_copy);107 !108returnstd::string(large_buffer.data());!109 }110 !111constchar*LoggerBridge::extractFilename(constchar* path) {112// Extract just the filename from full path using safer C++ string search!113if (!path) {!114return"";!115 }116117// Find last occurrence of Unix path separator!118constchar* filename =std::strrchr(path,'/');!119if (filename) {!120return filename +1;!121 }122123// Try Windows path separator!124 filename =std::strrchr(path,'\\');!125if (filename) {!126return filename +1;!127 }128129// No path separator found, return the whole string!130return path;!131 }132133voidLoggerBridge::log(int level,constchar* file,int line, !134constchar* format, ...) {135// Fast level check (should already be done by macro, but double-check)!136if (!isLoggable(level)) {!137return;!138 }139140// Check if initialized!141if (!initialized_ || !cached_logger_) {!142return;!143 }144145// Format the message!146 va_list args;!147va_start(args, format);!148 std::string message =formatMessage(format, args);!149va_end(args);150151// Extract filename from path!152constchar* filename =extractFilename(file);153154// Format the complete log message with file:line prefix using safe std::snprintf155// std::snprintf is safe: always null-terminates, never overflows buffer156// DevSkim warning is false positive - this is the recommended safe alternative!157char complete_message[4096];!158int written =std::snprintf(complete_message,sizeof(complete_message), !159"[DDBC] %s [%s:%d]", message.c_str(), filename, line);160161// Ensure null-termination (snprintf guarantees this, but be explicit)!162if (written >=static_cast<int>(sizeof(complete_message))) {!163 complete_message[sizeof(complete_message) -1] ='\0';!164 }165166// Lock for Python call (minimize critical section)!167 std::lock_guard<std::mutex>lock(mutex_);168 !169try {170// Acquire GIL for Python API call!171 py::gil_scoped_acquire gil;172173// Call Python logger's log method174// logger.log(level, message)!175 py::handlelogger_handle(cached_logger_);!176 py::object logger_obj = py::reinterpret_borrow<py::object>(logger_handle);177 !178 logger_obj.attr("_log")(level, complete_message);179 !180 }catch (const py::error_already_set& e) {181// Python error during logging - ignore to prevent cascading failures182// (Logging errors should not crash the application)!183 (void)e;// Suppress unused variable warning!184 }catch (const std::exception& e) {185// Other error - ignore!186 (void)e;!187 }!188 }189190 }// namespace logging191 }// namespace mssql_python mssql_python/pybind/logger_bridge.hppLines 146-156 146147 #defineLOG_FINEST(fmt, ...) \ 148 do { \149if (mssql_python::logging::LoggerBridge::isLoggable(mssql_python::logging::LOG_LEVEL_FINEST)) { \!150mssql_python::logging::LoggerBridge::log( \!151 mssql_python::logging::LOG_LEVEL_FINEST, __FILE__, __LINE__, fmt, ##__VA_ARGS__); \!152 } \153 }while(0)154155 #defineLOG_FINER(fmt, ...) \ 156 do { \ Lines 154-164 154155 #defineLOG_FINER(fmt, ...) \ 156 do { \157if (mssql_python::logging::LoggerBridge::isLoggable(mssql_python::logging::LOG_LEVEL_FINER)) { \!158mssql_python::logging::LoggerBridge::log( \!159 mssql_python::logging::LOG_LEVEL_FINER, __FILE__, __LINE__, fmt, ##__VA_ARGS__); \!160 } \161 }while(0)162163 #defineLOG_FINE(fmt, ...) \ 164 do { \ Lines 162-172 162163 #defineLOG_FINE(fmt, ...) \ 164 do { \165if (mssql_python::logging::LoggerBridge::isLoggable(mssql_python::logging::LOG_LEVEL_FINE)) { \!166mssql_python::logging::LoggerBridge::log( \!167 mssql_python::logging::LOG_LEVEL_FINE, __FILE__, __LINE__, fmt, ##__VA_ARGS__); \!168 } \169 }while(0)170171 #defineLOG_INFO(fmt, ...) \ 172 do { \ mssql_python/row.pyLines 138-153 138try:139# Remove braces if present140clean_value=value.strip("{}")141processed_values[i]=uuid.UUID(clean_value)!142conversion_count+=1143except (ValueError,AttributeError):!144logger.finer('_process_uuid_values: Conversion failed for index=%d',i)145pass# Keep original if conversion fails146logger.finest('_process_uuid_values: Converted %d UUID strings to UUID objects',conversion_count)147# Fallback to scanning all columns if indices weren't pre-identified148else:!149logger.finest('_process_uuid_values: Scanning all columns for GUID type')150fori,valueinenumerate(processed_values):151ifvalueisNone:152continue Lines 157-169 157ifsql_type==-11:# SQL_GUID158ifisinstance(value,str):159try:160processed_values[i]=uuid.UUID(value.strip("{}"))!161conversion_count+=1162except (ValueError,AttributeError):!163logger.finer('_process_uuid_values: Scan conversion failed for index=%d',i)164pass!165logger.finest('_process_uuid_values: Scan converted %d UUID strings',conversion_count)166# When native_uuid is False, convert UUID objects to strings167else:168string_conversion_count=0169fori,valueinenumerate(processed_values): Lines 186-194 186 """187frommssql_python.loggingimportlogger188189ifnotself._description:!190logger.finest('_apply_output_converters: No description - returning values as-is')191returnvalues192193logger.finest('_apply_output_converters: Applying converters - value_count=%d',len(values)) 📋 Files Needing Attention📉Files with overall lowest coverage (click to expand)mssql_python.pybind.logger_bridge.cpp: 19.1%mssql_python.pybind.logger_bridge.hpp: 57.1%mssql_python.pybind.ddbc_bindings.cpp: 69.9%mssql_python.row.py: 76.3%mssql_python.pybind.connection.connection.cpp: 81.6%mssql_python.connection.py: 83.2%mssql_python.pybind.connection.connection_pool.cpp: 85.5%mssql_python.auth.py: 87.3%mssql_python.helpers.py: 90.1%mssql_python.logging.py: 92.1% 🔗 Quick Links
|
| // Format the message using safe vsnprintf (always null-terminates) | ||
| va_list args_copy; | ||
| va_copy(args_copy, args); | ||
| int result =std::vsnprintf(buffer,sizeof(buffer), format, args_copy); |
Check warning
Code scanning / devskim
These functions are historically error-prone and have been associated with a significant number of vulnerabilities. Most of these functions have safer alternatives, such as replacing 'strcpy' with 'strlcpy' or 'strcpy_s'.
| // (This should be rare for typical log messages) | ||
| std::vector<char>large_buffer(result +1); | ||
| va_copy(args_copy, args); | ||
| std::vsnprintf(large_buffer.data(), large_buffer.size(), format, args_copy); |
Check warning
Code scanning / devskim
These functions are historically error-prone and have been associated with a significant number of vulnerabilities. Most of these functions have safer alternatives, such as replacing 'strcpy' with 'strlcpy' or 'strcpy_s'.
…nflict and include logger_bridge.hpp in ddbc_bindings.h
- Changed log filename format: timestamp now has no separator (YYYYMMDDHHMMSS)- Added CSV header with metadata: script name, PID, Python version, OS info- Converted log format to CSV: Timestamp, ThreadID, Level, Location, Source, Message- Replaced trace ID with OS native thread ID for debugger compatibility- Updated Python formatter to parse [Python]/[DDBC] tags into Source column- Updated C++ logger_bridge to use makeRecord() for proper file/line attribution- Logs are now easily parseable as CSV for analysis in Excel/pandas- Counter logic for connection/cursor tracking kept internally but not displayed
- Updated LOGGING.md: * Changed log format examples from trace ID to CSV format * Updated filename format (YYYYMMDDHHMMSS with no separators) * Replaced trace ID section with Thread Tracking section * Added CSV parsing examples with pandas * Updated all log output samples to show CSV columns- Updated MSSQL-Python-Logging-Design.md: * Changed file handler config to describe CSV format * Replaced Trace ID System with Thread Tracking System * Updated architecture to reflect OS native thread IDs * Added CSV formatter implementation details * Updated all code examples to use setup_logging() API * Changed log output examples to CSV format- Thread tracking now uses OS native thread IDs (threading.get_native_id())- CSV columns: Timestamp, ThreadID, Level, Location, Source, Message- File header includes metadata (PID, script name, Python version, etc.)- Easy analysis with pandas/Excel/CSV tools
- CSV format now mentioned only once as optional import capability- Focus on log structure and content, not format- Removed repetitive CSV parsing examples- Single section 'Importing Logs as CSV (Optional)' in LOGGING.md- Brief mention in design doc that format is importable as CSV
Features:- Whitelist log file extensions: .txt, .log, .csv only- Raise ValueError for invalid extensions- Export driver_logger for use in application code- Allow apps to use mssql-python's logger: from mssql_python.logging import driver_logger- Updated documentation with usage examples- Added validation in _setLevel() methodBenefits:- Prevents accidental use of wrong file types- Clear error messages for invalid extensions- Unified logging - apps can use same logger as driver- Same format and thread tracking for app logs
- Added 'Executing query:' log at start of execute() method- Removed duplicate log statement that was causing queries to appear twice- executemany() uses existing detailed log (shows parameter set count)- Each query now logged exactly once at DEBUG level- Parameters excluded from basic query log (pending PII review)
Uh oh!
There was an error while loading.Please reload this page.
Work Item / Issue Reference
Summary
This pull request introduces a new, comprehensive logging system to the
mssql_pythondriver, replacing the previous logging approach. The update provides JDBC-style log levels (FINEST, FINER, FINE), enhanced security through automatic sanitization of sensitive data, trace IDs, file rotation, and thread safety. The logging system is now used throughout the codebase, improving diagnostics and maintainability.Key changes include:
Logging System Enhancements:
FINEST,FINER,FINE), automatic sanitization, trace IDs, file rotation, and thread safety. The logging API is now documented in theREADME.mdwith usage examples and a link to detailed documentation.[1][2][3]Codebase Refactoring for Logging:
log()function inconnection.pywith the newloggerobject and its methods (info,debug,warning,error,finest,finer). This standardizes logging and leverages the new features.[1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26]__init__.py,auth.py,connection.py) to use the new logging system.[1][2][3][4]These changes make logging more robust, secure, and easier to use, while also improving the developer experience with better diagnostics and documentation.