- Notifications
You must be signed in to change notification settings - Fork126
SEA: Allow large metadata responses#653
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
* decouple session class from existing Connectionensure maintenance of current APIs of Connection while delegatingresponsibilitySigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add open property to Connection to ensure maintenance of existing APISigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* update unit tests to address ThriftBackend through session instead of through ConnectionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* chore: move session specific tests from test_client to test_sessionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)as in CONTRIBUTING.mdSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* use connection open property instead of long chain through sessionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* trigger integration workflowSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: ensure open attribute of Connection never failsin case the openSession takes long, the initialisation of the sessionwill not complete immediately. This could make the session attributeinaccessible. If the Connection is deleted in this time, the open()check will throw because the session attribute does not exist. Thus, wedefault to the Connection being closed in this case. This was not anissue before because open was a direct attribute of the Connectionclass. Caught in the integration tests.Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: de-complicate earlier connection open logicearlier, one of the integration tests was failing because 'session wasnot an attribute of Connection'. This is likely tied to a localconfiguration issue related to unittest that was causing an error in thetest suite itself. The tests are now passing without checking for thesession attribute.c676f9bSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* Revert "fix: de-complicate earlier connection open logic"This reverts commitd6b1b19.Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* [empty commit] attempt to trigger ci e2e workflowSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* Update CODEOWNERS (#562)new codeownersSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554)* Enhance Cursor close handling and context manager exception management* tests* fmt* Fix Cursor.close() to properly handle CursorAlreadyClosedError* Remove specific test message from Cursor.close() error handling* Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors.* add* addSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* PECOBLR-86 improve logging on python driver (#556)* PECOBLR-86 Improve logging for debug levelSigned-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>* PECOBLR-86 Improve logging for debug levelSigned-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>* fixed formatSigned-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>* used lazy loggingSigned-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>* changed debug to error logsSigned-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>* used lazy loggingSigned-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>---------Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session"This reverts commitdbb2ec5, reversingchanges made to7192f11.Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session"This reverts commitbdb8381.Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: separate session opening logic from instantiationensures correctness of self.session.open call in ConnectionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: use is_open attribute to denote session availabilitySigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: access thrift backend through sessionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* chore: use get_handle() instead of private session attribute in clientSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: remove accidentally removed assertionsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>---------Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>Co-authored-by: Jothi Prakash <jothi.prakash@databricks.com>Co-authored-by: Madhav Sainanee <madhav.sainanee@databricks.com>Co-authored-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>
NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiationslarge merge artifactsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black) + remove excess log (merge artifact)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix typingSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary checkSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary replace callSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* introduce __str__ methods for CommandId and SessionIdSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* docstrings for DatabricksClient interfaceSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* stronger typing of Cursor and ExecuteResponseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove utility functions from backend interface, fix circular importSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* rename info to propertiesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* newline for cleanlinessSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix circular importSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* to_hex_id -> get_hex_idSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* better comment on protocol version getterSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move guid to hex id to new utils moduleSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move staging allowed local path to connection propsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add strong return type for execute_commandSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* skip auth, error handling in databricksclient interfaceSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* chore: docstring + line widthSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* get_id -> get_guidSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* chore: docstringSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: to_hex_id -> to_hex_guidSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>---------Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
…574)* ensure backend client returns a ResultSet type in backend testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* newline for cleanlinessSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix circular importSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* to_hex_id -> get_hex_idSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* better comment on protocol version getterSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* stricter typing for cursorSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* correct typingSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* correct tests and merge artifactsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove accidentally modified workflow filesremnants of old mergeSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* chore: remove accidentally modified workflow filesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add back accidentally removed docstringsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* clean up docstringsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* log hexSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove unnecessary _replace callSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add __str__ for CommandIdSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* take TOpenSessionResp in get_protocol_version to maintain existing interfaceSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* active_op_handle -> active_mmand_idSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* ensure None returned for close_commandSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* account for ResultSet return in new pydocsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* pydoc for typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move common state to ResultSet aprentSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* stronger typing in resultSet behaviourSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove redundant patch in testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add has_been_closed_server_side assertionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove redundancies in testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* more robust close checkSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* use normalised state in e2e testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* simplify corrected testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add line gaps after multi-line pydocs for consistencySigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* use normalised CommandState type in ExecuteResponseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>---------Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* introduce http client (temp) and sea test fileSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce verbositySigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* redundant commentSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce redundancy, params and data separateSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* rename clientSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix type issuesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce repetition in request callsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary elifsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add newline at EOFSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>---------Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* [squashed from prev branch] introduce sea client with session open and close functionalitySigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove accidental changes to workflows (merge artifacts)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* pass test_input to get_protocol_version instead of session_id to maintain previous APISigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black + line gaps after multi-line pydocs)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* use factory for backend instantiationSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix type issuesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove redundant commentsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* introduce models for requests and responsesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove http client and test scriptto prevent diff from showing up post http-client mergeSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* Introduce Sea HTTP Client and test script (#583)* introduce http client (temp) and sea test fileSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce verbositySigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* redundant commentSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce redundancy, params and data separateSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* rename clientSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix type issuesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce repetition in request callsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary elifsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add newline at EOFSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>---------Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* CustomHttpClient -> SeaHttpClientSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* redundant comment in backend clientSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* regex for warehouse_id instead of .split, remove excess imports and behaviourSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove redundant attributesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* [nit] reduce nested codeSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* line gap after multi-line pydocSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* redundant importsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move sea backend and models into separate sea/ dirSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move http client into separate sea/ dirSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* change commands to include ones in docsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add link to sql-ref-parameters for session-confsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add client side filtering for session confs, add note on warehouses over endointsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* test unimplemented methods and max_download_threads propSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>---------Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* [squash from exec-sea] bring over execution phase changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add docstringSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remvoe exec func in sea backendSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess filesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess modelsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess sea backend testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* cleanupSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* re-introduce get_schema_descSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove SeaResultSetSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* clean imports and attributesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* pass CommandId to ExecRespSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove changes in typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add back essential types (ExecResponse, from_sea_state)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix fetch typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* excess importsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce diff by maintaining logsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix int test typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move guid_to_hex_id import to utilsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce diff in guid utils importSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move arrow_schema_bytes back into ExecuteResultSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* maintain logSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary assignmentSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary tuple responseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-ncessary verbose mockingSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move Queue construction to ResultSertSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move description to List[Tuple]Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* frmatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce diff (remove explicit tuple conversion)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove has_more_rows from ExecuteResponseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary has_more_rows aclcSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* default has_more_rows to TrueSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* return has_more_rows from ExecResponse conversion during GetRespMetadataSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove unnecessary replacementSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* better mocked backend namingSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove has_more_rows test in ExecuteResponseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* introduce replacement of original has_more_rows read testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* call correct method in test_use_arrow_schemaSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* call correct method in test_fall_back_to_hive_schemaSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* re-introduce result response read testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* simplify testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess fetch_results mocksSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* more minimal changes to thrift_backend testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move back to old table typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove outdated arrow_schema_bytes returnSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove duplicate importSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* rephrase model docstrings to explicitly denote that they are representations and not used over the wireSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* has_more_rows -> is_direct_resultsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* switch docstring format to align with Connection classSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>---------Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* [squash from exec-sea] bring over execution phase changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add docstringSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remvoe exec func in sea backendSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess filesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess modelsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess sea backend testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* cleanupSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* re-introduce get_schema_descSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove SeaResultSetSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* clean imports and attributesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* pass CommandId to ExecRespSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove changes in typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add back essential types (ExecResponse, from_sea_state)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix fetch typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* excess importsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce diff by maintaining logsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix int test typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* [squashed from exec-sea] init execution funcSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove irrelevant changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove ResultSetFilter functionalitySigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove more irrelevant changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove more irrelevant changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* even more irrelevant changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove sea response as init optionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* exec test example scriptsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* [squashed from sea-exec] merge sea stuffsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess removed docstringSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess changes in backendSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess importsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove accidentally removed _get_schema_descSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove unnecessary init with sea_response testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* rmeove unnecessary changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* improved models and filters from cloudfetch-sea branchSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* filters stuff (align with JDBC)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* backend from cloudfetch-seaSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove filtering, metadata opsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* raise NotImplementedErrror for metadata opsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary backend cahngesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-needed GetChunksResponseonly relevant in Fetch phaseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce code duplicationSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* more clear docstringsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* introduce strongly typed ChunkInfoSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove is_volume_operation from responseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add is_volume_op and more ResultData fieldsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>---------Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* [squash from exec-sea] bring over execution phase changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add docstringSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remvoe exec func in sea backendSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess filesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess modelsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess sea backend testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* cleanupSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* re-introduce get_schema_descSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove SeaResultSetSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* clean imports and attributesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* pass CommandId to ExecRespSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove changes in typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add back essential types (ExecResponse, from_sea_state)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix fetch typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* excess importsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce diff by maintaining logsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix int test typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* [squashed from exec-sea] init execution funcSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove irrelevant changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove ResultSetFilter functionalitySigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove more irrelevant changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove more irrelevant changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* even more irrelevant changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove sea response as init optionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move guid_to_hex_id import to utilsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce diff in guid utils importSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move arrow_schema_bytes back into ExecuteResultSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* maintain logSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary assignmentSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary tuple responseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-ncessary verbose mockingSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move Queue construction to ResultSertSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move description to List[Tuple]Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* frmatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce diff (remove explicit tuple conversion)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove has_more_rows from ExecuteResponseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary has_more_rows aclcSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* default has_more_rows to TrueSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* return has_more_rows from ExecResponse conversion during GetRespMetadataSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove unnecessary replacementSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* better mocked backend namingSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove has_more_rows test in ExecuteResponseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* introduce replacement of original has_more_rows read testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* call correct method in test_use_arrow_schemaSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* call correct method in test_fall_back_to_hive_schemaSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* re-introduce result response read testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* simplify testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess fetch_results mocksSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* more minimal changes to thrift_backend testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move back to old table typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove outdated arrow_schema_bytes returnSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* align SeaResultSet with new structureSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* correct sea res set testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove duplicate importSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* rephrase model docstrings to explicitly denote that they are representations and not used over the wireSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* has_more_rows -> is_direct_resultsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* switch docstring format to align with Connection classSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* has_more_rows -> is_direct_resultsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix type errors with arrow_schema_bytesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* spaces after multi line pydocsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove duplicate queue init (merge artifact)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce diff (remove newlines)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary changescovered by#588 anywaySigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* Revert "remove un-necessary changes"This reverts commita70a6ce.Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* b"" -> NoneSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>---------Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* decouple session class from existing Connectionensure maintenance of current APIs of Connection while delegatingresponsibilitySigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add open property to Connection to ensure maintenance of existing APISigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* update unit tests to address ThriftBackend through session instead of through ConnectionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* chore: move session specific tests from test_client to test_sessionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)as in CONTRIBUTING.mdSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* use connection open property instead of long chain through sessionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* trigger integration workflowSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: ensure open attribute of Connection never failsin case the openSession takes long, the initialisation of the sessionwill not complete immediately. This could make the session attributeinaccessible. If the Connection is deleted in this time, the open()check will throw because the session attribute does not exist. Thus, wedefault to the Connection being closed in this case. This was not anissue before because open was a direct attribute of the Connectionclass. Caught in the integration tests.Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: de-complicate earlier connection open logicearlier, one of the integration tests was failing because 'session wasnot an attribute of Connection'. This is likely tied to a localconfiguration issue related to unittest that was causing an error in thetest suite itself. The tests are now passing without checking for thesession attribute.c676f9bSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* Revert "fix: de-complicate earlier connection open logic"This reverts commitd6b1b19.Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* [empty commit] attempt to trigger ci e2e workflowSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* Update CODEOWNERS (#562)new codeownersSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554)* Enhance Cursor close handling and context manager exception management* tests* fmt* Fix Cursor.close() to properly handle CursorAlreadyClosedError* Remove specific test message from Cursor.close() error handling* Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors.* add* addSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* PECOBLR-86 improve logging on python driver (#556)* PECOBLR-86 Improve logging for debug levelSigned-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>* PECOBLR-86 Improve logging for debug levelSigned-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>* fixed formatSigned-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>* used lazy loggingSigned-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>* changed debug to error logsSigned-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>* used lazy loggingSigned-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>---------Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session"This reverts commitdbb2ec5, reversingchanges made to7192f11.Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session"This reverts commitbdb8381.Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: separate session opening logic from instantiationensures correctness of self.session.open call in ConnectionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: use is_open attribute to denote session availabilitySigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: access thrift backend through sessionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* chore: use get_handle() instead of private session attribute in clientSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: remove accidentally removed assertionsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>---------Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>Co-authored-by: Jothi Prakash <jothi.prakash@databricks.com>Co-authored-by: Madhav Sainanee <madhav.sainanee@databricks.com>Co-authored-by: Sai Shree Pradhan <saishree.pradhan@databricks.com>
NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiationslarge merge artifactsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black) + remove excess log (merge artifact)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix typingSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary checkSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary replace callSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* introduce __str__ methods for CommandId and SessionIdSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* docstrings for DatabricksClient interfaceSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* stronger typing of Cursor and ExecuteResponseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove utility functions from backend interface, fix circular importSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* rename info to propertiesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* newline for cleanlinessSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix circular importSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* to_hex_id -> get_hex_idSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* better comment on protocol version getterSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move guid to hex id to new utils moduleSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move staging allowed local path to connection propsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add strong return type for execute_commandSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* skip auth, error handling in databricksclient interfaceSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* chore: docstring + line widthSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* get_id -> get_guidSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* chore: docstringSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: to_hex_id -> to_hex_guidSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>---------Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
…574)* ensure backend client returns a ResultSet type in backend testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* newline for cleanlinessSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix circular importSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* to_hex_id -> get_hex_idSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* better comment on protocol version getterSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* stricter typing for cursorSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* correct typingSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* correct tests and merge artifactsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove accidentally modified workflow filesremnants of old mergeSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* chore: remove accidentally modified workflow filesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add back accidentally removed docstringsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* clean up docstringsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* log hexSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove unnecessary _replace callSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add __str__ for CommandIdSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* take TOpenSessionResp in get_protocol_version to maintain existing interfaceSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* active_op_handle -> active_mmand_idSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* ensure None returned for close_commandSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* account for ResultSet return in new pydocsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* pydoc for typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move common state to ResultSet aprentSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* stronger typing in resultSet behaviourSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove redundant patch in testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add has_been_closed_server_side assertionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove redundancies in testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* more robust close checkSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* use normalised state in e2e testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* simplify corrected testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add line gaps after multi-line pydocs for consistencySigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* use normalised CommandState type in ExecuteResponseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>---------Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
* [squash from exec-sea] bring over execution phase changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess testSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add docstringSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remvoe exec func in sea backendSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess filesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess modelsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess sea backend testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* cleanupSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* re-introduce get_schema_descSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove SeaResultSetSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* clean imports and attributesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* pass CommandId to ExecRespSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove changes in typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add back essential types (ExecResponse, from_sea_state)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix fetch typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* excess importsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce diff by maintaining logsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix int test typesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* [squashed from exec-sea] init execution funcSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove irrelevant changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove ResultSetFilter functionalitySigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove more irrelevant changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove more irrelevant changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* even more irrelevant changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove sea response as init optionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* exec test example scriptsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* [squashed from sea-exec] merge sea stuffsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess removed docstringSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess changes in backendSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess importsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove accidentally removed _get_schema_descSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove unnecessary init with sea_response testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* rmeove unnecessary changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* improved models and filters from cloudfetch-sea branchSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* filters stuff (align with JDBC)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* backend from cloudfetch-seaSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove filtering, metadata opsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* raise NotImplementedErrror for metadata opsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* change to valid table nameSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary changescovered by#588Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* simplify test moduleSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* logging -> debug levelSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* change table name in logSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary backend cahngesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-needed GetChunksResponseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-needed GetChunksResponseonly relevant in Fetch phaseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce code duplication in response parsingSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce code duplicationSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* more clear docstringsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* introduce strongly typed ChunkInfoSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove is_volume_operation from responseSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add is_volume_op and more ResultData fieldsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add test scriptsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* Revert "Merge branch 'exec-models-sea' into exec-phase-sea"This reverts commitbe1997e, reversingchanges made to37813ba.* change logging levelSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove excess changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove _get_schema_bytes (for now)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* redundant commentsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove fetch phase methodsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce code repetititon + introduce gaps after multi line pydocsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove unused importsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* move description extraction to helper funcSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* formatting (black)Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add more unit testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* streamline unit testsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* test getting the list of allowed configurationsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce diffSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* reduce diffSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* house constants in enums for readability and immutabilitySigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add note on hybrid dispositionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove redundant note on arrow_schema_bytesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove invalid importSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add strong typing for manifest in _extract_descriptionSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove un-necessary column skippingSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* remove parsing in backendSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* fix: convert sea statement id to CommandId typeSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* make polling interval a separate constantSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* align state checking with Thrift implementationSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* update unit tests according to changesSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add unit tests for added methodsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add spec to description extraction docstring, add strong typing to paramsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>* add strong typing for backend parameters argSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>---------Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
…nd forward refs, remove some unused importsSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
…nd CursorSigned-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
do we have integration tests for these?
varun-edachali-dbx commentedJul 31, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There are currently no e2e tests running for the metadata queries for SEA because there is some work to be done with regards to normalisation. I have added a note to add it into#639. For now, I have tested it manually. |
Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>
36d3ec4 intomainUh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
What type of PR is this?
Description
Currently, for metadata queries using the SEA backend,
use_cloud_fetchis hardcoded toFalse. However, this leads to it being less robust to large metadata queries. Thus, here, we re-useuse_cloud_fetchfrom the connection params instead of hardcoding it toFalseto aid robustness.How is this tested?
Related Tickets & Documents
Design Document