- Notifications
You must be signed in to change notification settings - Fork126
Enhance SEA HTTP Client#618
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Merged
Uh oh!
There was an error while loading.Please reload this page.
Merged
Changes fromall commits
Commits
Show all changes
453 commits Select commitHold shift + click to select a range
bf6d41c ensure command_id is not None
varun-edachali-dbx5afa733 line breaks after multi-line pyfocs
varun-edachali-dbxe3dfd36 ensure non null operationHandle for commandId creation
varun-edachali-dbx63360b3 use command_id methods instead of explicit guid_to_hex_id conversion
varun-edachali-dbx13ffb8d remove un-necessary artifacts in test_session, add back assertion
varun-edachali-dbxa74d279 Implement SeaDatabricksClient (Complete Execution Spec) (#590)
varun-edachali-dbxffd478e Merge branch 'sea-migration' into metadata-sea
varun-edachali-dbxf6d873d remove unimplemented metadata methods test, un-necessary imports
varun-edachali-dbx28675f5 introduce unit tests for metadata methods
varun-edachali-dbxd759050 add from __future__ import annotations to remove string literals arou…
varun-edachali-dbx3578659 remove verbosity in ResultSetFilter docstring
varun-edachali-dbx8713023 remove un-necessary info in ResultSetFilter docstring
varun-edachali-dbx22dc252 remove explicit type checking, string literals around forward annotat…
varun-edachali-dbx390f592 house SQL commands in constants
varun-edachali-dbxdd7dc6a convert complex types to string if not _use_arrow_native_complex_types
varun-edachali-dbx28308fe Merge branch 'metadata-sea' into fetch-json-inline
varun-edachali-dbx2712d1c introduce unit tests for altered functionality
varun-edachali-dbxdabba55 Merge branch 'fetch-json-inline' into ext-links-sea
varun-edachali-dbx48ad7b3 Revert "Merge branch 'fetch-json-inline' into ext-links-sea"
varun-edachali-dbxa1f9b9c reduce verbosity of ResultSetFilter docstring
varun-edachali-dbx984e8ee remove unused imports
varun-edachali-dbx3a999c0 Merge branch 'fetch-json-inline' into ext-links-sea
varun-edachali-dbxc313c2b Revert "Merge branch 'fetch-json-inline' into ext-links-sea"
varun-edachali-dbx3bc615e Revert "reduce verbosity of ResultSetFilter docstring"
varun-edachali-dbxb6e1a10 Reapply "Merge branch 'fetch-json-inline' into ext-links-sea"
varun-edachali-dbx2df3d39 Revert "Merge branch 'fetch-json-inline' into ext-links-sea"
varun-edachali-dbx5e75fb5 remove un-necessary filters changes
varun-edachali-dbx20822e4 remove un-necessary backend changes
varun-edachali-dbx802d045 remove constants changes
varun-edachali-dbxf3f795a remove changes in filters tests
varun-edachali-dbxf6c5950 remove unit test backend and JSON queue changes
varun-edachali-dbxd210ccd remove changes in sea result set testing
varun-edachali-dbx22a953e Revert "remove changes in sea result set testing"
varun-edachali-dbx3aed144 Revert "remove unit test backend and JSON queue changes"
varun-edachali-dbx0fe4da4 Revert "remove changes in filters tests"
varun-edachali-dbx0e3c0a1 Revert "remove constants changes"
varun-edachali-dbx93edb93 Revert "remove un-necessary backend changes"
varun-edachali-dbx871a44f Revert "remove un-necessary filters changes"
varun-edachali-dbx0ce144d remove unused imports
varun-edachali-dbx08ca60d Merge branch 'fetch-json-inline' into ext-links-sea
varun-edachali-dbx8c5cc77 working version
varun-edachali-dbx7f5c715 adopy _wait_until_command_done
varun-edachali-dbx9ef5fad introduce metadata commands
varun-edachali-dbx44183db use new backend structure
varun-edachali-dbxd59b351 constrain backend diff
varun-edachali-dbx1edc80a remove changes to filters
varun-edachali-dbxf82658a make _parse methods in models internal
varun-edachali-dbx54eb0a4 reduce changes in unit tests
varun-edachali-dbx50cc1e2 run small queries with SEA during integration tests
varun-edachali-dbx242307a run some tests for sea
varun-edachali-dbx1e21434 move docstring of DatabricksClient within class
varun-edachali-dbxcd4015b move ThriftResultSet import to top of file
varun-edachali-dbxed8b610 make backend/utils __init__ file empty
varun-edachali-dbx94d951e use from __future__ import annotations to remove string literals arou…
varun-edachali-dbxc20058e use lazy logging
varun-edachali-dbxfe3acb1 replace getters with property tag
varun-edachali-dbx9fb6a76 Merge branch 'main' into backend-refactors
varun-edachali-dbx61dfc4d set active_command_id to None, not active_op_handle
varun-edachali-dbx64fb9b2 align test_session with pytest instead of unittest
varun-edachali-dbx8a138e8 allow empty schema bytes for alignment with SEA
varun-edachali-dbx82f9d6b pass is_vl_op to Sea backend ExecuteResponse
varun-edachali-dbx35f1ef0 remove catalog requirement in get_tables
varun-edachali-dbxa515d26 move filters.py to SEA utils
varun-edachali-dbx59b1330 ensure SeaResultSet
varun-edachali-dbxcbf63f9 Merge branch 'main' into sea-migration
varun-edachali-dbx59b4825 remove duplicate test, correct active_command_id attribute
varun-edachali-dbx293e356 Merge branch 'sea-migration' into metadata-sea
varun-edachali-dbxdd40beb prevent circular imports
varun-edachali-dbx14057ac remove unused imports
varun-edachali-dbxa4d5bdb remove cast, throw error if not SeaResultSet
varun-edachali-dbx156421a Merge branch 'metadata-sea' into fetch-json-inline
varun-edachali-dbxeb1a9b4 pass param as TSparkParameterValue
varun-edachali-dbx9000666 Merge branch 'fetch-json-inline' into ext-links-sea
varun-edachali-dbxa3ca7c7 remove failing test (temp)
varun-edachali-dbx2c22010 remove SeaResultSet type assertion
varun-edachali-dbxc09508e change errors to align with spec, instead of arbitrary ValueError
varun-edachali-dbxe9b1314 make SEA backend methods return SeaResultSet
varun-edachali-dbx8ede414 use spec-aligned Exceptions in SEA backend
varun-edachali-dbx09a1b11 remove defensive row type check
varun-edachali-dbxe380654 SeaDatabricksClient: Add Metadata Commands (#593)
varun-edachali-dbx677a7b0 SEA volume operations fix: assign `manifest.is_volume_operation` to `…
varun-edachali-dbx5e01e7b Merge branch 'metadata-sea' into fetch-json-inline
varun-edachali-dbx3becefe Merge branch 'fetch-json-inline' into ext-links-sea
varun-edachali-dbxa026d31 raise ProgrammingError for invalid id
varun-edachali-dbx4446a9e make is_volume_operation strict bool
varun-edachali-dbx138359d remove complex types code
varun-edachali-dbxb99d0c4 Revert "remove complex types code"
varun-edachali-dbx21c389d introduce type conversion for primitive types for JSON + INLINE
varun-edachali-dbx734321a Merge branch 'sea-migration' into fetch-json-inline
varun-edachali-dbx9f0f969 remove SEA running on metadata queries (known failures
varun-edachali-dbx04a1936 remove un-necessary docstrings
varun-edachali-dbx11bc165 preliminary (robust) SEA HTTP Client
varun-edachali-dbxd389316 prevent catching of MaxRetryError and HttpError in client
varun-edachali-dbxcc48caf formatting (black)
varun-edachali-dbx6a1274f fix type annotations
varun-edachali-dbx278b8cd align expected types with databricks sdk
varun-edachali-dbx91b7f7f link rest api reference to validate types
varun-edachali-dbx7a5ae13 remove test_catalogs_returns_arrow_table test
varun-edachali-dbxf1776f3 fix fetchall_arrow and fetchmany_arrow
varun-edachali-dbx6143331 remove thrift aligned test_cancel_during_execute from SEA tests
varun-edachali-dbx45585d4 Introduce manual SEA test scripts for Exec Phase (#589)
varun-edachali-dbx8949d0c Merge branch 'sea-migration' into fetch-json-inline
varun-edachali-dbx5eaded4 remove un-necessary changes in example scripts
varun-edachali-dbxeeed9a1 remove un-necessary chagnes in example scripts
varun-edachali-dbxf233886 _convert_json_table -> _create_json_table
varun-edachali-dbx68ac437 remove accidentally removed test
varun-edachali-dbx7fd0845 remove new unit tests (to be re-added based on new arch)
varun-edachali-dbxea7ff73 remove changes in sea_result_set functionality (to be re-added)
varun-edachali-dbx563da71 introduce more integration tests
varun-edachali-dbxa018273 remove SEA tests in parameterized queries
varun-edachali-dbxc0e98f4 remove partial parameter fix changes
varun-edachali-dbx7343035 remove un-necessary timestamp tests
varun-edachali-dbxec500b6 slightly stronger typing of _convert_json_types
varun-edachali-dbx0b3e91d stronger typing of json utility func s
varun-edachali-dbx7664e44 stronger typing of fetch*_json
varun-edachali-dbxdb7b8e5 remove unused helper methods in SqlType
varun-edachali-dbxf75f2b5 line breaks after multi line pydocs, remove excess logs
varun-edachali-dbxe2d4ef5 line breaks after multi line pydocs, reduce diff of redundant changes
varun-edachali-dbx21e3078 reduce diff of redundant changes
varun-edachali-dbxbb015e6 mandate ResultData in SeaResultSet constructor
varun-edachali-dbx3944e39 Merge branch 'fetch-json-inline' into ext-links-sea
varun-edachali-dbxb3273c7 remove complex type conversion
varun-edachali-dbx38c2b88 correct fetch*_arrow
varun-edachali-dbxebc1915 Merge branch 'ext-links-sea' into sea-http-client
varun-edachali-dbx4651cd6 pass test_retry_exponential_backoff
varun-edachali-dbxd67eb7b prevent parsing empty response data (get test_retry_abort_non_recover…
varun-edachali-dbx2caf38d more defensive parsing, allow more method types in urllib3
varun-edachali-dbx3e55ddd allow Any values in session_conf, cast to String as done in Thrift ba…
varun-edachali-dbx4afff39 account for max_redirects in SEA backend
varun-edachali-dbx01d49cd return empty JsonQueue if no data
varun-edachali-dbx3d8aa7f do not preload content?
varun-edachali-dbx70c7dc8 Complete Fetch Phase (for `INLINE` disposition and `JSON_ARRAY` forma…
varun-edachali-dbxb77acbe Merge branch 'sea-migration' into ext-links-sea
varun-edachali-dbxfa2359d recover old sea tests
varun-edachali-dbxc07f709 move queue and result set into SEA specific dir
varun-edachali-dbx9e4ef2e pass ssl_options into CloudFetchQueue
varun-edachali-dbxb00c06c reduce diff
varun-edachali-dbx10f55f0 remove redundant conversion.py
varun-edachali-dbxcd119e9 fix type issues
varun-edachali-dbxd79638b ValueError not ProgrammingError
varun-edachali-dbxf84578a reduce diff
varun-edachali-dbxc621c0c introduce SEA cloudfetch e2e tests
varun-edachali-dbx7958cd9 allow empty cloudfetch result
varun-edachali-dbxe2d17ff add unit tests for CloudFetchQueue and SeaResultSet
varun-edachali-dbxd348b35 skip pyarrow dependent tests
varun-edachali-dbxabf9aab Merge branch 'main' into sea-migration
varun-edachali-dbx811205e Merge branch 'sea-migration' into ext-links-sea
varun-edachali-dbx9b4b606 Merge branch 'main' into backend-refactors
varun-edachali-dbx4bd290e simplify download process: no pre-fetching
varun-edachali-dbxdfbbf79 correct class name in logs
varun-edachali-dbxb12a8b0 Merge branch 'ext-links-sea' into sea-http-client
varun-edachali-dbx4f11ff0 Introduce `row_limit` param (#607)
varun-edachali-dbxed4d7ab Merge branch 'sea-migration' into ext-links-sea
varun-edachali-dbx51aa9be add sea tag on large queries
varun-edachali-dbxa5e9cdf align with old impl
varun-edachali-dbxbe16634 align next_n_rows with prev imple
varun-edachali-dbx6ec8656 align remaining_rows with prev impl
varun-edachali-dbx7ea7b75 remove un-necessary Optional params
varun-edachali-dbx64be07b remove un-necessary changes in thrift field if tests
varun-edachali-dbx165644c remove unused imports
varun-edachali-dbx3172ab8 Merge branch 'ext-links-sea' into sea-http-client
varun-edachali-dbx461e762 simplify error handling
varun-edachali-dbxfd1e6cf stop mypy complaints
varun-edachali-dbxabef941 run large queries
varun-edachali-dbx15378de run retry tests
varun-edachali-dbx83de7d9 Merge branch 'ext-links-sea' into sea-http-client
varun-edachali-dbx45f5c26 Merge branch 'main' into backend-refactors
varun-edachali-dbx2c9368a formatting (black)
varun-edachali-dbx9b1b1f5 remove repetition from Session.__init__
varun-edachali-dbxfce324b move link fetching immediately before table creation so link expiry i…
varun-edachali-dbx77e23d3 Merge branch 'backend-refactors' into sea-migration
varun-edachali-dbx3bd3aef fix merge artifacts
varun-edachali-dbx6d4701f correct patch paths
varun-edachali-dbx39469fa Merge branch 'sea-migration' into ext-links-sea
varun-edachali-dbxb2d1579 formatting (black)
varun-edachali-dbx077a71c fix types
varun-edachali-dbx7985639 fix param type in unit tests
varun-edachali-dbxdc1cb6d fix type issues
varun-edachali-dbxdfc32b4 Merge branch 'sea-migration' into ext-links-sea
varun-edachali-dbx0a0643b correct param extraction
varun-edachali-dbxf7fd1d9 remove common constructor for databricks client abc
varun-edachali-dbx510b0a3 make SEA Http Client instance a private member
varun-edachali-dbxdd2864b make GetChunksResponse model more robust
varun-edachali-dbxc32b281 add link to doc of GetChunk response model
varun-edachali-dbx0b1eba5 pass result_data instead of "initial links" into SeaCloudFetchQueue
varun-edachali-dbx777f7c1 move download_manager init into parent CloudFetchQueue
varun-edachali-dbx130b0d3 raise ServerOperationError for no 0th chunk
varun-edachali-dbx1920375 unused iports
varun-edachali-dbx5d04cd0 Merge branch 'main' into sea-migration
varun-edachali-dbx922c448 explicitly close result queue
varun-edachali-dbxd882c6e Merge branch 'sea-migration' into ext-links-sea
varun-edachali-dbx7fb59e4 Merge branch 'ext-links-sea' into sea-http-client
varun-edachali-dbx25596e7 simplify HTTP client using requests.session
varun-edachali-dbx2124fe0 explicitly close response object
varun-edachali-dbx404aadf formatting (black)
varun-edachali-dbx01c9b95 remove excess changes
varun-edachali-dbx5a43686 return None in case of empty respose
varun-edachali-dbx28c6bb1 ensure table is empty on no initial link s
varun-edachali-dbx41f1130 Merge branch 'ext-links-sea' into sea-http-client
varun-edachali-dbx1a0575a Complete Fetch Phase (`EXTERNAL_LINKS` disposition and `ARROW` format…
varun-edachali-dbxc07beb1 SEA Session Configuration Fix: Explicitly convert values to `str` (#…
varun-edachali-dbx97707f5 Merge branch 'sea-migration' into sea-http-client
varun-edachali-dbx36daee6 Revert "explicitly close response object"
varun-edachali-dbxb3fb5fa Revert "simplify HTTP client using requests.session"
varun-edachali-dbx7143eff pass error as 4th arg, not 3rd
varun-edachali-dbxe3528a1 add warning for legacy behaviour
varun-edachali-dbx112958d simplify error handling (attempt)
varun-edachali-dbxb7a4677 cleaner HTTP client using requests.sessions
varun-edachali-dbx640cc82 SEA: add support for `Hybrid` disposition (#631)
varun-edachali-dbx3b8e1ad Revert "cleaner HTTP client using requests.sessions"
varun-edachali-dbxc227739 clean up response (implicitly by using with)
varun-edachali-dbxa0aa023 improve docstrings, remove redundant comments and imports
varun-edachali-dbxef9283b unit tests for imp methods
varun-edachali-dbx8fbca9d SEA: Reduce network calls for synchronous commands (#633)
varun-edachali-dbxd8f46cb Merge branch 'sea-migration' into sea-http-client
varun-edachali-dbx806e5f5 SEA: Decouple Link Fetching (#632)
varun-edachali-dbxb57c3f3 Chunk download latency (#634)
saishreeeeeef5836b acquire lock before notif + formatting (black)
varun-edachali-dbx4fd2a3f Merge branch 'main' into sea-migration
varun-edachali-dbx26f8947 fix imports
varun-edachali-dbx2d44596 add get_chunk_link s
varun-edachali-dbx99e7435 simplify description extraction
varun-edachali-dbx54ec080 pass session_id_hex to ThriftResultSet
varun-edachali-dbxf9f9f31 revert to main's extract description
varun-edachali-dbx51cef2b validate row count for sync query tests as well
varun-edachali-dbx387102d guid_hex -> hex_guid
varun-edachali-dbxd53d1ea reduce diff
varun-edachali-dbxc7810aa reduce diff
varun-edachali-dbxb3072bd reduce diff
varun-edachali-dbx8be5264 set .value in compression
varun-edachali-dbx80692e3 reduce diff
varun-edachali-dbxf810ac1 Merge branch 'main' into sea-http-client
varun-edachali-dbx5a13925 formatting (black)
varun-edachali-dbx92ba484 remove redundant test
varun-edachali-dbx2d451b6 move extra_params to the back
varun-edachali-dbx971969a Merge branch 'sea-migration' into sea-http-client
varun-edachali-dbx0e87374 is_direct_results -> has_more_rows
varun-edachali-dbx20c705f Revert "is_direct_results -> has_more_rows"
varun-edachali-dbx2201765 Merge branch 'main' into sea-http-client
varun-edachali-dbxf277b07 stop passing session_id_hex
varun-edachali-dbxe58b6b5 remove redundant comment
varun-edachali-dbx9e086af add extra_params param
varun-edachali-dbx9cfa350 pass extra_params into test_...unset...
varun-edachali-dbx878a845 remove excess session_id_he
varun-edachali-dbx071ad4e reduce changes in DatabricksRetryPolicy
varun-edachali-dbxde339c0 reduce diff in DatabricksRetryPolicy
varun-edachali-dbx2edff78 simple comments on proxy setting
varun-edachali-dbx6c5b37f link docs for getproxies)(
varun-edachali-dbx328c3bd Merge branch 'main' into sea-http-client
varun-edachali-dbxdba08cd rename proxy specific attrs with proxy prefix
varun-edachali-dbxFile filter
Filter by extension
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading.Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading.Please reload this page.
Diff view
Diff view
There are no files selected for viewing
2 changes: 1 addition & 1 deletionsrc/databricks/sql/auth/retry.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -127,7 +127,7 @@ def __init__( | ||
| total=_attempts_remaining, | ||
| respect_retry_after_header=True, | ||
| backoff_factor=self.delay_min, | ||
| allowed_methods=["POST", "GET", "DELETE"], | ||
varun-edachali-dbx marked this conversation as resolved. Show resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| status_forcelist=[429, 503, *self.force_dangerous_codes], | ||
| ) | ||
Oops, something went wrong.
Uh oh!
There was an error while loading.Please reload this page.
Oops, something went wrong.
Uh oh!
There was an error while loading.Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.