Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Telemetry circuit breaker#658

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
saishreeeee wants to merge6 commits intomain
base:main
Choose a base branch
Loading
fromtele-circuit-breaker
Open
Show file tree
Hide file tree
Changes fromall commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletionREADME.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -11,7 +11,7 @@ You are welcome to file an issue here for general use cases. You can also contac

## Requirements

Python 3.8 or above is required.
Python 3.9 or above is required.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

@jprakash-db are we ok with python version upgrades?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This should be ok, given DBT tests pass.


## Documentation

Expand Down
152 changes: 14 additions & 138 deletionspoetry.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more abouthow customized files appear on GitHub.

7 changes: 4 additions & 3 deletionspyproject.toml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -9,7 +9,7 @@ packages = [{ include = "databricks", from = "src" }]
include = ["CHANGELOG.md"]

[tool.poetry.dependencies]
python = "^3.8.0"
python = "^3.9.0"
thrift = ">=0.16.0,<0.21.0"
pandas = [
{ version = ">=1.2.5,<2.3.0", python = ">=3.8,<3.13" },
Expand All@@ -22,10 +22,11 @@ openpyxl = "^3.0.10"
urllib3 = ">=1.26"
python-dateutil = "^2.8.0"
pyarrow = [
{ version = ">=14.0.1", python = ">=3.8,<3.13", optional=true },
{ version = ">=14.0.1", python = ">=3.9,<3.13", optional=true },
{ version = ">=18.0.0", python = ">=3.13", optional=true }
]
pyjwt = "^2.0.0"
pybreaker = "^1.4.0"


[tool.poetry.extras]
Expand All@@ -38,7 +39,7 @@ pylint = ">=2.12.0"
black = "^22.3.0"
pytest-dotenv = "^0.5.2"
numpy = [
{ version = ">=1.16.6", python = ">=3.8,<3.11" },
{ version = ">=1.16.6", python = ">=3.9,<3.11" },
{ version = ">=1.23.4", python = ">=3.11" },
]

Expand Down
10 changes: 9 additions & 1 deletionsrc/databricks/sql/common/http.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -9,6 +9,7 @@
import logging
from requests.adapters import HTTPAdapter
from databricks.sql.auth.retry import DatabricksRetryPolicy, CommandType
from pybreaker import CircuitBreaker, CircuitBreakerError

logger = logging.getLogger(__name__)

Expand DownExpand Up@@ -109,6 +110,9 @@ class TelemetryHttpClient: # TODO: Unify all the http clients in the PySQL Conn
TELEMETRY_RETRY_DELAY_MAX = 10.0
TELEMETRY_RETRY_STOP_AFTER_ATTEMPTS_DURATION = 30.0

CIRCUIT_BREAKER_FAIL_MAX = 5
CIRCUIT_BREAKER_RESET_TIMEOUT = 60

def __init__(self):
"""Initializes the session and mounts the custom retry adapter."""
retry_policy = DatabricksRetryPolicy(
Expand All@@ -123,6 +127,10 @@ def __init__(self):
self.session = requests.Session()
self.session.mount("https://", adapter)
self.session.mount("http://", adapter)
self.circuit_breaker = CircuitBreaker(
fail_max=self.CIRCUIT_BREAKER_FAIL_MAX,
reset_timeout=self.CIRCUIT_BREAKER_RESET_TIMEOUT,
)

@classmethod
def get_instance(cls) -> "TelemetryHttpClient":
Expand All@@ -141,7 +149,7 @@ def post(self, url: str, **kwargs) -> requests.Response:
This is a blocking call intended to be run in a background thread.
"""
logger.debug("Executing telemetry POST request to: %s", url)
return self.session.post(url, **kwargs)
return self.circuit_breaker.call(self.session.post,url, **kwargs)

def close(self):
"""Closes the underlying requests.Session."""
Expand Down
Loading
Loading

[8]ページ先頭

©2009-2025 Movatter.jp