- Notifications
You must be signed in to change notification settings - Fork61
feat: initializing the new data client#1238
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:v3_staging
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
| warnings.warn("pool_size no longer supported") | ||
| # set up client info headers for veneer library | ||
| self.client_info=DEFAULT_CLIENT_INFO | ||
| self.client_info=kwargs.get("client_info")orDEFAULT_CLIENT_INFO |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
nit: this can be `kwargs.get("client_info", DEFAULT_CLIENT_INFO)
Uh oh!
There was an error while loading.Please reload this page.
google/cloud/bigtable/client.py Outdated
| _table_data_client=None | ||
| _table_admin_client=None | ||
| _instance_admin_client=None | ||
| _new_table_data_client=None |
daniel-sancheDec 3, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
- We should avoid calling this "new", because someday it will be old
- We don't need internal references for both the gapic client and the veneer data client, since the veneer holds a copy of the gapic.
So we can probably remove _table_data_client and _new_table_data_client, and just have a_data_client, that holds the veneer.
Then indef table_data_client(self):, returnself._data_client._gapic_client
google/cloud/bigtable/client.py Outdated
| returnself._instance_admin_client | ||
| @property | ||
| defnew_table_data_client(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
it makes sense to make a lazy-loader for this, but I don't think we want to expose this publicly
google/cloud/bigtable/client.py Outdated
| """Getter for the new Data Table API. | ||
| TODO: Replace table_data_client with this implementation | ||
| when shimming legacy client is finished. |
daniel-sancheDec 3, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I don't think that's a goal. We should keep returning the gapic client, to keep things consistent (although I wouldn't actually expect many users to use the gapic data cleint direclty)
| warnings.warn("pool_size no longer supported") | ||
| # set up client info headers for veneer library | ||
| self.client_info=DEFAULT_CLIENT_INFO | ||
| self.client_info=kwargs.get("client_info")orDEFAULT_CLIENT_INFO |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
add a comment above this. Maybe something like# client_info argument intended for internal use only, to support legacy client shim
Uh oh!
There was an error while loading.Please reload this page.
| # set up client info headers for veneer library | ||
| self.client_info=DEFAULT_CLIENT_INFO | ||
| self.client_info=kwargs.get("client_info")orDEFAULT_CLIENT_INFO | ||
| self.client_info.client_library_version=self._client_version() |
daniel-sancheDec 3, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I wonder if we want to set a different user agent for the shim... We'll probably want to know how many people are using the shim, to tell us when it can be removed
Maybe that means we should build our own client_info in the legacy constructor, and only attach this version string if none was passed in?
if "_client_info" in kwargs: # use client_info passed in from legacy client self.client_info = kwargs["_client_info"]else: # set up client info headers for veneer library self.client_info = DEFAULT_CLIENT_INFO self.client_info.client_library_version = self._client_version()Uh oh!
There was an error while loading.Please reload this page.
| disable_background_channel_refresh=True, | ||
| ) | ||
| # should create background tasks for each channel | ||
| withmock.patch.object(client,"_ping_and_warm_instances",CrossSync.Mock()): |
daniel-sancheDec 3, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Do you need to mock this? I think that was done in other tests as a crude way of disabling background refresh, but maybe that's not needed with the new argument
| warnings.warn("pool_size no longer supported") | ||
| # set up client info headers for veneer library | ||
| self.client_info=DEFAULT_CLIENT_INFO | ||
| self.client_info=kwargs.get("client_info")orDEFAULT_CLIENT_INFO |
daniel-sancheDec 3, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Another thought: We should use a leading underscore for these undocumented kwargs, to make it extra clear that users shouldn't use them if they find them in the code
google/cloud/bigtable/client.py Outdated
| returnself._instance_admin_client | ||
| @property | ||
| def_data_client(self): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
nit: havingtable_data_client and_data_client is a little weird. Since this is just for internal use, maybe we should use_veneer_data_client to be more specific?
| project=self.project, | ||
| credentials=self._credentials, | ||
| client_options=self._client_options, | ||
| _client_info=self._client_info, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I was thinking we should customize theclient_library_versionlike in the veneer client, so we can specifically see which requests are going through the shim. Maybe `f"{google.cloud.bigtable.version}-shim"?
What do you think? (Maybe ping Mattie?)
google/cloud/bigtable/table.py Outdated
| self.table_id, | ||
| ) | ||
| ifself._instance | ||
| elseNone |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Why do you needif self._instance? It doesn't seem optional?
I don't think we'd ever want this to be None, right?
google/cloud/bigtable/table.py Outdated
| self._table_impl= ( | ||
| self._instance._client._data_client.get_table( | ||
| self._instance.instance_id, | ||
| self.table_id, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This is missing app_profile_id.
And maybemutation_timeout should be mapped todefault_mutate_rows_operation_timeout? Or maybe the timeout can just be passed in with each request. I guess you can leave that part for when you start looking at myutations
daniel-sanche left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I added a couple suggestions, but let me know if I'm missing context
| self.client_info=DEFAULT_CLIENT_INFO | ||
| # Private argument, for internal use only | ||
| self._is_legacy_client=bool(kwargs.get("_is_legacy_client",False)) |
daniel-sancheDec 18, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
If this is just for customizing the client_version/background refresh, I'd prefer to avoid tracking this extra state.
We're already passing in a client_info object, so I'd say we should set the value in the legacy client before passing it in, and then do something like:
if kwargs.get("_client_info): self.client_info = kwargs.get("_client_info")else: self.client_info = DEFAULT_CLIENT_INFO self.client_info.client_library_version = self._client_version()| notself._channel_refresh_task | ||
| andnotself._emulator_host | ||
| andnotself._is_closed.is_set() | ||
| andnotself._is_legacy_client |
daniel-sancheDec 18, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I did like the old version, where there was a flag specifically for disabling background refresh. That could be useful for other purposes too, like testing
Enabling _is_legacy_client could come with other side-effects
Uh oh!
There was an error while loading.Please reload this page.
Changes:
new_table_data_client(might need to be renamed) in the classic client for the new table data clientclient_infoanddisable_background_channel_refresh) to the new data client constructor to support the classic client.Fixes#1157