Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitfa76e31

Browse files
google-labs-jules[bot]chalmerloweLinchin
authored
feat: Add ExternalRuntimeOptions to BigQuery routine (#2311)
* feat: Add ExternalRuntimeOptions to BigQuery routineThis change introduces the `ExternalRuntimeOptions` class to the`google.cloud.bigquery.routine` module, allowing users to configureruntime options for external routines.Key changes:- Created the `ExternalRuntimeOptions` class with setters and getters for `container_memory`, `container_cpu`, `runtime_connection`, `max_batching_rows`, and `runtime_version`.- Updated the `Routine` class to include an `external_runtime_options` property that accepts an `ExternalRuntimeOptions` object.- Added comprehensive unit tests for the new class and its integration with the `Routine` class, including tests for both valid and invalid input values.* Update google/cloud/bigquery/routine/routine.py* feat: Add ExternalRuntimeOptions to BigQuery routineThis change introduces the `ExternalRuntimeOptions` class to the`google.cloud.bigquery.routine` module, allowing users to configureruntime options for external routines.Key changes:- Created the `ExternalRuntimeOptions` class with setters and getters for `container_memory`, `container_cpu`, `runtime_connection`, `max_batching_rows`, and `runtime_version`.- Updated the `Routine` class to include an `external_runtime_options` property that accepts an `ExternalRuntimeOptions` object.- Added comprehensive unit tests for the new class and its integration with the `Routine` class, including tests for both valid and invalid input values.* feat: Add ExternalRuntimeOptions to BigQuery routineThis change introduces the `ExternalRuntimeOptions` class to the`google.cloud.bigquery.routine` module, allowing users to configureruntime options for external routines.Key changes:- Created the `ExternalRuntimeOptions` class with setters and getters for `container_memory`, `container_cpu`, `runtime_connection`, `max_batching_rows`, and `runtime_version`.- Updated the `Routine` class to include an `external_runtime_options` property that accepts an `ExternalRuntimeOptions` object.- Added comprehensive unit tests for the new class and its integration with the `Routine` class, including tests for both valid and invalid input values.- Added additional tests to improve code coverage based on feedback.* feat: Add ExternalRuntimeOptions to BigQuery routineThis change introduces the `ExternalRuntimeOptions` class to the`google.cloud.bigquery.routine` module, allowing users to configureruntime options for external routines.Key changes:- Created the `ExternalRuntimeOptions` class with setters and getters for `container_memory`, `container_cpu`, `runtime_connection`, `max_batching_rows`, and `runtime_version`.- Updated the `Routine` class to include an `external_runtime_options` property that accepts an `ExternalRuntimeOptions` object.- Added comprehensive unit tests for the new class and its integration with the `Routine` class, including tests for both valid and invalid input values.- Added additional tests to improve code coverage based on feedback.- Addressed PyType errors by using helper functions for type conversion.* Update tests/unit/routine/test_external_runtime_options.py* feat: Add ExternalRuntimeOptions to BigQuery routineThis change introduces the `ExternalRuntimeOptions` class to the`google.cloud.bigquery.routine` module, allowing users to configureruntime options for external routines.Key changes:- Created the `ExternalRuntimeOptions` class with setters and getters for `container_memory`, `container_cpu`, `runtime_connection`, `max_batching_rows`, and `runtime_version`.- Updated the `Routine` class to include an `external_runtime_options` property that accepts an `ExternalRuntimeOptions` object.- Added comprehensive unit tests for the new class and its integration with the `Routine` class, including tests for both valid and invalid input values.- Added additional tests to improve code coverage based on feedback.- Addressed PyType errors by using helper functions for type conversion.- Addressed formatting nits from code review.---------Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>Co-authored-by: Chalmer Lowe <chalmerlowe@google.com>Co-authored-by: Lingqing Gan <lingqing.gan@gmail.com>
1 parent7fbd8c2 commitfa76e31

File tree

5 files changed

+421
-1
lines changed

5 files changed

+421
-1
lines changed

‎google/cloud/bigquery/__init__.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -98,6 +98,7 @@
9898
fromgoogle.cloud.bigquery.routineimportRoutineReference
9999
fromgoogle.cloud.bigquery.routineimportRoutineType
100100
fromgoogle.cloud.bigquery.routineimportRemoteFunctionOptions
101+
fromgoogle.cloud.bigquery.routineimportExternalRuntimeOptions
101102
fromgoogle.cloud.bigquery.schemaimportPolicyTagList
102103
fromgoogle.cloud.bigquery.schemaimportSchemaField
103104
fromgoogle.cloud.bigquery.schemaimportFieldElementType
@@ -181,6 +182,7 @@
181182
"RoutineArgument",
182183
"RoutineReference",
183184
"RemoteFunctionOptions",
185+
"ExternalRuntimeOptions",
184186
# Shared helpers
185187
"SchemaField",
186188
"FieldElementType",

‎google/cloud/bigquery/routine/__init__.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
fromgoogle.cloud.bigquery.routine.routineimportRoutineReference
2222
fromgoogle.cloud.bigquery.routine.routineimportRoutineType
2323
fromgoogle.cloud.bigquery.routine.routineimportRemoteFunctionOptions
24+
fromgoogle.cloud.bigquery.routine.routineimportExternalRuntimeOptions
2425

2526

2627
__all__= (
@@ -30,4 +31,5 @@
3031
"RoutineReference",
3132
"RoutineType",
3233
"RemoteFunctionOptions",
34+
"ExternalRuntimeOptions",
3335
)

‎google/cloud/bigquery/routine/routine.py‎

Lines changed: 184 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@
1515
# limitations under the License.
1616

1717
"""Define resources for the BigQuery Routines API."""
18-
18+
importtyping
1919
fromtypingimportAny,Dict,Optional,Union
2020

2121
importgoogle.cloud._helpers# type: ignore
@@ -69,6 +69,7 @@ class Routine(object):
6969
"determinism_level":"determinismLevel",
7070
"remote_function_options":"remoteFunctionOptions",
7171
"data_governance_type":"dataGovernanceType",
72+
"external_runtime_options":"externalRuntimeOptions",
7273
}
7374

7475
def__init__(self,routine_ref,**kwargs)->None:
@@ -349,6 +350,37 @@ def data_governance_type(self, value):
349350
)
350351
self._properties[self._PROPERTY_TO_API_FIELD["data_governance_type"]]=value
351352

353+
@property
354+
defexternal_runtime_options(self):
355+
"""Optional[google.cloud.bigquery.routine.ExternalRuntimeOptions]:
356+
Configures the external runtime options for a routine.
357+
358+
Raises:
359+
ValueError:
360+
If the value is not
361+
:class:`~google.cloud.bigquery.routine.ExternalRuntimeOptions` or
362+
:data:`None`.
363+
"""
364+
prop=self._properties.get(
365+
self._PROPERTY_TO_API_FIELD["external_runtime_options"]
366+
)
367+
ifpropisnotNone:
368+
returnExternalRuntimeOptions.from_api_repr(prop)
369+
370+
@external_runtime_options.setter
371+
defexternal_runtime_options(self,value):
372+
api_repr=value
373+
ifisinstance(value,ExternalRuntimeOptions):
374+
api_repr=value.to_api_repr()
375+
elifvalueisnotNone:
376+
raiseValueError(
377+
"value must be google.cloud.bigquery.routine.ExternalRuntimeOptions "
378+
"or None"
379+
)
380+
self._properties[
381+
self._PROPERTY_TO_API_FIELD["external_runtime_options"]
382+
]=api_repr
383+
352384
@classmethod
353385
deffrom_api_repr(cls,resource:dict)->"Routine":
354386
"""Factory: construct a routine given its API representation.
@@ -736,3 +768,154 @@ def __repr__(self):
736768
forproperty_nameinsorted(self._PROPERTY_TO_API_FIELD)
737769
]
738770
return"RemoteFunctionOptions({})".format(", ".join(all_properties))
771+
772+
773+
classExternalRuntimeOptions(object):
774+
"""Options for the runtime of the external system.
775+
776+
Args:
777+
container_memory (str):
778+
Optional. Amount of memory provisioned for a Python UDF container
779+
instance. Format: {number}{unit} where unit is one of "M", "G", "Mi"
780+
and "Gi" (e.g. 1G, 512Mi). If not specified, the default value is
781+
512Mi. For more information, see `Configure container limits for
782+
Python UDFs <https://cloud.google.com/bigquery/docs/user-defined-functions-python#configure-container-limits>`_
783+
container_cpu (int):
784+
Optional. Amount of CPU provisioned for a Python UDF container
785+
instance. For more information, see `Configure container limits
786+
for Python UDFs <https://cloud.google.com/bigquery/docs/user-defined-functions-python#configure-container-limits>`_
787+
runtime_connection (str):
788+
Optional. Fully qualified name of the connection whose service account
789+
will be used to execute the code in the container. Format:
790+
"projects/{projectId}/locations/{locationId}/connections/{connectionId}"
791+
max_batching_rows (int):
792+
Optional. Maximum number of rows in each batch sent to the external
793+
runtime. If absent or if 0, BigQuery dynamically decides the number of
794+
rows in a batch.
795+
runtime_version (str):
796+
Optional. Language runtime version. Example: python-3.11.
797+
"""
798+
799+
_PROPERTY_TO_API_FIELD= {
800+
"container_memory":"containerMemory",
801+
"container_cpu":"containerCpu",
802+
"runtime_connection":"runtimeConnection",
803+
"max_batching_rows":"maxBatchingRows",
804+
"runtime_version":"runtimeVersion",
805+
}
806+
807+
def__init__(
808+
self,
809+
container_memory:Optional[str]=None,
810+
container_cpu:Optional[int]=None,
811+
runtime_connection:Optional[str]=None,
812+
max_batching_rows:Optional[int]=None,
813+
runtime_version:Optional[str]=None,
814+
_properties:Optional[Dict]=None,
815+
)->None:
816+
if_propertiesisNone:
817+
_properties= {}
818+
self._properties=_properties
819+
820+
ifcontainer_memoryisnotNone:
821+
self.container_memory=container_memory
822+
ifcontainer_cpuisnotNone:
823+
self.container_cpu=container_cpu
824+
ifruntime_connectionisnotNone:
825+
self.runtime_connection=runtime_connection
826+
ifmax_batching_rowsisnotNone:
827+
self.max_batching_rows=max_batching_rows
828+
ifruntime_versionisnotNone:
829+
self.runtime_version=runtime_version
830+
831+
@property
832+
defcontainer_memory(self)->Optional[str]:
833+
"""Optional. Amount of memory provisioned for a Python UDF container instance."""
834+
return_helpers._str_or_none(self._properties.get("containerMemory"))
835+
836+
@container_memory.setter
837+
defcontainer_memory(self,value:Optional[str]):
838+
ifvalueisnotNoneandnotisinstance(value,str):
839+
raiseValueError("container_memory must be a string or None.")
840+
self._properties["containerMemory"]=value
841+
842+
@property
843+
defcontainer_cpu(self)->Optional[int]:
844+
"""Optional. Amount of CPU provisioned for a Python UDF container instance."""
845+
return_helpers._int_or_none(self._properties.get("containerCpu"))
846+
847+
@container_cpu.setter
848+
defcontainer_cpu(self,value:Optional[int]):
849+
ifvalueisnotNoneandnotisinstance(value,int):
850+
raiseValueError("container_cpu must be an integer or None.")
851+
self._properties["containerCpu"]=value
852+
853+
@property
854+
defruntime_connection(self)->Optional[str]:
855+
"""Optional. Fully qualified name of the connection."""
856+
return_helpers._str_or_none(self._properties.get("runtimeConnection"))
857+
858+
@runtime_connection.setter
859+
defruntime_connection(self,value:Optional[str]):
860+
ifvalueisnotNoneandnotisinstance(value,str):
861+
raiseValueError("runtime_connection must be a string or None.")
862+
self._properties["runtimeConnection"]=value
863+
864+
@property
865+
defmax_batching_rows(self)->Optional[int]:
866+
"""Optional. Maximum number of rows in each batch sent to the external runtime."""
867+
returntyping.cast(
868+
int,_helpers._int_or_none(self._properties.get("maxBatchingRows"))
869+
)
870+
871+
@max_batching_rows.setter
872+
defmax_batching_rows(self,value:Optional[int]):
873+
ifvalueisnotNoneandnotisinstance(value,int):
874+
raiseValueError("max_batching_rows must be an integer or None.")
875+
self._properties["maxBatchingRows"]=_helpers._str_or_none(value)
876+
877+
@property
878+
defruntime_version(self)->Optional[str]:
879+
"""Optional. Language runtime version."""
880+
return_helpers._str_or_none(self._properties.get("runtimeVersion"))
881+
882+
@runtime_version.setter
883+
defruntime_version(self,value:Optional[str]):
884+
ifvalueisnotNoneandnotisinstance(value,str):
885+
raiseValueError("runtime_version must be a string or None.")
886+
self._properties["runtimeVersion"]=value
887+
888+
@classmethod
889+
deffrom_api_repr(cls,resource:dict)->"ExternalRuntimeOptions":
890+
"""Factory: construct external runtime options given its API representation.
891+
Args:
892+
resource (Dict[str, object]): Resource, as returned from the API.
893+
Returns:
894+
google.cloud.bigquery.routine.ExternalRuntimeOptions:
895+
Python object, as parsed from ``resource``.
896+
"""
897+
ref=cls()
898+
ref._properties=resource
899+
returnref
900+
901+
defto_api_repr(self)->dict:
902+
"""Construct the API resource representation of this ExternalRuntimeOptions.
903+
Returns:
904+
Dict[str, object]: External runtime options represented as an API resource.
905+
"""
906+
returnself._properties
907+
908+
def__eq__(self,other):
909+
ifnotisinstance(other,ExternalRuntimeOptions):
910+
returnNotImplemented
911+
returnself._properties==other._properties
912+
913+
def__ne__(self,other):
914+
returnnotself==other
915+
916+
def__repr__(self):
917+
all_properties= [
918+
"{}={}".format(property_name,repr(getattr(self,property_name)))
919+
forproperty_nameinsorted(self._PROPERTY_TO_API_FIELD)
920+
]
921+
return"ExternalRuntimeOptions({})".format(", ".join(all_properties))

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp