gcloud beta metastore services create - create a Dataproc Metastore service

gcloud beta metastore services create(SERVICE :--location=LOCATION)[--async][--autoscaling-enabled][--data-catalog-sync][--database-type=DATABASE_TYPE; default="mysql"][--deletion-protection][--endpoint-protocol=ENDPOINT_PROTOCOL; default="thrift"][--hive-metastore-version=HIVE_METASTORE_VERSION][--labels=[KEY=VALUE,…]][--port=PORT; default=9083][--release-channel=RELEASE_CHANNEL; default="stable"][--tags=[KEY=VALUE,…]][--auxiliary-versions=[AUXILIARY_VERSIONS,…]    |--auxiliary-versions-from-file=AUXILIARY_VERSIONS_FROM_FILE][--consumer-subnetworks=[CONSUMER_SUBNETWORKS,…]    |--network=NETWORK    |--network-config-from-file=NETWORK_CONFIG_FROM_FILE][--encryption-kms-key=ENCRYPTION_KMS_KEY    |--encryption-kms-keys=[ENCRYPTION_KMS_KEYS,…]][--hive-metastore-configs=[KEY=VALUE,…]    |--hive-metastore-configs-from-file=PATH_TO_FILE][--instance-size=INSTANCE_SIZE    |--scaling-factor=SCALING_FACTOR    |--tier=TIER    |--max-scaling-factor=MAX_SCALING_FACTOR--min-scaling-factor=MIN_SCALING_FACTOR][--kerberos-principal=KERBEROS_PRINCIPAL--keytab=KEYTAB--krb5-config=KRB5_CONFIG][--maintenance-window-day=MAINTENANCE_WINDOW_DAY--maintenance-window-hour=MAINTENANCE_WINDOW_HOUR][--read-only-regions=[READ_ONLY_REGIONS,…]--read-write-regions=[READ_WRITE_REGIONS,…]][--scheduled-backup-configs-from-file=PATH_TO_FILE    |--enable-scheduled-backup--scheduled-backup-cron=SCHEDULED_BACKUP_CRON--scheduled-backup-location=SCHEDULED_BACKUP_LOCATION][GCLOUD_WIDE_FLAG …]

If run asynchronously with--async, exits after printing oneoperation name that can be used to poll the status of the creation via:

gcloudmetastoreoperationsdescribe

gcloudbetametastoreservicescreatemy-metastore-service--location=us-central1--port=9090

To create a Dataproc Metastore service with the namemy-metastore-service in locationus-central using anon default network foo, run:

gcloudbetametastoreservicescreatemy-metastore-service--location=us-central1--network=foo

Service resource - Arguments and flags that specify the Dataproc Metastoreservice you want to create. The arguments in this group can be used to specifythe attributes of this resource. (NOTE) Some attributes are not given argumentsin this group but can be set in other ways.

To set theproject attribute:

• provide the argumentservice on the command line with a fullyspecified name;
• provide the argument--project on the command line;
• set the propertycore/project.

This must be specified.

SERVICE

ID of the service or fully qualified identifier for the service.

To set theservice attribute:

• provide the argumentservice on the command line.

This positional argument must be specified if any of the other arguments in thisgroup are specified.

--location=LOCATION

The location of the Dataproc Metastore service.

If not specified, will usedefault metastore/location.To set thelocation attribute:

• provide the argumentservice on the command line with a fullyspecified name;
• provide the argument--location on the command line;
• set the propertymetastore/location.

--async

Return immediately, without waiting for the operation in progress to complete.

--autoscaling-enabled

A boolean flag to determine whether Dataproc Metastore autoscaling should beenabled, false if unspecified.

The default minimum and maximum scaling factors are 0.1 and 6.0, respectively.

The minimum and maximum scaling factors can be specified using--min-scaling-factor and --max-scaling-factor.

--data-catalog-sync

A boolean flag to determine whether Dataproc Metastore metadata sync to DataCatalog should be enabled, false if unspecified. Mutually exclusive with flag--encryption-kms-key.

--database-type=DATABASE_TYPE; default="mysql"

The type of database the Dataproc Metastore service will store data in.DATABASE_TYPE must be one of:

mysql

MYSQL database type is a Dataproc Metastore service backed by MySQLCloudSQL.

spanner

SPANNER database type is a Dataproc Metastore service backed byCloud Spanner.

--deletion-protection

Flag that enables delete protection on Dataproc Metastore instance to preventaccidental deletions of the instance. Use --deletion-protection to enable.

--endpoint-protocol=ENDPOINT_PROTOCOL; default="thrift"

The protocol to use for the metastore service endpoint. If unspecified, defaultstoTHRIFT.ENDPOINT_PROTOCOL must be oneof:

grpc

The modernizedGRPC protocol.

thrift

The legacy ApacheTHRIFT protocol.

--hive-metastore-version=HIVE_METASTORE_VERSION

The Hive metastore schema version. The supported versions of a location arelisted via:

gcloudmetastorelocationsdescribe

If unspecified, the default version chosen by the server will be used.

--labels=[KEY=VALUE,…]

List of label KEY=VALUE pairs to add.

Keys must start with a lowercase character and contain only hyphens(-), underscores (_), lowercase characters, andnumbers. Values must contain only hyphens (-), underscores(_), lowercase characters, and numbers.

--port=PORT; default=9083

The TCP port on which the Metastore service will listen. If unspecified, thedefault port 9083 will be used.

--release-channel=RELEASE_CHANNEL; default="stable"

The release channel of the service.RELEASE_CHANNEL mustbe one of:

canary

TheCANARY release channel contains the newest features, which maybe unstable and subject to unresolved issues with no known workarounds. Servicesusing theCANARY release channel are not subject to any SLAs.

stable

TheSTABLE release channel contains features that are consideredstable and have been validated for production use.

--tags=[KEY=VALUE,…]

List of tag KEY=VALUE pairs to add.

Auxiliary versions configuration for the Dataproc Metastore service. Whenspecified, a secondary Hive metastore service is created along with the primaryservice.At most one of these can be specified:

--auxiliary-versions=[AUXILIARY_VERSIONS,…]

Comma-separated list of auxiliary Hive metastore versions to deploy. AuxiliaryHive metastore versions must be less than the primary Hive metastore service'sversion.

--auxiliary-versions-from-file=AUXILIARY_VERSIONS_FROM_FILE

Path to a YAML file containing the auxiliary versions configuration for DataprocMetastore instance. The file should contain a unique auxiliary service name andauxiliary version that is lower than the primary service version. The primaryversion's Hive metastore configs are applied to the auxiliary version.Additional Hive metastore configs can be specified for the auxiliary versionusing "config_overrides". If a specified config property has already beenoverridden in the primary Hive metastore version's configs, the auxiliaryversion's override takes precedence. The contents of the file should bestructured as follows:

YAML:

-name:aux-service1version:x.y.zconfig_overrides:key1:value1key2:value2…-name:aux-service2version:x.y.zconfig_overrides:key1:value1key2:value2……

Network configuration for the metastore service.At most one of these can be specified:

--consumer-subnetworks=[CONSUMER_SUBNETWORKS,…]

The list of subnetworks from which the service can be accessed. This can be thesubnetwork's ID, fully-qualified URL, or relative name in the formprojects/{project_id}/regions/{region_id}/subnetworks/{subnetwork}.

--network=NETWORK

The name of the network on which the service can be accessed. This can be thenetwork's ID, fully-qualified URL, or relative name in the formprojects/{project_id}/global/networks/{network}. If "network","consumer-subnetworks" and "network-config" are unspecified, the "default"network will be used.

--network-config-from-file=NETWORK_CONFIG_FROM_FILE

Path to a YAML file containing the network configuration for Dataproc Metastoreinstance. The contents of the file should be structured as follows:

YAML: consumers:

• subnetwork:projects/{project_id}/regions/{region_id}/subnetworks/{subnetwork_id}
• subnetwork:projects/{project_id}/regions/{region_id}/subnetworks/{subnetwork_id} …

Encryption configuration for the metastore service.At most one of these can be specified:

--encryption-kms-key=ENCRYPTION_KMS_KEY

The name of the Cloud KMS key to use for customer data encryption. Cannot bechanged once the service is created. Mutually exclusive with flag--data-catalog-sync. This can be provided as a fully-qualified URL,or a relative name in the formprojects/{project_id}/locations/{location}/keyRings/{key_ring_id}/cryptoKeys/{crypto_key_id}.

--encryption-kms-keys=[ENCRYPTION_KMS_KEYS,…]

Comma-separated list of the Cloud KMS keys to use for customer data encryption.Cannot be changed once the service is created. Mutually exclusive with flag--data-catalog-sync and--encryption-kms-key. Each keycan be provided as a fully-qualified URL, or a relative name in the formprojects/{project_id}/locations/{location}/keyRings/{key_ring_id}/cryptoKeys/{crypto_key_id}.

Configuration properties specific to running Hive metastore software as themetastore service.At most one of these can be specified:

--hive-metastore-configs=[KEY=VALUE,…]

A mapping of Hive metastore configuration key-value pairs to apply to the Hivemetastore.

--hive-metastore-configs-from-file=PATH_TO_FILE

Path to a XML file containing a mapping of Hive metastore configurationkey-value pairs to apply to the Hive metastore.

For example:

hive-site.xml<configuration><property><name>hive.metastore.warehouse.dir</name><value>${test.warehouse.dir}</value><description></description></property></configuration>

Use a full or relative path to a local file containing the value ofhive_metastore_configs.

The instance size of the Dataproc Metastore instance.At most one of these can be specified:

--instance-size=INSTANCE_SIZE

The instance size of the service.INSTANCE_SIZE must beone of:

extra-large

Extra large instance size, maps to a scaling factor of 6.0.

extra-small

Extra small instance size, maps to a scaling factor of 0.1.

large

Large instance size, maps to a scaling factor of 3.0.

medium

Medium instance size, maps to a scaling factor of 1.0.

small

Small instance size, maps to a scaling factor of 0.5.

--scaling-factor=SCALING_FACTOR

The scaling factor of the service. Can be one of [0.1, 0.2, 0.3, 0.4, 0.5, 0.6,0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0].

--tier=TIER

The tier of the service.TIER must be one of:

developer

The developer tier provides limited scalability and no fault tolerance. Good forlow-cost proof-of-concept.

enterprise

The enterprise tier provides multi-zone high availability, and sufficientscalability for enterprise-level Dataproc Metastore workloads.

The autoscaling configuration of the Dataproc Metastore instance.

--max-scaling-factor=MAX_SCALING_FACTOR

The maximum scaling factor allowed by the autoscaler. Can be one of [0.1, 0.2,0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0]. The defaultvalue is 6.0.

--min-scaling-factor=MIN_SCALING_FACTOR

The minimum scaling factor allowed by the autoscaler. Can be one of [0.1, 0.2,0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0]. The defaultvalue is 0.1.

Information used to configure the Hive metastore service as a service principalin a Kerberos realm.

--kerberos-principal=KERBEROS_PRINCIPAL

A Kerberos principal that exists in the KDC to authenticate as. A typicalprincipal is of the form "primary/instance@REALM", but there is no exact format.

This flag argument must be specified if any of the other arguments in this groupare specified.

--keytab=KEYTAB

A Kerberos keytab file that can be used to authenticate a service principal witha Kerberos Key Distribution Center. This is a Secret Manager secret version, andcan be fully-qualified URL, or relative name in the formprojects/{project_id}/secrets/{secret_id}/versions/{version_id}.

This flag argument must be specified if any of the other arguments in this groupare specified.

--krb5-config=KRB5_CONFIG

A Cloud Storage URI that specifies the path to a krb5.conf file. It is of theform gs://{bucket_name}/path/krb5.conf, although the file does not need to benamed krb5.conf explicitly.

This flag argument must be specified if any of the other arguments in this groupare specified.

The one hour maintenance window that specifies when Dataproc Metastore mayperform system maintenance operation to the service, in UTC time.

--maintenance-window-day=MAINTENANCE_WINDOW_DAY

The day of week when the window starts, e.g.,sun.MAINTENANCE_WINDOW_DAY must be one of:fri,mon,sat,sun,thu,tue,wed.

This flag argument must be specified if any of the other arguments in this groupare specified.

--maintenance-window-hour=MAINTENANCE_WINDOW_HOUR

The hour of day (0-23) when the window starts.

This flag argument must be specified if any of the other arguments in this groupare specified.

Custom region configuration for the metastore service.

--read-only-regions=[READ_ONLY_REGIONS,…]

The list of read-only regions where the metastore service runs in. These regionsshould be part (or subset) of the multi-region.

--read-write-regions=[READ_WRITE_REGIONS,…]

The list of read-write regions where the metastore service runs in. Theseregions should be part (or subset) of the multi-region.

Scheduled backup configurations to enable a scheduled backup of the metastoreservice.At most one of these can be specified:

--scheduled-backup-configs-from-file=PATH_TO_FILE

Path to a JSON file containing a mapping of scheduled backup configurationskey-value pairs to apply to the scheduled backup.

For example:

backup_configs.json{"enabled":true,"cron_schedule":"0 * * * *","backup_location":"gs://example-bucket","time_zone":"UTC"}

Use a full or relative path to a local file containing the value ofscheduled_backup_configs.

Scheduled backup configurations to enable a scheduled backup of the metastoreservice.

--enable-scheduled-backup

Enable or disable scheduled backups for the Metastore service.

--scheduled-backup-cron=SCHEDULED_BACKUP_CRON

Specify the frequency of a scheduled backup of the metastore service.

--scheduled-backup-location=SCHEDULED_BACKUP_LOCATION

Set the Cloud Storage URI to store the scheduled backups of the metastoreservice.

Run$gcloud help for details.