Package google.cloud.speech.v1

Index

Adaptation

Service that implements Google Cloud Speech Adaptation API.

CreateCustomClass

rpc CreateCustomClass(CreateCustomClassRequest) returns (CustomClass)

Create a custom class.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see theAuthentication Overview.

CreatePhraseSet

rpc CreatePhraseSet(CreatePhraseSetRequest) returns (PhraseSet)

Create a set of phrase hints. Each item in the set can be a single word or a multi-word phrase. The items in the PhraseSet are favored by the recognition model when you send a call that includes the PhraseSet.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see theAuthentication Overview.

DeleteCustomClass

rpc DeleteCustomClass(DeleteCustomClassRequest) returns (Empty)

Delete a custom class.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see theAuthentication Overview.

DeletePhraseSet

rpc DeletePhraseSet(DeletePhraseSetRequest) returns (Empty)

Delete a phrase set.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see theAuthentication Overview.

GetCustomClass

rpc GetCustomClass(GetCustomClassRequest) returns (CustomClass)

Get a custom class.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see theAuthentication Overview.

GetPhraseSet

rpc GetPhraseSet(GetPhraseSetRequest) returns (PhraseSet)

Get a phrase set.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see theAuthentication Overview.

ListCustomClasses

rpc ListCustomClasses(ListCustomClassesRequest) returns (ListCustomClassesResponse)

List custom classes.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see theAuthentication Overview.

ListPhraseSet

rpc ListPhraseSet(ListPhraseSetRequest) returns (ListPhraseSetResponse)

List phrase sets.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see theAuthentication Overview.

UpdateCustomClass

rpc UpdateCustomClass(UpdateCustomClassRequest) returns (CustomClass)

Update a custom class.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see theAuthentication Overview.

UpdatePhraseSet

rpc UpdatePhraseSet(UpdatePhraseSetRequest) returns (PhraseSet)

Update a phrase set.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see theAuthentication Overview.

Speech

Service that implements Google Cloud Speech API.

LongRunningRecognize

rpc LongRunningRecognize(LongRunningRecognizeRequest) returns (Operation)

Performs asynchronous speech recognition: receive results via the google.longrunning.Operations interface. Returns either anOperation.error or anOperation.response which contains aLongRunningRecognizeResponse message. For more information on asynchronous speech recognition, see thehow-to.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see theAuthentication Overview.

Recognize

rpc Recognize(RecognizeRequest) returns (RecognizeResponse)

Performs synchronous speech recognition: receive results after all audio has been sent and processed.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see theAuthentication Overview.

StreamingRecognize

rpc StreamingRecognize(StreamingRecognizeRequest) returns (StreamingRecognizeResponse)

Performs bidirectional streaming speech recognition: receive results while sending audio. This method is only available via the gRPC API (not REST).

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see theAuthentication Overview.

CreateCustomClassRequest

Message sent by the client for theCreateCustomClass method.

Fields
parent

string

Required. The parent resource where this custom class will be created. Format:

projects/{project}/locations/{location}/customClasses

Speech-to-Text supports three locations:global,us (US North America), andeu (Europe). If you are calling thespeech.googleapis.com endpoint, use theglobal location. To specify a region, use aregional endpoint with matchingus oreu location value.

Authorization requires the followingIAM permission on the specified resourceparent:

  • speech.customClasses.create
custom_class_id

string

Required. The ID to use for the custom class, which will become the final component of the custom class' resource name.

This value should restrict to letters, numbers, and hyphens, with the first character a letter, the last a letter or a number, and be 4-63 characters.

custom_class

CustomClass

Required. The custom class to create.

CreatePhraseSetRequest

Message sent by the client for theCreatePhraseSet method.

Fields
parent

string

Required. The parent resource where this phrase set will be created. Format:

projects/{project}/locations/{location}

Speech-to-Text supports three locations:global,us (US North America), andeu (Europe). If you are calling thespeech.googleapis.com endpoint, use theglobal location. To specify a region, use aregional endpoint with matchingus oreu location value.

Authorization requires the followingIAM permission on the specified resourceparent:

  • speech.phraseSets.create
phrase_set_id

string

Required. The ID to use for the phrase set, which will become the final component of the phrase set's resource name.

This value should restrict to letters, numbers, and hyphens, with the first character a letter, the last a letter or a number, and be 4-63 characters.

phrase_set

PhraseSet

Required. The phrase set to create.

CustomClass

A set of words or phrases that represents a common concept likely to appear in your audio, for example a list of passenger ship names. CustomClass items can be substituted into placeholders that you set in PhraseSet phrases.

Fields
name

string

The resource name of the custom class.

custom_class_id

string

If this custom class is a resource, the custom_class_id is the resource id of the CustomClass. Case sensitive.

items[]

ClassItem

A collection of class items.

kms_key_name

string

Output only. TheKMS key name with which the content of the ClassItem is encrypted. The expected format isprojects/{project}/locations/{location}/keyRings/{key_ring}/cryptoKeys/{crypto_key}.

kms_key_version_name

string

Output only. TheKMS key version name with which content of the ClassItem is encrypted. The expected format isprojects/{project}/locations/{location}/keyRings/{key_ring}/cryptoKeys/{crypto_key}/cryptoKeyVersions/{crypto_key_version}.

uid

string

Output only. System-assigned unique identifier for the CustomClass. This field is not used.

display_name

string

Output only. User-settable, human-readable name for the CustomClass. Must be 63 characters or less. This field is not used.

state

State

Output only. The CustomClass lifecycle state. This field is not used.

delete_time

Timestamp

Output only. The time at which this resource was requested for deletion. This field is not used.

expire_time

Timestamp

Output only. The time at which this resource will be purged. This field is not used.

annotations

map<string, string>

Output only. Allows users to store small amounts of arbitrary data. Both the key and the value must be 63 characters or less each. At most 100 annotations. This field is not used.

etag

string

Output only. This checksum is computed by the server based on the value of other fields. This may be sent on update, undelete, and delete requests to ensure the client has an up-to-date value before proceeding. This field is not used.

reconciling

bool

Output only. Whether or not this CustomClass is in the process of being updated. This field is not used.

ClassItem

An item of the class.

Fields
value

string

The class item's value.

State

Set of states that define the lifecycle of a CustomClass.

Enums
STATE_UNSPECIFIEDUnspecified state. This is only used/useful for distinguishing unset values.
ACTIVEThe normal and active state.
DELETEDThis CustomClass has been deleted.

DeleteCustomClassRequest

Message sent by the client for theDeleteCustomClass method.

Fields
name

string

Required. The name of the custom class to delete. Format:

projects/{project}/locations/{location}/customClasses/{custom_class}

Speech-to-Text supports three locations:global,us (US North America), andeu (Europe). If you are calling thespeech.googleapis.com endpoint, use theglobal location. To specify a region, use aregional endpoint with matchingus oreu location value.

Authorization requires the followingIAM permission on the specified resourcename:

  • speech.customClasses.delete

DeletePhraseSetRequest

Message sent by the client for theDeletePhraseSet method.

Fields
name

string

Required. The name of the phrase set to delete. Format:

projects/{project}/locations/{location}/phraseSets/{phrase_set}

Authorization requires the followingIAM permission on the specified resourcename:

  • speech.phraseSets.delete

GetCustomClassRequest

Message sent by the client for theGetCustomClass method.

Fields
name

string

Required. The name of the custom class to retrieve. Format:

projects/{project}/locations/{location}/customClasses/{custom_class}

Authorization requires the followingIAM permission on the specified resourcename:

  • speech.customClasses.get

GetPhraseSetRequest

Message sent by the client for theGetPhraseSet method.

Fields
name

string

Required. The name of the phrase set to retrieve. Format:

projects/{project}/locations/{location}/phraseSets/{phrase_set}

Speech-to-Text supports three locations:global,us (US North America), andeu (Europe). If you are calling thespeech.googleapis.com endpoint, use theglobal location. To specify a region, use aregional endpoint with matchingus oreu location value.

Authorization requires the followingIAM permission on the specified resourcename:

  • speech.phraseSets.get

ListCustomClassesRequest

Message sent by the client for theListCustomClasses method.

Fields
parent

string

Required. The parent, which owns this collection of custom classes. Format:

projects/{project}/locations/{location}/customClasses

Speech-to-Text supports three locations:global,us (US North America), andeu (Europe). If you are calling thespeech.googleapis.com endpoint, use theglobal location. To specify a region, use aregional endpoint with matchingus oreu location value.

Authorization requires the followingIAM permission on the specified resourceparent:

  • speech.customClasses.list
page_size

int32

The maximum number of custom classes to return. The service may return fewer than this value. If unspecified, at most 50 custom classes will be returned. The maximum value is 1000; values above 1000 will be coerced to 1000.

page_token

string

A page token, received from a previousListCustomClass call. Provide this to retrieve the subsequent page.

When paginating, all other parameters provided toListCustomClass must match the call that provided the page token.

ListCustomClassesResponse

Message returned to the client by theListCustomClasses method.

Fields
custom_classes[]

CustomClass

The custom classes.

next_page_token

string

A token, which can be sent aspage_token to retrieve the next page. If this field is omitted, there are no subsequent pages.

ListPhraseSetRequest

Message sent by the client for theListPhraseSet method.

Fields
parent

string

Required. The parent, which owns this collection of phrase set. Format:

projects/{project}/locations/{location}

Speech-to-Text supports three locations:global,us (US North America), andeu (Europe). If you are calling thespeech.googleapis.com endpoint, use theglobal location. To specify a region, use aregional endpoint with matchingus oreu location value.

Authorization requires the followingIAM permission on the specified resourceparent:

  • speech.phraseSets.list
page_size

int32

The maximum number of phrase sets to return. The service may return fewer than this value. If unspecified, at most 50 phrase sets will be returned. The maximum value is 1000; values above 1000 will be coerced to 1000.

page_token

string

A page token, received from a previousListPhraseSet call. Provide this to retrieve the subsequent page.

When paginating, all other parameters provided toListPhraseSet must match the call that provided the page token.

ListPhraseSetResponse

Message returned to the client by theListPhraseSet method.

Fields
phrase_sets[]

PhraseSet

The phrase set.

next_page_token

string

A token, which can be sent aspage_token to retrieve the next page. If this field is omitted, there are no subsequent pages.

LongRunningRecognizeMetadata

Describes the progress of a long-runningLongRunningRecognize call. It is included in themetadata field of theOperation returned by theGetOperation call of thegoogle::longrunning::Operations service.

Fields
progress_percent

int32

Approximate percentage of audio processed thus far. Guaranteed to be 100 when the audio is fully processed and the results are available.

start_time

Timestamp

Time when the request was received.

last_update_time

Timestamp

Time of the most recent processing update.

uri

string

Output only. The URI of the audio file being transcribed. Empty if the audio was sent as byte content.

LongRunningRecognizeRequest

The top-level message sent by the client for theLongRunningRecognize method.

Fields
config

RecognitionConfig

Required. Provides information to the recognizer that specifies how to process the request.

audio

RecognitionAudio

Required. The audio data to be recognized.

output_config

TranscriptOutputConfig

Optional. Specifies an optional destination for the recognition results.

LongRunningRecognizeResponse

The only message returned to the client by theLongRunningRecognize method. It contains the result as zero or more sequentialSpeechRecognitionResult messages. It is included in theresult.response field of theOperation returned by theGetOperation call of thegoogle::longrunning::Operations service.

Fields
results[]

SpeechRecognitionResult

Sequential list of transcription results corresponding to sequential portions of audio.

total_billed_time

Duration

When available, billed audio seconds for the corresponding request.

output_config

TranscriptOutputConfig

Original output config if present in the request.

output_error

Status

If the transcript output fails this field contains the relevant error.

speech_adaptation_info

SpeechAdaptationInfo

Provides information on speech adaptation behavior in response

request_id

int64

The ID associated with the request. This is a unique ID specific only to the given request.

PhraseSet

Provides "hints" to the speech recognizer to favor specific words and phrases in the results.

Fields
name

string

The resource name of the phrase set.

phrases[]

Phrase

A list of word and phrases.

boost

float

Hint Boost. Positive value will increase the probability that a specific phrase will be recognized over other similar sounding phrases. The higher the boost, the higher the chance of false positive recognition as well. Negative boost values would correspond to anti-biasing. Anti-biasing is not enabled, so negative boost will simply be ignored. Thoughboost can accept a wide range of positive values, most use cases are best served with values between 0 (exclusive) and 20. We recommend using a binary search approach to finding the optimal value for your use case as well as adding phrases both with and without boost to your requests.

kms_key_name

string

Output only. TheKMS key name with which the content of the PhraseSet is encrypted. The expected format isprojects/{project}/locations/{location}/keyRings/{key_ring}/cryptoKeys/{crypto_key}.

kms_key_version_name

string

Output only. TheKMS key version name with which content of the PhraseSet is encrypted. The expected format isprojects/{project}/locations/{location}/keyRings/{key_ring}/cryptoKeys/{crypto_key}/cryptoKeyVersions/{crypto_key_version}.

uid

string

Output only. System-assigned unique identifier for the PhraseSet. This field is not used.

display_name

string

Output only. User-settable, human-readable name for the PhraseSet. Must be 63 characters or less. This field is not used.

state

State

Output only. The CustomClass lifecycle state. This field is not used.

delete_time

Timestamp

Output only. The time at which this resource was requested for deletion. This field is not used.

expire_time

Timestamp

Output only. The time at which this resource will be purged. This field is not used.

annotations

map<string, string>

Output only. Allows users to store small amounts of arbitrary data. Both the key and the value must be 63 characters or less each. At most 100 annotations. This field is not used.

etag

string

Output only. This checksum is computed by the server based on the value of other fields. This may be sent on update, undelete, and delete requests to ensure the client has an up-to-date value before proceeding. This field is not used.

reconciling

bool

Output only. Whether or not this PhraseSet is in the process of being updated. This field is not used.

Phrase

A phrases containing words and phrase "hints" so that the speech recognition is more likely to recognize them. This can be used to improve the accuracy for specific words and phrases, for example, if specific commands are typically spoken by the user. This can also be used to add additional words to the vocabulary of the recognizer. Seeusage limits.

List items can also include pre-built or custom classes containing groups of words that represent common concepts that occur in natural language. For example, rather than providing a phrase hint for every month of the year (e.g. "i was born in january", "i was born in febuary", ...), use the pre-built$MONTH class improves the likelihood of correctly transcribing audio that includes months (e.g. "i was born in $month"). To refer to pre-built classes, use the class' symbol prepended with$ e.g.$MONTH. To refer to custom classes that were defined inline in the request, set the class'scustom_class_id to a string unique to all class resources and inline classes. Then use the class' id wrapped in ${...} e.g. "${my-months}". To refer to custom classes resources, use the class' id wrapped in${} (e.g.${my-months}).

Speech-to-Text supports three locations:global,us (US North America), andeu (Europe). If you are calling thespeech.googleapis.com endpoint, use theglobal location. To specify a region, use aregional endpoint with matchingus oreu location value.

Fields
value

string

The phrase itself.

boost

float

Hint Boost. Overrides the boost set at the phrase set level. Positive value will increase the probability that a specific phrase will be recognized over other similar sounding phrases. The higher the boost, the higher the chance of false positive recognition as well. Negative boost will simply be ignored. Thoughboost can accept a wide range of positive values, most use cases are best served with values between 0 and 20. We recommend using a binary search approach to finding the optimal value for your use case as well as adding phrases both with and without boost to your requests.

State

Set of states that define the lifecycle of a CustomClass.

Enums
STATE_UNSPECIFIEDUnspecified state. This is only used/useful for distinguishing unset values.
ACTIVEThe normal and active state.
DELETEDThis CustomClass has been deleted.

RecognitionAudio

Contains audio data in the encoding specified in theRecognitionConfig. Eithercontent oruri must be supplied. Supplying both or neither returnsgoogle.rpc.Code.INVALID_ARGUMENT. Seecontent limits.

Fields
Union fieldaudio_source. The audio source, which is either inline content or a Google Cloud Storage uri.audio_source can be only one of the following:
content

bytes

The audio data bytes encoded as specified inRecognitionConfig. Note: as with all bytes fields, proto buffers use a pure binary representation, whereas JSON representations use base64.

uri

string

URI that points to a file that contains audio data bytes as specified inRecognitionConfig. The file must not be compressed (for example, gzip). Currently, only Google Cloud Storage URIs are supported, which must be specified in the following format:gs://bucket_name/object_name (other URI formats returngoogle.rpc.Code.INVALID_ARGUMENT). For more information, seeRequest URIs.

RecognitionConfig

Provides information to the recognizer that specifies how to process the request.

Fields
encoding

AudioEncoding

Encoding of audio data sent in allRecognitionAudio messages. This field is optional forFLAC andWAV audio files and required for all other audio formats. For details, seeAudioEncoding.

sample_rate_hertz

int32

Sample rate in Hertz of the audio data sent in allRecognitionAudio messages. Valid values are: 8000-48000. 16000 is optimal. For best results, set the sampling rate of the audio source to 16000 Hz. If that's not possible, use the native sample rate of the audio source (instead of re-sampling). This field is optional for FLAC and WAV audio files, but is required for all other audio formats. For details, seeAudioEncoding.

audio_channel_count

int32

The number of channels in the input audio data. ONLY set this for MULTI-CHANNEL recognition. Valid values for LINEAR16, OGG_OPUS and FLAC are1-8. Valid value for MULAW, AMR, AMR_WB and SPEEX_WITH_HEADER_BYTE is only1. If0 or omitted, defaults to one channel (mono). Note: We only recognize the first channel by default. To perform independent recognition on each channel setenable_separate_recognition_per_channel to 'true'.

enable_separate_recognition_per_channel

bool

This needs to be set totrue explicitly andaudio_channel_count > 1 to get each channel recognized separately. The recognition result will contain achannel_tag field to state which channel that result belongs to. If this is not true, we will only recognize the first channel. The request is billed cumulatively for all channels recognized:audio_channel_count multiplied by the length of the audio.

language_code

string

Required. The language of the supplied audio as aBCP-47 language tag. Example: "en-US". SeeLanguage Support for a list of the currently supported language codes.

alternative_language_codes[]

string

A list of up to 3 additionalBCP-47 language tags, listing possible alternative languages of the supplied audio. SeeLanguage Support for a list of the currently supported language codes. If alternative languages are listed, recognition result will contain recognition in the most likely language detected including the main language_code. The recognition result will include the language tag of the language detected in the audio. Note: This feature is only supported for Voice Command and Voice Search use cases and performance may vary for other use cases (e.g., phone call transcription).

max_alternatives

int32

Maximum number of recognition hypotheses to be returned. Specifically, the maximum number ofSpeechRecognitionAlternative messages within eachSpeechRecognitionResult. The server may return fewer thanmax_alternatives. Valid values are0-30. A value of0 or1 will return a maximum of one. If omitted, will return a maximum of one.

profanity_filter

bool

If set totrue, the server will attempt to filter out profanities, replacing all but the initial character in each filtered word with asterisks, e.g. "f***". If set tofalse or omitted, profanities won't be filtered out.

adaptation

SpeechAdaptation

Speech adaptation configuration improves the accuracy of speech recognition. For more information, see thespeech adaptation documentation. When speech adaptation is set it supersedes thespeech_contexts field.

transcript_normalization

TranscriptNormalization

Optional. Use transcription normalization to automatically replace parts of the transcript with phrases of your choosing. For StreamingRecognize, this normalization only applies to stable partial transcripts (stability > 0.8) and final transcripts.

speech_contexts[]

SpeechContext

Array ofSpeechContext. A means to provide context to assist the speech recognition. For more information, seespeech adaptation.

enable_word_time_offsets

bool

Iftrue, the top result includes a list of words and the start and end time offsets (timestamps) for those words. Iffalse, no word-level time offset information is returned. The default isfalse.

enable_word_confidence

bool

Iftrue, the top result includes a list of words and the confidence for those words. Iffalse, no word-level confidence information is returned. The default isfalse.

enable_automatic_punctuation

bool

If 'true', adds punctuation to recognition result hypotheses. This feature is only available in select languages. Setting this for requests in other languages has no effect at all. The default 'false' value does not add punctuation to result hypotheses.

enable_spoken_punctuation

BoolValue

The spoken punctuation behavior for the call If not set, uses default behavior based on model of choice e.g. command_and_search will enable spoken punctuation by default If 'true', replaces spoken punctuation with the corresponding symbols in the request. For example, "how are you question mark" becomes "how are you?". Seehttps://cloud.google.com/speech-to-text/docs/spoken-punctuation for support. If 'false', spoken punctuation is not replaced.

enable_spoken_emojis

BoolValue

The spoken emoji behavior for the call If not set, uses default behavior based on model of choice If 'true', adds spoken emoji formatting for the request. This will replace spoken emojis with the corresponding Unicode symbols in the final transcript. If 'false', spoken emojis are not replaced.

diarization_config

SpeakerDiarizationConfig

Config to enable speaker diarization and set additional parameters to make diarization better suited for your application. Note: When this is enabled, we send all the words from the beginning of the audio for the top alternative in every consecutive STREAMING responses. This is done in order to improve our speaker tags as our models learn to identify the speakers in the conversation over time. For non-streaming requests, the diarization results will be provided only in the top alternative of the FINAL SpeechRecognitionResult.

metadata

RecognitionMetadata

Metadata regarding this request.

model

string

Which model to select for the given request. Select the model best suited to your domain to get best results. If a model is not explicitly specified, then we auto-select a model based on the parameters in the RecognitionConfig.

ModelDescription

latest_long

Best for long form content like media or conversation.

latest_short

Best for short form content like commands or single shot directed speech.

command_and_search

Best for short queries such as voice commands or voice search.

phone_call

Best for audio that originated from a phone call (typically recorded at an 8khz sampling rate).

video

Best for audio that originated from video or includes multiple speakers. Ideally the audio is recorded at a 16khz or greater sampling rate. This is a premium model that costs more than the standard rate.

default

Best for audio that is not one of the specific audio models. For example, long-form audio. Ideally the audio is high-fidelity, recorded at a 16khz or greater sampling rate.

medical_conversation

Best for audio that originated from a conversation between a medical provider and patient.

medical_dictation

Best for audio that originated from dictation notes by a medical provider.

use_enhanced

bool

Set to true to use an enhanced model for speech recognition. Ifuse_enhanced is set to true and themodel field is not set, then an appropriate enhanced model is chosen if an enhanced model exists for the audio.

Ifuse_enhanced is true and an enhanced version of the specified model does not exist, then the speech is recognized using the standard version of the specified model.

AudioEncoding

The encoding of the audio data sent in the request.

All encodings support only 1 channel (mono) audio, unless theaudio_channel_count andenable_separate_recognition_per_channel fields are set.

For best results, the audio source should be captured and transmitted using a lossless encoding (FLAC orLINEAR16). The accuracy of the speech recognition can be reduced if lossy codecs are used to capture or transmit audio, particularly if background noise is present. Lossy codecs includeMULAW,AMR,AMR_WB,OGG_OPUS,SPEEX_WITH_HEADER_BYTE,MP3, andWEBM_OPUS.

TheFLAC andWAV audio file formats include a header that describes the included audio content. You can request recognition forWAV files that contain eitherLINEAR16 orMULAW encoded audio. If you sendFLAC orWAV audio file format in your request, you do not need to specify anAudioEncoding; the audio encoding format is determined from the file header. If you specify anAudioEncoding when you send sendFLAC orWAV audio, the encoding configuration must match the encoding described in the audio header; otherwise the request returns angoogle.rpc.Code.INVALID_ARGUMENT error code.

Enums
ENCODING_UNSPECIFIEDNot specified.
LINEAR16Uncompressed 16-bit signed little-endian samples (Linear PCM).
FLACFLAC (Free Lossless Audio Codec) is the recommended encoding because it is lossless--therefore recognition is not compromised--and requires only about half the bandwidth ofLINEAR16.FLAC stream encoding supports 16-bit and 24-bit samples, however, not all fields inSTREAMINFO are supported.
MULAW8-bit samples that compand 14-bit audio samples using G.711 PCMU/mu-law.
AMRAdaptive Multi-Rate Narrowband codec.sample_rate_hertz must be 8000.
AMR_WBAdaptive Multi-Rate Wideband codec.sample_rate_hertz must be 16000.
OGG_OPUSOpus encoded audio frames in Ogg container (OggOpus).sample_rate_hertz must be one of 8000, 12000, 16000, 24000, or 48000.
SPEEX_WITH_HEADER_BYTEAlthough the use of lossy encodings is not recommended, if a very low bitrate encoding is required,OGG_OPUS is highly preferred over Speex encoding. TheSpeex encoding supported by Cloud Speech API has a header byte in each block, as in MIME typeaudio/x-speex-with-header-byte. It is a variant of the RTP Speex encoding defined inRFC 5574. The stream is a sequence of blocks, one block per RTP packet. Each block starts with a byte containing the length of the block, in bytes, followed by one or more frames of Speex data, padded to an integral number of bytes (octets) as specified in RFC 5574. In other words, each RTP header is replaced with a single byte containing the block length. Only Speex wideband is supported.sample_rate_hertz must be 16000.
MP3MP3 audio. MP3 encoding is a Beta feature and only available in v1p1beta1. Support all standard MP3 bitrates (which range from 32-320 kbps). When using this encoding,sample_rate_hertz has to match the sample rate of the file being used.
WEBM_OPUSOpus encoded audio frames in WebM container (WebM).sample_rate_hertz must be one of 8000, 12000, 16000, 24000, or 48000.
ALAW8-bit samples that compand 13-bit audio samples using G.711 PCMU/a-law.

RecognitionMetadata

This item is deprecated!

Description of audio data to be recognized.

Fields
interaction_type

InteractionType

The use case most closely describing the audio content to be recognized.

industry_naics_code_of_audio

uint32

The industry vertical to which this speech recognition request most closely applies. This is most indicative of the topics contained in the audio. Use the 6-digit NAICS code to identify the industry vertical - seehttps://www.naics.com/search/.

microphone_distance

MicrophoneDistance

The audio type that most closely describes the audio being recognized.

original_media_type

OriginalMediaType

The original media the speech was recorded on.

recording_device_type

RecordingDeviceType

The type of device the speech was recorded with.

recording_device_name

string

The device used to make the recording. Examples 'Nexus 5X' or 'Polycom SoundStation IP 6000' or 'POTS' or 'VoIP' or 'Cardioid Microphone'.

original_mime_type

string

Mime type of the original audio file. For exampleaudio/m4a,audio/x-alaw-basic,audio/mp3,audio/3gpp. A list of possible audio mime types is maintained athttp://www.iana.org/assignments/media-types/media-types.xhtml#audio

audio_topic

string

Description of the content. Eg. "Recordings of federal supreme court hearings from 2012".

InteractionType

Use case categories that the audio recognition request can be described by.

Enums
INTERACTION_TYPE_UNSPECIFIEDUse case is either unknown or is something other than one of the other values below.
DISCUSSIONMultiple people in a conversation or discussion. For example in a meeting with two or more people actively participating. Typically all the primary people speaking would be in the same room (if not, see PHONE_CALL)
PRESENTATIONOne or more persons lecturing or presenting to others, mostly uninterrupted.
PHONE_CALLA phone-call or video-conference in which two or more people, who are not in the same room, are actively participating.
VOICEMAILA recorded message intended for another person to listen to.
PROFESSIONALLY_PRODUCEDProfessionally produced audio (eg. TV Show, Podcast).
VOICE_SEARCHTranscribe spoken questions and queries into text.
VOICE_COMMANDTranscribe voice commands, such as for controlling a device.
DICTATIONTranscribe speech to text to create a written document, such as a text-message, email or report.

MicrophoneDistance

Enumerates the types of capture settings describing an audio file.

Enums
MICROPHONE_DISTANCE_UNSPECIFIEDAudio type is not known.
NEARFIELDThe audio was captured from a closely placed microphone. Eg. phone, dictaphone, or handheld microphone. Generally if there speaker is within 1 meter of the microphone.
MIDFIELDThe speaker if within 3 meters of the microphone.
FARFIELDThe speaker is more than 3 meters away from the microphone.

OriginalMediaType

The original media the speech was recorded on.

Enums
ORIGINAL_MEDIA_TYPE_UNSPECIFIEDUnknown original media type.
AUDIOThe speech data is an audio recording.
VIDEOThe speech data originally recorded on a video.

RecordingDeviceType

The type of device the speech was recorded with.

Enums
RECORDING_DEVICE_TYPE_UNSPECIFIEDThe recording device is unknown.
SMARTPHONESpeech was recorded on a smartphone.
PCSpeech was recorded using a personal computer or tablet.
PHONE_LINESpeech was recorded over a phone line.
VEHICLESpeech was recorded in a vehicle.
OTHER_OUTDOOR_DEVICESpeech was recorded outdoors.
OTHER_INDOOR_DEVICESpeech was recorded indoors.

RecognizeRequest

The top-level message sent by the client for theRecognize method.

Fields
config

RecognitionConfig

Required. Provides information to the recognizer that specifies how to process the request.

audio

RecognitionAudio

Required. The audio data to be recognized.

RecognizeResponse

The only message returned to the client by theRecognize method. It contains the result as zero or more sequentialSpeechRecognitionResult messages.

Fields
results[]

SpeechRecognitionResult

Sequential list of transcription results corresponding to sequential portions of audio.

total_billed_time

Duration

When available, billed audio seconds for the corresponding request.

speech_adaptation_info

SpeechAdaptationInfo

Provides information on adaptation behavior in response

request_id

int64

The ID associated with the request. This is a unique ID specific only to the given request.

using_legacy_models

bool

Whether request used legacy asr models (was not automatically migrated to use conformer models).

SpeakerDiarizationConfig

Config to enable speaker diarization.

Fields
enable_speaker_diarization

bool

If 'true', enables speaker detection for each recognized word in the top alternative of the recognition result using a speaker_label provided in the WordInfo.

min_speaker_count

int32

Minimum number of speakers in the conversation. This range gives you more flexibility by allowing the system to automatically determine the correct number of speakers. If not set, the default value is 2.

max_speaker_count

int32

Maximum number of speakers in the conversation. This range gives you more flexibility by allowing the system to automatically determine the correct number of speakers. If not set, the default value is 6.

speaker_tag
(deprecated)

int32

This item is deprecated!

Output only. Unused.

SpeechAdaptation

Speech adaptation configuration.

Fields
phrase_sets[]

PhraseSet

A collection of phrase sets. To specify the hints inline, leave the phrase set'sname blank and fill in the rest of its fields. Any phrase set can use any custom class.

phrase_set_references[]

string

A collection of phrase set resource names to use.

custom_classes[]

CustomClass

A collection of custom classes. To specify the classes inline, leave the class'name blank and fill in the rest of its fields, giving it a uniquecustom_class_id. Refer to the inline defined class in phrase hints by itscustom_class_id.

abnf_grammar

ABNFGrammar

Augmented Backus-Naur form (ABNF) is a standardized grammar notation comprised by a set of derivation rules. See specifications:https://www.w3.org/TR/speech-grammar

ABNFGrammar

Fields
abnf_strings[]

string

All declarations and rules of an ABNF grammar broken up into multiple strings that will end up concatenated.

SpeechAdaptationInfo

Information on speech adaptation use in results

Fields
adaptation_timeout

bool

Whether there was a timeout when applying speech adaptation. If true, adaptation had no effect in the response transcript.

timeout_message

string

If set, returns a message specifying which part of the speech adaptation request timed out.

SpeechContext

Provides "hints" to the speech recognizer to favor specific words and phrases in the results.

Fields
phrases[]

string

A list of strings containing words and phrases "hints" so that the speech recognition is more likely to recognize them. This can be used to improve the accuracy for specific words and phrases, for example, if specific commands are typically spoken by the user. This can also be used to add additional words to the vocabulary of the recognizer. Seeusage limits.

List items can also be set to classes for groups of words that represent common concepts that occur in natural language. For example, rather than providing phrase hints for every month of the year, using the $MONTH class improves the likelihood of correctly transcribing audio that includes months.

boost

float

Hint Boost. Positive value will increase the probability that a specific phrase will be recognized over other similar sounding phrases. The higher the boost, the higher the chance of false positive recognition as well. Negative boost values would correspond to anti-biasing. Anti-biasing is not enabled, so negative boost will simply be ignored. Thoughboost can accept a wide range of positive values, most use cases are best served with values between 0 and 20. We recommend using a binary search approach to finding the optimal value for your use case.

SpeechRecognitionAlternative

Alternative hypotheses (a.k.a. n-best list).

Fields
transcript

string

Transcript text representing the words that the user spoke. In languages that use spaces to separate words, the transcript might have a leading space if it isn't the first result. You can concatenate each result to obtain the full transcript without using a separator.

confidence

float

The confidence estimate between 0.0 and 1.0. A higher number indicates an estimated greater likelihood that the recognized words are correct. This field is set only for the top alternative of a non-streaming result or, of a streaming result whereis_final=true. This field is not guaranteed to be accurate and users should not rely on it to be always provided. The default of 0.0 is a sentinel value indicatingconfidence was not set.

words[]

WordInfo

A list of word-specific information for each recognized word. Note: Whenenable_speaker_diarization is true, you will see all the words from the beginning of the audio.

SpeechRecognitionResult

A speech recognition result corresponding to a portion of the audio.

Fields
alternatives[]

SpeechRecognitionAlternative

May contain one or more recognition hypotheses (up to the maximum specified inmax_alternatives). These alternatives are ordered in terms of accuracy, with the top (first) alternative being the most probable, as ranked by the recognizer.

channel_tag

int32

For multi-channel audio, this is the channel number corresponding to the recognized result for the audio from that channel. For audio_channel_count = N, its output values can range from '1' to 'N'.

result_end_time

Duration

Time offset of the end of this result relative to the beginning of the audio.

language_code

string

Output only. TheBCP-47 language tag of the language in this result. This language code was detected to have the most likelihood of being spoken in the audio.

StreamingRecognitionConfig

Provides information to the recognizer that specifies how to process the request.

Fields
config

RecognitionConfig

Required. Provides information to the recognizer that specifies how to process the request.

single_utterance

bool

Iffalse or omitted, the recognizer will perform continuous recognition (continuing to wait for and process audio even if the user pauses speaking) until the client closes the input stream (gRPC API) or until the maximum time limit has been reached. May return multipleStreamingRecognitionResults with theis_final flag set totrue.

Iftrue, the recognizer will detect a single spoken utterance. When it detects that the user has paused or stopped speaking, it will return anEND_OF_SINGLE_UTTERANCE event and cease recognition. It will return no more than oneStreamingRecognitionResult with theis_final flag set totrue.

Thesingle_utterance field can only be used with specified models, otherwise an error is thrown. Themodel field inRecognitionConfig must be set to:

  • command_and_search
  • phone_call AND additional fielduseEnhanced=true
  • Themodel field is left undefined. In this case the API auto-selects a model based on any other parameters that you set inRecognitionConfig.
interim_results

bool

Iftrue, interim results (tentative hypotheses) may be returned as they become available (these interim results are indicated with theis_final=false flag). Iffalse or omitted, onlyis_final=true result(s) are returned.

enable_voice_activity_events

bool

Iftrue, responses with voice activity speech events will be returned as they are detected.

voice_activity_timeout

VoiceActivityTimeout

If set, the server will automatically close the stream after the specified duration has elapsed after the last VOICE_ACTIVITY speech event has been sent. The fieldvoice_activity_events must also be set to true.

VoiceActivityTimeout

Events that a timeout can be set on for voice activity.

Fields
speech_start_timeout

Duration

Duration to timeout the stream if no speech begins.

speech_end_timeout

Duration

Duration to timeout the stream after speech ends.

StreamingRecognitionResult

A streaming speech recognition result corresponding to a portion of the audio that is currently being processed.

Fields
alternatives[]

SpeechRecognitionAlternative

May contain one or more recognition hypotheses (up to the maximum specified inmax_alternatives). These alternatives are ordered in terms of accuracy, with the top (first) alternative being the most probable, as ranked by the recognizer.

is_final

bool

Iffalse, thisStreamingRecognitionResult represents an interim result that may change. Iftrue, this is the final time the speech service will return this particularStreamingRecognitionResult, the recognizer will not return any further hypotheses for this portion of the transcript and corresponding audio.

stability

float

An estimate of the likelihood that the recognizer will not change its guess about this interim result. Values range from 0.0 (completely unstable) to 1.0 (completely stable). This field is only provided for interim results (is_final=false). The default of 0.0 is a sentinel value indicatingstability was not set.

result_end_time

Duration

Time offset of the end of this result relative to the beginning of the audio.

channel_tag

int32

For multi-channel audio, this is the channel number corresponding to the recognized result for the audio from that channel. For audio_channel_count = N, its output values can range from '1' to 'N'.

language_code

string

Output only. TheBCP-47 language tag of the language in this result. This language code was detected to have the most likelihood of being spoken in the audio.

StreamingRecognizeRequest

The top-level message sent by the client for theStreamingRecognize method. MultipleStreamingRecognizeRequest messages are sent. The first message must contain astreaming_config message and must not containaudio_content. All subsequent messages must containaudio_content and must not contain astreaming_config message.

Fields
Union fieldstreaming_request. The streaming request, which is either a streaming config or audio content.streaming_request can be only one of the following:
streaming_config

StreamingRecognitionConfig

Provides information to the recognizer that specifies how to process the request. The firstStreamingRecognizeRequest message must contain astreaming_config message.

audio_content

bytes

The audio data to be recognized. Sequential chunks of audio data are sent in sequentialStreamingRecognizeRequest messages. The firstStreamingRecognizeRequest message must not containaudio_content data and all subsequentStreamingRecognizeRequest messages must containaudio_content data. The audio bytes must be encoded as specified inRecognitionConfig. Note: as with all bytes fields, proto buffers use a pure binary representation (not base64). Seecontent limits.

StreamingRecognizeResponse

StreamingRecognizeResponse is the only message returned to the client byStreamingRecognize. A series of zero or moreStreamingRecognizeResponse messages are streamed back to the client. If there is no recognizable audio, andsingle_utterance is set to false, then no messages are streamed back to the client.

Here's an example of a series ofStreamingRecognizeResponses that might be returned while processing audio:

  1. results { alternatives { transcript: "tube" } stability: 0.01 }

  2. results { alternatives { transcript: "to be a" } stability: 0.01 }

  3. results { alternatives { transcript: "to be" } stability: 0.9 } results { alternatives { transcript: " or not to be" } stability: 0.01 }

  4. results { alternatives { transcript: "to be or not to be" confidence: 0.92 } alternatives { transcript: "to bee or not to bee" } is_final: true }

  5. results { alternatives { transcript: " that's" } stability: 0.01 }

  6. results { alternatives { transcript: " that is" } stability: 0.9 } results { alternatives { transcript: " the question" } stability: 0.01 }

  7. results { alternatives { transcript: " that is the question" confidence: 0.98 } alternatives { transcript: " that was the question" } is_final: true }

Notes:

Fields
error

Status

If set, returns agoogle.rpc.Status message that specifies the error for the operation.

results[]

StreamingRecognitionResult

This repeated list contains zero or more results that correspond to consecutive portions of the audio currently being processed. It contains zero or oneis_final=true result (the newly settled portion), followed by zero or moreis_final=false results (the interim results).

speech_event_type

SpeechEventType

Indicates the type of speech event.

speech_event_time

Duration

Time offset between the beginning of the audio and event emission.

total_billed_time

Duration

When available, billed audio seconds for the stream. Set only if this is the last response in the stream.

speech_adaptation_info

SpeechAdaptationInfo

Provides information on adaptation behavior in response

request_id

int64

The ID associated with the request. This is a unique ID specific only to the given request.

SpeechEventType

Indicates the type of speech event.

Enums
SPEECH_EVENT_UNSPECIFIEDNo speech event specified.
END_OF_SINGLE_UTTERANCEThis event indicates that the server has detected the end of the user's speech utterance and expects no additional speech. Therefore, the server will not process additional audio (although it may subsequently return additional results). The client should stop sending additional audio data, half-close the gRPC connection, and wait for any additional results until the server closes the gRPC connection. This event is only sent ifsingle_utterance was set totrue, and is not used otherwise.
SPEECH_ACTIVITY_BEGINThis event indicates that the server has detected the beginning of human voice activity in the stream. This event can be returned multiple times if speech starts and stops repeatedly throughout the stream. This event is only sent ifvoice_activity_events is set to true.
SPEECH_ACTIVITY_ENDThis event indicates that the server has detected the end of human voice activity in the stream. This event can be returned multiple times if speech starts and stops repeatedly throughout the stream. This event is only sent ifvoice_activity_events is set to true.
SPEECH_ACTIVITY_TIMEOUTThis event indicates that the user-set timeout for speech activity begin or end has exceeded. Upon receiving this event, the client is expected to send a half close. Further audio will not be processed.

TranscriptNormalization

Transcription normalization configuration. Use transcription normalization to automatically replace parts of the transcript with phrases of your choosing. For StreamingRecognize, this normalization only applies to stable partial transcripts (stability > 0.8) and final transcripts.

Fields
entries[]

Entry

A list of replacement entries. We will perform replacement with one entry at a time. For example, the second entry in ["cat" => "dog", "mountain cat" => "mountain dog"] will never be applied because we will always process the first entry before it. At most 100 entries.

Entry

A single replacement configuration.

Fields
search

string

What to replace. Max length is 100 characters.

replace

string

What to replace with. Max length is 100 characters.

case_sensitive

bool

Whether the search is case sensitive.

TranscriptOutputConfig

Specifies an optional destination for the recognition results.

Fields

Union fieldoutput_type.

output_type can be only one of the following:

gcs_uri

string

Specifies a Cloud Storage URI for the recognition results. Must be specified in the format:gs://bucket_name/object_name, and the bucket must already exist.

UpdateCustomClassRequest

Message sent by the client for theUpdateCustomClass method.

Fields
custom_class

CustomClass

Required. The custom class to update.

The custom class'sname field is used to identify the custom class to be updated. Format:

projects/{project}/locations/{location}/customClasses/{custom_class}

Speech-to-Text supports three locations:global,us (US North America), andeu (Europe). If you are calling thespeech.googleapis.com endpoint, use theglobal location. To specify a region, use aregional endpoint with matchingus oreu location value.

Authorization requires the followingIAM permission on the specified resourcecustomClass:

  • speech.customClasses.update
update_mask

FieldMask

The list of fields to be updated.

UpdatePhraseSetRequest

Message sent by the client for theUpdatePhraseSet method.

Fields
phrase_set

PhraseSet

Required. The phrase set to update.

The phrase set'sname field is used to identify the set to be updated. Format:

projects/{project}/locations/{location}/phraseSets/{phrase_set}

Speech-to-Text supports three locations:global,us (US North America), andeu (Europe). If you are calling thespeech.googleapis.com endpoint, use theglobal location. To specify a region, use aregional endpoint with matchingus oreu location value.

Authorization requires the followingIAM permission on the specified resourcephraseSet:

  • speech.phraseSets.update
update_mask

FieldMask

The list of fields to be updated.

WordInfo

Word-specific information for recognized words.

Fields
start_time

Duration

Time offset relative to the beginning of the audio, and corresponding to the start of the spoken word. This field is only set ifenable_word_time_offsets=true and only in the top hypothesis. This is an experimental feature and the accuracy of the time offset can vary.

end_time

Duration

Time offset relative to the beginning of the audio, and corresponding to the end of the spoken word. This field is only set ifenable_word_time_offsets=true and only in the top hypothesis. This is an experimental feature and the accuracy of the time offset can vary.

word

string

The word corresponding to this set of information.

confidence

float

The confidence estimate between 0.0 and 1.0. A higher number indicates an estimated greater likelihood that the recognized words are correct. This field is set only for the top alternative of a non-streaming result or, of a streaming result whereis_final=true. This field is not guaranteed to be accurate and users should not rely on it to be always provided. The default of 0.0 is a sentinel value indicatingconfidence was not set.

speaker_tag
(deprecated)

int32

This item is deprecated!

Output only. A distinct integer value is assigned for every speaker within the audio. This field specifies which one of those speakers was detected to have spoken this word. Value ranges from '1' to diarization_speaker_count. speaker_tag is set if enable_speaker_diarization = 'true' and only for the top alternative. Note: Use speaker_label instead.

speaker_label

string

Output only. A label value assigned for every unique speaker within the audio. This field specifies which speaker was detected to have spoken this word. For some models, like medical_conversation this can be actual speaker role, for example "patient" or "provider", but generally this would be a number identifying a speaker. This field is only set if enable_speaker_diarization = 'true' and only for the top alternative.

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-10-23 UTC.