Method: locations.evaluateInstances

Full name: projects.locations.evaluateInstances

Evaluates instances based on a given metric.

Endpoint

posthttps://{service-endpoint}/v1/{location}:evaluateInstances

Where{service-endpoint} is one of thesupported service endpoints.

Path parameters

locationstring

Required. The resource name of the Location to evaluate the instances. Format:projects/{project}/locations/{location}

Request body

The request body contains data with the following structure:

Fields
metric_inputsUnion type
Instances and specs for evaluationmetric_inputs can be only one of the following:
exactMatchInputobject (ExactMatchInput)

Auto metric instances. Instances and metric spec for exact match metric.

bleuInputobject (BleuInput)

Instances and metric spec for bleu metric.

rougeInputobject (RougeInput)

Instances and metric spec for rouge metric.

fluencyInputobject (FluencyInput)

LLM-based metric instance. General text generation metrics, applicable to other categories. Input for fluency metric.

coherenceInputobject (CoherenceInput)

Input for coherence metric.

safetyInputobject (SafetyInput)

Input for safety metric.

groundednessInputobject (GroundednessInput)

Input for groundedness metric.

fulfillmentInputobject (FulfillmentInput)

Input for fulfillment metric.

summarizationQualityInputobject (SummarizationQualityInput)

Input for summarization quality metric.

pairwiseSummarizationQualityInputobject (PairwiseSummarizationQualityInput)

Input for pairwise summarization quality metric.

summarizationHelpfulnessInputobject (SummarizationHelpfulnessInput)

Input for summarization helpfulness metric.

summarizationVerbosityInputobject (SummarizationVerbosityInput)

Input for summarization verbosity metric.

questionAnsweringQualityInputobject (QuestionAnsweringQualityInput)

Input for question answering quality metric.

pairwiseQuestionAnsweringQualityInputobject (PairwiseQuestionAnsweringQualityInput)

Input for pairwise question answering quality metric.

questionAnsweringRelevanceInputobject (QuestionAnsweringRelevanceInput)

Input for question answering relevance metric.

questionAnsweringHelpfulnessInputobject (QuestionAnsweringHelpfulnessInput)

Input for question answering helpfulness metric.

questionAnsweringCorrectnessInputobject (QuestionAnsweringCorrectnessInput)

Input for question answering correctness metric.

pointwiseMetricInputobject (PointwiseMetricInput)

Input for pointwise metric.

pairwiseMetricInputobject (PairwiseMetricInput)

Input for pairwise metric.

toolCallValidInputobject (ToolCallValidInput)

Tool call metric instances. Input for tool call valid metric.

toolNameMatchInputobject (ToolNameMatchInput)

Input for tool name match metric.

toolParameterKeyMatchInputobject (ToolParameterKeyMatchInput)

Input for tool parameter key match metric.

toolParameterKvMatchInputobject (ToolParameterKVMatchInput)

Input for tool parameter key value match metric.

cometInputobject (CometInput)

Translation metrics. Input for Comet metric.

metricxInputobject (MetricxInput)

Input for Metricx metric.

trajectoryExactMatchInputobject (TrajectoryExactMatchInput)

Input for trajectory exact match metric.

trajectoryInOrderMatchInputobject (TrajectoryInOrderMatchInput)

Input for trajectory in order match metric.

trajectoryAnyOrderMatchInputobject (TrajectoryAnyOrderMatchInput)

Input for trajectory match any order metric.

trajectoryPrecisionInputobject (TrajectoryPrecisionInput)

Input for trajectory precision metric.

trajectoryRecallInputobject (TrajectoryRecallInput)

Input for trajectory recall metric.

trajectorySingleToolUseInputobject (TrajectorySingleToolUseInput)

Input for trajectory single tool use metric.

Response body

Response message for EvaluationService.EvaluateInstances.

If successful, the response body contains data with the following structure:

Fields
evaluation_resultsUnion type
Evaluation results will be served in the same order as presented in EvaluationRequest.instances.evaluation_results can be only one of the following:
exactMatchResultsobject (ExactMatchResults)

Auto metric evaluation results. Results for exact match metric.

bleuResultsobject (BleuResults)

Results for bleu metric.

rougeResultsobject (RougeResults)

Results for rouge metric.

fluencyResultobject (FluencyResult)

LLM-based metric evaluation result. General text generation metrics, applicable to other categories. result for fluency metric.

coherenceResultobject (CoherenceResult)

result for coherence metric.

safetyResultobject (SafetyResult)

result for safety metric.

groundednessResultobject (GroundednessResult)

result for groundedness metric.

fulfillmentResultobject (FulfillmentResult)

result for fulfillment metric.

summarizationQualityResultobject (SummarizationQualityResult)

Summarization only metrics. result for summarization quality metric.

pairwiseSummarizationQualityResultobject (PairwiseSummarizationQualityResult)

result for pairwise summarization quality metric.

summarizationHelpfulnessResultobject (SummarizationHelpfulnessResult)

result for summarization helpfulness metric.

summarizationVerbosityResultobject (SummarizationVerbosityResult)

result for summarization verbosity metric.

questionAnsweringQualityResultobject (QuestionAnsweringQualityResult)

Question answering only metrics. result for question answering quality metric.

pairwiseQuestionAnsweringQualityResultobject (PairwiseQuestionAnsweringQualityResult)

result for pairwise question answering quality metric.

questionAnsweringRelevanceResultobject (QuestionAnsweringRelevanceResult)

result for question answering relevance metric.

questionAnsweringHelpfulnessResultobject (QuestionAnsweringHelpfulnessResult)

result for question answering helpfulness metric.

questionAnsweringCorrectnessResultobject (QuestionAnsweringCorrectnessResult)

result for question answering correctness metric.

pointwiseMetricResultobject (PointwiseMetricResult)

Generic metrics. result for pointwise metric.

pairwiseMetricResultobject (PairwiseMetricResult)

result for pairwise metric.

toolCallValidResultsobject (ToolCallValidResults)

Tool call metrics. Results for tool call valid metric.

toolNameMatchResultsobject (ToolNameMatchResults)

Results for tool name match metric.

toolParameterKeyMatchResultsobject (ToolParameterKeyMatchResults)

Results for tool parameter key match metric.

toolParameterKvMatchResultsobject (ToolParameterKVMatchResults)

Results for tool parameter key value match metric.

cometResultobject (CometResult)

Translation metrics. result for Comet metric.

metricxResultobject (MetricxResult)

result for Metricx metric.

trajectoryExactMatchResultsobject (TrajectoryExactMatchResults)

result for trajectory exact match metric.

trajectoryInOrderMatchResultsobject (TrajectoryInOrderMatchResults)

result for trajectory in order match metric.

trajectoryAnyOrderMatchResultsobject (TrajectoryAnyOrderMatchResults)

result for trajectory any order match metric.

trajectoryPrecisionResultsobject (TrajectoryPrecisionResults)

result for trajectory precision metric.

trajectoryRecallResultsobject (TrajectoryRecallResults)

Results for trajectory recall metric.

trajectorySingleToolUseResultsobject (TrajectorySingleToolUseResults)

Results for trajectory single tool use metric.

JSON representation
{// evaluation_results"exactMatchResults":{object (ExactMatchResults)},"bleuResults":{object (BleuResults)},"rougeResults":{object (RougeResults)},"fluencyResult":{object (FluencyResult)},"coherenceResult":{object (CoherenceResult)},"safetyResult":{object (SafetyResult)},"groundednessResult":{object (GroundednessResult)},"fulfillmentResult":{object (FulfillmentResult)},"summarizationQualityResult":{object (SummarizationQualityResult)},"pairwiseSummarizationQualityResult":{object (PairwiseSummarizationQualityResult)},"summarizationHelpfulnessResult":{object (SummarizationHelpfulnessResult)},"summarizationVerbosityResult":{object (SummarizationVerbosityResult)},"questionAnsweringQualityResult":{object (QuestionAnsweringQualityResult)},"pairwiseQuestionAnsweringQualityResult":{object (PairwiseQuestionAnsweringQualityResult)},"questionAnsweringRelevanceResult":{object (QuestionAnsweringRelevanceResult)},"questionAnsweringHelpfulnessResult":{object (QuestionAnsweringHelpfulnessResult)},"questionAnsweringCorrectnessResult":{object (QuestionAnsweringCorrectnessResult)},"pointwiseMetricResult":{object (PointwiseMetricResult)},"pairwiseMetricResult":{object (PairwiseMetricResult)},"toolCallValidResults":{object (ToolCallValidResults)},"toolNameMatchResults":{object (ToolNameMatchResults)},"toolParameterKeyMatchResults":{object (ToolParameterKeyMatchResults)},"toolParameterKvMatchResults":{object (ToolParameterKVMatchResults)},"cometResult":{object (CometResult)},"metricxResult":{object (MetricxResult)},"trajectoryExactMatchResults":{object (TrajectoryExactMatchResults)},"trajectoryInOrderMatchResults":{object (TrajectoryInOrderMatchResults)},"trajectoryAnyOrderMatchResults":{object (TrajectoryAnyOrderMatchResults)},"trajectoryPrecisionResults":{object (TrajectoryPrecisionResults)},"trajectoryRecallResults":{object (TrajectoryRecallResults)},"trajectorySingleToolUseResults":{object (TrajectorySingleToolUseResults)}// Union type}

ExactMatchInput

Input for exact match metric.

Fields
metricSpecobject (ExactMatchSpec)

Required. Spec for exact match metric.

instances[]object (ExactMatchInstance)

Required. Repeated exact match instances.

JSON representation
{"metricSpec":{object (ExactMatchSpec)},"instances":[{object (ExactMatchInstance)}]}

ExactMatchSpec

This type has no fields.

Spec for exact match metric - returns 1 if prediction and reference exactly matches, otherwise 0.

ExactMatchInstance

Spec for exact match instance.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Required. Ground truth used to compare against the prediction.

JSON representation
{"prediction":string,"reference":string}

BleuInput

Input for bleu metric.

Fields
metricSpecobject (BleuSpec)

Required. Spec for bleu score metric.

instances[]object (BleuInstance)

Required. Repeated bleu instances.

JSON representation
{"metricSpec":{object (BleuSpec)},"instances":[{object (BleuInstance)}]}

BleuSpec

Spec for bleu score metric - calculates the precision of n-grams in the prediction as compared to reference - returns a score ranging between 0 to 1.

Fields
useEffectiveOrderboolean

Optional. Whether to useEffectiveOrder to compute bleu score.

JSON representation
{"useEffectiveOrder":boolean}

BleuInstance

Spec for bleu instance.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Required. Ground truth used to compare against the prediction.

JSON representation
{"prediction":string,"reference":string}

RougeInput

Input for rouge metric.

Fields
metricSpecobject (RougeSpec)

Required. Spec for rouge score metric.

instances[]object (RougeInstance)

Required. Repeated rouge instances.

JSON representation
{"metricSpec":{object (RougeSpec)},"instances":[{object (RougeInstance)}]}

RougeSpec

Spec for rouge score metric - calculates the recall of n-grams in prediction as compared to reference - returns a score ranging between 0 and 1.

Fields
rougeTypestring

Optional. Supported rouge types are rougen[1-9], rougeL, and rougeLsum.

useStemmerboolean

Optional. Whether to use stemmer to compute rouge score.

splitSummariesboolean

Optional. Whether to split summaries while using rougeLsum.

JSON representation
{"rougeType":string,"useStemmer":boolean,"splitSummaries":boolean}

RougeInstance

Spec for rouge instance.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Required. Ground truth used to compare against the prediction.

JSON representation
{"prediction":string,"reference":string}

FluencyInput

Input for fluency metric.

Fields
metricSpecobject (FluencySpec)

Required. Spec for fluency score metric.

instanceobject (FluencyInstance)

Required. Fluency instance.

JSON representation
{"metricSpec":{object (FluencySpec)},"instance":{object (FluencyInstance)}}

FluencySpec

Spec for fluency score metric.

Fields
versioninteger

Optional. Which version to use for evaluation.

JSON representation
{"version":integer}

FluencyInstance

Spec for fluency instance.

Fields
predictionstring

Required. Output of the evaluated model.

JSON representation
{"prediction":string}

CoherenceInput

Input for coherence metric.

Fields
metricSpecobject (CoherenceSpec)

Required. Spec for coherence score metric.

instanceobject (CoherenceInstance)

Required. Coherence instance.

JSON representation
{"metricSpec":{object (CoherenceSpec)},"instance":{object (CoherenceInstance)}}

CoherenceSpec

Spec for coherence score metric.

Fields
versioninteger

Optional. Which version to use for evaluation.

JSON representation
{"version":integer}

CoherenceInstance

Spec for coherence instance.

Fields
predictionstring

Required. Output of the evaluated model.

JSON representation
{"prediction":string}

SafetyInput

Input for safety metric.

Fields
metricSpecobject (SafetySpec)

Required. Spec for safety metric.

instanceobject (SafetyInstance)

Required. Safety instance.

JSON representation
{"metricSpec":{object (SafetySpec)},"instance":{object (SafetyInstance)}}

SafetySpec

Spec for safety metric.

Fields
versioninteger

Optional. Which version to use for evaluation.

JSON representation
{"version":integer}

SafetyInstance

Spec for safety instance.

Fields
predictionstring

Required. Output of the evaluated model.

JSON representation
{"prediction":string}

GroundednessInput

Input for groundedness metric.

Fields
metricSpecobject (GroundednessSpec)

Required. Spec for groundedness metric.

instanceobject (GroundednessInstance)

Required. Groundedness instance.

JSON representation
{"metricSpec":{object (GroundednessSpec)},"instance":{object (GroundednessInstance)}}

GroundednessSpec

Spec for groundedness metric.

Fields
versioninteger

Optional. Which version to use for evaluation.

JSON representation
{"version":integer}

GroundednessInstance

Spec for groundedness instance.

Fields
predictionstring

Required. Output of the evaluated model.

contextstring

Required. Background information provided in context used to compare against the prediction.

JSON representation
{"prediction":string,"context":string}

FulfillmentInput

Input for fulfillment metric.

Fields
metricSpecobject (FulfillmentSpec)

Required. Spec for fulfillment score metric.

instanceobject (FulfillmentInstance)

Required. Fulfillment instance.

JSON representation
{"metricSpec":{object (FulfillmentSpec)},"instance":{object (FulfillmentInstance)}}

FulfillmentSpec

Spec for fulfillment metric.

Fields
versioninteger

Optional. Which version to use for evaluation.

JSON representation
{"version":integer}

FulfillmentInstance

Spec for fulfillment instance.

Fields
predictionstring

Required. Output of the evaluated model.

instructionstring

Required. Inference instruction prompt to compare prediction with.

JSON representation
{"prediction":string,"instruction":string}

SummarizationQualityInput

Input for summarization quality metric.

Fields
metricSpecobject (SummarizationQualitySpec)

Required. Spec for summarization quality score metric.

Required. Summarization quality instance.

JSON representation
{"metricSpec":{object (SummarizationQualitySpec)},"instance":{object (SummarizationQualityInstance)}}

SummarizationQualitySpec

Spec for summarization quality score metric.

Fields
useReferenceboolean

Optional. Whether to use instance.reference to compute summarization quality.

versioninteger

Optional. Which version to use for evaluation.

JSON representation
{"useReference":boolean,"version":integer}

SummarizationQualityInstance

Spec for summarization quality instance.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Optional. Ground truth used to compare against the prediction.

contextstring

Required. Text to be summarized.

instructionstring

Required. Summarization prompt for LLM.

JSON representation
{"prediction":string,"reference":string,"context":string,"instruction":string}

PairwiseSummarizationQualityInput

Input for pairwise summarization quality metric.

Fields

Required. Spec for pairwise summarization quality score metric.

Required. Pairwise summarization quality instance.

JSON representation
{"metricSpec":{object (PairwiseSummarizationQualitySpec)},"instance":{object (PairwiseSummarizationQualityInstance)}}

PairwiseSummarizationQualitySpec

Spec for pairwise summarization quality score metric.

Fields
useReferenceboolean

Optional. Whether to use instance.reference to compute pairwise summarization quality.

versioninteger

Optional. Which version to use for evaluation.

JSON representation
{"useReference":boolean,"version":integer}

PairwiseSummarizationQualityInstance

Spec for pairwise summarization quality instance.

Fields
predictionstring

Required. Output of the candidate model.

baselinePredictionstring

Required. Output of the baseline model.

referencestring

Optional. Ground truth used to compare against the prediction.

contextstring

Required. Text to be summarized.

instructionstring

Required. Summarization prompt for LLM.

JSON representation
{"prediction":string,"baselinePrediction":string,"reference":string,"context":string,"instruction":string}

SummarizationHelpfulnessInput

Input for summarization helpfulness metric.

Fields
metricSpecobject (SummarizationHelpfulnessSpec)

Required. Spec for summarization helpfulness score metric.

Required. Summarization helpfulness instance.

JSON representation
{"metricSpec":{object (SummarizationHelpfulnessSpec)},"instance":{object (SummarizationHelpfulnessInstance)}}

SummarizationHelpfulnessSpec

Spec for summarization helpfulness score metric.

Fields
useReferenceboolean

Optional. Whether to use instance.reference to compute summarization helpfulness.

versioninteger

Optional. Which version to use for evaluation.

JSON representation
{"useReference":boolean,"version":integer}

SummarizationHelpfulnessInstance

Spec for summarization helpfulness instance.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Optional. Ground truth used to compare against the prediction.

contextstring

Required. Text to be summarized.

instructionstring

Optional. Summarization prompt for LLM.

JSON representation
{"prediction":string,"reference":string,"context":string,"instruction":string}

SummarizationVerbosityInput

Input for summarization verbosity metric.

Fields
metricSpecobject (SummarizationVerbositySpec)

Required. Spec for summarization verbosity score metric.

Required. Summarization verbosity instance.

JSON representation
{"metricSpec":{object (SummarizationVerbositySpec)},"instance":{object (SummarizationVerbosityInstance)}}

SummarizationVerbositySpec

Spec for summarization verbosity score metric.

Fields
useReferenceboolean

Optional. Whether to use instance.reference to compute summarization verbosity.

versioninteger

Optional. Which version to use for evaluation.

JSON representation
{"useReference":boolean,"version":integer}

SummarizationVerbosityInstance

Spec for summarization verbosity instance.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Optional. Ground truth used to compare against the prediction.

contextstring

Required. Text to be summarized.

instructionstring

Optional. Summarization prompt for LLM.

JSON representation
{"prediction":string,"reference":string,"context":string,"instruction":string}

QuestionAnsweringQualityInput

Input for question answering quality metric.

Fields
metricSpecobject (QuestionAnsweringQualitySpec)

Required. Spec for question answering quality score metric.

Required. Question answering quality instance.

JSON representation
{"metricSpec":{object (QuestionAnsweringQualitySpec)},"instance":{object (QuestionAnsweringQualityInstance)}}

QuestionAnsweringQualitySpec

Spec for question answering quality score metric.

Fields
useReferenceboolean

Optional. Whether to use instance.reference to compute question answering quality.

versioninteger

Optional. Which version to use for evaluation.

JSON representation
{"useReference":boolean,"version":integer}

QuestionAnsweringQualityInstance

Spec for question answering quality instance.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Optional. Ground truth used to compare against the prediction.

contextstring

Required. Text to answer the question.

instructionstring

Required. Question Answering prompt for LLM.

JSON representation
{"prediction":string,"reference":string,"context":string,"instruction":string}

PairwiseQuestionAnsweringQualityInput

Input for pairwise question answering quality metric.

Fields

Required. Spec for pairwise question answering quality score metric.

Required. Pairwise question answering quality instance.

JSON representation
{"metricSpec":{object (PairwiseQuestionAnsweringQualitySpec)},"instance":{object (PairwiseQuestionAnsweringQualityInstance)}}

PairwiseQuestionAnsweringQualitySpec

Spec for pairwise question answering quality score metric.

Fields
useReferenceboolean

Optional. Whether to use instance.reference to compute question answering quality.

versioninteger

Optional. Which version to use for evaluation.

JSON representation
{"useReference":boolean,"version":integer}

PairwiseQuestionAnsweringQualityInstance

Spec for pairwise question answering quality instance.

Fields
predictionstring

Required. Output of the candidate model.

baselinePredictionstring

Required. Output of the baseline model.

referencestring

Optional. Ground truth used to compare against the prediction.

contextstring

Required. Text to answer the question.

instructionstring

Required. Question Answering prompt for LLM.

JSON representation
{"prediction":string,"baselinePrediction":string,"reference":string,"context":string,"instruction":string}

QuestionAnsweringRelevanceInput

Input for question answering relevance metric.

Fields

Required. Spec for question answering relevance score metric.

Required. Question answering relevance instance.

JSON representation
{"metricSpec":{object (QuestionAnsweringRelevanceSpec)},"instance":{object (QuestionAnsweringRelevanceInstance)}}

QuestionAnsweringRelevanceSpec

Spec for question answering relevance metric.

Fields
useReferenceboolean

Optional. Whether to use instance.reference to compute question answering relevance.

versioninteger

Optional. Which version to use for evaluation.

JSON representation
{"useReference":boolean,"version":integer}

QuestionAnsweringRelevanceInstance

Spec for question answering relevance instance.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Optional. Ground truth used to compare against the prediction.

contextstring

Optional. Text provided as context to answer the question.

instructionstring

Required. The question asked and other instruction in the inference prompt.

JSON representation
{"prediction":string,"reference":string,"context":string,"instruction":string}

QuestionAnsweringHelpfulnessInput

Input for question answering helpfulness metric.

Fields

Required. Spec for question answering helpfulness score metric.

Required. Question answering helpfulness instance.

JSON representation
{"metricSpec":{object (QuestionAnsweringHelpfulnessSpec)},"instance":{object (QuestionAnsweringHelpfulnessInstance)}}

QuestionAnsweringHelpfulnessSpec

Spec for question answering helpfulness metric.

Fields
useReferenceboolean

Optional. Whether to use instance.reference to compute question answering helpfulness.

versioninteger

Optional. Which version to use for evaluation.

JSON representation
{"useReference":boolean,"version":integer}

QuestionAnsweringHelpfulnessInstance

Spec for question answering helpfulness instance.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Optional. Ground truth used to compare against the prediction.

contextstring

Optional. Text provided as context to answer the question.

instructionstring

Required. The question asked and other instruction in the inference prompt.

JSON representation
{"prediction":string,"reference":string,"context":string,"instruction":string}

QuestionAnsweringCorrectnessInput

Input for question answering correctness metric.

Fields

Required. Spec for question answering correctness score metric.

Required. Question answering correctness instance.

JSON representation
{"metricSpec":{object (QuestionAnsweringCorrectnessSpec)},"instance":{object (QuestionAnsweringCorrectnessInstance)}}

QuestionAnsweringCorrectnessSpec

Spec for question answering correctness metric.

Fields
useReferenceboolean

Optional. Whether to use instance.reference to compute question answering correctness.

versioninteger

Optional. Which version to use for evaluation.

JSON representation
{"useReference":boolean,"version":integer}

QuestionAnsweringCorrectnessInstance

Spec for question answering correctness instance.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Optional. Ground truth used to compare against the prediction.

contextstring

Optional. Text provided as context to answer the question.

instructionstring

Required. The question asked and other instruction in the inference prompt.

JSON representation
{"prediction":string,"reference":string,"context":string,"instruction":string}

PointwiseMetricInput

Input for pointwise metric.

Fields
metricSpecobject (PointwiseMetricSpec)

Required. Spec for pointwise metric.

instanceobject (PointwiseMetricInstance)

Required. Pointwise metric instance.

JSON representation
{"metricSpec":{object (PointwiseMetricSpec)},"instance":{object (PointwiseMetricInstance)}}

PointwiseMetricSpec

Spec for pointwise metric.

Fields
metricPromptTemplatestring

Required. Metric prompt template for pointwise metric.

JSON representation
{"metricPromptTemplate":string}

PointwiseMetricInstance

Pointwise metric instance. Usually one instance corresponds to one row in an evaluation dataset.

Fields
instanceUnion type
Instance for pointwise metric.instance can be only one of the following:
jsonInstancestring

Instance specified as a json string. String key-value pairs are expected in the jsonInstance to render PointwiseMetricSpec.instance_prompt_template.

JSON representation
{// instance"jsonInstance":string// Union type}

PairwiseMetricInput

Input for pairwise metric.

Fields
metricSpecobject (PairwiseMetricSpec)

Required. Spec for pairwise metric.

instanceobject (PairwiseMetricInstance)

Required. Pairwise metric instance.

JSON representation
{"metricSpec":{object (PairwiseMetricSpec)},"instance":{object (PairwiseMetricInstance)}}

PairwiseMetricSpec

Spec for pairwise metric.

Fields
metricPromptTemplatestring

Required. Metric prompt template for pairwise metric.

JSON representation
{"metricPromptTemplate":string}

PairwiseMetricInstance

Pairwise metric instance. Usually one instance corresponds to one row in an evaluation dataset.

Fields
instanceUnion type
Instance for pairwise metric.instance can be only one of the following:
jsonInstancestring

Instance specified as a json string. String key-value pairs are expected in the jsonInstance to render PairwiseMetricSpec.instance_prompt_template.

JSON representation
{// instance"jsonInstance":string// Union type}

ToolCallValidInput

Input for tool call valid metric.

Fields
metricSpecobject (ToolCallValidSpec)

Required. Spec for tool call valid metric.

instances[]object (ToolCallValidInstance)

Required. Repeated tool call valid instances.

JSON representation
{"metricSpec":{object (ToolCallValidSpec)},"instances":[{object (ToolCallValidInstance)}]}

ToolCallValidSpec

This type has no fields.

Spec for tool call valid metric.

ToolCallValidInstance

Spec for tool call valid instance.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Required. Ground truth used to compare against the prediction.

JSON representation
{"prediction":string,"reference":string}

ToolNameMatchInput

Input for tool name match metric.

Fields
metricSpecobject (ToolNameMatchSpec)

Required. Spec for tool name match metric.

instances[]object (ToolNameMatchInstance)

Required. Repeated tool name match instances.

JSON representation
{"metricSpec":{object (ToolNameMatchSpec)},"instances":[{object (ToolNameMatchInstance)}]}

ToolNameMatchSpec

This type has no fields.

Spec for tool name match metric.

ToolNameMatchInstance

Spec for tool name match instance.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Required. Ground truth used to compare against the prediction.

JSON representation
{"prediction":string,"reference":string}

ToolParameterKeyMatchInput

Input for tool parameter key match metric.

Fields
metricSpecobject (ToolParameterKeyMatchSpec)

Required. Spec for tool parameter key match metric.

instances[]object (ToolParameterKeyMatchInstance)

Required. Repeated tool parameter key match instances.

JSON representation
{"metricSpec":{object (ToolParameterKeyMatchSpec)},"instances":[{object (ToolParameterKeyMatchInstance)}]}

ToolParameterKeyMatchSpec

This type has no fields.

Spec for tool parameter key match metric.

ToolParameterKeyMatchInstance

Spec for tool parameter key match instance.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Required. Ground truth used to compare against the prediction.

JSON representation
{"prediction":string,"reference":string}

ToolParameterKVMatchInput

Input for tool parameter key value match metric.

Fields
metricSpecobject (ToolParameterKVMatchSpec)

Required. Spec for tool parameter key value match metric.

instances[]object (ToolParameterKVMatchInstance)

Required. Repeated tool parameter key value match instances.

JSON representation
{"metricSpec":{object (ToolParameterKVMatchSpec)},"instances":[{object (ToolParameterKVMatchInstance)}]}

ToolParameterKVMatchSpec

Spec for tool parameter key value match metric.

Fields
useStrictStringMatchboolean

Optional. Whether to use STRICT string match on parameter values.

JSON representation
{"useStrictStringMatch":boolean}

ToolParameterKVMatchInstance

Spec for tool parameter key value match instance.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Required. Ground truth used to compare against the prediction.

JSON representation
{"prediction":string,"reference":string}

CometInput

Input for Comet metric.

Fields
metricSpecobject (CometSpec)

Required. Spec for comet metric.

instanceobject (CometInstance)

Required. Comet instance.

JSON representation
{"metricSpec":{object (CometSpec)},"instance":{object (CometInstance)}}

CometSpec

Spec for Comet metric.

Fields
sourceLanguagestring

Optional. Source language in BCP-47 format.

targetLanguagestring

Optional. Target language in BCP-47 format. Covers both prediction and reference.

versionenum (CometVersion)

Required. Which version to use for evaluation.

JSON representation
{"sourceLanguage":string,"targetLanguage":string,"version":enum (CometVersion)}

CometVersion

Comet version options.

Enums
COMET_VERSION_UNSPECIFIEDComet version unspecified.
COMET_22_SRC_REFComet 22 for translation + source + reference (source-reference-combined).

CometInstance

Spec for Comet instance - The fields used for evaluation are dependent on the comet version.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Optional. Ground truth used to compare against the prediction.

sourcestring

Optional. Source text in original language.

JSON representation
{"prediction":string,"reference":string,"source":string}

MetricxInput

Input for MetricX metric.

Fields
metricSpecobject (MetricxSpec)

Required. Spec for Metricx metric.

instanceobject (MetricxInstance)

Required. Metricx instance.

JSON representation
{"metricSpec":{object (MetricxSpec)},"instance":{object (MetricxInstance)}}

MetricxSpec

Spec for MetricX metric.

Fields
sourceLanguagestring

Optional. Source language in BCP-47 format.

targetLanguagestring

Optional. Target language in BCP-47 format. Covers both prediction and reference.

versionenum (MetricxVersion)

Required. Which version to use for evaluation.

JSON representation
{"sourceLanguage":string,"targetLanguage":string,"version":enum (MetricxVersion)}

MetricxVersion

MetricX version options.

Enums
METRICX_VERSION_UNSPECIFIEDMetricX version unspecified.
METRICX_24_REFMetricX 2024 (2.6) for translation + reference (reference-based).
METRICX_24_SRCMetricX 2024 (2.6) for translation + source (QE).
METRICX_24_SRC_REFMetricX 2024 (2.6) for translation + source + reference (source-reference-combined).

MetricxInstance

Spec for MetricX instance - The fields used for evaluation are dependent on the MetricX version.

Fields
predictionstring

Required. Output of the evaluated model.

referencestring

Optional. Ground truth used to compare against the prediction.

sourcestring

Optional. Source text in original language.

JSON representation
{"prediction":string,"reference":string,"source":string}

TrajectoryExactMatchInput

Instances and metric spec for TrajectoryExactMatch metric.

Fields
metricSpecobject (TrajectoryExactMatchSpec)

Required. Spec for TrajectoryExactMatch metric.

instances[]object (TrajectoryExactMatchInstance)

Required. Repeated TrajectoryExactMatch instance.

JSON representation
{"metricSpec":{object (TrajectoryExactMatchSpec)},"instances":[{object (TrajectoryExactMatchInstance)}]}

TrajectoryExactMatchSpec

This type has no fields.

Spec for TrajectoryExactMatch metric - returns 1 if tool calls in the reference trajectory exactly match the predicted trajectory, else 0.

TrajectoryExactMatchInstance

Spec for TrajectoryExactMatch instance.

Fields
predictedTrajectoryobject (Trajectory)

Required. Spec for predicted tool call trajectory.

referenceTrajectoryobject (Trajectory)

Required. Spec for reference tool call trajectory.

JSON representation
{"predictedTrajectory":{object (Trajectory)},"referenceTrajectory":{object (Trajectory)}}

Trajectory

Spec for trajectory.

Fields
toolCalls[]object (ToolCall)

Required. Tool calls in the trajectory.

JSON representation
{"toolCalls":[{object (ToolCall)}]}

ToolCall

Spec for tool call.

Fields
toolNamestring

Required. Spec for tool name

toolInputstring

Optional. Spec for tool input

JSON representation
{"toolName":string,"toolInput":string}

TrajectoryInOrderMatchInput

Instances and metric spec for TrajectoryInOrderMatch metric.

Fields
metricSpecobject (TrajectoryInOrderMatchSpec)

Required. Spec for TrajectoryInOrderMatch metric.

instances[]object (TrajectoryInOrderMatchInstance)

Required. Repeated TrajectoryInOrderMatch instance.

JSON representation
{"metricSpec":{object (TrajectoryInOrderMatchSpec)},"instances":[{object (TrajectoryInOrderMatchInstance)}]}

TrajectoryInOrderMatchSpec

This type has no fields.

Spec for TrajectoryInOrderMatch metric - returns 1 if tool calls in the reference trajectory appear in the predicted trajectory in the same order, else 0.

TrajectoryInOrderMatchInstance

Spec for TrajectoryInOrderMatch instance.

Fields
predictedTrajectoryobject (Trajectory)

Required. Spec for predicted tool call trajectory.

referenceTrajectoryobject (Trajectory)

Required. Spec for reference tool call trajectory.

JSON representation
{"predictedTrajectory":{object (Trajectory)},"referenceTrajectory":{object (Trajectory)}}

TrajectoryAnyOrderMatchInput

Instances and metric spec for TrajectoryAnyOrderMatch metric.

Fields
metricSpecobject (TrajectoryAnyOrderMatchSpec)

Required. Spec for TrajectoryAnyOrderMatch metric.

Required. Repeated TrajectoryAnyOrderMatch instance.

JSON representation
{"metricSpec":{object (TrajectoryAnyOrderMatchSpec)},"instances":[{object (TrajectoryAnyOrderMatchInstance)}]}

TrajectoryAnyOrderMatchSpec

This type has no fields.

Spec for TrajectoryAnyOrderMatch metric - returns 1 if all tool calls in the reference trajectory appear in the predicted trajectory in any order, else 0.

TrajectoryAnyOrderMatchInstance

Spec for TrajectoryAnyOrderMatch instance.

Fields
predictedTrajectoryobject (Trajectory)

Required. Spec for predicted tool call trajectory.

referenceTrajectoryobject (Trajectory)

Required. Spec for reference tool call trajectory.

JSON representation
{"predictedTrajectory":{object (Trajectory)},"referenceTrajectory":{object (Trajectory)}}

TrajectoryPrecisionInput

Instances and metric spec for TrajectoryPrecision metric.

Fields
metricSpecobject (TrajectoryPrecisionSpec)

Required. Spec for TrajectoryPrecision metric.

instances[]object (TrajectoryPrecisionInstance)

Required. Repeated TrajectoryPrecision instance.

JSON representation
{"metricSpec":{object (TrajectoryPrecisionSpec)},"instances":[{object (TrajectoryPrecisionInstance)}]}

TrajectoryPrecisionSpec

This type has no fields.

Spec for TrajectoryPrecision metric - returns a float score based on average precision of individual tool calls.

TrajectoryPrecisionInstance

Spec for TrajectoryPrecision instance.

Fields
predictedTrajectoryobject (Trajectory)

Required. Spec for predicted tool call trajectory.

referenceTrajectoryobject (Trajectory)

Required. Spec for reference tool call trajectory.

JSON representation
{"predictedTrajectory":{object (Trajectory)},"referenceTrajectory":{object (Trajectory)}}

TrajectoryRecallInput

Instances and metric spec for TrajectoryRecall metric.

Fields
metricSpecobject (TrajectoryRecallSpec)

Required. Spec for TrajectoryRecall metric.

instances[]object (TrajectoryRecallInstance)

Required. Repeated TrajectoryRecall instance.

JSON representation
{"metricSpec":{object (TrajectoryRecallSpec)},"instances":[{object (TrajectoryRecallInstance)}]}

TrajectoryRecallSpec

This type has no fields.

Spec for TrajectoryRecall metric - returns a float score based on average recall of individual tool calls.

TrajectoryRecallInstance

Spec for TrajectoryRecall instance.

Fields
predictedTrajectoryobject (Trajectory)

Required. Spec for predicted tool call trajectory.

referenceTrajectoryobject (Trajectory)

Required. Spec for reference tool call trajectory.

JSON representation
{"predictedTrajectory":{object (Trajectory)},"referenceTrajectory":{object (Trajectory)}}

TrajectorySingleToolUseInput

Instances and metric spec for TrajectorySingleToolUse metric.

Fields
metricSpecobject (TrajectorySingleToolUseSpec)

Required. Spec for TrajectorySingleToolUse metric.

Required. Repeated TrajectorySingleToolUse instance.

JSON representation
{"metricSpec":{object (TrajectorySingleToolUseSpec)},"instances":[{object (TrajectorySingleToolUseInstance)}]}

TrajectorySingleToolUseSpec

Spec for TrajectorySingleToolUse metric - returns 1 if tool is present in the predicted trajectory, else 0.

Fields
toolNamestring

Required. Spec for tool name to be checked for in the predicted trajectory.

JSON representation
{"toolName":string}

TrajectorySingleToolUseInstance

Spec for TrajectorySingleToolUse instance.

Fields
predictedTrajectoryobject (Trajectory)

Required. Spec for predicted tool call trajectory.

JSON representation
{"predictedTrajectory":{object (Trajectory)}}

ExactMatchResults

Results for exact match metric.

Fields
exactMatchMetricValues[]object (ExactMatchMetricValue)

Output only. Exact match metric values.

JSON representation
{"exactMatchMetricValues":[{object (ExactMatchMetricValue)}]}

ExactMatchMetricValue

Exact match metric value for an instance.

Fields
scorenumber

Output only. Exact match score.

JSON representation
{"score":number}

BleuResults

Results for bleu metric.

Fields
bleuMetricValues[]object (BleuMetricValue)

Output only. Bleu metric values.

JSON representation
{"bleuMetricValues":[{object (BleuMetricValue)}]}

BleuMetricValue

Bleu metric value for an instance.

Fields
scorenumber

Output only. Bleu score.

JSON representation
{"score":number}

RougeResults

Results for rouge metric.

Fields
rougeMetricValues[]object (RougeMetricValue)

Output only. Rouge metric values.

JSON representation
{"rougeMetricValues":[{object (RougeMetricValue)}]}

RougeMetricValue

Rouge metric value for an instance.

Fields
scorenumber

Output only. Rouge score.

JSON representation
{"score":number}

FluencyResult

Spec for fluency result.

Fields
explanationstring

Output only. Explanation for fluency score.

scorenumber

Output only. Fluency score.

confidencenumber

Output only. confidence for fluency score.

JSON representation
{"explanation":string,"score":number,"confidence":number}

CoherenceResult

Spec for coherence result.

Fields
explanationstring

Output only. Explanation for coherence score.

scorenumber

Output only. Coherence score.

confidencenumber

Output only. confidence for coherence score.

JSON representation
{"explanation":string,"score":number,"confidence":number}

SafetyResult

Spec for safety result.

Fields
explanationstring

Output only. Explanation for safety score.

scorenumber

Output only. Safety score.

confidencenumber

Output only. confidence for safety score.

JSON representation
{"explanation":string,"score":number,"confidence":number}

GroundednessResult

Spec for groundedness result.

Fields
explanationstring

Output only. Explanation for groundedness score.

scorenumber

Output only. Groundedness score.

confidencenumber

Output only. confidence for groundedness score.

JSON representation
{"explanation":string,"score":number,"confidence":number}

FulfillmentResult

Spec for fulfillment result.

Fields
explanationstring

Output only. Explanation for fulfillment score.

scorenumber

Output only. Fulfillment score.

confidencenumber

Output only. confidence for fulfillment score.

JSON representation
{"explanation":string,"score":number,"confidence":number}

SummarizationQualityResult

Spec for summarization quality result.

Fields
explanationstring

Output only. Explanation for summarization quality score.

scorenumber

Output only. Summarization Quality score.

confidencenumber

Output only. confidence for summarization quality score.

JSON representation
{"explanation":string,"score":number,"confidence":number}

PairwiseSummarizationQualityResult

Spec for pairwise summarization quality result.

Fields
pairwiseChoiceenum (PairwiseChoice)

Output only. Pairwise summarization prediction choice.

explanationstring

Output only. Explanation for summarization quality score.

confidencenumber

Output only. confidence for summarization quality score.

JSON representation
{"pairwiseChoice":enum (PairwiseChoice),"explanation":string,"confidence":number}

PairwiseChoice

Pairwise prediction autorater preference.

Enums
PAIRWISE_CHOICE_UNSPECIFIEDUnspecified prediction choice.
BASELINEbaseline prediction wins
CANDIDATECandidate prediction wins
TIEWinner cannot be determined

SummarizationHelpfulnessResult

Spec for summarization helpfulness result.

Fields
explanationstring

Output only. Explanation for summarization helpfulness score.

scorenumber

Output only. Summarization Helpfulness score.

confidencenumber

Output only. confidence for summarization helpfulness score.

JSON representation
{"explanation":string,"score":number,"confidence":number}

SummarizationVerbosityResult

Spec for summarization verbosity result.

Fields
explanationstring

Output only. Explanation for summarization verbosity score.

scorenumber

Output only. Summarization Verbosity score.

confidencenumber

Output only. confidence for summarization verbosity score.

JSON representation
{"explanation":string,"score":number,"confidence":number}

QuestionAnsweringQualityResult

Spec for question answering quality result.

Fields
explanationstring

Output only. Explanation for question answering quality score.

scorenumber

Output only. Question Answering Quality score.

confidencenumber

Output only. confidence for question answering quality score.

JSON representation
{"explanation":string,"score":number,"confidence":number}

PairwiseQuestionAnsweringQualityResult

Spec for pairwise question answering quality result.

Fields
pairwiseChoiceenum (PairwiseChoice)

Output only. Pairwise question answering prediction choice.

explanationstring

Output only. Explanation for question answering quality score.

confidencenumber

Output only. confidence for question answering quality score.

JSON representation
{"pairwiseChoice":enum (PairwiseChoice),"explanation":string,"confidence":number}

QuestionAnsweringRelevanceResult

Spec for question answering relevance result.

Fields
explanationstring

Output only. Explanation for question answering relevance score.

scorenumber

Output only. Question Answering Relevance score.

confidencenumber

Output only. confidence for question answering relevance score.

JSON representation
{"explanation":string,"score":number,"confidence":number}

QuestionAnsweringHelpfulnessResult

Spec for question answering helpfulness result.

Fields
explanationstring

Output only. Explanation for question answering helpfulness score.

scorenumber

Output only. Question Answering Helpfulness score.

confidencenumber

Output only. confidence for question answering helpfulness score.

JSON representation
{"explanation":string,"score":number,"confidence":number}

QuestionAnsweringCorrectnessResult

Spec for question answering correctness result.

Fields
explanationstring

Output only. Explanation for question answering correctness score.

scorenumber

Output only. Question Answering Correctness score.

confidencenumber

Output only. confidence for question answering correctness score.

JSON representation
{"explanation":string,"score":number,"confidence":number}

PointwiseMetricResult

Spec for pointwise metric result.

Fields
explanationstring

Output only. Explanation for pointwise metric score.

scorenumber

Output only. Pointwise metric score.

JSON representation
{"explanation":string,"score":number}

PairwiseMetricResult

Spec for pairwise metric result.

Fields
pairwiseChoiceenum (PairwiseChoice)

Output only. Pairwise metric choice.

explanationstring

Output only. Explanation for pairwise metric score.

JSON representation
{"pairwiseChoice":enum (PairwiseChoice),"explanation":string}

ToolCallValidResults

Results for tool call valid metric.

Fields
toolCallValidMetricValues[]object (ToolCallValidMetricValue)

Output only. Tool call valid metric values.

JSON representation
{"toolCallValidMetricValues":[{object (ToolCallValidMetricValue)}]}

ToolCallValidMetricValue

Tool call valid metric value for an instance.

Fields
scorenumber

Output only. Tool call valid score.

JSON representation
{"score":number}

ToolNameMatchResults

Results for tool name match metric.

Fields
toolNameMatchMetricValues[]object (ToolNameMatchMetricValue)

Output only. Tool name match metric values.

JSON representation
{"toolNameMatchMetricValues":[{object (ToolNameMatchMetricValue)}]}

ToolNameMatchMetricValue

Tool name match metric value for an instance.

Fields
scorenumber

Output only. Tool name match score.

JSON representation
{"score":number}

ToolParameterKeyMatchResults

Results for tool parameter key match metric.

Fields
toolParameterKeyMatchMetricValues[]object (ToolParameterKeyMatchMetricValue)

Output only. Tool parameter key match metric values.

JSON representation
{"toolParameterKeyMatchMetricValues":[{object (ToolParameterKeyMatchMetricValue)}]}

ToolParameterKeyMatchMetricValue

Tool parameter key match metric value for an instance.

Fields
scorenumber

Output only. Tool parameter key match score.

JSON representation
{"score":number}

ToolParameterKVMatchResults

Results for tool parameter key value match metric.

Fields
toolParameterKvMatchMetricValues[]object (ToolParameterKVMatchMetricValue)

Output only. Tool parameter key value match metric values.

JSON representation
{"toolParameterKvMatchMetricValues":[{object (ToolParameterKVMatchMetricValue)}]}

ToolParameterKVMatchMetricValue

Tool parameter key value match metric value for an instance.

Fields
scorenumber

Output only. Tool parameter key value match score.

JSON representation
{"score":number}

CometResult

Spec for Comet result - calculates the comet score for the given instance using the version specified in the spec.

Fields
scorenumber

Output only. Comet score. Range depends on version.

JSON representation
{"score":number}

MetricxResult

Spec for MetricX result - calculates the MetricX score for the given instance using the version specified in the spec.

Fields
scorenumber

Output only. MetricX score. Range depends on version.

JSON representation
{"score":number}

TrajectoryExactMatchResults

Results for TrajectoryExactMatch metric.

Fields
trajectoryExactMatchMetricValues[]object (TrajectoryExactMatchMetricValue)

Output only. TrajectoryExactMatch metric values.

JSON representation
{"trajectoryExactMatchMetricValues":[{object (TrajectoryExactMatchMetricValue)}]}

TrajectoryExactMatchMetricValue

TrajectoryExactMatch metric value for an instance.

Fields
scorenumber

Output only. TrajectoryExactMatch score.

JSON representation
{"score":number}

TrajectoryInOrderMatchResults

Results for TrajectoryInOrderMatch metric.

Fields
trajectoryInOrderMatchMetricValues[]object (TrajectoryInOrderMatchMetricValue)

Output only. TrajectoryInOrderMatch metric values.

JSON representation
{"trajectoryInOrderMatchMetricValues":[{object (TrajectoryInOrderMatchMetricValue)}]}

TrajectoryInOrderMatchMetricValue

TrajectoryInOrderMatch metric value for an instance.

Fields
scorenumber

Output only. TrajectoryInOrderMatch score.

JSON representation
{"score":number}

TrajectoryAnyOrderMatchResults

Results for TrajectoryAnyOrderMatch metric.

Fields
trajectoryAnyOrderMatchMetricValues[]object (TrajectoryAnyOrderMatchMetricValue)

Output only. TrajectoryAnyOrderMatch metric values.

JSON representation
{"trajectoryAnyOrderMatchMetricValues":[{object (TrajectoryAnyOrderMatchMetricValue)}]}

TrajectoryAnyOrderMatchMetricValue

TrajectoryAnyOrderMatch metric value for an instance.

Fields
scorenumber

Output only. TrajectoryAnyOrderMatch score.

JSON representation
{"score":number}

TrajectoryPrecisionResults

Results for TrajectoryPrecision metric.

Fields
trajectoryPrecisionMetricValues[]object (TrajectoryPrecisionMetricValue)

Output only. TrajectoryPrecision metric values.

JSON representation
{"trajectoryPrecisionMetricValues":[{object (TrajectoryPrecisionMetricValue)}]}

TrajectoryPrecisionMetricValue

TrajectoryPrecision metric value for an instance.

Fields
scorenumber

Output only. TrajectoryPrecision score.

JSON representation
{"score":number}

TrajectoryRecallResults

Results for TrajectoryRecall metric.

Fields
trajectoryRecallMetricValues[]object (TrajectoryRecallMetricValue)

Output only. TrajectoryRecall metric values.

JSON representation
{"trajectoryRecallMetricValues":[{object (TrajectoryRecallMetricValue)}]}

TrajectoryRecallMetricValue

TrajectoryRecall metric value for an instance.

Fields
scorenumber

Output only. TrajectoryRecall score.

JSON representation
{"score":number}

TrajectorySingleToolUseResults

Results for TrajectorySingleToolUse metric.

Fields
trajectorySingleToolUseMetricValues[]object (TrajectorySingleToolUseMetricValue)

Output only. TrajectorySingleToolUse metric values.

JSON representation
{"trajectorySingleToolUseMetricValues":[{object (TrajectorySingleToolUseMetricValue)}]}

TrajectorySingleToolUseMetricValue

TrajectorySingleToolUse metric value for an instance.

Fields
scorenumber

Output only. TrajectorySingleToolUse score.

JSON representation
{"score":number}

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-06-27 UTC.