DataQualityRule

A rule captures data quality intent about a data source.

JSON representation
{"column":string,"ignoreNull":boolean,"dimension":string,"threshold":number,"name":string,"description":string,"suspended":boolean,// Union fieldrule_type can be only one of the following:"rangeExpectation":{object (RangeExpectation)},"nonNullExpectation":{object (NonNullExpectation)},"setExpectation":{object (SetExpectation)},"regexExpectation":{object (RegexExpectation)},"uniquenessExpectation":{object (UniquenessExpectation)},"statisticRangeExpectation":{object (StatisticRangeExpectation)},"rowConditionExpectation":{object (RowConditionExpectation)},"tableConditionExpectation":{object (TableConditionExpectation)},"sqlAssertion":{object (SqlAssertion)}// End of list of possible types for union fieldrule_type.}
Fields
column

string

Optional. The unnested column which this rule is evaluated against.

ignoreNull

boolean

Optional. Rows withnull values will automatically fail a rule, unlessignoreNull istrue. In that case, suchnull rows are trivially considered passing.

This field is only valid for the following type of rules:

  • RangeExpectation
  • RegexExpectation
  • SetExpectation
  • UniquenessExpectation
dimension

string

Required. The dimension a rule belongs to. Results are also aggregated at the dimension level. Custom dimension name is supported with all uppercase letters and maximum length of 30 characters.

threshold

number

Optional. The minimum ratio ofpassing_rows / total_rows required to pass this rule, with a range of [0.0, 1.0].

0 indicates default value (i.e. 1.0).

This field is only valid for row-level type rules.

name

string

Optional. A mutable name for the rule.

  • The name must contain only letters (a-z, A-Z), numbers (0-9), or hyphens (-).
  • The maximum length is 63 characters.
  • Must start with a letter.
  • Must end with a number or a letter.
description

string

Optional. Description of the rule.

  • The maximum length is 1,024 characters.
suspended

boolean

Optional. Whether the Rule is active or suspended. Default is false.

Union fieldrule_type. The rule-specific configuration.rule_type can be only one of the following:
rangeExpectation

object (RangeExpectation)

Row-level rule which evaluates whether each column value lies between a specified range.

nonNullExpectation

object (NonNullExpectation)

Row-level rule which evaluates whether each column value is null.

setExpectation

object (SetExpectation)

Row-level rule which evaluates whether each column value is contained by a specified set.

regexExpectation

object (RegexExpectation)

Row-level rule which evaluates whether each column value matches a specified regex.

uniquenessExpectation

object (UniquenessExpectation)

Row-level rule which evaluates whether each column value is unique.

statisticRangeExpectation

object (StatisticRangeExpectation)

Aggregate rule which evaluates whether the column aggregate statistic lies between a specified range.

rowConditionExpectation

object (RowConditionExpectation)

Row-level rule which evaluates whether each row in a table passes the specified condition.

tableConditionExpectation

object (TableConditionExpectation)

Aggregate rule which evaluates whether the provided expression is true for a table.

sqlAssertion

object (SqlAssertion)

Aggregate rule which evaluates the number of rows returned for the provided statement. If any rows are returned, this rule fails.

RangeExpectation

Evaluates whether each column value lies between a specified range.

JSON representation
{"minValue":string,"maxValue":string,"strictMinEnabled":boolean,"strictMaxEnabled":boolean}
Fields
minValue

string

Optional. The minimum column value allowed for a row to pass this validation. At least one ofminValue andmaxValue need to be provided.

maxValue

string

Optional. The maximum column value allowed for a row to pass this validation. At least one ofminValue andmaxValue need to be provided.

strictMinEnabled

boolean

Optional. Whether each value needs to be strictly greater than ('>') the minimum, or if equality is allowed.

Only relevant if aminValue has been defined. Default = false.

strictMaxEnabled

boolean

Optional. Whether each value needs to be strictly lesser than ('<') the maximum, or if equality is allowed.

Only relevant if amaxValue has been defined. Default = false.

NonNullExpectation

This type has no fields.

Evaluates whether each column value is null.

SetExpectation

Evaluates whether each column value is contained by a specified set.

JSON representation
{"values":[string]}
Fields
values[]

string

Optional. Expected values for the column value.

RegexExpectation

Evaluates whether each column value matches a specified regex.

JSON representation
{"regex":string}
Fields
regex

string

Optional. A regular expression the column value is expected to match.

UniquenessExpectation

This type has no fields.

Evaluates whether the column has duplicates.

StatisticRangeExpectation

Evaluates whether the column aggregate statistic lies between a specified range.

JSON representation
{"statistic":enum (ColumnStatistic),"minValue":string,"maxValue":string,"strictMinEnabled":boolean,"strictMaxEnabled":boolean}
Fields
statistic

enum (ColumnStatistic)

Optional. The aggregate metric to evaluate.

minValue

string

Optional. The minimum column statistic value allowed for a row to pass this validation.

At least one ofminValue andmaxValue need to be provided.

maxValue

string

Optional. The maximum column statistic value allowed for a row to pass this validation.

At least one ofminValue andmaxValue need to be provided.

strictMinEnabled

boolean

Optional. Whether column statistic needs to be strictly greater than ('>') the minimum, or if equality is allowed.

Only relevant if aminValue has been defined. Default = false.

strictMaxEnabled

boolean

Optional. Whether column statistic needs to be strictly lesser than ('<') the maximum, or if equality is allowed.

Only relevant if amaxValue has been defined. Default = false.

ColumnStatistic

The list of aggregate metrics a rule can be evaluated against.

Enums
STATISTIC_UNDEFINEDUnspecified statistic type
MEANEvaluate the column mean
MINEvaluate the column min
MAXEvaluate the column max

RowConditionExpectation

Evaluates whether each row passes the specified condition.

The SQL expression needs to useGoogleSQL syntax and should produce a boolean value per row as the result.

Example: col1 >= 0 AND col2 < 10

JSON representation
{"sqlExpression":string}
Fields
sqlExpression

string

Optional. The SQL expression.

TableConditionExpectation

Evaluates whether the provided expression is true.

The SQL expression needs to useGoogleSQL syntax and should produce a scalar boolean result.

Example: MIN(col1) >= 0

JSON representation
{"sqlExpression":string}
Fields
sqlExpression

string

Optional. The SQL expression.

SqlAssertion

A SQL statement that is evaluated to return rows that match an invalid state. If any rows are are returned, this rule fails.

The SQL statement must useGoogleSQL syntax, and must not contain any semicolons.

You can use the data reference parameter${data()} to reference the source table with all of its precondition filters applied. Examples of precondition filters include row filters, incremental data filters, and sampling. For more information, seeData reference parameter.

Example:SELECT * FROM ${data()} WHERE price < 0

JSON representation
{"sqlStatement":string}
Fields
sqlStatement

string

Optional. The SQL statement.

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-07-21 UTC.