The ML.DECODE_IMAGE function

This document describes theML.DECODE_IMAGE scalar function, which lets youconvert image bytes to a multi-dimensionalARRAY representation. Thisconversion lets you do inference using theML.PREDICT functionon vision models that require images to be inARRAY format.

Syntax

ML.DECODE_IMAGE(data) AS input_field

Arguments

ML.DECODE_IMAGE takes the following argument:

• data: this argument must be a reference to one of the following:

  • ABYTES value that represents an image in one of the followingformats:

    • JPEG
    • PNG
    • BMP

  • Thedata pseudocolumn of an object table.

    If you are referencingonly the object table in thequery_statementof theML.PREDICT function, you can specifydata as the value. If you arejoining the object table with other tables in thequery_statement of theML.PREDICT function, then you should fully qualify thedata value asdataset_name.object_table_name.data.This lets you avoid ambiguity if the tables share any column names.

  Thedata argument value must be <= 10 MB (10,000,000 bytes).

Output

ML.DECODE_IMAGE returns aSTRUCT<ARRAY<INT64>, ARRAY<FLOAT64>> value thatrepresents the image.

The first array in the struct represents the dimensions of the image.It contains threeINT64 values, which represent the image height (H),width (W), and number of channels (C).

The second array in the struct contains the decoded image data. The length ofthe array is equivalent to H x W x C from the preceding array. Each valuein the array is between[0, 1).

For example, a 1x2 image in theRGB color space might have a formsimilar to[[0.1,0.2,0.3],[0.4,0.5,0.6]], which would be representedas a struct in the form([1,2,3],[0.1,0.2,0.3,0.4,0.5,0.6]).

If the struct value is larger than 60 MB, it is downscaled to that sizewhile preserving aspect ratio.

You can useML.DECODE_IMAGE output directly in anML.PREDICT function,or you can write the results to a table column and reference that columnwhen you callML.PREDICT. You can also passML.DECODE_IMAGE output toanother image processing function for additional preprocessing.

If you are usingML.DECODE_IMAGE output directlyin anML.PREDICT function, you must alias the output of this functionwith the name of an input field for the model that is referenced in theML.PREDICT function. You can find this information byinspecting the modeland looking at the field names in theFeatures section. For moreinformation, seeRun inference on image object tables.

Examples

The following examples show different ways you can use theML.DECODE_IMAGEfunction.

Example 1

The following example uses theML.DECODE_IMAGE function directly in theML.PREDICT function. It returns the inference results for all images in theobject table, for a model with an input field ofinput and an outputfield offeature:

SELECT*FROMML.PREDICT(MODEL`my_dataset.vision_model`,(SELECTuri,ML.RESIZE_IMAGE(ML.DECODE_IMAGE(data),480,480,FALSE)ASinputFROM`my_dataset.object_table`));

Example 2

The following example uses theML.DECODE_IMAGE function directly in theML.PREDICT function, and uses theML.CONVERT_COLOR_SPACE function in theML.PREDICT function to convertthe image color space fromRBG toYIQ. It also shows how touse object table fields to filter the objects included in inference.It returns the inference results for all JPG images in theobject table, for a model with an input field ofinput and an outputfield offeature:

SELECT*FROMML.PREDICT(MODEL`my_dataset.vision_model`,(SELECTuri,ML.CONVERT_COLOR_SPACE(ML.RESIZE_IMAGE(ML.DECODE_IMAGE(data),224,280,TRUE),'YIQ')ASinputFROM`my_dataset.object_table`WHEREcontent_type='image/jpeg'));

Example 3

The following example uses results fromML.DECODE_IMAGE that have beenwritten to a table column but not processed any further. It usesML.RESIZE_IMAGE andML.CONVERT_IMAGE_TYPE in theML.PREDICT function toprocess the image data. It returns the inference results for all images in thedecoded images table, for a model with an input field ofinput and an outputfield offeature.

Create the decoded images table:

CREATEORREPLACETABLE`my_dataset.decoded_images`AS(SELECTML.DECODE_IMAGE(data)ASdecoded_imageFROM`my_dataset.object_table`);

Run inference on the decoded images table:

SELECT*FROMML.PREDICT(MODEL`my_dataset.vision_model`,(SELECTuri,ML.CONVERT_IMAGE_TYPE(ML.RESIZE_IMAGE(decoded_image,480,480,FALSE))ASinputFROM`my_dataset.decoded_images`));

Example 4

The following example uses results fromML.DECODE_IMAGE that have beenwritten to a table column and preprocessed usingML.RESIZE_IMAGE. It returns the inference results for all images in thedecoded images table, for a model with an input field ofinput and an outputfield offeature.

Create the table:

CREATEORREPLACETABLE`my_dataset.decoded_images`AS(SELECTML.RESIZE_IMAGE(ML.DECODE_IMAGE(data)480,480,FALSE)ASdecoded_imageFROM`my_dataset.object_table`);

Run inference on the decoded images table:

SELECT*FROMML.PREDICT(MODEL`my_dataset.vision_model`,(SELECTuri,decoded_imageASinputFROM`my_dataset.decoded_images`));

Example 5

The following example uses theML.DECODE_IMAGE function directly in theML.PREDICT function. In this example, the model has an output field ofembeddings and two input fields: one that expects animage,f_img, and one that expects a string,f_txt. The imageinput comes from the object table and the string input comes from astandard BigQuery table that is joined with the object tableby using theuri column.

SELECT*FROMML.PREDICT(MODEL`my_dataset.mixed_model`,(SELECTuri,ML.RESIZE_IMAGE(ML.DECODE_IMAGE(my_dataset.my_object_table.data),224,224,FALSE)ASf_img,my_dataset.image_description.descriptionASf_txtFROM`my_dataset.object_table`JOIN`my_dataset.image_description`ONobject_table.uri=image_description.uri));

What's next

• For information about feature preprocessing, seeFeature preprocessing overview.

• For more information about supported SQL statements and functions for eachmodel type, see the following documents:

  • End-to-end user journeys for generative AI models
  • End-to-end user journeys for time series forecasting models
  • End-to-end user journeys for ML models
  • End-to-end user journeys for imported models
  • Contribution analysis user journey