Console

<a href="https://console.cloud.google.com/vertex-ai/studio/media/generate;tab=image"target="console" track-name="consoleLink" track-type="task">Go to MediaStudio</a>

1. ClickUpload. In the displayed file dialog, select a file to upload.

2. ClickInpaint.

3. In theParameters panel, clickInpaint (Remove).

4. Do one of the following:

   • Upload your own mask:

     1. Create a mask on your computer.

     2. ClickUpload mask. In the displayed dialog, select a mask to upload.

   • Define your own mask: in the editing toolbar, use the mask tools (check_box_outline_blankbox,brushbrush, ormasked_transitionsinvert tool) to specify the area or areas to add content to.

5. Optional: In theParameters panel, adjust the following options:

   • Model: the Imagen model to use

   • Number of results: the number of result to generate

   • Negative prompt: items to avoid generating

6. In the prompt field, enter a prompt to modify the image.

7. Clickplay_arrowGenerate.

Python

Install

pip install --upgrade google-genai

To learn more, see the SDK reference documentation.

Set environment variables to use the Gen AI SDK with Vertex AI:

# Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values# with appropriate values for your project.exportGOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECTexportGOOGLE_CLOUD_LOCATION=us-central1exportGOOGLE_GENAI_USE_VERTEXAI=True

fromgoogleimportgenaifromgoogle.genai.typesimport(RawReferenceImage,MaskReferenceImage,MaskReferenceConfig,EditImageConfig,)client=genai.Client()# TODO(developer): Update and un-comment below line# output_file = "output-image.png"raw_ref=RawReferenceImage(reference_image=Image.from_file(location="test_resources/fruit.png"),reference_id=0,)mask_ref=MaskReferenceImage(reference_id=1,reference_image=Image.from_file(location="test_resources/fruit_mask.png"),config=MaskReferenceConfig(mask_mode="MASK_MODE_USER_PROVIDED",mask_dilation=0.01,),)image=client.models.edit_image(model="imagen-3.0-capability-001",prompt="",reference_images=[raw_ref,mask_ref],config=EditImageConfig(edit_mode="EDIT_MODE_INPAINT_REMOVAL",),)image.generated_images[0].image.save(output_file)print(f"Created output image using{len(image.generated_images[0].image.image_bytes)} bytes")# Example response:# Created output image using 1234567 bytes

REST

Before using any of the request data, make the following replacements:

• PROJECT_ID: Your Google Cloudproject ID.
• LOCATION: Your project's region. For example,us-central1,europe-west2, orasia-northeast3. For a list of available regions, seeGenerative AI on Vertex AI locations.
• prompt: For image outpainting you can provide an empty string to create the edited images. If you choose to provide a prompt, use a description of the masked area for best results. For example, "a blue sky" instead of "insert a blue sky".
• referenceType: AReferenceImage is an image that provides additional context for image editing. A normal RGB raw reference image (REFERENCE_TYPE_RAW) is required for editing use cases. At most one raw reference image may exist in one request. The output image has the same height and width as raw reference image. A mask reference image (REFERENCE_TYPE_MASK) is required for masked editing use cases. If a raw reference image is present, the mask image has to be the same height and width as the raw reference image. If the mask reference image is empty andmaskMode is not set toMASK_MODE_USER_PROVIDED, the mask is computed based on the raw reference image.
• B64_BASE_IMAGE: The base image to edit or upscale. The image must be specified as abase64-encoded byte string. Size limit: 10 MB.
• B64_OUTPAINTING_MASK: The black and white image you want to use as a mask layer to edit the original image. The mask should be same resolution as input image. The output image will be the same resolution as the input image. This mask image must be specified as abase64-encoded byte string. Size limit: 10 MB.
• MASK_DILATION - float. The percentage of image width to dilate this mask by. A value of0.03 is recommended for outpainting. Setting"dilation": 0.0 might result in obvious borders at the extension point, or might cause a white border effect.
• EDIT_STEPS - integer. The number of sampling steps for the base model. For outpainting, start at35 steps. Increase steps if the quality doesn't meet your requirements.
• EDIT_IMAGE_COUNT - The number of edited images. Accepted integer values: 1-4. Default value: 4.

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/imagen-3.0-capability-001:predict

Request JSON body:

{  "instances": [    {      "prompt": "",      "referenceImages": [        {          "referenceType": "REFERENCE_TYPE_RAW",          "referenceId": 1,          "referenceImage": {            "bytesBase64Encoded": "B64_BASE_IMAGE"          }        },        {          "referenceType": "REFERENCE_TYPE_MASK",          "referenceId": 2,          "referenceImage": {            "bytesBase64Encoded": "B64_OUTPAINTING_MASK"          },          "maskImageConfig": {            "maskMode": "MASK_MODE_USER_PROVIDED",            "dilation":MASK_DILATION          }        }      ]    }  ],  "parameters": {    "editConfig": {      "baseSteps":EDIT_STEPS    },    "editMode": "EDIT_MODE_OUTPAINT",    "sampleCount":EDIT_IMAGE_COUNT  }}

To send your request, choose one of these options:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/imagen-3.0-capability-001:predict"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/imagen-3.0-capability-001:predict" | Select-Object -Expand Content

{  "predictions": [    {      "bytesBase64Encoded": "BASE64_IMG_BYTES",      "mimeType": "image/png"    },    {      "mimeType": "image/png",      "bytesBase64Encoded": "BASE64_IMG_BYTES"    }  ]}

Console

<a href="https://console.cloud.google.com/vertex-ai/studio/media/generate;tab=image"target="console" track-name="consoleLink" track-type="task">Go to MediaStudio</a>

1. ClickUpload. In the displayed file dialog, select a file to upload.

2. ClickInpaint.

3. In theParameters panel, clickInpaint (Remove).

4. In the editing toolbar, clickbackground_replaceExtract.

   Select one of the mask extraction options:

   • Background elements: detects the background elements and creates a mask around them.

   • wallpaperForeground elements: detects the foreground objects and creates a mask around them.

   • background_replacePeople: detects people and creates a mask around them.

5. Optional: In theParameters panel, adjust the following options:

   • Model: the Imagen model to use

   • Number of results: the number of result to generate

   • Negative prompt: items to avoid generating

6. In the prompt field, enter a new prompt to modify the image.

7. ClicksendGenerate.

Python

Install

pip install --upgrade google-genai

To learn more, see the SDK reference documentation.

Set environment variables to use the Gen AI SDK with Vertex AI:

# Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values# with appropriate values for your project.exportGOOGLE_CLOUD_PROJECT=GOOGLE_CLOUD_PROJECTexportGOOGLE_CLOUD_LOCATION=us-central1exportGOOGLE_GENAI_USE_VERTEXAI=True

fromgoogleimportgenaifromgoogle.genai.typesimport(RawReferenceImage,MaskReferenceImage,MaskReferenceConfig,EditImageConfig,)client=genai.Client()# TODO(developer): Update and un-comment below line# output_file = "output-image.png"raw_ref=RawReferenceImage(reference_image=Image.from_file(location="test_resources/fruit.png"),reference_id=0,)mask_ref=MaskReferenceImage(reference_id=1,reference_image=None,config=MaskReferenceConfig(mask_mode="MASK_MODE_FOREGROUND",),)image=client.models.edit_image(model="imagen-3.0-capability-001",prompt="",reference_images=[raw_ref,mask_ref],config=EditImageConfig(edit_mode="EDIT_MODE_INPAINT_REMOVAL",),)image.generated_images[0].image.save(output_file)print(f"Created output image using{len(image.generated_images[0].image.image_bytes)} bytes")# Example response:# Created output image using 1234567 bytes

REST

Before using any of the request data, make the following replacements:

• PROJECT_ID: Your Google Cloudproject ID.
• LOCATION: Your project's region. For example,us-central1,europe-west2, orasia-northeast3. For a list of available regions, seeGenerative AI on Vertex AI locations.
• prompt: For best results, omit a prompt andnegativePrompt when you use inpainting for removal.
• B64_BASE_IMAGE: The base image to edit or upscale. The image must be specified as abase64-encoded byte string. Size limit: 10 MB.
• MASK_MODE - A string that sets the type of automatic mask creation the model uses. Available values:
  • MASK_MODE_BACKGROUND: Automatically generates a mask using background segmentation. Use this setting for modifying background content.
  • MASK_MODE_FOREGROUND: Automatically generates a mask using foreground segmentation. Use this setting to modify foreground content, such as removing these foreground objects (removal using inpainting).
  • MASK_MODE_SEMANTIC: Automatically generates a mask using semantic segmentation based on thesegmentation classes you specify in themaskImageConfig.maskClasses array. For example:

              "maskImageConfig": {            "maskMode": "MASK_MODE_SEMANTIC",            "maskClasses": [175, 176], // bicycle, car            "dilation": 0.01          }

• MASK_DILATION - float. The percentage of image width to dilate this mask by. A value of0.01 is recommended to compensate for imperfect input masks.
• EDIT_STEPS - integer. The number of sampling steps for the base model. For inpainting removal, start at12 steps. Increase steps to upper limit of75 if the quality doesn't meet your requirements. Increasing steps also increases request latency.
• EDIT_IMAGE_COUNT - The number of edited images. Accepted integer values: 1-4. Default value: 4.

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/imagen-3.0-capability-001:predict

Request JSON body:

{  "instances": [    {      "prompt": "",      "referenceImages": [        {          "referenceType": "REFERENCE_TYPE_RAW",          "referenceId": 1,          "referenceImage": {            "bytesBase64Encoded": "B64_BASE_IMAGE"          }        },        {          "referenceType": "REFERENCE_TYPE_MASK",          "referenceId": 2,          "maskImageConfig": {            "maskMode": "MASK_MODE",            "dilation":MASK_DILATION          }        }      ]    }  ],  "parameters": {    "editConfig": {      "baseSteps":EDIT_STEPS    },    "editMode": "EDIT_MODE_INPAINT_REMOVAL",    "sampleCount":EDIT_IMAGE_COUNT  }}

To send your request, choose one of these options:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/imagen-3.0-capability-001:predict"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/imagen-3.0-capability-001:predict" | Select-Object -Expand Content

{  "predictions": [    {      "bytesBase64Encoded": "BASE64_IMG_BYTES",      "mimeType": "image/png"    },    {      "mimeType": "image/png",      "bytesBase64Encoded": "BASE64_IMG_BYTES"    }  ]}