Controlled Customization lets you to create new images based on a source imageor a source image signal (canny edge or scribble). This pageshows you how to send two types of Controlled Customization requests:

1. Imagen 3 Controlled Customization (GA feature)

   View Imagen for Editing and Customization model card

2. Imagen 2 Controlled Customization - canny edge or scribble source image signal(Preview feature)

Use cases

Imagen 3 Controlled Customization offers free-style prompting, which can giveyou the impression that the model can do more than it is trained to do. Thefollowing sections describe use cases and examples forImagen 3 Controlled Customization.

The model has been trained on use cases that we provide, and we expect goodresults when you use Imagen 3 Controlled Customization. If you push the model torespond in unexpected ways, we don't expect good results.

Examples of intended use cases

The following are use cases that Imagen 3 Controlled Customization is trained todo and produces good results for:

• Generate an image that follows the prompt and the canny edge controlimages.

• Generate an image that follows the prompt and the scribble images.

• Stylize a photo of a person while preserving the facial expression.

Examples of unintended use cases

The following are use cases that Imagen 3 Controlled Customization isn't trainedto do and produces poor results for:

• Generate an image using a style specified in the prompt.

• Generate an image from text that follows a specific style provided by areference image, with some level of control on the image composition using acontrol image.

• Generate an image from text that follows a specific style provided by areference image, with some level of control on the image composition using acontrol scribble.

• Generate an image from text that follows a specific style provided by thereference image, with some level of control on the image composition using acontrol image. The person in the image has a specific facial expression.

• Stylize a photo of two or more people, and preserve their facialexpressions.

• Stylize a photo of a pet, and turn it into a drawing. Preserve or specifythe composition of the image (for example, watercolor).

Before you begin

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the Vertex AI API.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

Enable the API

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the Vertex AI API.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

Enable the API

1. Set up authentication for your environment.

   Select the tab for how you plan to use the samples on this page:

   Console

   When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

   REST

   To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

   Install the Google Cloud CLI. After installation,initialize the Google Cloud CLI by running the following command:

   gcloudinit

   If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

   For more information, seeAuthenticate for using REST in the Google Cloud authentication documentation.

Customization prompt writing

The prompt you use with Imagen 3 Customization might affect the qualityof your generated images. Use the following prompt templates as a startingpoint for writing customization prompts. You might need to send several requeststo get your targeted output.

Send a Imagen 3 Controlled Customization request

Use the following samples to send a Imagen 3 Controlled Customization request:

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/imagen-3.0-capability-001:predict

{  "instances": [    {      "prompt": "TEXT_PROMPT",      "referenceImages": [        {          "referenceType": "REFERENCE_TYPE_CONTROL",          "referenceId":1,          "referenceImage": {            "bytesBase64Encoded": "BASE64_CONTROL_IMAGE"          },          "controlImageConfig": {            "controlType": "CONTROL_TYPE",            "enableControlImageComputation":false          }        }      ]    }  ],  "parameters": {    "sampleCount":IMAGE_COUNT  }}

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/imagen-3.0-capability-001:predict"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/imagen-3.0-capability-001:predict" | Select-Object -Expand Content

{  "predictions": [    {      "bytesBase64Encoded": "BASE64_IMG_BYTES",      "mimeType": "image/png"    },    {      "mimeType": "image/png",      "bytesBase64Encoded": "BASE64_IMG_BYTES"    }  ]}

fromgoogleimportgenaifromgoogle.genai.typesimport(ControlReferenceConfig,ControlReferenceImage,EditImageConfig,Image,)client=genai.Client()# TODO(developer): Update and un-comment below line# output_gcs_uri = "gs://your-bucket/your-prefix"# Create a reference image out of an existing scribble image signal# using https://storage.googleapis.com/cloud-samples-data/generative-ai/image/car_scribble.pngcontrol_reference_image=ControlReferenceImage(reference_id=1,reference_image=Image(gcs_uri="gs://cloud-samples-data/generative-ai/image/car_scribble.png"),config=ControlReferenceConfig(control_type="CONTROL_TYPE_SCRIBBLE"),)image=client.models.edit_image(model="imagen-3.0-capability-001",prompt="an oil painting showing the side of a red car[1]",reference_images=[control_reference_image],config=EditImageConfig(edit_mode="EDIT_MODE_CONTROLLED_EDITING",number_of_images=1,safety_filter_level="BLOCK_MEDIUM_AND_ABOVE",person_generation="ALLOW_ADULT",output_gcs_uri=output_gcs_uri,),)# Example response:# gs://your-bucket/your-prefixprint(image.generated_images[0].image.gcs_uri)

fromgoogleimportgenaifromgoogle.genai.typesimport(ControlReferenceConfig,ControlReferenceImage,EditImageConfig,Image,)client=genai.Client()# TODO(developer): Update and un-comment below line# output_gcs_uri = "gs://your-bucket/your-prefix"# Create a reference image out of an existing canny edge image signal# using https://storage.googleapis.com/cloud-samples-data/generative-ai/image/car_canny.pngcontrol_reference_image=ControlReferenceImage(reference_id=1,reference_image=Image(gcs_uri="gs://cloud-samples-data/generative-ai/image/car_canny.png"),config=ControlReferenceConfig(control_type="CONTROL_TYPE_CANNY"),)image=client.models.edit_image(model="imagen-3.0-capability-001",prompt="a watercolor painting of a red car[1] driving on a road",reference_images=[control_reference_image],config=EditImageConfig(edit_mode="EDIT_MODE_CONTROLLED_EDITING",number_of_images=1,safety_filter_level="BLOCK_MEDIUM_AND_ABOVE",person_generation="ALLOW_ADULT",output_gcs_uri=output_gcs_uri,),)# Example response:# gs://your-bucket/your-prefixprint(image.generated_images[0].image.gcs_uri)

Send an Imagen 2 Controlled Customization request

Preview

This Imagen feature is a Preview offering, subject to the "Pre-GA Offerings Terms" of theGoogle Cloud Service Specific Terms. Pre-GA products and features may have limited support, and changes to pre-GA products and features may not be compatible with other pre-GA versions. For more information, see thelaunch stage descriptions. Further, by using this Imagen feature, you agree to the Generative AI Previewterms and conditions ("Preview Terms").

Input image	Other parameters	Output image
	Prompt: "digital art version" Negative prompt: "black and white" Guide image type: Standard RGB Control condition: canny edge Imagen Control scale: 0.95
	Prompt: "digital art version" Negative prompt: "black and white" Guide image type: canny edge Imagen Control scale: 0.95

Use the following samples to send a Imagen 2 Controlled Customization request:

POST https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/imagen-2.0-edit-preview-0627:predict

{  "instances": [    {      "prompt": "TEXT_PROMPT",      "image": {        "bytesBase64Encoded": "B64_BASE_IMAGE"      }    }  ],  "parameters": {    "sampleCount":EDIT_IMAGE_COUNT,    "negativePrompt": "NEGATIVE_PROMPT",    "controlPluginConfig":  {      "conditions": [        {          "conditionName": "CONDITION",          "controlScale":CONTROL_SCALE,          "samplingSteps":SAMPLING_STEPS,          "computeConditionMap":COMPUTE_CONDITION_MAP        }      ]    }  }}

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/imagen-2.0-edit-preview-0627:predict"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/publishers/google/models/imagen-2.0-edit-preview-0627:predict" | Select-Object -Expand Content

{  "predictions": [    {      "bytesBase64Encoded": "BASE64_IMG_BYTES",      "mimeType": "image/png"    },    {      "mimeType": "image/png",      "bytesBase64Encoded": "BASE64_IMG_BYTES"    }  ]}

Product usage

To view usage standards and content restrictions associated withImagen on Vertex AI, see theusage guidelines.

Model versions

There are multiple image generation models that you can use. For moreinformation, seeImagenmodels.

What's next

Read articles about Imagen and other Generative AI on Vertex AIproducts:

• A developer's guide to getting started with Imagen 3 onVertex AI
• New generative media models and tools, built with and for creators
• New in Gemini: Custom Gems and improved image generation withImagen 3
• Google DeepMind: Imagen 3 - Our highest quality text-to-image model