In the Google Cloud console, you can create apublic endpointand deploy a model to it.

Models can be deployed from theOnline prediction page or the Model Registrypage.

Deploy a model from the Online prediction page

In the Online prediction page, you can create an endpoint and deployone or more models to it as follows:

1. In the Google Cloud console, in the Vertex AI section, goto theOnline prediction page.

   Go to the Online prediction page

2. ClickaddCreate.

3. In theNew endpoint pane:

   1. Enter theEndpoint name.

   2. SelectStandard for the access type.

   3. To create a dedicated (not shared) public endpoint, select theEnable dedicated DNS checkbox.

   4. ClickContinue.

4. In theModel settings pane:

   1. Select your model from the drop-down list.

   2. Choose the model version from the drop-down list.

   3. Enter theTraffic split percentage for the model.

   4. ClickDone.

   5. Repeat these steps for any additional models to be deployed.

Deploy a model from the Model Registry page

In the Model Registry page, you can deploy a model to oneor more new or existing endpoints as follows:

1. In the Google Cloud console, in the Vertex AI section, goto theModels page.

   Go to the Models page

2. Click the name and version ID of the model you want to deploy to openits details page.

3. Select theDeploy & Test tab.

   If your model is already deployed to any endpoints, they are listed in theDeploy your model section.

4. ClickDeploy to endpoint.

5. To deploy your model to a new endpoint:

   1. Selectradio_button_checkedCreate new endpoint
   2. Provide a name for the new endpoint.
   3. To create a dedicated (not shared) public endpoint, select theEnable dedicated DNS checkbox.
   4. ClickContinue.

   To deploy your model to an existing endpoint:

   1. Selectradio_button_checkedAdd to existing endpoint.
   2. Select the endpoint from the drop-down list.
   3. ClickContinue.

   You can deploy multiple models to an endpoint, or you can deploy thesame model to multiple endpoints.

6. If you deploy your model to an existing endpoint that has one or moremodels deployed to it, you must update theTraffic split percentagefor the model you are deploying and the already deployed models so that allof the percentages add up to 100%.

7. If you're deploying your model to a new endpoint, accept 100 for theTraffic split. Otherwise, adjust the traffic split values forall models on the endpoint so they add up to 100.

8. Enter theMinimum number of compute nodes you want to provide foryour model.

   This is the number of nodes that need to be available to the model at all times.

   You are charged for the nodes used, whether to handle inference load or forstandby (minimum) nodes, even without inference traffic. See thepricing page.

   The number of compute nodes can increase if needed to handle inferencetraffic, but it will never go higher than the maximum number of nodes.

9. To use autoscaling, enter theMaximum number of compute nodes youwant Vertex AI to scale up to.

10. Select yourMachine type.

    Larger machine resources increase your inference performance andincrease costs.Compare the available machine types.

11. Select anAccelerator type and anAccelerator count.

    If you enabled accelerator use when youimportedor created the model, this option displays.

    For the accelerator count, refer to theGPUtable to check for valid numbersof GPUs that you can use with each CPU machine type. The acceleratorcount refers to the number of accelerators per node, not the totalnumber of accelerators in your deployment.

12. If you want to use acustom serviceaccount for the deployment, selecta service account in theService account drop-down box.

13. Learn how tochange thedefault settings for inference logging.

14. ClickDone for your model, and when all theTraffic splitpercentages are correct, clickContinue.

    The region where your model deploys is displayed. This must be the region where you created your model.

15. ClickDeploy to deploy your model to the endpoint.

What's next

• Learn how toget an online inference.
• Learn how tochange thedefault settings for inference logging.