Vertex AI offersprebuilt containersto serve inferences and explanations from models trained using the followingmachine learning (ML) frameworks:

• TensorFlow
• PyTorch
• XGBoost
• scikit-learn

To use one of these prebuilt containers, you must save your model as one ormoremodel artifacts that comply with the requirements of the prebuiltcontainer. These requirements apply whether or not your model artifacts arecreated on Vertex AI.

If you use acustom container to serve inferences, youdon't need to comply with the requirements in this page, but you can still usethem as guidelines.

Framework-specific requirements for exporting to prebuilt containers

Depending onwhich ML framework you plan to use forinference, you must export model artifactsin different formats. The following sections describe the acceptable modelformats for each ML framework.

TensorFlow

If youuse TensorFlow to train amodel, export your model as aTensorFlow SavedModeldirectory.

There are several ways to export SavedModels from TensorFlow training code. Thefollowing list describes a few ways that work for various TensorFlow APIs:

• If you use Keras for training,usetf.keras.Model.save to export aSavedModel.

• If you use an Estimator for training,usetf.estimator.Estimator.export_saved_model to export aSavedModel.

• Otherwise,usetf.saved_model.save orusetf.compat.v1.saved_model.SavedModelBuilder.

  If you aren't using Keras or an Estimator, make sure touse theservetag andserving_default signature when you export yourSavedModelin order to make sure Vertex AI can use yourmodel artifacts to serve inferences. Keras and Estimator handle thisautomatically. Learn more aboutspecifying signatures duringexport.

To serve inferences using these artifacts, create aModel with theprebuiltcontainer for inference matchingthe version of TensorFlow that you used for training.

TensorFlow for Vertex Explainable AI

If you want toget explanations from aModel that usesa TensorFlow prebuilt container to serve inferences, read theadditionalrequirements for exporting a TensorFlow model forVertex Explainable AI.

Enable server-side request batching for TensorFlow

If you want to enable request batching for aModel that uses a TensorFlowprebuilt container to serve inferences, includeconfig/batching_parameters_config in the same Cloud Storage directory assaved_model.pbfile. To configure the batching config file, see theTensorFlow'sofficial documentation.

PyTorch

You must package your model artifacts into a model archive (.mar) file.This archive must include a handler script, eitherdefault orcustom, thatdefines how the model processes inference requests. For more advanced workflows,you should consider using custom containers instead of relying on the prebuiltPyTorch serving image.

The prebuilt PyTorch images expect the archive to be namedmodel.mar, so makesure you set the model-name to 'model' when creating the archive.

For information about optimizing the memory usage, latency or throughput of aPyTorch model served with TorchServe, see thePyTorch performance guide.

XGBoost

If you use anXGBoost prebuilt containerto train a model, you can export the trained model in the following ways:

• Usexgboost.Booster'ssave_model methodto export a file namedmodel.bst.
• Use thejoblib libraryto export a file namedmodel.joblib.

Your model artifact's filename must exactly match one of these options.

The following examples show how to train and export a model:

importosfromgoogle.cloudimportstoragefromsklearnimportdatasetsimportxgboostasxgbdigits=datasets.load_digits()dtrain=xgb.DMatrix(digits.data,label=digits.target)bst=xgb.train({},dtrain,20)artifact_filename='model.bst'# Save model artifact to local filesystem (doesn't persist)local_path=artifact_filenamebst.save_model(local_path)# Upload model artifact to Cloud Storagemodel_directory=os.environ['AIP_MODEL_DIR']storage_path=os.path.join(model_directory,artifact_filename)blob=storage.blob.Blob.from_string(storage_path,client=storage.Client())blob.upload_from_filename(local_path)

importosfromgoogle.cloudimportstoragefromsklearnimportdatasetsimportjoblibimportxgboostasxgbdigits=datasets.load_digits()dtrain=xgb.DMatrix(digits.data,label=digits.target)bst=xgb.train({},dtrain,20)artifact_filename='model.joblib'# Save model artifact to local filesystem (doesn't persist)local_path=artifact_filenamejoblib.dump(bst,local_path)# Upload model artifact to Cloud Storagemodel_directory=os.environ['AIP_MODEL_DIR']storage_path=os.path.join(model_directory,artifact_filename)blob=storage.blob.Blob.from_string(storage_path,client=storage.Client())blob.upload_from_filename(local_path)

To serve inferences using this artifact, create aModel with theprebuiltcontainer for inference matchingthe version of XGBoost that you used for training.

scikit-learn

If youuse ascikit-learn prebuilt model to train amodel, you can export itby using thejoblib libraryto export a file namedmodel.joblib.

Your model artifact's filename must exactly match one of these options. You canexport standard scikit-learn estimators orscikit-learnpipelines.

The following example shows how to train and export a model:

importosfromgoogle.cloudimportstoragefromsklearnimportdatasetsfromsklearn.ensembleimportRandomForestClassifierimportjoblibdigits=datasets.load_digits()classifier=RandomForestClassifier()classifier.fit(digits.data,digits.target)artifact_filename='model.joblib'# Save model artifact to local filesystem (doesn't persist)local_path=artifact_filenamejoblib.dump(classifier,local_path)# Upload model artifact to Cloud Storagemodel_directory=os.environ['AIP_MODEL_DIR']storage_path=os.path.join(model_directory,artifact_filename)blob=storage.blob.Blob.from_string(storage_path,client=storage.Client())blob.upload_from_filename(local_path)

To serve inferences using this artifact, create aModel with theprebuiltcontainer for inferencematching the version of scikit-learn that you used for training.

What's next

• Read aboutadditional requirements for your trainingcode that you must consider when performingserverless training on Vertex AI.

• Learn how tocreate a customTrainingPipelineresource in order to run your customtraining code and create aModel from the resulting model artifacts.

• Learn how toimport aModelfrom model artifacts in Cloud Storage. This applies to model artifacts thatyoucreate using aCustomJobresource or aHyperparameterTuningJobresource, as well as modelartifacts that you train outside of Vertex AI.