Run notebook files with the executor

Vertex AI Workbench managed notebooks isdeprecated. On April 14, 2025, support for managed notebooks ended and the ability to create managed notebooks instances was removed. Existing instances will continue to function until March 30, 2026, but patches, updates, and upgrades won't be available. To continue using Vertex AI Workbench, we recommend that youmigrate your managed notebooks instances to Vertex AI Workbench instances.

This page describes how to use the executor ina Vertex AI Workbench managed notebooks instance to run notebook filesas a one-time execution and on a schedule.

Overview

The executor lets you submit a notebook (ipynb) file to run onVertex AI custom training.You can set parameter values for each execution of a notebook file.You can also run a notebook file on a recurring schedule.After the execution completes, you can view the execution results andshare them with others.

When a notebook file is submitted to Vertex AI custom training,Vertex AI creates a new custom training jobthat executes your notebook file following thelifecycle of a trainingjob.

Requirements for notebook code run by the executor

When you write notebook code to run in the executor, keep in mindthat the code will run in atenant projectseparate from your managed notebooks instance's project.This section describes how this affects your code when it is runin the executor.

Ensure package installations are available to the executor

If your notebook depends on package installations that are not alreadyincluded in the managed notebooks kernel that you are using,make sure your packages are available to your notebook codein the executor in one of the following ways:

• Use acustom containerthat already has the package installed, and then execute your notebookon that custom container. See therequirements for usinga custom container with the executor.

• Install the package within your notebook file's code.The package is installed every time the notebook file is executed,but this ensures that the package is available onthe container that you select when you execute your notebook.

Use explicit project selection

When you access resources through code run by the executor,the executor might not connect to the correctGoogle Cloud project. If you encounter permission errors,connecting to the wrong project might be the problem.

This problem occurs because the executor does not run your codedirectly in your managed notebooks instance'sGoogle Cloud project. Instead, the executorruns your code in Vertex AI custom trainingwithin a tenant project managed by Google.Therefore, don't try to infer a project ID from the environment inyour notebook code; specify project IDs explicitly.

If you don't want to hardcode a project ID in your code, you canreference theCLOUD_ML_PROJECT_ID environment variable.Vertex AIsets this environment variable in every custom training container to contain theproject number of the project where you initiatedcustom training. Many Google Cloud tools can accept a projectnumber wherever they take a project ID.

For example, if you want to use thePython Client for GoogleBigQuery to access aBigQuery table in the same project, then do not infer theproject in your notebook code:

Implicit project selection

fromgoogle.cloudimportbigqueryclient=bigquery.Client()

Instead use code that explicitly selects a project:

Explicit project selection

importosfromgoogle.cloudimportbigqueryproject_number=os.environ["CLOUD_ML_PROJECT_ID"]client=bigquery.Client(project=project_number)

Authenticate access using service accounts

By default, your managed notebooks instancecan have access to resources that exist in the same project.Therefore, when you run your notebook file's code manually,these resources do not need additional authentication. However,because the executor runs in a separate tenant project, it does nothave the same default access.

Also, the executor cannot use end-user credentials to authenticate accessto resources, for example, thegcloud auth login command.

To resolve these issues, in your notebook file's code,authenticate access to resources through a service account.

Then when you create an execution or schedule, specify theservice account.

For example, while youcreate an execution,complete these steps:

1. In theSubmit notebooks to Executor dialog,expandAdvanced options.

2. In theIdentity and API access section,clear the check mark next toUse Vertex AI Training's default service accountand enter the specific service account to use.

See the full list ofsteps for creating an execution.

Requirements when using a custom container

You can use the executor to run notebook code on a custom container.Your custom container must include thenbexecutor extension, whichenables the executor to run notebook code asa Vertex AI custom training job.To ensure that your custom container has thenbexecutor extension,you can modify one of theDeep Learning Containers container images tocreate aderivative container image.Deep Learning Containers imagesinclude thenbexecutor extension.

Before you begin

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the Notebooks and Vertex AI APIs.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

Enable the APIs

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

Enable the Notebooks and Vertex AI APIs.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

Enable the APIs

Required roles

To ensure that your instance's service account has the necessary permissions to interact with the Vertex AI Workbench executor, ask your administrator to grant your instance's service account the following IAM roles on the project:

• Notebooks Viewer (roles/notebooks.viewer)
• Vertex AI User (roles/aiplatform.user)
• Storage Admin (roles/storage.admin)

For more information about granting roles, seeManage access to projects, folders, and organizations.

Your administrator might also be able to give your instance's service account the required permissions throughcustom roles or otherpredefined roles.

Open JupyterLab

To open JupyterLab and prepare a notebook file to run,complete the following steps.

1. Open JupyterLab.

2. Upload a notebook (ipynb) file, open an existing file,oropen a new notebookfileand add the code that you want to run to the notebook.

3. Make sure your notebook file's code meets therequirementsfor using the executor.

Create an execution

To create an execution that runs your notebook file,complete the following steps. These steps cover both schedulingexecutions and creating a one-time execution.

1. In your managed notebooks instance'sJupyterLab user interface, open the notebook file that you want to run.

2. Click the Execute button.

3. In theSubmit notebooks to Executor dialog,in theExecution name field, enter a name for your execution.

4. Select aMachine type andAccelerator type.

5. Select anEnvironment.

6. In theType field,selectOne-time execution orselectSchedule-based recurring executions and completethe dialog for scheduling executions.

7. InAdvanced options,select theRegion where you want to run your notebook.

8. In theCloud Storage bucket field,select an available Cloud Storage bucket orenter a name for a new bucket and clickCreate and select.The executor stores your notebook outputin this Cloud Storage bucket.

9. Optional: in theNotebook parameterization section,in theInput parameters text box,add notebook parameters separated by commas, for exampleoptimizer="SGD",learning_rate=0.01.

   Learn more abouthow to usenotebook parameters.

10. Optional: in theIdentity and API access section,selectUse Vertex AI Training's default service accountor clear the check mark and enter a specific service account to use.

    Note: If your notebook file's code authenticates access to servicesby using end-user credentials or default access withinyour managed notebooks instance's project,see theRequirements for notebook code runby the executor and consider specifying a service account.

11. Optional: in theNetworking section,specify a Virtual Private Cloud network. Using a VPC networkfor your execution requires aprivate servicesaccess connection.

12. ClickSubmit.

    One-time executions begin immediately.Scheduled executions run automaticallyon the schedule that you set.

    Note: If your managed notebooks instance is shut down, theexecutor still runs your notebook file on schedule.

In the Google Cloud console, on theVertex AI Workbench page,you can view your completed executions ontheExecutions taband view your schedules on theSchedules tab.

View, share, and import an executed notebook file

By using your managed notebooks instance'sJupyterLab user interface, you can view an executed notebook's output,share the results with others,and import the executed notebook file into JupyterLab.

View a notebook execution's results

You can view a notebook execution's results in the Google Cloud consoleor in the JupyterLab user interface.

Share a notebook execution's results

You can share execution results by providing access to theCloud Storage bucket that contains your notebook execution.Providing this access also grants users access to any other resourcesin the same Cloud Storage bucket.To share execution results, complete the following steps.

Import an executed notebook into JupyterLab

To import an executed notebook into JupyterLab, complete the following steps.

1. In your managed notebooks instance'sJupyterLab user interface,in the navigation menu, click the Notebook Executor button.

2. Click theExecutions tab.

3. Select theRegion that contains your executed notebook.

4. Next to the execution that you want to import,click themore_vert options menu,and selectImport executed notebook.

5. Select the kernel that you want to open the notebook.

   The executor opens the executed notebook filein JupyterLab and stores this notebook file inthe JupyterLab File Browser in a folder namedimported_notebook_jobs.

View or delete a schedule

You can view and delete schedules by using either the Google Cloud console oryour managed notebooks instance's JupyterLab user interface.

View a schedule

View a schedule to see the frequency settings of the scheduleor to view results of your notebook executions.

Delete a schedule

Deleting a schedule does not delete the executions that weregenerated from that schedule.

Jobs on Vertex AI custom training

Because notebook executions are run on Vertex AI custom training,they are exposed as custom training jobs in Vertex AI.You can view these custom training jobsin the Google Cloud console, on theCustom jobs tab oftheVertex AI Training page.Learn more abouthow to work withVertex AI custom training jobs.

What's next

• Learn how torun notebook executionswith parameters.

• Learn more aboutVertex AI custom training.