Cloud RDMA enables low-latency reliable messaging capabilities by usingthe IRDMA RDMA driver. This driver supports Remote Direct MemoryAccess (RDMA) between Compute Engine instances. RDMA transfers data betweenremote machines and local memory through the network interface without usinghost CPU or intermediate host buffers.

Cloud RDMA capable instances require at least two virtual networkinterfaces (vNICs):

• A vNIC for Cloud RDMA communication that uses the IDPF network andIRDMA RDMA driver. This vNIC can't connect to the internet. An instance canhave only one vNIC that uses Cloud RDMA.
• A vNIC for normal network traffic. This vNIC is fully connected to theGoogle Cloud network and can connect to the internet. This vNIC uses the gVNICnetwork driver. You can add up to eight additional gVNIC network interfaces,for a total of 10 vNICs per instance.

The IRDMA driver is supported withH4D instances.

When configuring a vNIC that uses the IRDMA driver, you must specify aVPC network that was created by using theFalcon RDMA network profile.

Before you begin

• If you haven't already, set upauthentication. Authentication verifies your identity for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

  Select the tab for how you plan to use the samples on this page:

  Console

  When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

  gcloud

  1. Install the Google Cloud CLI. After installation,initialize the Google Cloud CLI by running the following command:

     gcloudinit

     If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

     Note: If you installed the gcloud CLI previously, make sure you have the latest version by runninggcloud components update.
  2. Set a default region and zone.

  REST

  To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

  Install the Google Cloud CLI. After installation,initialize the Google Cloud CLI by running the following command:

  gcloudinit

  If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

  Note: If you installed the gcloud CLI previously, make sure you have the latest version by runninggcloud components update.

  For more information, seeAuthenticate for using REST in the Google Cloud authentication documentation.

Operating system support

Google recommends using theHPC VM image,which comes with Cloud RDMA drivers pre-installed.

Of the public OS images provided for Google Cloud,the IRDMA RDMA driver is supported with the following:

• Container-Optimized OS 117 LTS or later
• Rocky Linux 8 optimized for Google Cloud and later optimized for Google Cloudversions

Overview of using Cloud RDMA with Compute Engine instances

To create a compute instance that uses Cloud RDMA, you must have atleast one regular VPC network and oneFalcon VPC network. TheFalcon VPC network uses the Falcon RDMA network profile to enableCloud RDMA traffic between H4D instances. This network is separate fromthe regular VPC network that carries non-RDMA traffic to otherGoogle Cloud services or the internet.

The tasks to complete to create a compute instance that uses Cloud RDMAare as follows:

1. Choose apublic OS image thatsupports Cloud RDMA, or create a custom OS image that is tagged touse IRDMA.

2. Identify or create at least two VPC networks:

   • A regular VPC network for the traffic that goes through thegVNIC network interface
   • A Falcon VPC network for the Cloud RDMA traffic

3. Create a compute instance using the public or custom OS image. During theinstance creation, configure at least two network interfaces: one that usesthe gVNIC network driver and one that uses the IRDMA RDMA driver.

4. Verify that Cloud RDMA is enabled.

Create a custom OS image that supports Cloud RDMA

You can create the OS image using either theGoogle Cloud CLI orREST. For detailedinformation and best practices for creating custom OS images, seeCreate custom OS images.

Create a VM with IRDMA driver support

You can create a VM using one of thesupported public OS images orusing a custom OS image you created using the steps inCreate a custom OS image that supports IRDMA.

To use Cloud RDMA with your instance, you must configure multiplenetwork interfaces (NICs) for the instance. One NIC must use the GVNIC driver(specifynic-type=GVNIC) and another NIC must use the IRDMA driver (specifynic-type=IRDMA).

Create a VM using a public OS image

To create VMs using the HPC VM image, or a public OS image that supportsCloud RDMA, follow the instructions on the following pages:

• To create an on-demand instance, seeCreate an instance that uses Cloud RDMA.
• To create Cloud RDMA-enabled instances in bulk, seeBulk create HPC-optimized instances that use Cloud RDMA.
• To use Slurm to create a cluster of Cloud RDMA-enabled instances, seeCreate an Cloud RDMA-enabled HPC Slurm cluster with H4D instances.
• To create a MIG, seeCreate a HPC MIG with H4D machine series.

Create a VM using a custom OS image

If you created acustom OS image that supports Cloud RDMA,you can use that custom OS image to create a VM by usingtheGoogle Cloud console,Google Cloud CLI, orREST.

For instructions on how to create an instance using a custom image, seeCreate an instance from a custom image.

Verify that Cloud RDMA is enabled

After the VM is deployed, run the following performance bandwidth test to ensurethat you are running on a healthy Cloud RDMA-enabled VM instance.

ib_send_bw-aF &# start the serverib_send_bw-aF$(hostname)# start the client (can be done on the same machine)

The reported single-connection bandwidthBW average [MB/s] should reach atleast 11,000 MBps for rows higher than 4096 B. Smaller values areexpected for lower message sizes.

What's next

• Connect to your instance.
• Change the MTU setting of a VPC network.
• Overview of HPC cluster creation
• Set up and scale MPI applications on H4D VMs with Cloud RDMA.