Cloud Composer 3 | Cloud Composer 2 | Cloud Composer 1

This page explains the difference between Private IP and Public IP environmentnetworking types in Cloud Composer 3 and provides instructions for switchingthe networking type of your environment.

If you want to disable or enable internet access only when installing PyPI packages, seeConfigure internet access when installing PyPI packages.

If you want to enable access to your VPC network from your environment,seeConnect an environment to a VPC network.

About environment networking types

Cloud Composer 3 uses two environment networking types:

• Public IP networking:

  • Airflow components of the environment can access theinternet. This is the default networking type.

  • Airflow components in Public IP environments establish outboundconnections from automatically allocated public IP addresses and ports.If you want your Public IP environments to use predetermined IPaddresses and ports, you can do so throughconnecting a VPC network to your environmentandswitching it to Private IP.In this case, Cloud Composer routes all traffic excepttraffic to Google services through this network.

• Private IP networking:

  • Private IP networking is a more secure option compared to Public IPnetworking.

  • Airflow components of the environment don't have access tothe internet. For environmentsattached to a VPC network, the network configurationcontrols internet access.

  • Private IP environments configure Private Google Access through theprivate.googleapis.com range, which enables access toGoogle APIs, services, and domains supported by this range.

    For more information and the list of services and domains availablethroughprivate.googleapis.com, seeNetwork configurationin the Virtual Private Cloud documentation.

  • Private IP environments with VPC Service Controls configurePrivate Google Access through therestricted.googleapis.comrange, which enables access to Google APIs, services, and domainssupported by this range.

    For more information and the list of services and domains availablethroughrestricted.googleapis.com, seeNetwork configurationin the Virtual Private Cloud documentation.

  • If a Private IP environment isattached to a custom VPC network, then all internaltraffic is routed to the VPC network, except the traffic to GoogleAPIs, services, and domains that are available to Private IPenvironments through Private Google Access.

Cloud Composer 2 networking compared to Cloud Composer 3

In Cloud Composer 3, Private IP environments require no configuration.

The following Cloud Composer 2 networking features are no longer relevant inCloud Composer 3:

• Configuring Private IP networking. You don'tneed to specify IP ranges, networks, or configure connectivity and firewallrules.

• Configuring Private Service Connect.You don't need to set ranges for Private Service Connectin Cloud Composer 3.

  Note:Custom VPC network attachments inCloud Composer 3 are based on Private Service Connect.This implementation isnot related to howPrivate Service Connect is used in Cloud Composer 2 (forconnectivity between the environment's cluster and the resources located inthe tenant project). In Cloud Composer 3, network attachments are useddifferently, you don't need to reflect your Cloud Composer 2 networkingsetup in them.

• Using privately used public IP ranges. This feature provided anoption to extend the available IP ranges, which are not required inCloud Composer 3.

• Using the IP Masquerade agent. You don't need to configurecluster connectivity in Cloud Composer 3.

• Configuring authorized networks. It is notpossible to access the environment's cluster in Cloud Composer 3.

The following DNS configuration isn't supported in Cloud Composer 3:

• Cloud Composer 3 doesn't support a user-defined.internal DNS zone. If you create a DNS zone for.internal, it won't be possible to reach that zone.

Change environment networking type

gcloudbetacomposerenvironmentsupdateENVIRONMENT_NAME\--locationLOCATION\--enable-private-environment

gcloudbetacomposerenvironmentsupdateENVIRONMENT_NAME\--locationLOCATION\--disable-private-environment

gcloudbetacomposerenvironmentsupdateexample-environment\--locationus-central1\--enable-private-environment

gcloudbetacomposerenvironmentsupdateexample-environment\--locationus-central1\--disable-private-environment

// PATCH https://composer.googleapis.com/v1beta1/projects/example-project/// locations/us-central1/environments/example-environment?updateMask=// config.private_environment_config.enable_private_environment"config":{"privateEnvironmentConfig":{"enablePrivateEnvironment":true}}

resource"google_composer_environment""example"{provider=google-betaname="ENVIRONMENT_NAME"region="LOCATION"config{enable_private_environment=PRIVATE_IP_STATUS}}

resource"google_composer_environment""example"{provider=google-betaname="example-environment"region="us-central1"config{enable_private_environment=true...otherconfigurationparameters}}

Configure proxy server variables

You can sethttp_proxy andhttps_proxyenvironment variablesin your environment. These standard Linux variables are used by web clientsthat run in containers of your environment's cluster to route traffic throughthe specified proxies.

TheNO_PROXY variable by default is set to a list of Google domains,the IP address of the environment's cluster Compute Engine metadataserver, andlocalhost so that they are excluded from proxying:

.google.com,.googleapis.com,metadata.google.internal,169.254.169.254,localhost

This configuration makes it possible to create an environment with sethttp_proxy andhttps_proxy environment variables in cases when the proxyisn't configured to handle traffic to Google services.

What's next

• Configure VPC Service Controls