VPC Flow Logs

VPC Flow Logs samples packets in your Virtual Private Cloud (VPC)network to generate flow logs. Flow logs are aggregated by IP connection(5-tuple). VPC Flow Logs samples the following packets:

• Packets that are sent from and received byvirtual machine (VM) instances, includinginstances used asGoogle Kubernetes Engine nodes
• Packets that are sent from and received by Cloud Run resourcesconfigured withDirect VPC egress
• Packets that are sent through VLAN attachments forCloud InterconnectandCloud VPN tunnels

You can view flow logs inCloud Logging, and youcan export logs to any destination that Cloud Logging export supports.These logs can be used for network monitoring, forensics, security analysis,and expense optimization.

For more information, seeSupported configurations.

Use cases

The following are use cases for VPC Flow Logs.

Network monitoring

VPC Flow Logs provides you with visibility into networkthroughput and performance. You can:

• Monitor the VPC network
• Perform network diagnosis
• Filter the flow logs by VMs, serverless endpoints, VLAN attachments, andCloud VPN tunnels to understand traffic changes
• Understand traffic growth for capacity forecasting

Understanding network usage and optimizing network traffic expenses

You can analyze network usage with VPC Flow Logs tooptimize network traffic expenses. For example, you cananalyze the network flows for the following:

• Traffic between regions and zones
• Traffic to specific countries on the internet
• Traffic to on-premises and other cloud networks
• Top talkers in the network, including VMs, serverless endpoints,VLAN attachments, and Cloud VPN tunnels

Network forensics

You can use VPC Flow Logs for network forensics. For example,if an incident occurs, you can examine the following:

• Which IP addresses have communicated with each other and when
• Which IPs are compromised by analyzing all the incoming and outgoing network flows

Supported configurations

You can enable VPC Flow Logs at the organization and projectlevels. An organization-level VPC Flow Logs configuration enablesflow logs for all subnets, VLAN attachments, and Cloud VPN tunnels inall VPC networks in the organization.

At the project level, you can enable VPC Flow Logs for specificVPC networks, subnets, VLAN attachments, and Cloud VPNtunnels.

Configuration scope	Generates flow logs for these resources	Steps to enable
Organization	All VM instances and Cloud Run resources in all subnets in the organization All VLAN attachments in the organization All Cloud VPN tunnels in the organization	Enable VPC Flow Logs for an organization
VPC network	All VM instances and Cloud Run resources in all subnets in the VPC network All VLAN attachments in the VPC network All Cloud VPN tunnels in the VPC network	Enable VPC Flow Logs for a VPC network
Subnet	All VM instances and Cloud Run resources in a specificsubnet	Enable VPC Flow Logs for a subnet: Recommended:Enable VPC Flow Logs for a subnet (Network Management API) Enable VPC Flow Logs for a subnet (Compute Engine API)
VLAN attachment	A specific VLAN attachment	Enable VPC Flow Logs for a VLAN attachment
Cloud VPN tunnel	A specific Cloud VPN tunnel	Enable VPC Flow Logs for a Cloud VPN tunnel

You can use filtering to customize these configuration scopes. For moreinformation, seeLog sampling and processing.

Logs collection

Packets are sampled within an aggregation interval. All packets collected fora given IP connection within the aggregation interval are aggregated into asingle flow log entry. This data is then sent toLogging in the Google Cloud project of theVPC network that reported the flow.

Logs are stored in Logging for 30 days by default. Ifyou want to keep logs longer than that, you can eitherset a customretention period orexport them to a supporteddestination.

Log sampling and processing

To generate flow logs, VPC Flow Logs samples packets in yourVPC network, including packets that are sent from and received byVMs and serverless endpoints and packets that pass through gateways such asVLAN attachments or Cloud VPN tunnels. After the flow logs aregenerated, VPC Flow Logs processes them by following the proceduredescribed in this section.

VPC Flow Logs samples packets using aprimary sampling rate.The primary sampling rate is dynamic and varies depending on the load of thephysical host running the reporting resource at the time of sampling. Theprobability of sampling any single IP connection increases with the volume ofpackets. You can't control the primary flow log sampling process or adjust theprimary sampling rate.

After the flow logs are generated, VPC Flow Logs processes themaccording to the following procedure:

1. Filtering. You can specify that only logs that match specified criteriaare generated. For example, you can filter so that only logsfor a particular VM or only logs with a particular metadata valueare generated and the rest are discarded. For more information, seeLog filtering.
2. Aggregation. Information for sampled packets is aggregated overa configurableaggregation interval to produce aflow log entry.
3. Secondary flow log sampling. This is a second sampling process. Flow log entriesare further sampled according to a configurablesecondary sampling rate parameter.The secondary sampling is performed on the flow logs generated by theprimary flow log sampling process. For example, if the secondary samplingrate is set to 1.0, or 100%, VPC Flow Logs samples 100% ofthe flow logs generated by the primary flow log sampling.
4. Metadata. If disabled, all metadata annotations are discarded. If youwant to keep metadata, you can retain all fields or a specific set offields. For more information, seeMetadataannotations.
5. Write to Logging. The final log entries are written toCloud Logging.

Note: You can't change how VPC Flow Logs collects samples.However, you can control the secondary flow log sampling with theSecondary sampling rate parameter, as described inEnable VPC Flow Logs.If you need to analyze all packets, you can usePacket Mirroring and collector instancesrunning third-party software.

Because VPC Flow Logs doesn't capture every packet, it compensatesfor missed packets by interpolating from the captured packets. This happens forpackets missed because of initial and user-configurable sampling settings.

Even though Google Cloud doesn't capture every packet, log record capturescan be quite large. You can balance your traffic visibility and storage costneeds by adjusting the following aspects of logs collection:

• Aggregation interval. Sampled packets for a time interval are aggregatedinto a single log entry. This time interval can be 5 seconds(default), 30 seconds, 1 minute, 5 minutes, 10 minutes, or 15 minutes.
• Secondary sampling rate.
  • For configurations created with the Compute Engine API, 50% of logentries are kept by default. You can set thisparameter from1.0 (100%, all log entries are kept)to0.0 (0%, no logs are kept).
  • For configurations created with the Network Management API, 100% oflog entries are kept by default. You can set thisparameter from1.0 to greater than0.0.
• Metadata annotations. By default, flow log entries are annotated withmetadata information, such as the names of the source anddestination within Google Cloud or the geographic region of externalsources and destinations. Metadata annotations can be turned off, or youcan specify only certain annotations, to save storage space.
• Filtering. By default, logs are generated for every sampled flow.You can setfilters togenerate logs that only match certain criteria.

Specifications

• VPC Flow Logs introduces no delayor performance penalty when enabled.
• VPC Flow Logs works with VPC networks, not legacynetworks.
• VPC Flow Logssamples TCP, UDP, ICMP, ESP, GRE,and RDMA flows:
  • Both inbound and outbound flows are sampled. For RDMA over ConvergedEthernet (RoCE), only outbound flows are sampled.
  • Flows can be within Google Cloud or between Google Cloud andother networks.
  • If a flow is captured by sampling,VPC Flow Logs generates a log for the flow. Each flow recordincludes the information described in theRecord format section.
• VPC Flow Logs interacts with firewall rules in the followingways:
  • Egress packets are sampledbeforeegress firewall rules. Even if anegress firewall rule denies outbound packets, those packets can besampled by VPC Flow Logs.
  • Ingress packets are sampledafteringress firewall rules. If aningress firewall rule denies inbound packets, those packets aren'tsampled by VPC Flow Logs.
• You can usefilters inVPC Flow Logs to generate only certain logs.
• VPC Flow Logs supports VMs that have multiple network interfaces.In each VPC, you need to enable VPC Flow Logsfor each subnet that contains a network interface.
• To log flows between Pods on the same Google Kubernetes Engine (GKE) node, youmust enableintranode visibilityfor the cluster.
• VPC Flow Logs isn't supported for subnets with purposeINTERNAL_HTTPS_LOAD_BALANCER because these subnets are used as proxy-onlysubnets and have no VM instances or serverless endpoints.
• VPC Flow Logs writes logs to the project of the reportingVPC network. For resources in Shared VPC networks,logs are reported in the host project.

Pricing and billing

Standard pricing for Logging,BigQuery, or Pub/Sub apply.VPC Flow Logs pricing is described inNetwork Telemetry pricing.

VPC Flow Logs charges are billed to the Google Cloud project of theresource that reports flow logs. If VPC Flow Logs is enabled for anorganization, each project is billed separately.

What's next

• To learn more about the VPC Flow Logs record format and whichmetadata annotations are available, seeAbout VPC Flow Logs records.
• To see examples of VPC Flow Logs that are collected for varioususe cases, seeAbout traffic flows.
• To start reporting flows for a subnet, seeConfigure VPC Flow Logs.