Packet Mirroring

This page is an overview of Packet Mirroring in theVirtual Private Cloud (VPC) network.If you want to analyze your workloads' networktraffic at scale and monitor network traffic using third-partyvirtual appliances, useNetwork Security Integration Packet Mirroring. For more information,seeOut-of-band integration overview.

Note: We recommend that you use Network Security Integration Packet Mirroring for new deployments. For more information, seeComparison of VPC Packet Mirroring and Network Security Integration Packet Mirroring.

Packet Mirroring clones the traffic of specified instances in yourVPC network and forwards it for examination.Packet Mirroring captures all traffic and packet data, includingpayloads and headers. The capture can be configured for both egress and ingresstraffic, only ingress traffic, or only egress traffic.

The mirroring happens on the virtual machine (VM) instances, not on the network.Consequently, Packet Mirroring consumes additional bandwidth on theVMs.

Packet Mirroring is useful when you need to monitor and analyze your securitystatus. It exports all traffic, not only the traffic between sampling periods.For example, you can use security software that analyzes mirrored traffic todetect all threats or anomalies. Additionally, you can inspect the full trafficflow to detect application performance issues. For more information, see theexampleuse cases.

How it works

Packet Mirroring copies traffic frommirrored sources and sends it to acollector destination. To configure Packet Mirroring, you create apacketmirroring policy that specifies the source and destination.

• Mirrored sources are Compute Engine VM instances that you can select byspecifying subnets, network tags, or instance names. If you specify a subnet,all existing and future instances in that subnet are mirrored. You can specifyone or more source types—if an instance matches at least one of them, it'smirrored.

  Packet Mirroring collects traffic from an instance's network interfacein the network where the packet mirroring policy applies. In cases where an instancehas multiple network interfaces, the other interfaces aren't mirrored unlessanother policy has been configured to do so.

• Acollector destination is an instance group that is behind an internal loadbalancer. Instances in the instance group are referred to ascollectorinstances.

  When you specify the collector destination, you enter the name of a forwardingrule that is associated with the internal passthrough Network Load Balancer. Google Cloudthen forwards the mirrored traffic to the collector instances. An internalload balancer for Packet Mirroring is similar to other internal loadbalancers except that the forwarding rule must be configured forPacket Mirroring. Any non-mirrored traffic that is sent to the loadbalancer is dropped.

Filtering

By default, Packet Mirroring collects all IPv4 traffic of mirroredinstances. Instead of collecting all IPv4 traffic, you can use filters to expandthe traffic that's collected to include all or some IPv6 traffic. You can alsouse filters to narrow the traffic that's mirrored, which can help you limitthe bandwidth that's used by mirrored instances.

You can configure filters to collect traffic based on protocol, CIDR ranges(IPv4, IPv6, or both), direction of traffic (ingress-only, egress-only, orboth), or a combination.

Policy order

Multiple packet mirroring policies can apply to an instance. The priority of apacket mirroring policy is always1000 and cannot bechanged. Identical policies are not supported. Google Cloud can sendtraffic to any of the load balancers that have been configured with identicalpacket mirroring policies. To predictably and consistently send mirrored trafficto a single load balancer, create policies that have filters withnon-overlapping address ranges. If ranges overlap, set unique filter protocols.

Depending on each policy's filter, Google Cloud chooses a policy for eachflow. If you have distinct policies, Google Cloud uses thecorresponding policy that matches the mirrored traffic. For example, you mighthave one policy that has the filter198.51.100.3/24:TCP and another policythat has the filter2001:db8::/64:TCP:UDP. Because the policies are distinct,there's no ambiguity about which policy Google Cloud uses.

However, if you have overlapping policies, Google Cloud evaluatestheir filters to choose which policy to use. For example, you might have twopolicies, one that has a filter for10.0.0.0/24:TCP and another for10.0.0.0/16:TCP. These policies overlap because their CIDR ranges overlap.

When choosing a policy, Google Cloud prioritizes policies bycomparing their filter's CIDR range size.

Google Cloud chooses a policy based on a filter:

• If policies have different but overlapping CIDR ranges and the same exactprotocols, Google Cloud chooses the policy that uses the most specificCIDR range. Suppose the destination for a TCP packet leavinga mirrored instance is10.240.1.4, and there are two policies with thefollowing filters:10.240.1.0/24:ALL and10.240.0.0/16:TCP. Because themost specific match for10.240.1.4 is10.240.1.0/24:ALL,Google Cloud uses the policy that has the filter10.240.1.0/24:ALL.

• If policies specify the same exact CIDR range with overlapping protocols,Google Cloud chooses a policy with the most specific protocol.For example, the following filters have the same range but overlappingprotocols:10.240.1.0/24:TCP and10.240.1.0/24:ALL. For matching TCPtraffic, Google Cloud uses the10.240.1.0/24:TCP policy.The10.240.1.0/24:ALL policy applies to matching traffic for all otherprotocols.

• If policies have the same exact CIDR range but distinct protocols, thesepolicies don't overlap. Google Cloud uses the policy that correspondsto the mirrored traffic's protocol. For example, you might have a policy for2001:db8::/64:TCP and another for2001:db8::/64:UDP. Depending on themirrored traffic's protocol, Google Cloud uses either the TCP or UDPpolicy.

• If overlapping policies have the same exact filter, they are identical. Inthis case, Google Cloud might choose the same policy or a differentpolicy each time that matching traffic is re-evaluated against thesepolicies. We recommend that you avoid creating identical packet mirroringpolicies.

VPC Flow Logs

VPC Flow Logs doesn't log mirrored packets. If a collector instance is on asubnet that has VPC Flow Logs enabled, traffic that is sent directly to thecollector instance is logged, including traffic from mirrored instances. Thatis, if the original destination IPv4 or IPv6 address matches the IPv4 or IPv6address of the collector instance, the flow is logged.

For more information about VPC Flow Logs, seeUsing VPC Flow Logs.

Key properties

The following list describes constraints or behaviors withPacket Mirroring that are important to understand before you use it:

• Each packet mirroring policy definesmirrored sources and acollectordestination. You must adhere to the following rules:

  • All mirrored sources must be in the same project, VPCnetwork, and Google Cloud region.
  • A collector destination must be in the same region as the mirrored sources.A collector destination can be located in either the same VPCnetwork as the mirrored sources or a VPC network connectedto the mirrored sources' network using VPC Network Peering.
  • Each mirroring policy can only reference a single collector destination.However, a single collector destination can be referenced by multiplemirroring policies.

• Alllayer 4 protocols are supported by Packet Mirroring.

• You cannot mirror and collect traffic on the same network interface of a VMinstance because doing this would cause a mirroring loop.

• To mirror traffic passing between Pods on the same Google Kubernetes Engine (GKE)node, you must enableIntranodevisibility forthe cluster.

• To mirror IPv6 traffic, use filters to specify the IPv6 CIDR rangesof the IPv6 traffic that you want to mirror. You can mirror all IPv6 trafficby using a CIDR range filter of::/0. You can mirror all IPv4 and IPv6traffic by using the following comma-separated CIDR range filter:0.0.0.0/0,::/0.

• Mirroring traffic consumes bandwidth on the mirrored instance. For example,if a mirrored instance experiences 1 Gbps of ingress traffic and 1 Gbps ofegress traffic, the total traffic on the instances is 1 Gbps of ingress and 3Gbps of egress (1 Gbps of normal egress traffic and 2 Gbps of mirrored egresstraffic). To limit what traffic is collected, you can use filters.

• The cost of Packet Mirroring varies depending on the amount of egresstraffic traveling from a mirrored instance to an instance group and whether thetraffic travels between zones.

• Packet Mirroring applies to both ingress and egress direction. If twoVM instances that are being mirrored send traffic to each other,Google Cloud collects two versions of the same packet. You can alterthis behaviour by specifying that only ingress or only egress packets aremirrored.

• There is a maximum number of packet mirroring policies that you can create fora project. For more information, see the per-project quotas on thequotas page.

• For each packet mirroring policy, the maximum number of mirrored sources thatyou can specify depends on the source type:

  • 5 subnets
  • 5 tags
  • 50 instances

• The maximum number of packet mirroring filters is 30, which is the number ofIPv4 and IPv6 address ranges multiplied by the number of protocols. Forexample, you can specify 30 ranges and 1 protocol, which would be 30 filters.However, you cannot specify 30 ranges and 2 protocols, which would be 60filters and greater than the maximum.

• Mirrored traffic is encrypted only if the VM encrypts that traffic at theapplication layer. While VM-to-VM connections within VPC networksand peered VPC networks areencrypted,the encryption and decryption happens in the hypervisors. From the perspectiveof the VM, this traffic is not encrypted.

Use cases

The following sections describe real-world scenarios that demonstrate why youmight use Packet Mirroring.

Enterprise security

Security and network engineering teams must ensure that they are catching allanomalies and threats that might indicate security breaches and intrusions. Theymirror all traffic so that they can complete a comprehensive inspection ofsuspicious flows. Because attacks can span multiple packets, security teams mustbe able to get all packets for each flow.

For example, the following security tools require you to capture multiplepackets:

• Intrusion detection system(IDS) tools requiremultiple packets of a single flow to match asignature so that the tools can detect persistent threats.

• Deep Packet Inspection engines inspect packet payloads to detect protocolanomalies.

• Network forensics for PCI compliance and other regulatory use cases requirethat most packets be examined. Packet Mirroringprovides a solution for capturing different attack vectors, such asinfrequent communication or attempted but unsuccessful communication.

Application performance monitoring

Network engineers can use mirrored traffic to troubleshoot performance issuesreported by application and database teams. To check for networking issues,network engineers can view what's going over the wire rather than relying onapplication logs.

For example, network engineers can use data from Packet Mirroring to completethe following tasks:

• Analyze protocols and behaviors so that they can find and fix issues, such aspacket loss or TCP resets.

• Analyze (in real time) traffic patterns from remote desktop, VoIP, and otherinteractive applications. Network engineers can search for issues that affectthe application's user experience, such as multiple packet resends or morethan expected reconnections.

Example collector destination topologies

You can use Packet Mirroring in various setups. The following examplesshow the location of collector destinations and their policies for differentpacket mirroring configurations, such as VPC Network Peering andShared VPC.

Collector destination in the same network

The following example shows a packet mirroring configuration where the mirroredsource and collector destination are in the same VPC network.

In the preceding diagram, the packet mirroring policy is configured to mirrormirrored-subnet and send mirrored traffic to the internal passthrough Network Load Balancer.Google Cloud mirrors the traffic on existing and future instances in thesubnet. All traffic to and from the internet, on-premises hosts, or Googleservices is mirrored.

Collector destination in a peer network

You can build a centralized collector model, where instances in differentVPC networks send mirrored traffic to a collector destination ina central VPC network. That way, you can use a singledestination collector.

In the following example, thecollector-load-balancer internal passthrough Network Load Balancer isin theus-central1 region in thenetwork-a VPC network inproject-a. This destination collector can be used by two packet mirroringpolicies:

• policy-1 collects packets from mirrored sources in theus-central1 regionin thenetwork-a VPC network inproject-a and sends themto thecollector-load-balancer destination.

• policy-2 collects packets from mirrored sources in theus-central1 regionin thenetwork-b VPC network inproject-b and sends themto the samecollector-load-balancer destination.

Two mirroring policies are required because mirrored sources exist in differentVPC networks.

In the preceding diagram, the collector destination collects mirrored trafficfrom subnets in two different networks. All resources (the source anddestination) must be in the same region. The setup innetwork-a is similar tothe example where themirrored source and collector destination are in the sameVPC network.policy-1 is configured to collecttraffic fromsubnet-a and send it tocollector-ilb.

policy-2 is configured inproject-a but specifiessubnet-b as a mirroredsource. Becausenetwork-a andnetwork-b are peered, the destinationcollector can collect traffic fromsubnet-b.

The networks are in different projects and might have different owners. It'spossible for either owner to create the packet mirroring policy if they have theright permissions:

• If theowners ofproject-a create the packet mirroring policy, they musthave thecompute.packetMirroringAdmin role on the network, subnet, orinstances to mirror inproject-b.

• If theowners ofproject-b create the packet mirroring policy, they musthavecompute.packetMirroringUser role inproject-a.

For more information about enabling private connectivity across twoVPC networks, seeVPC Network Peering.

Shared VPC

In the following Shared VPC scenarios, the mirrored instances for thecollector destination are all in the same Shared VPC network. Even though theresources are all in the same network, they can be in different projects, suchas the host project or several different service projects. The followingexamples show where packet mirroring policies must be created and who can createthem.

If both the mirrored sources and collector destination are in the same project,either in a host project or service project, the setup is similar to havingeverything in thesame VPC network. The projectowner can create all the resources and set the required permissions in thatproject.

For more information, seeShared VPC overiew.

Collector destination in service project

In the following example, the collector destination is in a service project thatuses a subnet in the host project. In this case, the policy is also in theservice project. The policy could also be in the host project.

In the preceding diagram, the service project contains the collector instancesthat use the collector subnet in the Shared VPC network. The packetmirroring policy was created in the service project and is configured to mirrorinstances that have a network interface insubnet-mirrored.

Service or host project users can create the packet mirroring policy. To do so,users must have thecompute.packetMirroringUser role in the service projectwhere the collector destination is located. Users must also have thecompute.packetMirroringAdmin role on the mirrored sources.

Collector destination in host project

In the following example, the collector destination is in the host project andmirrored instances are in the service projects.

This example might apply to scenarios where developers deploy applications inservice projects and use the Shared VPC network. They don't have tomanage the networking infrastructure or Packet Mirroring. Instead, acentralized networking or security team, who have control over the host projectand Shared VPC network, are responsible for provisioning packetmirroring policies.

In the preceding diagram, the packet mirroring policy is created in the hostproject, where the collector destination is located. The policy is configured tomirror instances in the mirrored subnet. VM instances in service projects canuse the mirrored subnet, and their traffic is mirrored.

Service or host project users can create the packet mirroring policy. To do so,users in the service project must have thecompute.packetMirroringUser role inthe host project. Users in the host project require thecompute.packetMirroringAdmin role for mirrored sources in the serviceprojects.

Multi-interface VM instances

You can include VM instances that have multiple network interfaces in a packetmirroring policy. Because a policy can mirror resources from a single network,you cannot create one policy to mirror traffic for all network interfaces of aninstance. If you need to mirror more than one network interface of a multiplenetwork interface instance, you must create one packet mirroring policy for eachinterface because each interface connects to a unique VPC network.

Pricing

You are charged for the amount of data processed by Packet Mirroring.For details, seePacket Mirroring pricing.

You are also charged for all the prerequisite components and egress trafficthat are related to Packet Mirroring. For example, the instancesthat collect traffic are charged at the regular rate. Also, if packetmirroring traffic travels between zones, you are charged for the egresstraffic. For pricing details, see the relatedpricing page.

What's next

• Use Packet Mirroring.
• Monitor Packet Mirroring.
• Internal passthrough Network Load Balancer overview.
• Packet Mirroring partner providers.
• Out-of-band integration overview.