Repository files navigation

This is not an officially supported Google product.

For network emulation, there are many approaches using VM's for emulation of ahardware router. Arista, Cisco, Juniper, Drivenets, and Nokia have multiple implementationsof their network operating system and various generations of hardware emulation.These systems are very good for most validation of vendor control planeimplementations and data plane for limited certifications. The idea of thisproject is to provide a standard "interface" so that vendors can produce astandard container implementation which can be used to build complex topologies.

• Have standard lifecycle management infrastructure for allowing multiple vendordevice emulations to be present in a single "topology"
• Allow for control plane access via standard k8s networking
• Provide a common networking interface for the forwarding plane between networkpods.
  • Data plane wires between pods
  • Control plane wires between topology manager
• Define service implementation for allowing interaction with the topologymanager service.
  • Topology manager is the public API for allowing external users to manipulatethe link state in the topology.
  • The topology manager will run as a service in k8s environment.
  • It will provide a gRPC interface for tests to interact with
  • It will listen to CRDs published via the network device pods for discovery
• Data plane connections for connectivity between pods must be a publictransport mechanism
  • This can't be implemented as just exposing "x eth devices on the pod"because Linux doesn't understand the associated control messages which areneeded to make this work like a wire.
  • Transceiver state, optical characteristics, wire state, packet filtering /shaping / drops
  • LACP or other port aggregation protocols or APS cannot be simulatedcorrectly
  • The topology manager will start a topology agent on each host for the pod todirectly interact with.
  • The topology agent will provide the connectivity between nodes
• Define how pods boot an initial configuration
  • Ideally, this method would allow for dynamic
• Define how pods express services for use in-cluster as well as externalservices

The main use case of this infrastructure is for the development of tests tovalidate control plane / configuration of network devices without needing realhardware.

The main use case we are interested in is the ability to bring up arbitrarytopologies to represent a production topology. This would require multiplevendors as well as traffic generation and end hosts.

In support of the testing we need to be able to provide every tester, engineerand continuous automated run a set of environments to validate test scenariosused in production. These can also be used to pre-validate hardware testing aswell. This can reduce cycle time as there will be no contention for the virtualtestbed vs. the hardware testbed. This also allows for "unit testing" theintegration test.

For the development of new services or for offering a better environment todevelopers for existing services, virtual testbeds would allow for betterscaling of resources and easier to use testbeds that would be customized for ateam's needs. Specifically, workflow automation struggles to have physicalrepresentations of metros that need to be validated for workflows. A virtualtestbed would allow for the majority of workflows to be validated against anynumber of production topologies.

See the collection ofdocs for in depth guides on how useKubernetes Network Emulation (KNE).

The KNE CLI optionally collects anonymous usage metrics.This is turned OFFby default. We use the metrics to gauge the health and performance of variousKNE operations (i.e. cluster deployment, topology creation) on anopt-inbasis. There is a global flag--report_usage that when provided sharesanonymous details about certain KNE CLI commands. Collected data can be seen intheevent proto definition.Usage metrics are NOT sharedby default. Additionally the PubSub project and topic the events are publishedto are configurable. If you want to track your own private metrics about yourKNE usage then that is supported by providing a Cloud PubSub project/topic ofyour choosing. Full details about how/when usage events are published can befound in the codebasehere. We appreciate usage metricreporting as it helps us develop a better KNE experience for all of our users.Whether that be detecting an abnormally high number of cluster deploymentfailures due to an upgrade to an underlying dependency introduced by a newcommit, or detecting a bug from a scenario where the failure rate for topologiesovern links is far greater thann-1 links. Usage metric reporting ishelpful tool for the KNE developers.

This project is mainly based on the k8s-topo from github.com/networkop/k8s-topoand meshnet-cni plugin from github.com/networkop/meshnet-cni.