History

README.md

This document outlines the testing strategy forkro, focusing onintegration tests andend-to-end testing. It defines the purpose,approach and boundaries of each type of testing, providing clear guidelines forcontributors and maintainers.

kro is a complex controller that interacts with multiple Kubernetes resourcesand depends on various Kubernetes features, such as custom resources, customcontrollers, informers, and client-go. The testing strategy aims to ensure thatkro works as expected in a Kubernetes environment, and that it can be safelydeployed in production clusters.

1. Use existing Kubernetes testing frameworks (when possible): Don't reinventthe wheel. If a feature is not covered by existing frameworks, contribute tothem.
2. Focus on kro's logic, not on other controllers or Kubernetes components. e.gavoid testing native controllers, ACK or Karpenter's behaviour...
3. Prioritize integration tests, validate with end to end tests.
4. Maintain seperation of concerns, controller logic, integration tests, and e2etests
5. Ensure readability: similar to the codebase, tests should be easy to read,understand and maintain.

• integration/: Contains integration test suites for kro.
• e2e/: Contains e2e test suites for kro.
• testdata/: Directory for test data, such as Kubernetes manifests,resourcegroups ...

In integration tests, the focus should be on your custom controller's logic andits interactions with the Kubernetes API server. Instead of emulating othercontrollers, you should:

1. Mock the Kubernetes API server responses
2. Verify that your controller makes the correct API calls
3. Test your controller's reconciliation logic
4. Check status updates on your custom resources

1. Create a ResourceGroup
2. Trigger ResourceGroup reconciliation
3. Check that the ResourceGroup status was updated correctly
4. Verify that the correct Create (CRD) call was made to the API server
5. Check that the controller is watching the correct RG instances
6. Create a ResourceGroup instance
7. Trigger the ResourceGroup instance reconciliation
8. Check that the ResourceGroup instance status was updated correctly
9. Verify that the some resources were created in the cluster
10. Trigger a second reconciliation and check that the status was updatedcorrectly
11. Repeat until all the RG instances are created
12. Do the same for updates and deletions

E2E tests for kro should focus on validating the entire system's behavior in areal Kubernetes environment. These tests ensure that kro works correctly withactual Kubernetes resources and other controllers. The approach for E2E testsshould:

1. Use a real Kubernetes cluster (e.g. kind, minikube, or EKS)
2. Deploy kro controller and it's CRDs
3. Deploy other controllers or resources that will interact with kro resources.
4. Create kro ResourceGroups and ResourceGroupInstances and verify their fulllifecycle.

1. Deploy kro controller and CRDs
2. Deploy a sample application that uses kro
3. Create aResourceGroup custom resource
4. Verify that the corresponding CRD is created in the cluster
5. Create an instance of theResourceGroup
6. Wait for all resources defined in theResourceGroup to be created
7. Verify that all resources are correctly created and configured
8. Update theResourceGroup instance
9. Verify that the changes are correctly propagated to the managed resources
10. Delete theResourceGroup instance
11. Verify that all managed resources are correctly deleted
12. Delete theResourceGroup custom resource
13. Verify that the corresponding CRD is removed from the cluster

1. Cross namespace resource management
2. Scaling testing: Create a large number of ResourceGroups and ResourceGroupinstances to test kro at scale.
3. Failure recovery: Simulate failures in the controller or the Kubernetes APIserver and verify that kro recovers correctly
4. Controller upgrade testing: Deploy a new version of kro and verify that itcan handle existingResourceGroups andResourceGroup instances
5. ResourceGroup conflict testing: Create multipleResourceGroups withconflicting resources and verify that kro handles the conflicts correctly
6. Integration with other controllers: Deploy other controllers that interactwith kro resources and verify that they work correctly together