- Notifications
You must be signed in to change notification settings - Fork63
Skipping MCAD CPU Preemption Test#696
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
[APPROVALNOTIFIER] This PR isNOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be foundhere. Needs approval from an approver in each of these files: Approvers can indicate their approval by writing |
ronensc commentedDec 7, 2023
For future reference, the root cause analysis of the test's failure has been conducted by@dgrove-oss, and it can be found here: |
0a8c451 to5c3d3edCompare5c3d3ed to701e8e9CompareFiona-Waters commentedDec 8, 2023
Thanks@ronensc That's good to know! |
dgrove-oss commentedDec 8, 2023
I don't think it's worth backporting, but I did redo these test cases for mcad v2 to be robust against different cluster sizes inproject-codeflare/mcad#83 |
Fiona-Waters commentedDec 11, 2023
More investigation is required as to why these tests are failing. Closing this PR. |
KPostOffice left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Looks good. I like the move to more generic tests. One question.
| //aw := createDeploymentAWwith550CPU(context, appendRandomString("aw-deployment-2-550cpu")) | ||
| cap:=getClusterCapacitycontext(context) | ||
| resource:=cpuDemand(cap,0.275).String() | ||
| aw:=createGenericDeploymentCustomPodResourcesWithCPUAW( |
KPostOfficeDec 19, 2023 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
What happens if the cluster has many smaller nodes resulting a a highcap but inability to schedule AppWrappers becauase they do not fit on the individual nodes? Do we care about that at all in this test case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
From a test case perspective, the cluster is assumed to have homogenous nodes and it requests deployments that fit on a node in the cluster in CPU dimension.
Fiona-Waters left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This looks great, so happy to move forward with this improvement. Just a couple of small comments.
| funcgetClusterCapacitycontext(context*context)*clusterstateapi.Resource { | ||
| capacity:=clusterstateapi.EmptyResource() | ||
| nodes,_:=context.kubeclient.CoreV1().Nodes().List(context.ctx, metav1.ListOptions{}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
We should handle the error here.
| podList,err:=context.kubeclient.CoreV1().Pods("").List(context.ctx, metav1.ListOptions{FieldSelector:labelSelector}) | ||
| // TODO: when no pods are listed, do we send entire node capacity as available | ||
| // this will cause false positive dispatch. | ||
| iferr!=nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Should the error be caught like this instead?
| iferr!=nil { | |
| Expect(err).NotTo(HaveOccurred() |
Skipping the MCAD CPU Preemption Test which is failing intermittently on PRs so that we can get some outstanding PRs merged.