From 3cc43521ce3ee242b2ecef488429f73f29112f3e Mon Sep 17 00:00:00 2001 From: Hippie Hacker Date: Thu, 2 Apr 2020 09:03:42 +1300 Subject: [PATCH 1/7] Add roadmap that includes Q2 OKRs and status on Q1 OKRs --- docs/roadmap.org | 88 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 docs/roadmap.org diff --git a/docs/roadmap.org b/docs/roadmap.org new file mode 100644 index 000000000..b3bfe257e --- /dev/null +++ b/docs/roadmap.org @@ -0,0 +1,88 @@ +# -*- ii: mode; -*- +#+TODO: ORG(o) MOCK(m) TRIAGE(r) BACKLOG(b) TEST(t) PROMOTION(p) FLAKING(f) COMMENTS(c) | DONE(d) +#+TITLE: APISnoop OKRs +* 2020 Q1 +** Increase Stable Test Coverage Velocity 100% over 2019 +*** KR1 (26/+27) new conformant stable endpoints +**** Done = 5 +***** DONE +3 Promote: Secret patching test #87262 + CLOSED: [2020-04-02 Thu 06:49] +***** DONE +1 Promote: find Kubernetes Service in default Namespace #87260 + CLOSED: [2020-04-02 Thu 06:50] +***** DONE +1 Promote: Namespace patch test #87256 + CLOSED: [2020-04-02 Thu 06:50] +**** Needs Approval +3 +***** PROMOTION +3 Promote: pod PreemptionExecutionPath verification +Messaged belamaric + ? #issue + ? #test + https://github.com/kubernetes/kubernetes/pull/83378 # promotion +**** Needs Review +9 +***** TEST +3 Promote: PodTemplate Lifecycle test #88036 (possibly flakes) + https://github.com/kubernetes/kubernetes/issues/86141 #issue + Needs reopening and checkboxes for current state.. + https://github.com/kubernetes/kubernetes/pull/88036#ref-pullrequest-571656281 # promotion + https://github.com/kubernetes/kubernetes/pull/88588#issuecomment-606957802 # related flaking comments +***** COMMENTS +2 Promote: ConfigMap Lifecycle test #88034 (needs addressing) + https://github.com/kubernetes/kubernetes/pull/88034#discussion_r398728147 + https://github.com/kubernetes/kubernetes/pull/89707 PR to possibly handle timeouts + Find an example +***** COMMENTS +4 Pod and PodStatus + https://github.com/kubernetes/kubernetes/issues/88545 #issue / mock-test + https://github.com/kubernetes/kubernetes/pull/89453 #test + https://github.com/kubernetes/kubernetes/pull/89453#discussion_r400346746 + Not sure this will work, you will be racing with the kubelet, I think. That is, kubelet may mark it ready again. +**** In Progress +18 +***** TEST +4 Promote: Endpoints (needs promotion) + https://github.com/kubernetes/kubernetes/issues/87762 #issue + https://github.com/kubernetes/kubernetes/pull/88778 # test + ? # promotion +***** TEST +5 Promote: Event Lifecycle test #86858 (needs promote) + https://github.com/kubernetes/kubernetes/issues/86288 #issue + https://github.com/kubernetes/kubernetes/pull/86858 #test + ??? # promotion +***** FLAKING +7 ReplicationController lifecycle + https://github.com/kubernetes/kubernetes/issues/88302 #issue / mock test + Needs reopening and checkboxes for current state... + https://github.com/kubernetes/kubernetes/pull/88588 # test is flaking +**** Sorted Backlog +5 +***** BACKLOG +2 ServiceStatus lifecycle + https://github.com/cncf/apisnoop/pull/298 # apisnoop needs link to ticket? + https://github.com/kubernetes/kubernetes/issues/89135 # new issue? + Currently, this test is having issues writing to the ServiceStatus endpoints (via patch and update). + The data is patched without errors, but the data when fetched is no different to before the patching. +***** BACKLOG +3 ServiceAccount lifecycle + https://github.com/kubernetes/kubernetes/issues/89071 # issue + @johnbelamaric You don't need to check the status of the secret as part of the test. In other places we check that the resource in question happens, we don't have to follow. +**** Triage +12 +***** TRIAGE +5 Apps DaemonSet lifecycle + https://github.com/cncf/apisnoop/pull/305 # apisnoop + https://github.com/kubernetes/kubernetes/issues/89637 # issue +***** TRIAGE +5 Apps Deployment lifecycle + ? # apisnoop + https://github.com/kubernetes/kubernetes/issues/89340 # issue +***** TRIAGE +2 NodeStatus :deprioritized: + Needs these comments addressed + https://github.com/kubernetes/kubernetes/issues/88358#issuecomment-591062171 + +*** KR2 +6% Coverage Increase + Should be displayed on apisnoop.cncf.io +** Complete cncf/apisnoop prow.k8s.io + EKS migration +*** KR1 All cncf/apisnoop artifacts created by prow.k8s.io +*** KR2 All cncf/apisnoop github workflow managed by prow.k8s.io +*** KR3 All cncf/apisnoop non-prow infra moved to EKS +** Mentor/Teach test-writing workflow at Contributer Summit / KubeConEU +*** KR1 Caleb and Hippie Mentoring at Contributor Summit +I am pairing weekly with Guin and Mallian to ensure the workflow is accessible. +*** KR2 Riaan teaching test writing +* 2020 Q2 +** Increase Stable Test Coverage Velocity 50% over Q1 +*** KR1 (0/+40) new conformant stable endpoints +*** KR2 +9% Coverage Increase +*** KR3 (stretch) 50% stable endpoints hit by conformance tests +** Prepare to Gate k/k PRs touching test/e2e or API +*** KR1 comment w/ list of increase/decrease of stable endpoints +*** KR2 gate w/ comment +** Prepare to Gate cncf/k8s-conformance PRs touching v*.*/ +*** KR1 comment w/ list of unrun conformance tests +*** KR2 gate w/ comment From 5648aba1df039b7c002e9ccd516351c0055807a1 Mon Sep 17 00:00:00 2001 From: Hippie Hacker Date: Thu, 2 Apr 2020 10:27:38 +1300 Subject: [PATCH 2/7] Moving from roadmap.org from docs to org --- {docs => org}/roadmap.org | 1 + 1 file changed, 1 insertion(+) rename {docs => org}/roadmap.org (99%) diff --git a/docs/roadmap.org b/org/roadmap.org similarity index 99% rename from docs/roadmap.org rename to org/roadmap.org index b3bfe257e..be9955a1e 100644 --- a/docs/roadmap.org +++ b/org/roadmap.org @@ -1,6 +1,7 @@ # -*- ii: mode; -*- #+TODO: ORG(o) MOCK(m) TRIAGE(r) BACKLOG(b) TEST(t) PROMOTION(p) FLAKING(f) COMMENTS(c) | DONE(d) #+TITLE: APISnoop OKRs + * 2020 Q1 ** Increase Stable Test Coverage Velocity 100% over 2019 *** KR1 (26/+27) new conformant stable endpoints From 215ae188f8b8f68fb2e4a0fac30c43e687831d0d Mon Sep 17 00:00:00 2001 From: Caleb Woodbine Date: Thu, 2 Apr 2020 15:51:47 +1300 Subject: [PATCH 3/7] Update progress of tests --- org/roadmap.org | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/org/roadmap.org b/org/roadmap.org index be9955a1e..901c0b7c9 100644 --- a/org/roadmap.org +++ b/org/roadmap.org @@ -34,14 +34,20 @@ Messaged belamaric https://github.com/kubernetes/kubernetes/pull/89453#discussion_r400346746 Not sure this will work, you will be racing with the kubelet, I think. That is, kubelet may mark it ready again. **** In Progress +18 -***** TEST +4 Promote: Endpoints (needs promotion) +***** TEST +4 Promote: Endpoints https://github.com/kubernetes/kubernetes/issues/87762 #issue https://github.com/kubernetes/kubernetes/pull/88778 # test - ? # promotion -***** TEST +5 Promote: Event Lifecycle test #86858 (needs promote) + https://github.com/kubernetes/kubernetes/pull/89752 # promotion + + [[https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%2520test%2520the%2520lifecycle%2520of%2520an%2520Endpoint][TestGrid reference]] + +***** TEST +5 Promote: Event Lifecycle test #86858 https://github.com/kubernetes/kubernetes/issues/86288 #issue https://github.com/kubernetes/kubernetes/pull/86858 #test - ??? # promotion + https://github.com/kubernetes/kubernetes/pull/89753 # promotion + + [[https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%2520ensure%2520that%2520an%2520event%2520can%2520be%2520fetched%252C%2520patched%252C%2520deleted%252C%2520and%2520listed][TestGrid reference]] + ***** FLAKING +7 ReplicationController lifecycle https://github.com/kubernetes/kubernetes/issues/88302 #issue / mock test Needs reopening and checkboxes for current state... From 98459add46576c51af4d8215cd9ce08d926eac3c Mon Sep 17 00:00:00 2001 From: Caleb Woodbine Date: Mon, 6 Apr 2020 11:25:55 +1200 Subject: [PATCH 4/7] Update progress on tickets/prs --- org/roadmap.org | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/org/roadmap.org b/org/roadmap.org index 901c0b7c9..d2b120381 100644 --- a/org/roadmap.org +++ b/org/roadmap.org @@ -24,6 +24,7 @@ Messaged belamaric Needs reopening and checkboxes for current state.. https://github.com/kubernetes/kubernetes/pull/88036#ref-pullrequest-571656281 # promotion https://github.com/kubernetes/kubernetes/pull/88588#issuecomment-606957802 # related flaking comments + [[https://github.com/kubernetes/kubernetes/pull/89746][https://github.com/kubernetes/kubernetes/pull/89746]] to address flakes ***** COMMENTS +2 Promote: ConfigMap Lifecycle test #88034 (needs addressing) https://github.com/kubernetes/kubernetes/pull/88034#discussion_r398728147 https://github.com/kubernetes/kubernetes/pull/89707 PR to possibly handle timeouts @@ -32,7 +33,8 @@ Messaged belamaric https://github.com/kubernetes/kubernetes/issues/88545 #issue / mock-test https://github.com/kubernetes/kubernetes/pull/89453 #test https://github.com/kubernetes/kubernetes/pull/89453#discussion_r400346746 - Not sure this will work, you will be racing with the kubelet, I think. That is, kubelet may mark it ready again. + Addressed the comment: + "Not sure this will work, you will be racing with the kubelet, I think. That is, kubelet may mark it ready again." **** In Progress +18 ***** TEST +4 Promote: Endpoints https://github.com/kubernetes/kubernetes/issues/87762 #issue @@ -52,6 +54,8 @@ Messaged belamaric https://github.com/kubernetes/kubernetes/issues/88302 #issue / mock test Needs reopening and checkboxes for current state... https://github.com/kubernetes/kubernetes/pull/88588 # test is flaking + + [[https://github.com/kubernetes/kubernetes/pull/89746][https://github.com/kubernetes/kubernetes/pull/89746]] created to address flaking **** Sorted Backlog +5 ***** BACKLOG +2 ServiceStatus lifecycle https://github.com/cncf/apisnoop/pull/298 # apisnoop needs link to ticket? From f7d02cd6486aa1a2961618d7610107564ac04f55 Mon Sep 17 00:00:00 2001 From: Caleb Woodbine Date: Mon, 6 Apr 2020 16:10:03 +1200 Subject: [PATCH 5/7] Update ReplicationController resource lifecycle test reference information --- org/roadmap.org | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/org/roadmap.org b/org/roadmap.org index d2b120381..7af2b7ab7 100644 --- a/org/roadmap.org +++ b/org/roadmap.org @@ -55,7 +55,7 @@ Messaged belamaric Needs reopening and checkboxes for current state... https://github.com/kubernetes/kubernetes/pull/88588 # test is flaking - [[https://github.com/kubernetes/kubernetes/pull/89746][https://github.com/kubernetes/kubernetes/pull/89746]] created to address flaking + [[https://github.com/kubernetes/kubernetes/pull/89746][https://github.com/kubernetes/kubernetes/pull/89746]] created to [[https://github.com/kubernetes/kubernetes/issues/89740][address flaking]] - relies on it's own update response data **** Sorted Backlog +5 ***** BACKLOG +2 ServiceStatus lifecycle https://github.com/cncf/apisnoop/pull/298 # apisnoop needs link to ticket? From bb56683f736280d282baf881f09c5623ad3b32f7 Mon Sep 17 00:00:00 2001 From: Hippie Hacker Date: Tue, 7 Apr 2020 07:20:32 +1200 Subject: [PATCH 6/7] Add Q1 Scoring and generate org/roadmap.md Co-Authored-By: James Grahas Co-Authored-By: Riaan Kleinhans --- org/roadmap.md | 166 ++++++++++++++++++++++++++++++++++++++++++++++++ org/roadmap.org | 107 ++++++++++++++++--------------- 2 files changed, 221 insertions(+), 52 deletions(-) create mode 100644 org/roadmap.md diff --git a/org/roadmap.md b/org/roadmap.md new file mode 100644 index 000000000..7430e1d03 --- /dev/null +++ b/org/roadmap.md @@ -0,0 +1,166 @@ +- [2020 Q1 (Jan-Mar)](#sec-1) + - [Increase Stable Test Coverage Velocity 100% over 2019 (Score: 0.9)](#sec-1-1) + - [KR1=0.9 (26/+27) new conformant stable endpoints](#sec-1-1-1) + - [KR2=0.9 +6% Coverage Increase](#sec-1-1-2) + - [Complete cncf/apisnoop prow.k8s.io + EKS migration (Score: 0.5)](#sec-1-2) + - [KR1=0.5 All cncf/apisnoop artifacts created by prow.k8s.io](#sec-1-2-1) + - [KR2=0.0 All cncf/apisnoop github workflow managed by prow.k8s.io](#sec-1-2-2) + - [KR3=1.0 All cncf/apisnoop non-prow infra moved to EKS](#sec-1-2-3) + - [Mentor/Teach test-writing workflow at Contributer Summit / KubeConEU (Score: 1.0)](#sec-1-3) + - [KR1 Caleb and Hippie Mentoring at Contributor Summit](#sec-1-3-1) + - [KR2 Zach and Stephen teaching test writing](#sec-1-3-2) +- [2020 Q2 (Apr-Jun)](#sec-2) + - [Increase Stable Test Coverage Velocity 50% over Q1](#sec-2-1) + - [KR1 (0/+40) new conformant stable endpoints](#sec-2-1-1) + - [KR2 +9% Coverage Increase](#sec-2-1-2) + - [KR3 (stretch) 50% stable endpoints hit by conformance tests](#sec-2-1-3) + - [Prepare to Gate k/k PRs touching test/e2e or API](#sec-2-2) + - [KR1 comment w/ list of increase/decrease of stable endpoints](#sec-2-2-1) + - [KR2 gate w/ comment](#sec-2-2-2) + - [Prepare to Gate cncf/k8s-conformance PRs touching v\*.\*/](#sec-2-3) + - [KR1 comment w/ list of unrun conformance tests](#sec-2-3-1) + - [KR2 gate w/ comment](#sec-2-3-2) + + +# 2020 Q1 (Jan-Mar) + +## Increase Stable Test Coverage Velocity 100% over 2019 (Score: 0.9) + +### KR1=0.9 (26/+27) new conformant stable endpoints + +1. SCORE: 0.9 + + Done = 5 Needs Approval = 3 Needs Review = 9 In Progress (no flakes) = 9 + +2. Done = 5 + + 1. DONE +3 Promote: Secret patching test #87262 + + 2. DONE +1 Promote: find Kubernetes Service in default Namespace #87260 + + 3. DONE +1 Promote: Namespace patch test #87256 + +3. Needs Approval +3 + + 1. PROMOTION +3 Promote: pod PreemptionExecutionPath verification + + - ? #issue + - ? #test + - Promotion: + + Clayton says: "I got it" Has a failing test… /retest + +4. Needs Review +9 + + 1. TEST +3 Promote: PodTemplate Lifecycle test #88036 (removing flakes) + + - Issue: #issue Needs reopening and checkboxes for current state.. + - Promotion: + - Flakes: + - Addressing Flakes: [https://github.com/kubernetes/kubernetes/pull/89746](https://github.com/kubernetes/kubernetes/pull/89746) + + 2. COMMENTS +2 Promote: ConfigMap Lifecycle test #88034 (comments addressed) + + - Promotion: + - Addressing Comments: (addresed) + - PR to handle timeouts: + + 3. COMMENTS +4 Pod and PodStatus + + - mock-test: + - test: Addressed the [comment](https://github.com/kubernetes/kubernetes/pull/89453#discussion_r400346746): "Not sure this will work, you will be racing with the kubelet, I think. That is, kubelet may mark it ready again." + +5. In Progress +18 + + 1. TEST +4 Promote: Endpoints + + - mock-test: + - test: + - promotion: [TestGrid reference](https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%20test%20the%20lifecycle%20of%20an%20Endpoint) still looks green! + + 2. TEST +5 Promote: Event Lifecycle test #86858 + + - mock-test: + - test: + - promotion: [TestGrid reference](https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%20ensure%20that%20an%20event%20can%20be%20fetched%2C%20patched%2C%20deleted%2C%20and%20listed) + + 3. FLAKING +7 ReplicationController lifecycle + + - mock-test: Needs reopening and checkboxes for current state… + - test: + - [address flaking comment](https://github.com/kubernetes/kubernetes/issues/89740) : [https://github.com/kubernetes/kubernetes/pull/89746](https://github.com/kubernetes/kubernetes/pull/89746) + + relies on it's own update response data + +6. Sorted Backlog +5 + + 1. BACKLOG +2 ServiceStatus lifecycle + + - org-file: + - mock-test: Currently, this test is having issues writing to the ServiceStatus endpoints (via patch and update). The data is patched without errors, but the data when fetched is no different to before the patching. + + 2. BACKLOG +3 ServiceAccount lifecycle + + - mock-test: @johnbelamaric You don't need to check the status of the secret as part of the test. In other places we check that the resource in question happens, we don't have to follow. + +7. Triage +12 + + 1. TRIAGE +5 Apps DaemonSet lifecycle + + - org-file: + - mock-test: + + 2. TRIAGE +5 Apps Deployment lifecycle + + - org-file: + - mock-test: + + 3. TRIAGE +2 NodeStatus :deprioritized: + + Needs these comments addressed, and we voted to de-priorize + +### KR2=0.9 +6% Coverage Increase + +1. SCORE: 0.9 + + Based on the same KR above… it's not merged, but these are non-flakey tests that are ready to merge. + +## Complete cncf/apisnoop prow.k8s.io + EKS migration (Score: 0.5) + +### KR1=0.5 All cncf/apisnoop artifacts created by prow.k8s.io + +### KR2=0.0 All cncf/apisnoop github workflow managed by prow.k8s.io + +### KR3=1.0 All cncf/apisnoop non-prow infra moved to EKS + +## Mentor/Teach test-writing workflow at Contributer Summit / KubeConEU (Score: 1.0) + +### KR1 Caleb and Hippie Mentoring at Contributor Summit + +I am pairing weekly with Guin and Mallian to ensure the workflow is accessible. + +### KR2 Zach and Stephen teaching test writing + +They are teaching Riaan, all remote, using our org-flow. + +# 2020 Q2 (Apr-Jun) + +## Increase Stable Test Coverage Velocity 50% over Q1 + +### KR1 (0/+40) new conformant stable endpoints + +### KR2 +9% Coverage Increase + +### KR3 (stretch) 50% stable endpoints hit by conformance tests + +## Prepare to Gate k/k PRs touching test/e2e or API + +### KR1 comment w/ list of increase/decrease of stable endpoints + +### KR2 gate w/ comment + +## Prepare to Gate cncf/k8s-conformance PRs touching v\*.\*/ + +### KR1 comment w/ list of unrun conformance tests + +### KR2 gate w/ comment diff --git a/org/roadmap.org b/org/roadmap.org index 7af2b7ab7..1fa1e772f 100644 --- a/org/roadmap.org +++ b/org/roadmap.org @@ -2,9 +2,14 @@ #+TODO: ORG(o) MOCK(m) TRIAGE(r) BACKLOG(b) TEST(t) PROMOTION(p) FLAKING(f) COMMENTS(c) | DONE(d) #+TITLE: APISnoop OKRs -* 2020 Q1 -** Increase Stable Test Coverage Velocity 100% over 2019 -*** KR1 (26/+27) new conformant stable endpoints +* 2020 Q1 (Jan-Mar) +** Increase Stable Test Coverage Velocity 100% over 2019 (Score: 0.9) +*** KR1=0.9 (26/+27) new conformant stable endpoints +**** SCORE: 0.9 +Done = 5 +Needs Approval = 3 +Needs Review = 9 +In Progress (no flakes) = 9 **** Done = 5 ***** DONE +3 Promote: Secret patching test #87262 CLOSED: [2020-04-02 Thu 06:49] @@ -14,79 +19,77 @@ CLOSED: [2020-04-02 Thu 06:50] **** Needs Approval +3 ***** PROMOTION +3 Promote: pod PreemptionExecutionPath verification -Messaged belamaric - ? #issue - ? #test - https://github.com/kubernetes/kubernetes/pull/83378 # promotion +- ? #issue +- ? #test +- Promotion: https://github.com/kubernetes/kubernetes/pull/83378 +Clayton says: "I got it" +Has a failing test... /retest **** Needs Review +9 -***** TEST +3 Promote: PodTemplate Lifecycle test #88036 (possibly flakes) - https://github.com/kubernetes/kubernetes/issues/86141 #issue +***** TEST +3 Promote: PodTemplate Lifecycle test #88036 (removing flakes) +- Issue: https://github.com/kubernetes/kubernetes/issues/86141 #issue Needs reopening and checkboxes for current state.. - https://github.com/kubernetes/kubernetes/pull/88036#ref-pullrequest-571656281 # promotion - https://github.com/kubernetes/kubernetes/pull/88588#issuecomment-606957802 # related flaking comments - [[https://github.com/kubernetes/kubernetes/pull/89746][https://github.com/kubernetes/kubernetes/pull/89746]] to address flakes -***** COMMENTS +2 Promote: ConfigMap Lifecycle test #88034 (needs addressing) - https://github.com/kubernetes/kubernetes/pull/88034#discussion_r398728147 - https://github.com/kubernetes/kubernetes/pull/89707 PR to possibly handle timeouts - Find an example +- Promotion: https://github.com/kubernetes/kubernetes/pull/88036#ref-pullrequest-571656281 +- Flakes: https://github.com/kubernetes/kubernetes/pull/88588#issuecomment-606957802 +- Addressing Flakes: [[https://github.com/kubernetes/kubernetes/pull/89746][https://github.com/kubernetes/kubernetes/pull/89746]] +***** COMMENTS +2 Promote: ConfigMap Lifecycle test #88034 (comments addressed) +- Promotion: https://github.com/kubernetes/kubernetes/pull/88034#discussion_r398728147 +- Addressing Comments: https://github.com/kubernetes/kubernetes/pull/88034#issuecomment-607430447 (addresed) +- PR to handle timeouts: https://github.com/kubernetes/kubernetes/pull/89707 ***** COMMENTS +4 Pod and PodStatus - https://github.com/kubernetes/kubernetes/issues/88545 #issue / mock-test - https://github.com/kubernetes/kubernetes/pull/89453 #test - https://github.com/kubernetes/kubernetes/pull/89453#discussion_r400346746 - Addressed the comment: +- mock-test: https://github.com/kubernetes/kubernetes/issues/88545 +- test: https://github.com/kubernetes/kubernetes/pull/89453 + Addressed the [[https://github.com/kubernetes/kubernetes/pull/89453#discussion_r400346746][comment]]: "Not sure this will work, you will be racing with the kubelet, I think. That is, kubelet may mark it ready again." **** In Progress +18 ***** TEST +4 Promote: Endpoints - https://github.com/kubernetes/kubernetes/issues/87762 #issue - https://github.com/kubernetes/kubernetes/pull/88778 # test - https://github.com/kubernetes/kubernetes/pull/89752 # promotion - - [[https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%2520test%2520the%2520lifecycle%2520of%2520an%2520Endpoint][TestGrid reference]] - +- mock-test: https://github.com/kubernetes/kubernetes/issues/87762 +- test: https://github.com/kubernetes/kubernetes/pull/88778 +- promotion: https://github.com/kubernetes/kubernetes/pull/89752 + [[https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%2520test%2520the%2520lifecycle%2520of%2520an%2520Endpoint][TestGrid reference]] still looks green! ***** TEST +5 Promote: Event Lifecycle test #86858 - https://github.com/kubernetes/kubernetes/issues/86288 #issue - https://github.com/kubernetes/kubernetes/pull/86858 #test - https://github.com/kubernetes/kubernetes/pull/89753 # promotion - +- mock-test: https://github.com/kubernetes/kubernetes/issues/86288 +- test: https://github.com/kubernetes/kubernetes/pull/86858 +- promotion: https://github.com/kubernetes/kubernetes/pull/89753 [[https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%2520ensure%2520that%2520an%2520event%2520can%2520be%2520fetched%252C%2520patched%252C%2520deleted%252C%2520and%2520listed][TestGrid reference]] - ***** FLAKING +7 ReplicationController lifecycle - https://github.com/kubernetes/kubernetes/issues/88302 #issue / mock test +- mock-test: https://github.com/kubernetes/kubernetes/issues/88302 Needs reopening and checkboxes for current state... - https://github.com/kubernetes/kubernetes/pull/88588 # test is flaking - - [[https://github.com/kubernetes/kubernetes/pull/89746][https://github.com/kubernetes/kubernetes/pull/89746]] created to [[https://github.com/kubernetes/kubernetes/issues/89740][address flaking]] - relies on it's own update response data +- test: https://github.com/kubernetes/kubernetes/pull/88588 +- [[https://github.com/kubernetes/kubernetes/issues/89740][address flaking comment]] : [[https://github.com/kubernetes/kubernetes/pull/89746][https://github.com/kubernetes/kubernetes/pull/89746]] +relies on it's own update response data **** Sorted Backlog +5 ***** BACKLOG +2 ServiceStatus lifecycle - https://github.com/cncf/apisnoop/pull/298 # apisnoop needs link to ticket? - https://github.com/kubernetes/kubernetes/issues/89135 # new issue? +- org-file: https://github.com/cncf/apisnoop/pull/298 +- mock-test: https://github.com/kubernetes/kubernetes/issues/89135 Currently, this test is having issues writing to the ServiceStatus endpoints (via patch and update). The data is patched without errors, but the data when fetched is no different to before the patching. ***** BACKLOG +3 ServiceAccount lifecycle - https://github.com/kubernetes/kubernetes/issues/89071 # issue +- mock-test: https://github.com/kubernetes/kubernetes/issues/89071 @johnbelamaric You don't need to check the status of the secret as part of the test. In other places we check that the resource in question happens, we don't have to follow. **** Triage +12 ***** TRIAGE +5 Apps DaemonSet lifecycle - https://github.com/cncf/apisnoop/pull/305 # apisnoop - https://github.com/kubernetes/kubernetes/issues/89637 # issue +- org-file: https://github.com/cncf/apisnoop/pull/305 +- mock-test: https://github.com/kubernetes/kubernetes/issues/89637 ***** TRIAGE +5 Apps Deployment lifecycle - ? # apisnoop - https://github.com/kubernetes/kubernetes/issues/89340 # issue +- org-file: +- mock-test: https://github.com/kubernetes/kubernetes/issues/89340 ***** TRIAGE +2 NodeStatus :deprioritized: - Needs these comments addressed + Needs these comments addressed, and we voted to de-priorize https://github.com/kubernetes/kubernetes/issues/88358#issuecomment-591062171 -*** KR2 +6% Coverage Increase - Should be displayed on apisnoop.cncf.io -** Complete cncf/apisnoop prow.k8s.io + EKS migration -*** KR1 All cncf/apisnoop artifacts created by prow.k8s.io -*** KR2 All cncf/apisnoop github workflow managed by prow.k8s.io -*** KR3 All cncf/apisnoop non-prow infra moved to EKS -** Mentor/Teach test-writing workflow at Contributer Summit / KubeConEU +*** KR2=0.9 +6% Coverage Increase +**** SCORE: 0.9 +Based on the same KR above... it's not merged, but these are non-flakey tests that are ready to merge. +** Complete cncf/apisnoop prow.k8s.io + EKS migration (Score: 0.5) +*** KR1=0.5 All cncf/apisnoop artifacts created by prow.k8s.io +*** KR2=0.0 All cncf/apisnoop github workflow managed by prow.k8s.io +*** KR3=1.0 All cncf/apisnoop non-prow infra moved to EKS +** Mentor/Teach test-writing workflow at Contributer Summit / KubeConEU (Score: 1.0) *** KR1 Caleb and Hippie Mentoring at Contributor Summit I am pairing weekly with Guin and Mallian to ensure the workflow is accessible. -*** KR2 Riaan teaching test writing -* 2020 Q2 +*** KR2 Zach and Stephen teaching test writing +They are teaching Riaan, all remote, using our org-flow. +* 2020 Q2 (Apr-Jun) ** Increase Stable Test Coverage Velocity 50% over Q1 *** KR1 (0/+40) new conformant stable endpoints *** KR2 +9% Coverage Increase From 9d4990e6bf0578e7b83846e650acf58f0acf4d7a Mon Sep 17 00:00:00 2001 From: Hippie Hacker Date: Wed, 8 Apr 2020 06:50:24 +1200 Subject: [PATCH 7/7] Get ready for conformance subgroup sig-arch meeting. --- org/roadmap.md | 164 +++++++++++++++++++++++++++--------------------- org/roadmap.org | 164 +++++++++++++++++++++++++++--------------------- 2 files changed, 186 insertions(+), 142 deletions(-) diff --git a/org/roadmap.md b/org/roadmap.md index 7430e1d03..a0e7b67bb 100644 --- a/org/roadmap.md +++ b/org/roadmap.md @@ -1,14 +1,14 @@ - [2020 Q1 (Jan-Mar)](#sec-1) - - [Increase Stable Test Coverage Velocity 100% over 2019 (Score: 0.9)](#sec-1-1) - - [KR1=0.9 (26/+27) new conformant stable endpoints](#sec-1-1-1) - - [KR2=0.9 +6% Coverage Increase](#sec-1-1-2) - - [Complete cncf/apisnoop prow.k8s.io + EKS migration (Score: 0.5)](#sec-1-2) + - [Increase Stable Test Coverage Velocity 100% over 2019 (Score: 0.2)](#sec-1-1) + - [KR1=0.3 (8/+27) new conformant stable endpoints](#sec-1-1-1) + - [kr2=0.3 +6% coverage increase](#sec-1-1-2) + - [complete cncf/apisnoop prow.k8s.io + EKS migration (Score: 0.5)](#sec-1-2) - [KR1=0.5 All cncf/apisnoop artifacts created by prow.k8s.io](#sec-1-2-1) - [KR2=0.0 All cncf/apisnoop github workflow managed by prow.k8s.io](#sec-1-2-2) - - [KR3=1.0 All cncf/apisnoop non-prow infra moved to EKS](#sec-1-2-3) - - [Mentor/Teach test-writing workflow at Contributer Summit / KubeConEU (Score: 1.0)](#sec-1-3) - - [KR1 Caleb and Hippie Mentoring at Contributor Summit](#sec-1-3-1) - - [KR2 Zach and Stephen teaching test writing](#sec-1-3-2) + - [KR3=1.0 All cncf/apisnoop non-prow infra moved to EKS/Packet](#sec-1-2-3) + - [Mentor/Teach test-writing workflow at Contributer Summit / KubeConEU (Score: 0.5)](#sec-1-3) + - [KR1=0.0 Caleb and Hippie Mentoring at Contributor Summit](#sec-1-3-1) + - [KR2 1.0 Zach and Stephen teaching test writing](#sec-1-3-2) - [2020 Q2 (Apr-Jun)](#sec-2) - [Increase Stable Test Coverage Velocity 50% over Q1](#sec-2-1) - [KR1 (0/+40) new conformant stable endpoints](#sec-2-1-1) @@ -24,122 +24,138 @@ # 2020 Q1 (Jan-Mar) -## Increase Stable Test Coverage Velocity 100% over 2019 (Score: 0.9) +## Increase Stable Test Coverage Velocity 100% over 2019 (Score: 0.2) -### KR1=0.9 (26/+27) new conformant stable endpoints +### KR1=0.3 (8/+27) new conformant stable endpoints -1. SCORE: 0.9 +1. SCORE CALCULATION: 0.3 -> 1.0 - Done = 5 Needs Approval = 3 Needs Review = 9 In Progress (no flakes) = 9 + Done(8) + + - Needs Approval(3) = 11 -> 0.4 + - Needs Two Weeks(16) = 27 -> 1.0 -2. Done = 5 +2. done = 8 - 1. DONE +3 Promote: Secret patching test #87262 + 1. done +3 promote: secret patching test #87262 + + 2. done +1 promote: find kubernetes service in default namespace #87260 + + 3. done +1 promote: namespace patch test #87256 - 2. DONE +1 Promote: find Kubernetes Service in default Namespace #87260 + 4. done +3 promote: pod preemptionexecutionpath verification - 3. DONE +1 Promote: Namespace patch test #87256 + - promotion: -3. Needs Approval +3 +3. needs approval +3 - 1. PROMOTION +3 Promote: pod PreemptionExecutionPath verification + 1. promotion +3 promote: podtemplate lifecycle test #88036 (removing flakes) - - ? #issue - - ? #test - - Promotion: - - Clayton says: "I got it" Has a failing test… /retest + - issue: needs reopening and checkboxes for current state.. + - promotion: + - flakes: + - addressing flakes: [https://github.com/kubernetes/kubernetes/pull/89746](https://github.com/kubernetes/kubernetes/pull/89746) -4. Needs Review +9 +4. needs two weeks (no flakes) +16 - 1. TEST +3 Promote: PodTemplate Lifecycle test #88036 (removing flakes) + 1. soak +5 promote: event lifecycle test - - Issue: #issue Needs reopening and checkboxes for current state.. - - Promotion: - - Flakes: - - Addressing Flakes: [https://github.com/kubernetes/kubernetes/pull/89746](https://github.com/kubernetes/kubernetes/pull/89746) + - mock-test: jan 6th + - test: april 1st + - promotion: + + give the reviewer all the information all we need + + - [testgrid reference](https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%20ensure%20that%20an%20event%20can%20be%20fetched%2c%20patched%2c%20deleted%2c%20and%20listed) - 2. COMMENTS +2 Promote: ConfigMap Lifecycle test #88034 (comments addressed) + 2. soak +7 promote: replicationcontroller lifecycle - - Promotion: - - Addressing Comments: (addresed) - - PR to handle timeouts: + - mock-test: needs reopening and checkboxes for current state… + - test: + - promotion: + + - [address flaking comment](https://github.com/kubernetes/kubernetes/issues/89740) : [https://github.com/kubernetes/kubernetes/pull/89746](https://github.com/kubernetes/kubernetes/pull/89746) + + relies on it's own update response data > i have the same concern as #89707 that this test will not fail if the watch times out - 3. COMMENTS +4 Pod and PodStatus + 3. soak +4 promote: endpoints - - mock-test: - - test: Addressed the [comment](https://github.com/kubernetes/kubernetes/pull/89453#discussion_r400346746): "Not sure this will work, you will be racing with the kubelet, I think. That is, kubelet may mark it ready again." + - mock-test: feb 3rd + - test: mar 27th + - promotion: april 10th? + - [testgrid reference](https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%20test%20the%20lifecycle%20of%20an%20endpoint) still looks green! + + fixme: create shows +5^, mock+promotion shows +4 same issue as configmap lificle: this doesn't verify that the endpoints is deleted. it just watches for an endpoints deletion event. would this test fail if it didn't see a deletion event? -5. In Progress +18 +5. needs review +6 - 1. TEST +4 Promote: Endpoints + 1. comments +2 promote: configmap lifecycle test #88034 (comments addressed) - - mock-test: - - test: - - promotion: [TestGrid reference](https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%20test%20the%20lifecycle%20of%20an%20Endpoint) still looks green! - - 2. TEST +5 Promote: Event Lifecycle test #86858 - - - mock-test: - - test: - - promotion: [TestGrid reference](https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%20ensure%20that%20an%20event%20can%20be%20fetched%2C%20patched%2C%20deleted%2C%20and%20listed) + conceptually this pr adds watches there's no gaurantee that we will see the watch. let's ensure what happens in the negative case. when your waiting for the config map to be deleted, how do you know it's not deleted. for each watch: what happens if the watch times out… when you setup a watch to timeout after 60 seconds…. pretend it's running on a super slow processor what if it times out for every single test…. would i want the watch to be considered a failure…. probably… if it doesn't execute to completion. it's not clear that that happens + + - promotion: + - addressing comments: (addresed) + - pr to handle timeouts: - 3. FLAKING +7 ReplicationController lifecycle + 2. comments +4 pod and podstatus - - mock-test: Needs reopening and checkboxes for current state… - - test: - - [address flaking comment](https://github.com/kubernetes/kubernetes/issues/89740) : [https://github.com/kubernetes/kubernetes/pull/89746](https://github.com/kubernetes/kubernetes/pull/89746) - - relies on it's own update response data + - mock-test: + - test: addressed the [comment](https://github.com/kubernetes/kubernetes/pull/89453#discussion_r400346746): "not sure this will work, you will be racing with the kubelet, i think. that is, kubelet may mark it ready again." -6. Sorted Backlog +5 +6. sorted backlog +5 - 1. BACKLOG +2 ServiceStatus lifecycle + 1. backlog +2 servicestatus lifecycle - org-file: - - mock-test: Currently, this test is having issues writing to the ServiceStatus endpoints (via patch and update). The data is patched without errors, but the data when fetched is no different to before the patching. + - mock-test: currently, this test is having issues writing to the servicestatus endpoints (via patch and update). the data is patched without errors, but the data when fetched is no different to before the patching. - 2. BACKLOG +3 ServiceAccount lifecycle + 2. backlog +3 serviceaccount lifecycle - - mock-test: @johnbelamaric You don't need to check the status of the secret as part of the test. In other places we check that the resource in question happens, we don't have to follow. + - mock-test: @johnbelamaric you don't need to check the status of the secret as part of the test. in other places we check that the resource in question happens, we don't have to follow. -7. Triage +12 +7. triage +12 - 1. TRIAGE +5 Apps DaemonSet lifecycle + 1. triage +5 apps daemonset lifecycle - org-file: - mock-test: - 2. TRIAGE +5 Apps Deployment lifecycle + 2. triage +5 apps deployment lifecycle - org-file: - mock-test: - 3. TRIAGE +2 NodeStatus :deprioritized: + 3. triage +2 nodestatus :deprioritized: - Needs these comments addressed, and we voted to de-priorize + needs these comments addressed, and we voted to de-priorize -### KR2=0.9 +6% Coverage Increase +### kr2=0.3 +6% coverage increase -1. SCORE: 0.9 +1. score: 0.3 - Based on the same KR above… it's not merged, but these are non-flakey tests that are ready to merge. + based on the same kr above… but depends on the two week soak. -## Complete cncf/apisnoop prow.k8s.io + EKS migration (Score: 0.5) +## complete cncf/apisnoop prow.k8s.io + EKS migration (Score: 0.5) ### KR1=0.5 All cncf/apisnoop artifacts created by prow.k8s.io +Definitions in prow, but need to do our Q1 release… this week. + ### KR2=0.0 All cncf/apisnoop github workflow managed by prow.k8s.io -### KR3=1.0 All cncf/apisnoop non-prow infra moved to EKS +- [ ] PR Merged managed via prow (VS pushing to master or manual merging) + +### KR3=1.0 All cncf/apisnoop non-prow infra moved to EKS/Packet -## Mentor/Teach test-writing workflow at Contributer Summit / KubeConEU (Score: 1.0) +Anything not on prow or EKS, not on Google. -### KR1 Caleb and Hippie Mentoring at Contributor Summit +## Mentor/Teach test-writing workflow at Contributer Summit / KubeConEU (Score: 0.5) -I am pairing weekly with Guin and Mallian to ensure the workflow is accessible. +### KR1=0.0 Caleb and Hippie Mentoring at Contributor Summit -### KR2 Zach and Stephen teaching test writing +I am pairing weekly with Guin and Mallian to ensure the workflow is accessible, if they like it I'll bring it as a mentoring session for others soon. Caleb is mentoring Zach and Stephen. + +### KR2 1.0 Zach and Stephen teaching test writing They are teaching Riaan, all remote, using our org-flow. @@ -157,10 +173,14 @@ They are teaching Riaan, all remote, using our org-flow. ### KR1 comment w/ list of increase/decrease of stable endpoints +(maybe gate on behaviours) + ### KR2 gate w/ comment ## Prepare to Gate cncf/k8s-conformance PRs touching v\*.\*/ ### KR1 comment w/ list of unrun conformance tests +Dims has something helpful going forward. What is the version + ### KR2 gate w/ comment diff --git a/org/roadmap.org b/org/roadmap.org index 1fa1e772f..af460072b 100644 --- a/org/roadmap.org +++ b/org/roadmap.org @@ -1,93 +1,113 @@ # -*- ii: mode; -*- -#+TODO: ORG(o) MOCK(m) TRIAGE(r) BACKLOG(b) TEST(t) PROMOTION(p) FLAKING(f) COMMENTS(c) | DONE(d) +#+TODO: ORG(o) MOCK(m) TRIAGE(r) BACKLOG(b) TEST(t) SOAK(s) PROMOTION(p) FLAKING(f) COMMENTS(c) | DONE(d) #+TITLE: APISnoop OKRs * 2020 Q1 (Jan-Mar) -** Increase Stable Test Coverage Velocity 100% over 2019 (Score: 0.9) -*** KR1=0.9 (26/+27) new conformant stable endpoints -**** SCORE: 0.9 -Done = 5 -Needs Approval = 3 -Needs Review = 9 -In Progress (no flakes) = 9 -**** Done = 5 -***** DONE +3 Promote: Secret patching test #87262 - CLOSED: [2020-04-02 Thu 06:49] -***** DONE +1 Promote: find Kubernetes Service in default Namespace #87260 - CLOSED: [2020-04-02 Thu 06:50] -***** DONE +1 Promote: Namespace patch test #87256 - CLOSED: [2020-04-02 Thu 06:50] -**** Needs Approval +3 -***** PROMOTION +3 Promote: pod PreemptionExecutionPath verification -- ? #issue -- ? #test -- Promotion: https://github.com/kubernetes/kubernetes/pull/83378 -Clayton says: "I got it" -Has a failing test... /retest -**** Needs Review +9 -***** TEST +3 Promote: PodTemplate Lifecycle test #88036 (removing flakes) -- Issue: https://github.com/kubernetes/kubernetes/issues/86141 #issue - Needs reopening and checkboxes for current state.. -- Promotion: https://github.com/kubernetes/kubernetes/pull/88036#ref-pullrequest-571656281 -- Flakes: https://github.com/kubernetes/kubernetes/pull/88588#issuecomment-606957802 -- Addressing Flakes: [[https://github.com/kubernetes/kubernetes/pull/89746][https://github.com/kubernetes/kubernetes/pull/89746]] -***** COMMENTS +2 Promote: ConfigMap Lifecycle test #88034 (comments addressed) -- Promotion: https://github.com/kubernetes/kubernetes/pull/88034#discussion_r398728147 -- Addressing Comments: https://github.com/kubernetes/kubernetes/pull/88034#issuecomment-607430447 (addresed) -- PR to handle timeouts: https://github.com/kubernetes/kubernetes/pull/89707 -***** COMMENTS +4 Pod and PodStatus -- mock-test: https://github.com/kubernetes/kubernetes/issues/88545 -- test: https://github.com/kubernetes/kubernetes/pull/89453 - Addressed the [[https://github.com/kubernetes/kubernetes/pull/89453#discussion_r400346746][comment]]: - "Not sure this will work, you will be racing with the kubelet, I think. That is, kubelet may mark it ready again." -**** In Progress +18 -***** TEST +4 Promote: Endpoints -- mock-test: https://github.com/kubernetes/kubernetes/issues/87762 -- test: https://github.com/kubernetes/kubernetes/pull/88778 -- promotion: https://github.com/kubernetes/kubernetes/pull/89752 - [[https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%2520test%2520the%2520lifecycle%2520of%2520an%2520Endpoint][TestGrid reference]] still looks green! -***** TEST +5 Promote: Event Lifecycle test #86858 -- mock-test: https://github.com/kubernetes/kubernetes/issues/86288 -- test: https://github.com/kubernetes/kubernetes/pull/86858 +** Increase Stable Test Coverage Velocity 100% over 2019 (Score: 0.2) +*** KR1=0.3 (8/+27) new conformant stable endpoints +**** SCORE CALCULATION: 0.3 -> 1.0 +Done(8) ++ Needs Approval(3) = 11 -> 0.4 ++ Needs Two Weeks(16) = 27 -> 1.0 +**** done = 8 +***** done +3 promote: secret patching test #87262 + closed: [2020-04-02 thu 06:49] +***** done +1 promote: find kubernetes service in default namespace #87260 + closed: [2020-04-02 thu 06:50] +***** done +1 promote: namespace patch test #87256 + closed: [2020-04-02 thu 06:50] +***** done +3 promote: pod preemptionexecutionpath verification +- promotion: https://github.com/kubernetes/kubernetes/pull/83378 +**** needs approval +3 +***** promotion +3 promote: podtemplate lifecycle test #88036 (removing flakes) +- issue: https://github.com/kubernetes/kubernetes/issues/86141 + needs reopening and checkboxes for current state.. +- promotion: https://github.com/kubernetes/kubernetes/pull/88036#ref-pullrequest-571656281 +- flakes: https://github.com/kubernetes/kubernetes/pull/88588#issuecomment-606957802 +- addressing flakes: [[https://github.com/kubernetes/kubernetes/pull/89746][https://github.com/kubernetes/kubernetes/pull/89746]] +**** needs two weeks (no flakes) +16 +***** soak +5 promote: event lifecycle test +- mock-test: jan 6th https://github.com/kubernetes/kubernetes/issues/86288 +- test: april 1st https://github.com/kubernetes/kubernetes/pull/86858 - promotion: https://github.com/kubernetes/kubernetes/pull/89753 - [[https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%2520ensure%2520that%2520an%2520event%2520can%2520be%2520fetched%252C%2520patched%252C%2520deleted%252C%2520and%2520listed][TestGrid reference]] -***** FLAKING +7 ReplicationController lifecycle +give the reviewer all the information all we need +- [[https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%2520ensure%2520that%2520an%2520event%2520can%2520be%2520fetched%252c%2520patched%252c%2520deleted%252c%2520and%2520listed][testgrid reference]] +***** soak +7 promote: replicationcontroller lifecycle - mock-test: https://github.com/kubernetes/kubernetes/issues/88302 - Needs reopening and checkboxes for current state... + needs reopening and checkboxes for current state... - test: https://github.com/kubernetes/kubernetes/pull/88588 +- promotion: + - [[https://github.com/kubernetes/kubernetes/issues/89740][address flaking comment]] : [[https://github.com/kubernetes/kubernetes/pull/89746][https://github.com/kubernetes/kubernetes/pull/89746]] relies on it's own update response data -**** Sorted Backlog +5 -***** BACKLOG +2 ServiceStatus lifecycle +> i have the same concern as #89707 that this test will not fail if the watch times out +***** soak +4 promote: endpoints +- mock-test: feb 3rd https://github.com/kubernetes/kubernetes/issues/87762 +- test: mar 27th https://github.com/kubernetes/kubernetes/pull/88778 +- promotion: april 10th? https://github.com/kubernetes/kubernetes/pull/89752 +- [[https://testgrid.k8s.io/sig-release-master-blocking#gce-cos-master-default&include-filter-by-regex=should%2520test%2520the%2520lifecycle%2520of%2520an%2520endpoint][testgrid reference]] still looks green! +fixme: create shows +5^, mock+promotion shows +4 +same issue as configmap lificle: +this doesn't verify that the endpoints is deleted. +it just watches for an endpoints deletion event. +would this test fail if it didn't see a deletion event? +**** needs review +6 +***** comments +2 promote: configmap lifecycle test #88034 (comments addressed) +conceptually this pr adds watches +there's no gaurantee that we will see the watch. +let's ensure what happens in the negative case. +when your waiting for the config map to be deleted, how do you know it's not deleted. +for each watch: +what happens if the watch times out... +when you setup a watch to timeout after 60 seconds.... +pretend it's running on a super slow processor +what if it times out for every single test.... would i want the watch to be considered a failure.... +probably... if it doesn't execute to completion. +it's not clear that that happens +- promotion: https://github.com/kubernetes/kubernetes/pull/88034#discussion_r398728147 +- addressing comments: https://github.com/kubernetes/kubernetes/pull/88034#issuecomment-607430447 (addresed) +- pr to handle timeouts: https://github.com/kubernetes/kubernetes/pull/89707 +***** comments +4 pod and podstatus +- mock-test: https://github.com/kubernetes/kubernetes/issues/88545 +- test: https://github.com/kubernetes/kubernetes/pull/89453 + addressed the [[https://github.com/kubernetes/kubernetes/pull/89453#discussion_r400346746][comment]]: + "not sure this will work, you will be racing with the kubelet, i think. that is, kubelet may mark it ready again." +**** sorted backlog +5 +***** backlog +2 servicestatus lifecycle - org-file: https://github.com/cncf/apisnoop/pull/298 - mock-test: https://github.com/kubernetes/kubernetes/issues/89135 - Currently, this test is having issues writing to the ServiceStatus endpoints (via patch and update). - The data is patched without errors, but the data when fetched is no different to before the patching. -***** BACKLOG +3 ServiceAccount lifecycle + currently, this test is having issues writing to the servicestatus endpoints (via patch and update). + the data is patched without errors, but the data when fetched is no different to before the patching. +***** backlog +3 serviceaccount lifecycle - mock-test: https://github.com/kubernetes/kubernetes/issues/89071 - @johnbelamaric You don't need to check the status of the secret as part of the test. In other places we check that the resource in question happens, we don't have to follow. -**** Triage +12 -***** TRIAGE +5 Apps DaemonSet lifecycle + @johnbelamaric you don't need to check the status of the secret as part of the test. in other places we check that the resource in question happens, we don't have to follow. +**** triage +12 +***** triage +5 apps daemonset lifecycle - org-file: https://github.com/cncf/apisnoop/pull/305 - mock-test: https://github.com/kubernetes/kubernetes/issues/89637 -***** TRIAGE +5 Apps Deployment lifecycle +***** triage +5 apps deployment lifecycle - org-file: - mock-test: https://github.com/kubernetes/kubernetes/issues/89340 -***** TRIAGE +2 NodeStatus :deprioritized: - Needs these comments addressed, and we voted to de-priorize +***** triage +2 nodestatus :deprioritized: + needs these comments addressed, and we voted to de-priorize https://github.com/kubernetes/kubernetes/issues/88358#issuecomment-591062171 -*** KR2=0.9 +6% Coverage Increase -**** SCORE: 0.9 -Based on the same KR above... it's not merged, but these are non-flakey tests that are ready to merge. -** Complete cncf/apisnoop prow.k8s.io + EKS migration (Score: 0.5) +*** kr2=0.3 +6% coverage increase +**** score: 0.3 +based on the same kr above... but depends on the two week soak. +** complete cncf/apisnoop prow.k8s.io + EKS migration (Score: 0.5) *** KR1=0.5 All cncf/apisnoop artifacts created by prow.k8s.io +Definitions in prow, but need to do our Q1 release... this week. *** KR2=0.0 All cncf/apisnoop github workflow managed by prow.k8s.io -*** KR3=1.0 All cncf/apisnoop non-prow infra moved to EKS -** Mentor/Teach test-writing workflow at Contributer Summit / KubeConEU (Score: 1.0) -*** KR1 Caleb and Hippie Mentoring at Contributor Summit -I am pairing weekly with Guin and Mallian to ensure the workflow is accessible. -*** KR2 Zach and Stephen teaching test writing +- [ ] PR Merged managed via prow (VS pushing to master or manual merging) +*** KR3=1.0 All cncf/apisnoop non-prow infra moved to EKS/Packet +Anything not on prow or EKS, not on Google. +** Mentor/Teach test-writing workflow at Contributer Summit / KubeConEU (Score: 0.5) +*** KR1=0.0 Caleb and Hippie Mentoring at Contributor Summit +I am pairing weekly with Guin and Mallian to ensure the workflow is accessible, +if they like it I'll bring it as a mentoring session for others soon. +Caleb is mentoring Zach and Stephen. +*** KR2 1.0 Zach and Stephen teaching test writing They are teaching Riaan, all remote, using our org-flow. * 2020 Q2 (Apr-Jun) ** Increase Stable Test Coverage Velocity 50% over Q1 @@ -96,7 +116,11 @@ They are teaching Riaan, all remote, using our org-flow. *** KR3 (stretch) 50% stable endpoints hit by conformance tests ** Prepare to Gate k/k PRs touching test/e2e or API *** KR1 comment w/ list of increase/decrease of stable endpoints +(maybe gate on behaviours) *** KR2 gate w/ comment ** Prepare to Gate cncf/k8s-conformance PRs touching v*.*/ *** KR1 comment w/ list of unrun conformance tests +Dims has something helpful going forward. +What is the version *** KR2 gate w/ comment +