Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor controller code #41

Open
wants to merge 36 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
0228de6
added initial implementation to resolve the bug
amirbavand Oct 5, 2024
28dcabc
fixed some bugs
amirbavand Oct 5, 2024
412936a
addd required files to implement integration test for deletion scenario
amirbavand Oct 6, 2024
5c1a286
addd required files to implement integration test for deletion scenario
amirbavand Oct 6, 2024
5028c0b
added required integration test
amirbavand Oct 6, 2024
f16a8ae
Merge branch 'main' into fix-bug-deletion
amirbavand Oct 6, 2024
2cea244
added new line to file
amirbavand Oct 7, 2024
13875bc
reset the file
amirbavand Oct 7, 2024
b8e52a1
delete some of log statements
amirbavand Oct 7, 2024
fd8b799
fixed some lint issues
amirbavand Oct 7, 2024
6e1dbf7
fixed some more lint issues
amirbavand Oct 7, 2024
5996c64
addressed review comments
amirbavand Oct 12, 2024
38ff5be
used map instead of list to delete the resources
amirbavand Oct 12, 2024
9f83ada
refactord code for dynamicreconciler
amirbavand Nov 1, 2024
57adccc
refactored code for dynamicreconciler and removed unnecessary codes
amirbavand Nov 1, 2024
cdfec43
more refactoring
amirbavand Nov 1, 2024
cb3d04a
Merge branch 'main' into refactor-controller-code
amirbavand Nov 10, 2024
b772291
fixed a but in dynamicreconciler
amirbavand Nov 15, 2024
daf91d5
Merge branch 'main' into refactor-controller-code
amirbavand Nov 15, 2024
24e8500
fixed a runtime error
amirbavand Nov 15, 2024
135cea3
remove unused function CheckIfResourceExists
amirbavand Nov 15, 2024
c326f0d
refactored code for testing
amirbavand Nov 15, 2024
cdea02f
added unit tests for util functions
amirbavand Nov 15, 2024
4113333
added some new changes for make code testable
amirbavand Nov 16, 2024
335094a
removed singleton object for sqliterepository and pass inject it as d…
amirbavand Nov 16, 2024
7d7a1a7
Merge branch 'main' into refactor-controller-code
amirbavand Nov 16, 2024
99ce2cf
pass watchmanager interface instead of watchmanager
amirbavand Nov 17, 2024
502768a
added unit tests for dynamic_reconciler
amirbavand Nov 17, 2024
e2fdf40
fix lint issues
amirbavand Nov 17, 2024
06b4593
changed package names for test files
amirbavand Nov 17, 2024
cd3ac7e
applied some of the review comments
amirbavand Nov 19, 2024
6c74a8b
fix issue with unit test
amirbavand Nov 19, 2024
60df66a
remove some not needed lint errors
amirbavand Nov 25, 2024
40e2efe
changed watchmanagerinterface to watchmanager
amirbavand Nov 25, 2024
182fc3f
Update watchmanager.go
amirbavand Nov 25, 2024
b31ed27
remove duplicated method
amirbavand Nov 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions cmd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import (
"crypto/tls"
"flag"
_ "kumquat/renderer/cue"
"kumquat/repository"
"os"

// Import all Kubernetes client auth plugins (e.g. Azure, GCP, OIDC, etc.)
Expand Down Expand Up @@ -150,11 +151,17 @@ func main() {
setupLog.Error(err, "unable to create dynamic k8s client")
os.Exit(1)
}
rep, err := repository.NewSQLiteRepository()
if err != nil {
setupLog.Error(err, "unable to create repository")
os.Exit(1)
}

if err = (&controller.TemplateReconciler{
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
K8sClient: k8sClient,
Client: mgr.GetClient(),
Scheme: mgr.GetScheme(),
K8sClient: k8sClient,
Repository: rep,
}).SetupWithManager(mgr); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "Template")
os.Exit(1)
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ require (
google.golang.org/genproto/googleapis/rpc v0.0.0-20240930140551-af27646dc61f // indirect
google.golang.org/grpc v1.67.1 // indirect
google.golang.org/protobuf v1.35.1 // indirect
gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
amirbavand marked this conversation as resolved.
Show resolved Hide resolved
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
k8s.io/apiextensions-apiserver v0.31.1 // indirect
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package controller
package controller_test

import (
"context"
Expand Down
131 changes: 131 additions & 0 deletions internal/controller/dynamicReconciler.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
package controller

import (
"context"
kumquatv1beta1 "kumquat/api/v1beta1"
"kumquat/repository"

"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/runtime/schema"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
)

type DynamicReconciler struct {
client.Client
amirbavand marked this conversation as resolved.
Show resolved Hide resolved
GVK schema.GroupVersionKind
K8sClient K8sClient
amirbavand marked this conversation as resolved.
Show resolved Hide resolved
WatchManager WatchManagerInterface
repository repository.Repository
amirbavand marked this conversation as resolved.
Show resolved Hide resolved
}

func NewDynamicReconciler(client client.Client, gvk schema.GroupVersionKind, k8sClient K8sClient, wm WatchManagerInterface, repo repository.Repository) *DynamicReconciler { // nolint:errcheck
amirbavand marked this conversation as resolved.
Show resolved Hide resolved
return &DynamicReconciler{
Client: client,
GVK: gvk,
K8sClient: k8sClient,
WatchManager: wm,
repository: repo,
}
}

func (r *DynamicReconciler) Reconcile(ctx context.Context, req reconcile.Request) (reconcile.Result, error) {
log := log.FromContext(ctx)
log.Info("Reconciling dynamic resource", "GVK", r.GVK, "name", req.Name, "namespace", req.Namespace)

resource, err := r.fetchResource(ctx, req)
if err != nil {
return reconcile.Result{}, err
}

if resource == nil {
log.Info("Resource deleted", "GVK", r.GVK, "name", req.Name, "namespace", req.Namespace)
// set group to core if it is empty
group := r.GVK.Group
if r.GVK.Group == "" {
group = "core"
}

err = DeleteResourceFromDatabaseByNameAndNameSpace(r.repository, r.GVK.Kind, group, req.Namespace, req.Name) // nolint:errcheck
amirbavand marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
return reconcile.Result{}, err
}
r.findAndReProcessAffectedTemplates(ctx) // nolint:errcheck
amirbavand marked this conversation as resolved.
Show resolved Hide resolved

return reconcile.Result{}, nil

}

err = UpsertResourceToDatabase(r.repository, resource, ctx) // nolint:errcheck
amirbavand marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
return reconcile.Result{}, err
}

err = r.findAndReProcessAffectedTemplates(ctx) // nolint:errcheck

if err != nil {
return reconcile.Result{}, err
}

return reconcile.Result{}, nil
}

// reconcileTemplates reconciles the templates associated with the resource.
func (r *DynamicReconciler) findAndReProcessAffectedTemplates(ctx context.Context) error {
log := log.FromContext(ctx)
var templates []string

templatesMap := r.WatchManager.GetManagedTemplates()
for templateName, gvks := range templatesMap {
if _, exists := gvks[r.GVK]; exists {
log.Info("Reconciling template", "templateName", templateName)
templates = append(templates, templateName)
}
}

for _, templateName := range templates {
if err := r.processTemplate(ctx, templateName); err != nil {
return err
}
}

return nil
}

// fetchResource fetches the resource from the cluster.
func (r *DynamicReconciler) fetchResource(
ctx context.Context,
req reconcile.Request,
) (*unstructured.Unstructured, error) {
resource := &unstructured.Unstructured{}
resource.SetGroupVersionKind(r.GVK)
err := r.Client.Get(ctx, req.NamespacedName, resource)
if err != nil {
if client.IgnoreNotFound(err) != nil {
amirbavand marked this conversation as resolved.
Show resolved Hide resolved
return nil, err
}
return nil, nil
}
return resource, nil
}

// processTemplate processes a single template.
func (r *DynamicReconciler) processTemplate(ctx context.Context, templateName string) error {
log := log.FromContext(ctx)

template, err := r.K8sClient.Get(ctx, "kumquat.guidewire.com", "Template", "templates", templateName)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was hoping we wouldn't have to query Kubernetes to get templates, we should be able to track them all in memory.

If we do need to query Kubernetes here, can you use the TemplateAPIGroup constant and others from the template package? And avoid hard-coding a namespace--maybe now is the time to make Template.kumquat.guidewire.com a cluster-scoped CRD instead of namespace-scoped?

if err != nil {
log.Error(err, "unable to get template", "templateName", templateName)
return err
}

templateObj := &kumquatv1beta1.Template{}
if err := runtime.DefaultUnstructuredConverter.FromUnstructured(template.Object, templateObj); err != nil {
log.Error(err, "unable to convert unstructured to template")
return err
}
return ProcessTemplateResources(templateObj, r.repository, log, r.K8sClient, r.WatchManager)

}
Loading