Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add NATS publisher support to reminder #4829

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

Vyom-Yadav
Copy link
Collaborator

Summary

Provide a brief overview of the changes and the issue being addressed.
Explain the rationale and any background necessary for understanding the changes.
List dependencies required by this change, if any.

Add NATS as an option for reminder to publish events.

Change Type

Mark the type of change your PR introduces:

  • Bug fix (resolves an issue without affecting existing features)
  • Feature (adds new functionality without breaking changes)
  • Breaking change (may impact existing functionalities or require documentation updates)
  • Documentation (updates or additions to documentation)
  • Refactoring or test improvements (no bug fixes or new functionality)

Testing

Outline how the changes were tested, including steps to reproduce and any relevant configurations.
Attach screenshots if helpful.

Tested locally, working fine (nats stream view output):

...
[269] Subject: minder.internal.repo.reconciler.event Received: 1970-01-01T05:30:02+05:30

  ce-source: minder
  ce-specversion: 1.0
  ce-subject: TODO
  ce-type: minder.internal.repo.reconciler.event
  ce-datacontenttype: application/json
  ce-id: 36f8fc06-6b36-48e8-ad92-3cde66759a13
  ce-minderpublished0at: 2024-10-26T18:39:07Z

{"entity_id":"ca6f63c7-672b-4906-ae2d-ce050cd0b6bc","project":"f6bf23d3-ace6-4652-b858-8f9addf7bea5","provider":"a3c63545-59d7-4d46-9a4c-5028876ad236"}

...

Review Checklist:

  • Reviewed my own code for quality and clarity.
  • Added comments to complex or tricky code sections.
  • Updated any affected documentation.
  • Included tests that validate the fix or feature.
  • Checked that related changes are merged.

Signed-off-by: Vyom Yadav <[email protected]>
@Vyom-Yadav Vyom-Yadav requested a review from a team as a code owner October 26, 2024 18:46
Comment on lines +129 to +133
_, err = js.CreateOrUpdateStream(ctx, jetstream.StreamConfig{
Name: c.cfg.Prefix,
Subjects: []string{c.cfg.Prefix + ".>"},
})
return err
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Newer jetstream API, cleaner code, context support. https://github.com/nats-io/nats.go/blob/main/jetstream/README.md

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 if this works; I think when I started looking, this wasn't cleaned up / operable with CloudEvents yet.

@Vyom-Yadav Vyom-Yadav force-pushed the add-nats-publisher-to-reminder branch from 3d399bf to 748cb2c Compare October 27, 2024 11:34
Copy link
Member

@evankanderson evankanderson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a separate fix going in for the one test failure.

In general, I think you could have separated the changes to internal/events/nats from the changes to reminder, which makes it easier to roll back or hold one change or the other if there needs to be discussion. But I'm willing to approve this and then iterate more going forward, modulo the comments, particularly about re-using eventer.New, rather than duplicating the setup code at this point.

@@ -153,7 +150,7 @@ func instantiateDriver(
return eventersql.BuildPostgreSQLDriver(ctx, cfg)
case constants.NATSDriver:
zerolog.Ctx(ctx).Info().Msg("Using NATS driver")
return nats.BuildNatsChannelDriver(cfg)
return nats.BuildNatsChannelDriver(&cfg.Nats)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the other constructors take the entire cfg, so that's the model I followed here.

Comment on lines +20 to +29
func (r *reminder) getMessagePublisher(ctx context.Context) (message.Publisher, common.DriverCloser, error) {
switch r.cfg.EventConfig.Driver {
case constants.NATSDriver:
return r.setupNATSPublisher(ctx)
case constants.SQLDriver:
return r.setupSQLPublisher(ctx)
default:
return nil, nil, fmt.Errorf("unknown publisher type: %s", r.cfg.EventConfig.Driver)
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm nervous about having this diverge from internal/events/events.go. Can we use an eventer.Interface from eventer.New() here?

(I need to do some refactoring in that package to move eventer/interface into eventer, but I'll wait until this code is landed to reduce the amount of conflicts.)

Comment on lines +152 to +162

// NatsConfig is the configuration when using NATS as the event driver
type NatsConfig struct {
// URL is the URL for the NATS server
URL string `mapstructure:"url" default:"nats://localhost:4222"`
// Prefix is the prefix for the NATS subjects to subscribe to
Prefix string `mapstructure:"prefix" default:"minder"`
// Queue is the name of the queue group to join when consuming messages
// queue groups allow multiple process to round-robin process messages.
Queue string `mapstructure:"queue" default:"minder"`
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to move this up to common (rather than just having a split between client and server configuration), let's put it in its own file. It feels kind of peculiar to have this in common while having the rest of the event configuration still in server, so I'd argue for moving the code back, particularly in the context of having smaller, more focused PRs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of moving the reminder configuration into the same package as server, and calling them all "server-side components`? It feels like it would allow for more sharing between the two implementations.

Comment on lines +129 to +133
_, err = js.CreateOrUpdateStream(ctx, jetstream.StreamConfig{
Name: c.cfg.Prefix,
Subjects: []string{c.cfg.Prefix + ".>"},
})
return err
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 if this works; I think when I started looking, this wasn't cleaned up / operable with CloudEvents yet.

Copy link
Contributor

This PR needs additional information before we can continue. It is now marked as stale because it has been open for 30 days with no activity. Please provide the necessary details to continue or it will be closed in 30 days.

@github-actions github-actions bot added the Stale label Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants