feat(api): Allow Events/Workflows to be processed/triggered with priority

[isaiahdahl]

What changed? Why was the change needed?

Allows workflow trigger to specify a priority with a value that's then passed onto BullMQ and determines the priority processing of the items on the queue.

Simply put I want to use Novu for system notifications that are critical ie. "Password Reset" or "You've Been Invited to Join "Person's" Team". But then I also want to use it to power some marketing drip/bucket campaigns. I would have like a "new-music-for-you" workflow that is triggered from our API with personalization context to generate an email with personalized songs (context: we're a Music Publishing company) . That "new-music-for-you" notification could trigger to 100k+ people. And when that happens, I wouldn't want the next "password-reset" notification to be stuck in line behind all of those "new-music-for-you" emails.

https://discord.com/channels/895029566685462578/1282721751922638858

[isaiahdahl]

Unsure of what branch to make this against to have the highest probability of getting it merged. Are you doing anymore 0.2x releases before 2.0?

[SokratisVidros]

Unsure of what branch to make this against to have the highest probability of getting it merged. Are you doing anymore 0.2x releases before 2.0?

Which version are you using? We are still doing hotfixes for v0.2x and we plan to have the first v2.0 docker release in October.

[SokratisVidros]

Thanks for the contribution. Please take a look at the comment and as soon as it's resolved we will be happy to merge this.

[isaiahdahl]

Unsure of what branch to make this against to have the highest probability of getting it merged. Are you doing anymore 0.2x releases before 2.0?

Which version are you using? We are still doing hotfixes for v0.2x and we plan to have the first v2.0 docker release in October.

I'm currently using 0.23. Are you hotfixing existing builds so a new image pull would include hotfixes?

Would be super ideal if this could come into 0.23 as we had issues going to 0.24 back in late march but didn't spend much time debugging or trying to figure out what it was because 0.23 had what we needed.

We would plan to evaluate and move to 2.x when that comes out? How would this PR end up in that release? Will you guys move it in or should I do a similar PR against next?

[SokratisVidros]

I will merge this PR into next for now. The v2.0 docker release is scheduled in early October.

Can you please open the same one using the v0.24.x branch as a base?

[isaiahdahl]

This was based against 0.24 originally. But if the changes are easy to rebase to next then yes I'll open it up again based from 0.24. What branch should the PR be against?

[SokratisVidros]

There is a v0.24.x branch. Please use that. This will be merged in next.

[isaiahdahl]

Sounds good. Thanks!

[isaiahdahl]

To resolve conflicts I just re-created the PR based from next in a separate branch and force pushed that to this branch.

There is a v0.24.x branch. Please use that. This will be merged in next.

Here is that PR: #6528

[djabarovgeorge]

Great job on the priority addon super clean solution! if the platform is not fully scalable. triggering a large number of notifications at once could lead to latency.

I have two quick questions about this pull request:

1. Am I missing any other PRs related to this feature? I'm asking because I don't see how the ParseEventRequest use case will receive the priority parameter.

2. Was there a decision made not to include this feature in the Novu Cloud solution?

[isaiahdahl]

@djabarovgeorge

Great job on the priority addon super clean solution! if the platform is not fully scalable. triggering a large number of notifications at once could lead to latency.

I have two quick questions about this pull request:

1. Am I missing any other PRs related to this feature? I'm asking because I don't see how the ParseEventRequest use case will receive the priority parameter.

There is one other PR referenced further up but it's the exact same change just against the 0.2x branch. No other PR with any more features.

The parseEventRequest is taking the payload from the API endpoint. An example for how passing priority would look like this:

import { Novu } from "@novu/node";

const novu = new Novu("<NOVU_SECRET_KEY>");

await novu.trigger("<WORKFLOW_TRIGGER_IDENTIFIER>", {
  to: {
    subscriberId: "<UNIQUE_SUBSCRIBER_IDENTIFIER>",
    email: "[email protected]",
    firstName: "John",
    lastName: "Doe",
  },
  priority: 2,
  payload: {
    name: "Hello World",
    organization: {
      logo: "https://happycorp.com/logo.png",
    },
  },
});

This give all the control to the developer who explicitly decides whether or not to make use of priority. Events triggered without setting a priority would be processed before any events that have a priority as per how bull works.

---

2. Was there a decision made not to include this feature in the Novu Cloud solution?

@SokratisVidros left these comments further up in a review comment.

Leveraging BullMQ priorities makes sense in a self hosted setup but not in a cloud deployment as jobs are multiplexed across different organizations.

I suggest adding a if (process.env.IS_DOCKER_HOSTED === 'true') { check before passing the priority option.

[isaiahdahl]

@rifont is this still waiting on you? seems like this has gone stale since I made the changes you requested.

[rifont]

We will approve and release this with the caveat as listed here: #6528 (review)

[rifont]

Needed to resolve the import order linting problem here.

Suggested change

	} from '@novu/shared';
	import { EnvironmentWithUserCommand } from '../../../shared/commands/project.command';
	import { ApiHideProperty } from '@nestjs/swagger';
	} from '@novu/shared';
	import { ApiHideProperty } from '@nestjs/swagger';
	import { EnvironmentWithUserCommand } from '../../../shared/commands/project.command';