feat: background pooler with retry

[CermakM]

This PR shifts the initial implementation towards a backend pooler
with retry functionality.

The assumptions and implementation decisions have been made when implementing the pooler:

• the background manager's primary purpose is to run (ideally)
  permananent background goroutines, as such, there is no need for
  lifecycle hooks such as OnTaskSucceeded or onTaskFailed, those were
  removed in favor of the retrier.
• the manager is context-based and all go routines spawned by the
  manager share a single cancellable context (cancellable independantly
  of their parent context)
• when stopped or cancelled, the conc pooler gracefully shuts down
  dangling goroutines
• by default, the pooler returns joined errors, this behaviour can be
  modifier with WithFirstError Option.

Example usage:

	ctx := context.Background()
	// Create a new manager.
	// The manager will cancel its context and all its tasks if any of the tasks returns an error.
	// The manager will return the first error encountered by any of its tasks.
	backgroundManager := manager.New(ctx, manager.WithCancelOnError(), manager.WithFirstError())
	//nolint
	backgroundManager.Run(
		func(ctx context.Context) error {
			select {
			case <-ctx.Done():
				return ctx.Err()
			case <-time.After(2 * time.Second):
				// Fail after 2 seconds.
				return context.DeadlineExceeded
			}
		},
	)
	backgroundManager.Run(
		func(ctx context.Context) error {
			select {
			case <-ctx.Done():
				return ctx.Err()
			case <-time.After(5 * time.Second):
				fmt.Printf("This won't be executed\n")
			}
		},
	)
	if err := backgroundManager.Wait(); err != nil {
		//nolint
		fmt.Printf("Error: %v\n", err) // Output: Error: context deadline exceeded
	}

[Fazt01]

First impression it doesn't really do much either, but will review more thoroughly tomorrow.
Just found 1 obvious point.

[Fazt01]

I see a race condition here (not atomic operation).
simple: don't have stat (no adding and subtracting 1)
less-but-still-simple: add mutex or use atomic operation

[Fazt01]

Even after more thorough review, I still think that this approach doesn't really add value.
Most of the functionality is in options which:

• forwards 3 of the conc options
• forwards 2 of the retrier options.

If we consider the use case of long running tasks (e.g. SQS handlers), I don't really want WithCancelOnError or WithFirstError. If I know all the tasks that will need to run at the start, I also don't want WithMaxTasks. Considering the retriers: in SQS handler I'll have a for (while true) loop, that will probably handle any errors and retries in its body - I don't really expect it to ever return an error (maybe apart from context.Cancelled) - maybe panic could happen, which is the only think that the retrier would help with.

My possible idea for a "feature request" would be to handle such outer loop of for (while true): user would provide an inner body of that loop (e.g. read from SQS and handle the message), optionally return and error. This package could then reasonably handle retries (even with infinite attempts - why not) - wait a bit on error, rerun immediately on success (read next message).

Then, a follow-up "feature-request" would be to gracefully complete the iteration on .Stop(): consider .Stop() is called on SIGTERM - the SQS handler should have a possibility to (without cancelling the ctx) complete the iteration (handle all db and external-api stuff) - hopefuly within few seconds. But next iteration would not start - there would be no possible context.Cancelled when doing db or external-api stuff.

However, even with these features, it would most likely still be less ergonomic to include such module, than to handle these in the app itself - not sure without seeing such potential usage.

I prefer this version infinitesimally to the master version. I prefer nothing infinitely to either version.

[Fazt01]

I don't see a reason to add all these, apart from .idea and .vscode.
For example, .env.test is something that we would want to have in repo, right? (if it did exist at all)

[Fazt01]

I think it would be simpler to have just type Option func(m *Manager) - most options could be simplified to inline the function. And there isn't really any other reasonable method into this interface 🤔

[TomasKocman]

Functional options pattern is implemented here for example