Skip to content
This repository has been archived by the owner on Oct 14, 2024. It is now read-only.

MG-234 - Improve Logging Middleware #272

Merged
merged 23 commits into from
Jan 24, 2024
Merged

Conversation

Musilah
Copy link
Contributor

@Musilah Musilah commented Jan 15, 2024

What type of PR is this?

This is an optimisation for Logging.

What does this do?

This improves Logging by updating the filtering and searchability through structured logs.

Which issue(s) does this PR fix/relate to?

Have you included tests for your changes?

No.

Did you document any new/modified feature?

No.

Notes

N/A.

bootstrap/api/logging.go Outdated Show resolved Hide resolved
@Musilah Musilah marked this pull request as ready for review January 16, 2024 22:28
users/api/logging.go Outdated Show resolved Hide resolved
users/api/logging.go Outdated Show resolved Hide resolved
users/api/logging.go Outdated Show resolved Hide resolved
@Musilah Musilah force-pushed the Middleware branch 3 times, most recently from 340a646 to 0525973 Compare January 19, 2024 09:47
@dborovcanin dborovcanin force-pushed the Middleware branch 3 times, most recently from 2142349 to 1db7b76 Compare January 21, 2024 15:20
if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s.", message, err))
args = append(args, slog.String("error", err.Error()))
lm.logger.Error("Register client failed to complete successfully", args...)
Copy link
Contributor

@dborovcanin dborovcanin Jan 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use Warn. The correct log level, however, can only be set properly by inspecting the error; we'll not do that in this PR.

if err != nil {
lm.logger.Warn(fmt.Sprintf("%s with error: %s.", message, err))
args = append(args, slog.String("error", err.Error()))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
args = append(args, slog.String("error", err.Error()))
args = append(args, slog.Any("error", err))

Do this for the rest. It returns a better error format

Copy link
Member

@rodneyosodo rodneyosodo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have reviewed logging in ws, users, things, twins, readers, provision and messaging use that information to change the other services

Also have a look at this case

func (lm *loggingMiddleware) Cert(token, thingID, duration string) (cert, key string, err error) {
	defer func(begin time.Time) {
		args := []interface{}{
			slog.String("duration", time.Since(begin).String()),
			slog.String("thing_id", thingID),
			slog.String("duration", duration),
		}
		if err != nil {
			args = append(args, slog.String("error", err.Error()))
			lm.logger.Warn("Cert failed to complete successfully", args...)
			return
		}
		lm.logger.Info("Cert completed successfully", args...)
	}(time.Now())

	return lm.svc.Cert(token, thingID, duration)
}

This is found in provision. This logs is not informative or clear on what it does Cert completed successfully because the underlying function creates a certificate. Use go docs to infer what a function does and make the log statemes clear and concise as to what is happening

ws/api/logging.go Outdated Show resolved Hide resolved
ws/api/logging.go Outdated Show resolved Hide resolved
ws/api/logging.go Outdated Show resolved Hide resolved
users/api/logging.go Outdated Show resolved Hide resolved
users/api/logging.go Outdated Show resolved Hide resolved
readers/api/logging.go Outdated Show resolved Hide resolved
provision/api/logging.go Outdated Show resolved Hide resolved
provision/api/logging.go Outdated Show resolved Hide resolved
pkg/messaging/handler/logging.go Outdated Show resolved Hide resolved
pkg/messaging/handler/logging.go Outdated Show resolved Hide resolved
Copy link
Member

@rodneyosodo rodneyosodo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some packages have not been updated for example consumers/writers/api/logging.go

auth/api/logging.go Outdated Show resolved Hide resolved
auth/api/logging.go Show resolved Hide resolved
bootstrap/api/logging.go Outdated Show resolved Hide resolved
certs/api/logging.go Outdated Show resolved Hide resolved
coap/api/logging.go Show resolved Hide resolved
consumers/notifiers/api/logging.go Outdated Show resolved Hide resolved
lora/api/logging.go Outdated Show resolved Hide resolved
lora/api/logging.go Outdated Show resolved Hide resolved
auth/api/logging.go Outdated Show resolved Hide resolved
auth/api/logging.go Outdated Show resolved Hide resolved
auth/api/logging.go Show resolved Hide resolved
auth/api/logging.go Outdated Show resolved Hide resolved
auth/api/logging.go Show resolved Hide resolved
@@ -28,314 +27,533 @@ func LoggingMiddleware(svc auth.Service, logger *slog.Logger) auth.Service {

func (lm *loggingMiddleware) ListObjects(ctx context.Context, pr auth.PolicyReq, nextPageToken string, limit int32) (p auth.PolicyPage, err error) {
defer func(begin time.Time) {
message := fmt.Sprintf("Method list_objects took %s to complete", time.Since(begin))
args := []interface{}{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use any instead of interface{}.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do the same for all the other cases.

Comment on lines 49 to 56
slog.Group(
"object",
slog.String("type", pr.ObjectType),
),
slog.Group(
"subject",
slog.String("id", pr.Subject),
slog.String("type", pr.SubjectType),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's flatten Policy by using a single Group here, like:

Suggested change
slog.Group(
"object",
slog.String("type", pr.ObjectType),
),
slog.Group(
"subject",
slog.String("id", pr.Subject),
slog.String("type", pr.SubjectType),
slog.Group(
"policy_request",
slog.String("object_type", pr.ObjectType),
slog.String("subject_id", pr.Subject),
slog.String("subject_type", pr.SubjectType))

users/api/logging.go Outdated Show resolved Hide resolved
users/api/logging.go Outdated Show resolved Hide resolved
users/api/logging.go Outdated Show resolved Hide resolved
twins/api/logging.go Outdated Show resolved Hide resolved
twins/api/logging.go Outdated Show resolved Hide resolved
certs/api/logging.go Show resolved Hide resolved
certs/api/logging.go Outdated Show resolved Hide resolved
certs/api/logging.go Show resolved Hide resolved
bootstrap/api/logging.go Outdated Show resolved Hide resolved
bootstrap/api/logging.go Outdated Show resolved Hide resolved
slog.String("type", pr.ObjectType),
),
slog.Group("subject",
slog.String("id", pr.Subject),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to make this user ID, rather than the token. Let's open a ticket for this, it's out of the scope for this PR.

@dborovcanin dborovcanin merged commit a1a6701 into absmach:main Jan 24, 2024
4 checks passed
dborovcanin pushed a commit that referenced this pull request Jan 24, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants