Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report aeflex services discovered before errors #37

Open
stephen-soltesz opened this issue Jul 27, 2022 · 0 comments
Open

Report aeflex services discovered before errors #37

stephen-soltesz opened this issue Jul 27, 2022 · 0 comments

Comments

@stephen-soltesz
Copy link
Contributor

The current logic in the aeflex service discovery collects services incrementally. Unfortunately, if any one encounters an error, all services discovered up to that point are ignored and only the error is returned.

This behavior was observed in production, when metrics from the locate AEFlex service were not discovered b/c of an error scanning the "locate-platform" service.

Ideally, the discovery process could log errors, and report any previously discovered services.

2022/07/27 16:29:04 apps/mlab-sandbox/services/default versions: 6 active: 0 inactive: 0
2022/07/27 16:29:05 apps/mlab-sandbox/services/locate versions: 2 active: 2 inactive: 0
2022/07/27 16:29:05 apps/mlab-sandbox/services/locate-platform versions: 79 active: 0 inactive: 0
2022/07/27 16:29:05 Error: *aeflex.Service: googleapi: Error 400: The requested Compute resource is not found.
Details:
[
  {
    "@type": "type.googleapis.com/google.rpc.PreconditionFailure",
    "violations": [
      {
        "description": "Request to https://compute.googleapis.com/compute/v1/projects/projects/mlab-sandbox/regions/us-east1/instanceGroupManagers/aef-locate--platform-20220420t140553-00/listManagedInstances?key\u0026maxResults=50 failed, details: The resource 'projects/mlab-sandbox/regions/us-east1/instanceGroupManagers/aef-locate--platform-20220420t140553-00' was not found",
        "subject": "COMPUTE",
        "type": "RESOURCE_NOT_FOUND"
      }
    ]
  }
]
, failedPrecondition

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant