Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Trivial Bug] Adding wildcards to index to scrape entire patterns #90

Merged
merged 1 commit into from
Apr 5, 2024

Conversation

vishnuchalla
Copy link
Collaborator

@vishnuchalla vishnuchalla commented Apr 4, 2024

Type of change

  • Refactor
  • New feature
  • Bug fix
  • Optimization
  • Documentation Update

Description

Adding wildcard pattern to scan all the indices in cases when no timestamp field is specified.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.

Testing

Tested in local by doing a sh local-compose.sh.

Before this change

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/anyio/streams/memory.py", line 98, in receive
    return self.receive_nowait()
  File "/usr/local/lib/python3.9/site-packages/anyio/streams/memory.py", line 93, in receive_nowait
    raise WouldBlock
anyio.WouldBlock

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/base.py", line 78, in call_next
    message = await recv_stream.receive()
  File "/usr/local/lib/python3.9/site-packages/anyio/streams/memory.py", line 118, in receive
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 371, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/usr/local/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 59, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 1106, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/base.py", line 108, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/backend/./app/main.py", line 54, in some_middleware
    return await call_next(request)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/base.py", line 84, in call_next
    raise app_exc
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/base.py", line 70, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/cors.py", line 91, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 274, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/backend/./app/api/v1/endpoints/ocp/graph.py", line 118, in graph
    meta = await getMetadata(uuid, 'ocp.elasticsearch')
  File "/backend/./app/api/v1/commons/utils.py", line 17, in getMetadata
    return meta[0]
IndexError: list index out of range
INFO:     10.0.2.100:57656 - "GET /api/v1/quay/jobs HTTP/1.1" 200 OK

After this change

{'query': {'query_string': {'query': 'uuid: "08b4d5bf-1c51-4abd-b80c-144edf65bd3f"'}}}
{'ciSystem': 'JENKINS', 'uuid': '08b4d5bf-1c51-4abd-b80c-144edf65bd3f', 'releaseStream': '4.13.27', 'platform': 'AWS', 'clusterType': 'self-managed', 'benchmark': 'amadeus-usecase', 'masterNodesCount': 3, 'workerNodesCount': 100, 'infraNodesCount': 3, 'masterNodesType': 'm5.4xlarge', 'workerNodesType': 'm5.xlarge', 'infraNodesType': 'r5.2xlarge', 'totalNodesCount': 107, 'clusterName': 'liqcui-amadeus13-vqcpr', 'ocpVersion': '4.13.27', 'networkType': 'OVNKubernetes', 'buildTag': '282', 'jobStatus': 'success', 'buildUrl': 'https://mastern-jenkins-csb-openshift-qe.apps.ocp-c1.prod.psi.redhat.com/job/scale-ci/job/liqcui-e2e-benchmarking-multibranch-pipeline/job/kube-burner/282/', 'upstreamJob': 'kube-burner', 'upstreamJobBuild': '', 'executionDate': '2024-01-02T03:47:41Z', 'jobDuration': '3357', 'startDate': '2024-01-02T03:47:41Z', 'endDate': '2024-01-02T04:43:38Z', 'timestamp': '2024-01-02T03:47:41Z'}
{'query': {'bool': {'must': [{'query_string': {'query': 'benchmark: "amadeus-usecase$" AND workerNodesType: "m5.xlarge" AND masterNodesType: "m5.4xlarge" AND masterNodesCount: "3" AND workerNodesCount: "100" AND platform: "AWS" AND ocpVersion: 4.13* AND jobStatus: success'}}]}}}
{'query': {'query_string': {'query': '( uuid: "08b4d5bf-1c51-4abd-b80c-144edf65bd3f" ) AND metricName: "jobSummary"'}}}
{'query': {'query_string': {'query': '( uuid: "08b4d5bf-1c51-4abd-b80c-144edf65bd3f" OR uuid: "108141b2-471f-4226-a05b-7f32f27a7fe4" ) AND metricName: "jobSummary"'}}}
08b4d5bf-1c51-4abd-b80c-144edf65bd3f
{'query': {'query_string': {'query': '( uuid: "08b4d5bf-1c51-4abd-b80c-144edf65bd3f" ) AND metricName: "podLatencyQuantilesMeasurement" AND quantileName: "Ready"'}}}
[{'P50': 8793,
  'P95': 40252,
  'P99': 49540,
  'avg': 18464,
  'jobName': 'amadeus-mgob-usecase',
  'max': 79895,
  'metricName': 'podLatencyQuantilesMeasurement',
  'quantileName': 'Ready',
  'timestamp': '2024-01-02T04:13:57.032357642Z',
  'uuid': '08b4d5bf-1c51-4abd-b80c-144edf65bd3f'}]
  quantileName  ...               jobName
0        Ready  ...  amadeus-mgob-usecase

[1 rows x 10 columns]
08b4d5bf-1c51-4abd-b80c-144edf65bd3f
{'query': {'query_string': {'query': '( uuid: "08b4d5bf-1c51-4abd-b80c-144edf65bd3f" ) AND metricName: "podLatencyQuantilesMeasurement" AND quantileName: "Ready"'}}}
[{'P50': 8793,
  'P95': 40252,
  'P99': 49540,
  'avg': 18464,
  'jobName': 'amadeus-mgob-usecase',
  'max': 79895,
  'metricName': 'podLatencyQuantilesMeasurement',
  'quantileName': 'Ready',
  'timestamp': '2024-01-02T04:13:57.032357642Z',
  'uuid': '08b4d5bf-1c51-4abd-b80c-144edf65bd3f'}]
  quantileName  ...               jobName
0        Ready  ...  amadeus-mgob-usecase

[1 rows x 10 columns]
INFO:     10.0.2.100:44700 - "GET /api/v1/ocp/graph/08b4d5bf-1c51-4abd-b80c-144edf65bd3f HTTP/1.1" 200 OK

Copy link
Member

@chentex chentex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chentex chentex merged commit afd14b8 into cloud-bulldozer:main Apr 5, 2024
1 check passed
@shashank-boyapally
Copy link

/lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants