Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Errors when trying to count_over_time using Grafana #585

Closed
covalschi opened this issue Oct 7, 2024 · 2 comments
Closed

Errors when trying to count_over_time using Grafana #585

covalschi opened this issue Oct 7, 2024 · 2 comments

Comments

@covalschi
Copy link

covalschi commented Oct 7, 2024

Description:

Executing count_over_time({job="heplify-server"}[5m]) using Grafana 11.2 results in 500 Internal Server Error from qryn.

Full query parameters:

{
  "request": {
    "url": "api/ds/query?ds_type=loki&requestId=explore_law_1",
    "method": "POST",
    "data": {
      "queries": [
        {
          "refId": "A",
          "expr": "count_over_time({job=\"heplify-server\"}[5m])",
          "queryType": "range",
          "datasource": {
            "type": "loki",
            "uid": "ee0567uwmc9a8f"
          },
          "editorMode": "code",
          "step": "",
          "legendFormat": "",
          "datasourceId": 171,
          "intervalMs": 1000,
          "maxDataPoints": 2523
        }
      ],
      "from": "1728328890000",
      "to": "1728332490102"
    },
    "hideFromInspector": false
  },
  "response": {
    "results": {
      "A": {
        "error": "Internal Server Error",
        "errorSource": "downstream",
        "status": 500,
        "refId": "A"
      }
    }
  }
}

Notes:

  • Removing count_over_time returns logs as usual (valid data)
  • Adding post-processing, like count_over_time({job="heplify-server"} |~ `$search` [$__interval]), also removes the error (did not check for data validity)
  • Switching query type from "Range" to "Instant" in Grafana UI also removes the error (did not check for data validity)
  • Issue not present in [email protected], [email protected] demonstrates similar behaviour

System settings:
qryn+clickhouse server

sse4.2 supported, proxmox virtual machine
6.8.0-45-generic
Ubuntu 24.04.1 LTS
ClickHouse local version 24.9.2.42 (official build). (via clickhouse repository)
node v18.19.1
[email protected] (intalled via npm as global package)
NODE_PORT=3100
CLICKHOUSE_SERVER="localhost"
CLICKHOUSE_AUTH="default:PASSWD" (sha256)
CLICKHOUSE_DB="qryn"
NODE_OPTIONS="--max-old-space-size=8192"
LOG_LEVEL="debug"

grafana server

proxmox virtual machine
5.4.0-174-generic
Ubuntu 20.04.6 LTS
Grafana Version 11.2.2 (commit: c47b921ef40c0c411e9bad2cac05eb6c540c0979, branch: HEAD) (recently updated from 10.4 via Grafana apt repo)
@akvlad
Copy link
Collaborator

akvlad commented Oct 9, 2024

fixed in 3.2.34 #587
Feel free to try.

@akvlad
Copy link
Collaborator

akvlad commented Oct 14, 2024

closing issue as fixed.

@akvlad akvlad closed this as completed Oct 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants