Incorrect Metric Values in KEDA HPA Status

[adityalifi]

Report

The scaling behavior is working correctly, but in the status section of the KEDA-managed HPA, I am seeing different metric values than expected. Specifically:

Instead of _active_fastify_connections, the status is showing values in milliCPU, which is incorrect.
The expected metric should reflect active Fastify connections, not CPU usage.

This is my keda scaled object configurations

Expected Behavior

The HPA should correctly display _active_fastify_connections in the status, matching the metric used in the ScaledObject.

Actual Behavior

The HPA status is reporting values in milliCPU instead of the correct external metric.

Steps to Reproduce the Problem

1. Define a ScaledObject with a Prometheus trigger using _active_fastify_connections.
2. Set a threshold of 4 for scaling.
3. Observe the status section of the KEDA HPA.

Logs from KEDA operator

2025-01-30T09:20:12Z	INFO	Detected resource targeted for scaling	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"scaler-v1-advanced-steptransaction","namespace":"backend"}, "namespace": "backend", "name": "scaler-v1-advanced-steptransaction", "reconcileID": "55ccd8d4-c42e-4b88-bc46-7a39fe1822e2", "resource": "apps/v1.Deployment", "name": "api-v1-advanced-steptransaction"}
2025-01-30T09:20:13Z	INFO	Creating a new HPA	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"scaler-v1-advanced-steptransaction","namespace":"backend"}, "namespace": "backend", "name": "scaler-v1-advanced-steptransaction", "reconcileID": "55ccd8d4-c42e-4b88-bc46-7a39fe1822e2", "HPA.Namespace": "backend", "HPA.Name": "keda-hpa-scaler-v1-advanced-steptransaction"}
2025-01-30T09:20:13Z	INFO	Initializing Scaling logic according to ScaledObject Specification	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"scaler-v1-advanced-steptransaction","namespace":"backend"}, "namespace": "backend", "name": "scaler-v1-advanced-steptransaction", "reconcileID": "55ccd8d4-c42e-4b88-bc46-7a39fe1822e2"}
2025-01-30T09:20:13Z	INFO	Reconciling ScaledObject	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"scaler-v1-advanced-steptransaction","namespace":"backend"}, "namespace": "backend", "name": "scaler-v1-advanced-steptransaction", "reconcileID": "a86831b2-d811-469c-94b7-991438f4b7c5"}
2025-01-30T09:20:13Z	INFO	Detected resource targeted for scaling	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"scaler-v1-advanced-steptransaction","namespace":"backend"}, "namespace": "backend", "name": "scaler-v1-advanced-steptransaction", "reconcileID": "a86831b2-d811-469c-94b7-991438f4b7c5", "resource": "apps/v1.Deployment", "name": "api-v1-advanced-steptransaction"}
2025-01-30T09:20:16Z	INFO	Reconciling ScaledObject	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"scaler-v1-quote","namespace":"backend"}, "namespace": "backend", "name": "scaler-v1-quote", "reconcileID": "e58c872a-9d47-4f08-875a-2368d536b36f"}
2025-01-30T09:20:16Z	INFO	Detected resource targeted for scaling	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"scaler-v1-quote","namespace":"backend"}, "namespace": "backend", "name": "scaler-v1-quote", "reconcileID": "e58c872a-9d47-4f08-875a-2368d536b36f", "resource": "apps/v1.Deployment", "name": "api-v1-quote"}
2025-01-30T09:20:16Z	INFO	Creating a new HPA	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"scaler-v1-quote","namespace":"backend"}, "namespace": "backend", "name": "scaler-v1-quote", "reconcileID": "e58c872a-9d47-4f08-875a-2368d536b36f", "HPA.Namespace": "backend", "HPA.Name": "keda-hpa-scaler-v1-quote"}
2025-01-30T09:20:16Z	INFO	Initializing Scaling logic according to ScaledObject Specification	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"scaler-v1-quote","namespace":"backend"}, "namespace": "backend", "name": "scaler-v1-quote", "reconcileID": "e58c872a-9d47-4f08-875a-2368d536b36f"}
2025-01-30T09:20:16Z	INFO	Reconciling ScaledObject	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"scaler-v1-quote","namespace":"backend"}, "namespace": "backend", "name": "scaler-v1-quote", "reconcileID": "e2f62053-f57c-4e7b-9261-0752db333002"}
2025-01-30T09:20:16Z	INFO	Detected resource targeted for scaling	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"scaler-v1-quote","namespace":"backend"}, "namespace": "backend", "name": "scaler-v1-quote", "reconcileID": "e2f62053-f57c-4e7b-9261-0752db333002", "resource": "apps/v1.Deployment", "name": "api-v1-quote"}
2025-01-30T09:21:47Z	INFO	cert-rotation	no cert refresh needed
2025-01-30T09:21:47Z	INFO	cert-rotation	Ensuring CA cert	{"name": "keda-admission", "gvk": "admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration", "name": "keda-admission", "gvk": "admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration"}
2025-01-30T09:21:47Z	INFO	cert-rotation	Ensuring CA cert	{"name": "v1beta1.external.metrics.k8s.io", "gvk": "apiregistration.k8s.io/v1, Kind=APIService", "name": "v1beta1.external.metrics.k8s.io", "gvk": "apiregistration.k8s.io/v1, Kind=APIService"}
2025-01-30T10:59:07Z	INFO	cert-rotation	no cert refresh needed
2025-01-30T10:59:07Z	INFO	cert-rotation	Ensuring CA cert	{"name": "keda-admission", "gvk": "admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration", "name": "keda-admission", "gvk": "admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration"}
2025-01-30T10:59:07Z	INFO	cert-rotation	Ensuring CA cert	{"name": "v1beta1.external.metrics.k8s.io", "gvk": "apiregistration.k8s.io/v1, Kind=APIService", "name": "v1beta1.external.metrics.k8s.io", "gvk": "apiregistration.k8s.io/v1, Kind=APIService"}

This is how I am getting the values

KEDA Version

2.16.1

Kubernetes Version

1.30

Platform

Amazon Web Services

Scaler Details

prometheus scale

Anything else?

kind: HorizontalPodAutoscaler
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"keda.sh/v1alpha1","kind":"ScaledObject","metadata":{"annotations":{},"labels":{"app.kubernetes.io/instance":"lifi-backend-staging-eu-central-1","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"lifi-backend","app.kubernetes.io/part-of":"lifi-backend","app.kubernetes.io/version":"1.0.0","argocd.argoproj.io/instance":"lifi-backend-staging-eu-central-1","helm.sh/chart":"lifi-backend"},"name":"scaler-v1-advanced-routes","namespace":"backend"},"spec":{"advanced":{"horizontalPodAutoscalerConfig":{"behavior":{"scaleDown":{"policies":[{"periodSeconds":30,"type":"Pods","value":1}],"selectPolicy":"Max","stabilizationWindowSeconds":180},"scaleUp":{"policies":[{"periodSeconds":30,"type":"Pods","value":2}],"stabilizationWindowSeconds":0}}}},"cooldownPeriod":180,"maxReplicaCount":10,"minReplicaCount":3,"pollingInterval":15,"scaleTargetRef":{"apiVersion":"apps/v1","kind":"Deployment","name":"api-v1-advanced-routes"},"triggers":[{"metadata":{"metricName":"v1-advanced-routes_active_fastify_connections","query":"max(lifi_backend_http_active_fastify_connections{namespace=\"backend\",
      pod=~\"api-v1-advanced-routes-.*\"})","serverAddress":"http://prometheus-staging-eu-cent-prometheus.prometheus.svc.cluster.local:9090","threshold":"4"},"type":"prometheus"}]}}
  creationTimestamp: '2025-01-30T09:20:09Z'
  labels:
    app.kubernetes.io/instance: lifi-backend-staging-eu-central-1
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: lifi-backend
    app.kubernetes.io/part-of: lifi-backend
    app.kubernetes.io/version: 1.0.0
    argocd.argoproj.io/instance: lifi-backend-staging-eu-central-1
    helm.sh/chart: lifi-backend
    scaledobject.keda.sh/name: scaler-v1-advanced-routes
  name: keda-hpa-scaler-v1-advanced-routes
  namespace: backend
  ownerReferences:
    - apiVersion: keda.sh/v1alpha1
      blockOwnerDeletion: true
      controller: true
      kind: ScaledObject
      name: scaler-v1-advanced-routes
      uid: 43bcde07-c818-4ca1-b561-14b3d50c2363
  resourceVersion: '336951337'
  uid: 5448494b-c0b5-4163-94cc-709a150c0528
spec:
  behavior:
    scaleDown:
      policies:
        - periodSeconds: 30
          type: Pods
          value: 1
      selectPolicy: Max
      stabilizationWindowSeconds: 180
    scaleUp:
      policies:
        - periodSeconds: 30
          type: Pods
          value: 2
      selectPolicy: Max
      stabilizationWindowSeconds: 0
  maxReplicas: 10
  metrics:
    - external:
        metric:
          name: s0-prometheus
          selector:
            matchLabels:
              scaledobject.keda.sh/name: scaler-v1-advanced-routes
        target:
          averageValue: '4'
          type: AverageValue
      type: External
  minReplicas: 3
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-v1-advanced-routes
status:
  conditions:
    - lastTransitionTime: '2025-01-30T09:20:11Z'
      message: recommended size matches current size
      reason: ReadyForNewScale
      status: 'True'
      type: AbleToScale
    - lastTransitionTime: '2025-01-30T09:20:11Z'
      message: >-
        the HPA was able to successfully calculate a replica count from external
        metric
        s0-prometheus(&LabelSelector{MatchLabels:map[string]string{scaledobject.keda.sh/name:
        scaler-v1-advanced-routes,},MatchExpressions:[]LabelSelectorRequirement{},})
      reason: ValidMetricFound
      status: 'True'
      type: ScalingActive
    - lastTransitionTime: '2025-01-30T09:20:11Z'
      message: the desired replica count is less than the minimum replica count
      reason: TooFewReplicas
      status: 'True'
      type: ScalingLimited
  currentMetrics:
    - external:
        current:
          averageValue: 667m
        metric:
          name: s0-prometheus
          selector:
            matchLabels:
              scaledobject.keda.sh/name: scaler-v1-advanced-routes
      type: External
  currentReplicas: 3
  desiredReplicas: 3