Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust prometheus query to use token #520

Merged
merged 14 commits into from
Dec 12, 2023
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ ifdef::include_when_16[]
//include::../modules/proc_overriding-the-default-grafana-container-image.adoc[leveloffset=+2]
include::../modules/proc_importing-dashboards.adoc[leveloffset=+2]
endif::include_when_16[]
include::../modules/proc_connecting-an-external-dashboard-system.adoc[leveloffset=+2]

//Editing the metrics retention time period
include::../modules/con_metrics-retention-time-period.adoc[leveloffset=+1]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,18 @@
[role="_abstract"]
In {OpenShift}, applications are exposed to the external network through a route. For more information about routes, see https://docs.openshift.com/container-platform/{NextSupportedOpenShiftVersion}/networking/configuring_ingress_cluster_traffic/overview-traffic.html[Configuring ingress cluster traffic].

In {Project} ({ProjectShort}), HTTPS routes are exposed for each service that has a web-based interface. These routes are protected by {OpenShift} RBAC and any user that has a `ClusterRoleBinding` that enables them to view {OpenShift} Namespaces can log in. For more information about RBAC, see https://docs.openshift.com/container-platform/{NextSupportedOpenShiftVersion}/authentication/using-rbac.html[Using RBAC to define and apply permissions].
In {Project} ({ProjectShort}), HTTPS routes are exposed for each service that has a web-based interface. These routes are protected by {OpenShift} RBAC and any user that has read access to the component can log in.
csibbitt marked this conversation as resolved.
Show resolved Hide resolved

Specifically, the following permissions are required in order to access the corresponding component UIs:
csibbitt marked this conversation as resolved.
Show resolved Hide resolved

[source,json,options="nowrap"]
----
{"namespace":"service-telemetry", "resource":"grafana", "group":"integreatly.org", "verb":"get"}
{"namespace":"service-telemetry", "resource":"prometheus", "group":"monitoring.rhobs", "verb":"get"}
{"namespace":"service-telemetry", "resource":"alertmanager", "group":"monitoring.rhobs", "verb":"get"}
----

For more information about RBAC, see https://docs.openshift.com/container-platform/{NextSupportedOpenShiftVersion}/authentication/using-rbac.html[Using RBAC to define and apply permissions].

.Procedure

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@

[id="connecting-an-external-dashboard-system_{context}"]
= Connecting an external dashboard system

It is possible to configure third-party visualization tools to connect to the {ProjectShort} Prometheus for metrics retrieval. Access is controlled via an OAuth token, and a ServiceAccount is already created that has (only) the required permissions. A new OAuth token can be generated against this account for the external system to use.

To use the authentication token, the third-party tool must be configured to supply an HTTP Bearer Token Authorization header as described in RFC6750. Consult the documentation of the third-party tool for how to configure this header. For example link:https://grafana.com/docs/grafana/latest/datasources/prometheus/configure-prometheus-data-source/#custom-http-headers[Configure Prometheus - Custom HTTP Headers] in the _Grafana Documentation_.

.Procedure

. Log in to {OpenShift}.

. Change to the `service-telemetry` namespace:
+
[source,bash]
----
$ oc project service-telemetry
----

. Create a new token secret for the stf-prometheus-reader service account
+
[source,bash]
----
$ oc create -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
name: my-prometheus-reader-token
namespace: service-telemetry
annotations:
kubernetes.io/service-account.name: stf-prometheus-reader
type: kubernetes.io/service-account-token
EOF
----

. Retrieve the token from the secret
+
[source,bash]
----
$ TOKEN=$(oc get secret my-prometheus-reader-token -o template='{{.data.token}}' | base64 -d)
----

. Retrieve the Prometheus host name
+
[source,bash]
----
$ PROM_HOST=$(oc get route default-prometheus-proxy -ogo-template='{{ .spec.host }}')
----

. Test the access token
+
[source,bash]
----
$ curl -k -H "Authorization: Bearer ${TOKEN}" https://${PROM_HOST}/api/v1/query?query=up

{"status":"success",[...]
----

. Configure your third-party tool with the PROM_HOST and TOKEN values from above
+
[source,bash]
----
$ echo $PROM_HOST
$ echo $TOKEN
----

. The token remains valid as long as the secret exists. You can revoke the token by deleting the secret.
csibbitt marked this conversation as resolved.
Show resolved Hide resolved
+
[source,bash]
----
$ oc delete secret my-prometheus-reader-token
secret "my-prometheus-reader-token" deleted
----

.Additional information

For more information about service account token secrets, see link:https://docs.openshift.com/container-platform/{NextSupportedOpenShiftVersion}/nodes/pods/nodes-pods-secrets.html#nodes-pods-secrets-creating-sa_nodes-pods-secrets[Creating a service account token secret] in the _OpenShift Container Platform Documentation_.
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ To change the rule, edit the value of the `expr` parameter.
+
[source,bash,options="nowrap"]
----
$ curl -k --user "internal:$(oc get secret default-prometheus-htpasswd -ogo-template='{{ .data.password | base64decode }}')" https://$(oc get route default-prometheus-proxy -ogo-template='{{ .spec.host }}')/api/v1/rules
$ curl -k -H "Authorization: Bearer $(oc create token stf-prometheus-reader)" https://$(oc get route default-prometheus-proxy -ogo-template='{{ .spec.host }}')/api/v1/rules
{"status":"success","data":{"groups":[{"name":"./openstack.rules","file":"/etc/prometheus/rules/prometheus-default-rulefiles-0/service-telemetry-prometheus-alarm-rules.yaml","rules":[{"state":"inactive","name":"Collectd metrics receive count is zero","query":"rate(sg_total_collectd_msg_received_count[1m]) == 0","duration":0,"labels":{},"annotations":{},"alerts":[],"health":"ok","evaluationTime":0.00034627,"lastEvaluation":"2021-12-07T17:23:22.160448028Z","type":"alerting"}],"interval":30,"evaluationTime":0.000353787,"lastEvaluation":"2021-12-07T17:23:22.160444017Z"}]}}
----
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -73,4 +73,6 @@ grafana-route-service-telemetry.apps.infra.watch

. In a web browser, navigate to https://_<grafana_route_address>_. Replace _<grafana_route_address>_ with the value that you retrieved in the previous step.

. Log in with OpenShift credentials. For more information about logging in, see xref:accessing-uis-for-stf-components_assembly-installing-the-core-components-of-stf[].

. To view the dashboard, click *Dashboards* and *Manage*.