Observability Laboratory

This repository contains a docker compose configuration that will make possible for you to develop grafana dashboards with a prometheus datasource.

One of the hardest things when trying to learn how prometheus works is having consistent metrics. All the exporters are different, you have to configure them, etc. That's why in this laboratory the metrics can be created arbitrarly, even though there's some exporters and instrumented programs.

The fake metrics are done with prometheus-data-generator. It's a fake Prometheus exporter that let's you create the metrics.

The laboratory consists in:

• grafana
• prometheus
• prometheus-data-generator
• python and java example microservices
• linux node exporter
• blackbox exporter
• confimap-reloader

Prometheus

Generate Metrics

You can generate arbitrary metrics. To do so, you may edit the prometheus-data-generator/config.yml to configure it. The changes will be detected live, so you don't have to do anything more than edit it. You may get more details on how to configure this exporter in it's repository.

Alerts

You may edit the prometheus/rules.yml. It will also be detected automatically and prometheus will reload its configuration.

Grafana

It will import any json dashboard that grafana detects in the grafana/dashboards/ directory. The dashboards will be imported and detected automatically.

It has the grafonnet)submodule, in case you want to generate the dashboards with it. The blackbox dashboard is an example of it.

Linux node exporter

There's a grafana dashboard.

Windows node exporter

It can't be dockerized, so there's none.

Microservices

Python

There's two python microservices, one with plain flask and the other with pyms.

Java

There's a java microservice with SpringBoot.

Blackbox exporter

Used to make requests to the microservices and create metrics.

How to use it

Clone the repository (with the recursive parameter if you want grafonnet):

git clone --recursive https://github.com/little-angry-clouds/observability-laboratory

Launch the containers:

docker-compose up

And that's it. You only have to edit the metrics, the rules and the dashboards and access the services:

• Grafana: Default credentials are admin:admin
• Prometheus
• Prometheus Data Generator
• Linux Node Exporter
• Python Flask
• Python Pyms
• Java Springboot
• Blackbox Exporter