Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
  • Loading branch information
mattyx14 committed Dec 13, 2023
1 parent 9a503fe commit 9519cee
Show file tree
Hide file tree
Showing 70 changed files with 1,856 additions and 813 deletions.
65 changes: 65 additions & 0 deletions metrics/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Canary Metrics (OpenTelemetry)

By default, no metrics are collected or exported. To enable metrics, you must setup a metrics exporter. The following example shows how to setup a Prometheus exporter.

config.lua

```lua
metricsEnablePrometheus = true
metricsPrometheusAddress = "0.0.0.0:9464"
```

This, in and of itself will expose a Prometheus endpoint at `http://localhost:9464/metrics`. However, you will need to configure Prometheus to scrape this endpoint.

The easiest, batteries included way, to do this is using the provided `docker-compose.yml` file provided in this `metrics` directory. Simply run `docker-compose up` and you will have a Prometheus instance running and scraping the Canary metrics endpoint.

The `docker-compose.yml` file also includes a Grafana instance that is preconfigured to use the Prometheus instance as a data source. The Grafana instance is exposed at `http://localhost:3000` and the default username and password are `admin` and `admin` respectively (you will be prompted to change the password on first login).

## Usage

This is an **advanced** feature. While you can simply enable OStream and get metrics in your logs, that is not recommended to do in production. Prometheus can be run efficiently in production with minimal impact to server performance.

_Enabling OStream:_

```config.lua
metricsEnableOstream = true
metricsOstreamInterval = 1000
```

If you **don't** how what Prometheus and Grafana are, you need to learn that first: https://prometheus.io/ is your starting point. You can come back to this feature once you've understood how to install and run this software.

## Metrics

We export all kinds of metrics, but the most important ones are:

Here's an interactive demo of a dashboard from a real production server: https://snapshots.raintank.io/dashboard/snapshot/bpiq45inK3I2Xixa2d7oNHWekdiDE6zr

- Latency metrics for C++ methods
- Latency metrics for Lua functions
- Latency metrics for SQL queries
- Latency metrics for Dispatcher tasks
- Latency metrics for DB Lock contention

**Screenshot**
![grafana](https://github.com/opentibiabr/canary/assets/223760/b307c335-9af9-4c1a-bf7e-5c3dc86a016d)

## Analytics

We also export analytic event, counters and other useful data. This is useful for debugging and understanding the behavior of the server. Some interesting ones are:

- Stats around monsters killed (per monster type, player, etc)
- Stats around raw exp and total exp gained
- Stats around wealth gained (based on gold and item drops, with their NPC value)

### Examples:

_Note: you can normally see player names here, I've hidden those for privacy._

**Raw exp/h**
![exp-per-hour](https://github.com/opentibiabr/canary/assets/223760/3a873aca-f2e4-4d19-8e61-ed20c176a30f)

**Raw gold/h**
![gold-per-hour](https://github.com/opentibiabr/canary/assets/223760/1c0d1e99-c4b9-4d9a-aced-75ac376b4673)

**Monsters killed/h**
![monsters-per-hour](https://github.com/opentibiabr/canary/assets/223760/4d8c9e19-d579-4405-a018-fc69c79a11c2)
43 changes: 43 additions & 0 deletions metrics/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
version: "3"

services:
prometheus:
image: prom/prometheus:latest
restart: unless-stopped
volumes:
- ./prometheus:/etc/prometheus
- prometheus-data:/prometheus
command:
- "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.path=/prometheus"
- "--web.enable-lifecycle"
- "--log.level=debug"
ports:
- "9090:9090"
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
- monitoring-net

grafana:
image: grafana/grafana:latest
restart: unless-stopped
volumes:
- grafana-data:/var/lib/grafana
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
- GF_USERS_ALLOW_SIGN_UP=false
depends_on:
- prometheus
ports:
- "4444:3000"
networks:
- monitoring-net

volumes:
prometheus-data:
grafana-data:

networks:
monitoring-net:
9 changes: 9 additions & 0 deletions metrics/prometheus/prometheus.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
global:
scrape_interval: 5s
scrape_timeout: 2s
evaluation_interval: 5s
scrape_configs:
- job_name: canary
static_configs:
- targets: ['host.docker.internal:9464']
3 changes: 2 additions & 1 deletion src/account/account.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,9 @@ namespace account {
}

void Account::addPremiumDays(const int32_t &days) {
auto timeLeft = static_cast<int32_t>((m_account.premiumLastDay - getTimeNow()) % 86400);
auto timeLeft = std::max(0, static_cast<int>((m_account.premiumLastDay - getTimeNow()) % 86400));
setPremiumDays(m_account.premiumRemainingDays + days);
m_account.premiumDaysPurchased += days;

if (timeLeft > 0) {
m_account.premiumLastDay += timeLeft;
Expand Down
3 changes: 2 additions & 1 deletion src/account/account.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
#include "config/configmanager.hpp"
#include "utils/definitions.hpp"
#include "security/argon.hpp"
#include "utils/tools.hpp"

namespace account {
class Account {
Expand Down Expand Up @@ -106,7 +107,7 @@ namespace account {
void addPremiumDays(const int32_t &days);
void setPremiumDays(const int32_t &days);
[[nodiscard]] inline uint32_t getPremiumRemainingDays() const {
return m_account.premiumRemainingDays;
return m_account.premiumLastDay > getTimeNow() ? static_cast<uint32_t>((m_account.premiumLastDay - getTimeNow()) / 86400) : 0;
}

[[nodiscard]] inline uint32_t getPremiumDaysPurchased() const {
Expand Down
2 changes: 1 addition & 1 deletion src/account/account_repository_db.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -160,11 +160,11 @@ namespace account {

acc.id = result->getNumber<uint32_t>("id");
acc.accountType = static_cast<AccountType>(result->getNumber<int32_t>("type"));
acc.premiumRemainingDays = result->getNumber<uint32_t>("premdays");
acc.premiumLastDay = result->getNumber<time_t>("lastday");
acc.sessionExpires = result->getNumber<time_t>("expires");
acc.premiumDaysPurchased = result->getNumber<uint32_t>("premdays_purchased");
acc.creationTime = result->getNumber<uint32_t>("creation");
acc.premiumRemainingDays = acc.premiumLastDay > getTimeNow() ? (acc.premiumLastDay - getTimeNow()) / 86400 : 0;

setupLoyaltyInfo(acc);

Expand Down
13 changes: 12 additions & 1 deletion src/canary_server.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,16 @@ int CanaryServer::run() {
loadConfigLua();

logger.info("Server protocol: {}.{}{}", CLIENT_VERSION_UPPER, CLIENT_VERSION_LOWER, g_configManager().getBoolean(OLD_PROTOCOL, __FUNCTION__) ? " and 10x allowed!" : "");
metrics::Options metricsOptions;
metricsOptions.enablePrometheusExporter = g_configManager().getBoolean(METRICS_ENABLE_PROMETHEUS, __FUNCTION__);
if (metricsOptions.enablePrometheusExporter) {
metricsOptions.prometheusOptions.url = g_configManager().getString(METRICS_PROMETHEUS_ADDRESS, __FUNCTION__);
}
metricsOptions.enableOStreamExporter = g_configManager().getBoolean(METRICS_ENABLE_OSTREAM, __FUNCTION__);
if (metricsOptions.enableOStreamExporter) {
metricsOptions.ostreamOptions.export_interval_millis = std::chrono::milliseconds(g_configManager().getNumber(METRICS_OSTREAM_INTERVAL, __FUNCTION__));
}
g_metrics().init(metricsOptions);

rsa.start();
initializeDatabase();
Expand Down Expand Up @@ -209,7 +219,7 @@ void CanaryServer::logInfos() {
/**
*It is preferable to keep the close button off as it closes the server without saving (this can cause the player to lose items from houses and others informations, since windows automatically closes the process in five seconds, when forcing the close)
* Choose to use "CTROL + C" or "CTROL + BREAK" for security close
* To activate/desactivate window;
* To activate/deactivate window;
* \param MF_GRAYED Disable the "x" (force close) button
* \param MF_ENABLED Enable the "x" (force close) button
*/
Expand Down Expand Up @@ -376,4 +386,5 @@ void CanaryServer::modulesLoadHelper(bool loaded, std::string moduleName) {
void CanaryServer::shutdown() {
inject<ThreadPool>().shutdown();
g_dispatcher().shutdown();
g_metrics().shutdown();
}
Loading

0 comments on commit 9519cee

Please sign in to comment.