A lot of restarts of the report service, because JDBC connection timeout, because we have a lot of threads

[polarnik]

Hello!

Default thread pool settings are not the best:

report:
  replicaCount: 1
  cache:
    enabled: false
    ttlByDefault: 1m
    ttlWidgetsAutomationTrend: 1h
    ttlWidgetsProjectMetricTrend: 1h
    ttlWidgetsLaunchDurationHistogram: 1h
    ttlWidgetsAnalyticPieChart: 1h
    ttlWidgetsTrComplexTrend: 1h
    ttlWidgetsTrStatisticTrend: 1h
    ttlWidgetsTcLastResult: 1h
  image: allure-report
  maxDBConn: 10
  maxConcurrency: 5
  maxS3Concurrency: 200
  taskExecutorCorePoolSize: 200

The source:
https://github.com/qameta/allure-testops-deployment/blob/master/charts/allure-testops/values.yaml

200 threads rabbitConnectionFactorySharedExecutor (taskExecutorCorePoolSize: 200) can not work with 10 JDBC connections (maxDBConn: 10), only. Allure health check will not get a free connection, and Kubernetes will restart the service, because the service is not healthy.

And 200 threads rabbitConnectionFactorySharedExecutor will allocate all CPU resources in PostgreSQL.

The best settings are:

  maxDBConn: 50
  maxConcurrency: 5
  taskExecutorCorePoolSize: 10

[polarnik]

For example, we used big taskExecutorCorePoolSize (rabbitConnectionFactorySharedExecutor)

We had 100% CPU Usage in PostgreSQL and the service restarts:

Take a look on the day: 08/15

The config was updated on 08/21:

  maxDBConn: 50
  maxConcurrency: 5
  taskExecutorCorePoolSize: 10

The service is working well, without restarts and without 100% CPU Usage in PostgreSQL