Column family and metrics cause a very high object (and name) cound in mbean server. #135
Milestone
Comments
elcallio
added a commit
to elcallio/scylla-jmx
that referenced
this issue
Sep 1, 2020
Fixes scylladb#134 Refs scylladb#135 Replaces previous refresh calls with ones bound to registration check objects, which provides some sync between threads doing refresh, and reduced redundant calls. Also adds repeated reaping of dead objects, i.e. every 5 minutes we try to remove dead CF:s (not adding new ones), to reduce idle footprint.
Merged
elcallio
added a commit
to elcallio/scylla-jmx
that referenced
this issue
Sep 1, 2020
Fixes scylladb#134 Refs scylladb#135 Replaces previous refresh calls with ones bound to registration check objects, which provides some sync between threads doing refresh, and reduced redundant calls. Also adds repeated reaping of dead objects, i.e. every 5 minutes we try to remove dead CF:s (not adding new ones), to reduce idle footprint.
elcallio
added a commit
to elcallio/scylla-jmx
that referenced
this issue
Sep 7, 2020
Fixes scylladb#134 Refs scylladb#135 Replaces previous refresh calls with ones bound to registration check objects, which provides some sync between threads doing refresh, and reduced redundant calls. Also adds repeated reaping of dead objects, i.e. every 5 minutes we try to remove dead CF:s (not adding new ones), to reduce idle footprint.
penberg
added a commit
that referenced
this issue
Sep 7, 2020
" Fixes #133 Fixes #134 Refs #135 Makes CF mbean refresh code synchronized and tries to remove reductant calls if we contend. Adds background reaping of dead objects to reduce memory load in (test) scenarios where we manage to refresh to add, but not cause removal (i.e. no wildcard queries). TableMetricsObjectName serialization is fixed in the series because without it we see loads of exceptions when refreshing the mbean set. " * elcallio-jmx-fixes: scylla-jmx: Use registration checker objects scylla-jmx: Introduce a registration check object scylla-jmx: Fix TableMetricObjectName serialization
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
For every CF, we bind some 50+ metrics, each under two different names (Table and ColumnFamily).
This leads to rather high object counts if the associated server has many CF:s.
Since objects are also only reaped when a wildcard query is issued, it is somewhat possible to cause
objects to remain bound that represent dropped CF:s for quite some time, depending on query pattern
or JMX commands run.
The text was updated successfully, but these errors were encountered: