Java agent causes Java logging (JUL) issues on various platforms

[dhoard]

Issues

The JMX Exporter Java agent's use of standard Java logging (JUL). This causes various issues due to initialization/classloader issues because a Java agent uses a different classloader/classpath than an application.

Related Issues

JMX Exporter Java agent issues...

• Cannot run JMX Exporter in Wildfly 17 running on Java 11 #455
• JMX agent interfering with logging format of application #976
• Issue in Integrating jmx_prometheus_javaagent with Jboss #1000

... and GlashFish issues...

• JDK loggers can initialize the JDK LogManager before GlassFishLogManager is set eclipse-ee4j/glassfish#25133
• Redesign of GlassFish bootstrap eclipse-ee4j/glassfish#25183

Investigate using a native logging implementation

The JMX Exporter already has a native logging implementation that is used for development/developer debugging.

https://github.com/prometheus/jmx_exporter/tree/main/collector/src/main/java/io/prometheus/jmx/logger

Pros

Migrating all code using standard Java logging (JUL) to the native implementation will resolve these issues since the code is packaged in the Java agent.

This makes integration into applications that use a custom JUL LogManager/logging implementation a non-issue.

Cons

The native logging solution, which is currently only used for development/developer debugging, is very limited.

• all or nothing
• only logs to System.out
• doesn't log to a file
• doesn't have per class configuration
• only a single logging level
• is not dynamic

In most scenarios, JMX Exporter logging is typically only used when there are issues with the exporter. We may be able to solve some of the issue.