Thinking of moving to slf4j?

[sveba]

My team is using spring boot with logback and slf4j as abstraction. We are including the norconex crawler as mvn dependency and running it programmaticaly. The problem is that you are using the log4j directly and thats why we have 2 logging frameworks in our project. Do you think of moving to SLF4J, so the user can pickup the logging fw by himself?

I already tried the log4j-over-sljf4j but I get a java.lang.IncompatibleClassChangeError because their
https://github.com/qos-ch/slf4j/blob/v_1.7.22/log4j-over-slf4j/src/main/java/org/apache/log4j/FileAppender.java
Doesn't implement the org.apache.log4j.Appender interface.

jef/norconex-jef/src/main/java/com/norconex/jef4/suite/JobSuite.java

Line 281 in 1deb8a5

appender.setLayout(new ThreadSafeLayout(appender.getLayout()));

[essiembre]

I see the value. The challenge is addressing the current need of accessing log4j specific classes to support some lower-level operations we want to do with logging. This will require some investigation. If we find out SLF4J can be used as a replacement, we will consider it for sure. I am marking this as a feature request.

[essiembre]

After investigating, it turns out to be more challenging to do than I hoped for. There is one issue I have yet to resolve before we can make this happen.

The JEF library needs to control how logs are built. People can define their own logger in the log4j.properties file, but JEF will create one at runtime that gives it the level of control it needs. A few requirements for JEF to control part of the logging:

• To have JEF handle log backups the same way it handles other files it backs up.
• To eventually allow log cleaning via startup option.
• To control the log format so we can add runtime custom information like the thread or which crawl job wrote the line. This is important for integration into other tools, such as JEF Monitor. JEF API has to be able to return the logs and they have to have a predictable format for parsing.
• To allow configuring logs location via application XML configuration (e.g. Norconex HTTP Collector), and not have regular users having to learn about an underlying logging framework.

The goal of SLF4J is to provide an abstraction, so it does not offer that low-level control that is needed.

A possible option is to still use SLF4J and simply use log4j.properties as the implementation for JEF, but since JEF is a generic library, requiring log4j as the logging framework would defy the purpose of using SLF4J in the first place (we would not want to impose any logging framework if we adopt SLF4J).

The other approach would be to have JEF not rely on log4j for writing its logs and use a custom logging solution (irk!).

The best would be if SLF4J provided a way to control low-level logging via code, but the abstraction is too high-level (rightfully so).

One possible option is maybe to create an implementation of SLF4J just for JEF, that does not get loaded on start-up (to avoid conflicts), but rather programmatically, with a life on its own besides actual logging framework provided by implementors. I still have to research that.

If you have ideas how to meet the above requirements with SLF4J, please share your thoughts.

[dgomesbr]

@essiembre, a couple of comments:

The JEF library needs to control how logs are built.
No problem at all, SLF4J is a facade, you could have a underlining performant implementation such as logback, which is the default implementation of it. With that you still could control everything while working with its Interface instead of its implementation.

All the dotted points are addressed, one in special worth the comment:

To allow configuring logs location via application XML configuration (e.g. Norconex HTTP Collector), and not have regular users having to learn about an underlying logging framework.
Sample logback.xml: https://github.com/pedestal/pedestal/blob/master/service/dev/logback.xml
Extended version used by spring boot: https://github.com/spring-projects/spring-boot/tree/master/spring-boot/src/main/resources/org/springframework/boot/logging/logback

The goal of SLF4J is to provide an abstraction, so it does not offer that low-level control that is needed.
That's not the way it is. It does provide the interface for all the implementation that you're doing right now and with that in mind, it allows you to bridge it with existing or newer implementations

A possible option is to still use SLF4J and simply use log4j.properties as the implementation for JEF, but since JEF is a generic library, requiring log4j as the logging framework would defy the purpose of using SLF4J in the first place (we would not want to impose any logging framework if we adopt SLF4J).
You could use SLF4J-LOG4J Bridge and don't change a thing, think about a first step into migrating to SLF4J completely

The other approach would be to have JEF not rely on log4j for writing its logs and use a custom logging solution (irk!).

The best would be if SLF4J provided a way to control low-level logging via code, but the abstraction is too high-level (rightfully so).

One possible option is maybe to create an implementation of SLF4J just for JEF, that does not get loaded on start-up (to avoid conflicts), but rather programmatically, with a life on its own besides actual logging framework provided by implementors. I still have to research that.
No need at all, drop-in replacement with Logback

I could open a branch on JEF to show you how to do it.

[essiembre]

Replacing log4j with logback or else is the easy part. The difficult part is part is JEF needs low-level logging control. So whether we use SLF4J with log4j or logback, we do not want to impose either for projects wanting to use JEF.

If you mean you could create a branch that allows using SLF4J with JEF using log4j/logback while projects consuming JEF can use their own logging without impacting the one used by JEF, sure go ahead.

My plan was to custom-wire a specific implementation of SLF4J for JEF only that is not "detected" by SLF4J from the classpath to avoid forcing its use on consuming projects. But contributions are welcomed!

[dgomesbr]

I think it could be a little bit of over engineering (as I still think you can use SLF4J and not the impl), but we could have norconex-logging module as dependencies on other modules and per default using log4j, but replaceable with norconex-logging-slf4j.

Thoughts?

@sveba your inputs are welcomed here as well to improve that idea.