Skip to content

Logging

Jump to bottom
Michael Hillman edited this page Aug 11, 2021 · 29 revisions

All codes within The World Avatar project (TWA) should ensure they adequately use logging to record their actions and the occurrences of any warnings/errors. To that end, a common logging approach is presented below. Unless a specialist approach is needed for a particular code base (if so, please consult senior developers), then it's expected that this approach is followed in all TWA projects.

Note: A previous logging framework is present in some older code bases; previously, the JPSBaseLogger class within the JPS Base Library was created with the intention that all codes would log through this central location, and that logs were remotely sent to a Servlet running using the JPS_BASE project. Whilst this JPSBaseLogger class still exists, it is now deprecated and should be avoided wherever possible. The logging Servlet provided by the JPS_BASE project is also not currently in use and should be avoided.


Logging in Java projects

Logging within all Java projects should now be handled via the use of Log4j2. Older versions of Log4j, and any use of SLF4J are now discouraged.

Log4j2 uses a configuration file to define the destination of logging statements, their format, and the minimum logging level. Two Log4j configuration files (one for development environments, one for production) have been created for use with TWA Java code bases and can be downloaded from the package repository using Maven. See the Packages page of the Wiki for details on how to connect to the package repository.

The configuration files for each environment are currently configured as such:

Development Production
Logs to Console Logs to Console
Logs to Rolling File Logs to Rolling File
Minimum level: DEBUG Minimum level: WARN

Log files will be written to ~/.jps/logs; if the JPSAgent class is initialised at any point (i.e. by referencing or inheriting from it), then the standard out and err streams will also be redirected to the logging system, developers can call methods like, System.out.println() as normal and the contents should get redirected to the logger (and to the console and log file in turn).

Setting up new code bases

If you're starting a new Java code base, it's highly suggested that you take a copy of the example Java agent. This has already been configured to include the required libraries, download the config files, and contains example logging statements. Comments are provided within the pom.xml, source code, and Docker configuration of this example agent; it's worth taking the time to read and understand what's going on before adding your own changes.

Updating existing code bases

If adding logging to (or updating logging in) an existing code base, simply follow the below steps.

  1. Add the parent pom to your project.
  2. Ensure the below dependencies are added to your project (any other logging libraries can be removed).
<dependency>
	<groupId>org.apache.logging.log4j</groupId>
	<artifactId>log4j-api</artifactId>
</dependency>
<dependency>
	<groupId>org.apache.logging.log4j</groupId>
	<artifactId>log4j-core</artifactId>
</dependency>
  1. If using a Java project that builds as a WAR file (i.e. a Servlet), also include the following dependency.
<dependency>
	<groupId>org.apache.logging.log4j</groupId>
	<artifactId>log4j-web</artifactId>
</dependency>
  1. Add the following plugins to your project's pom.xml file. If definitions of these plugins already exist in your project, their configuration should automatically be merged with the one declared in the parent pom (so you don't need to add these elements). If this causes issues, please contact senior developers.
<!-- Used to build into a WAR file and ensures everything in ./WEB-INF
gets copied into the final WAR file's internal WEB-INF directory. -->
<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-war-plugin</artifactId>
	<!-- Version, configuration, and executions should be pulled from the 
	parent POM unless overridden here. -->
</plugin>

<!-- Downloads and extracts ZIP archives from Maven repository -->
<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-dependency-plugin</artifactId>
	<!-- Version, configuration, and executions should be pulled from the 
	parent POM unless overridden here. -->
</plugin>
  1. Add the following profiles to your project's pom.xml file. This will allow you to choose which logging configuration file (development or production) will be used in your project via a command line argument.
<!-- Profiles are used to switch between building for development and production 
environments. Use "-P profile-id" within an mvn command to build with a profile -->
<profiles>
	<!-- This profile should be used for development builds. -->
	<profile>
		<id>dev-profile</id>
		<activation>
			<activeByDefault>true</activeByDefault>
		</activation>
		<properties>
			<!-- Set property to download development logging config -->
			<log.classifier>java-log-config-dev</log.classifier>
		</properties>
	</profile>
	
	<!-- This profile should be used for production builds. -->
	<profile>
		<id>prod-profile</id>
		<properties>
			<!-- Set property to download production logging config -->
			<log.classifier>java-log-config-prod</log.classifier>
		</properties>
	</profile>
</profiles>

Once these steps are completed, you should be able to initialise and call Log4j2 Logger objects directly within your classes. It's recommended that each class contains a private static final Logger instance with the input class/class name passed during construction.

An example of a simple Java class using a Log4j2 logger is shown below.

package uk.ac.cam.cares.jps.agent;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

/**
 * Example logging usage.
 */
public class LoggingExample {

    /**
     * Logger for reporting info/errors.
     */
    private static final Logger LOGGER = LogManager.getLogger(LoggingExample.class);

    /**
     * Test the logging.
     */
    public void logSomething() {
        LOGGER.debug("This is a debug message.");
        LOGGER.info("This is an info message.");
        LOGGER.warn("This is a warn message.");
        LOGGER.error("This is an error message.");
        LOGGER.fatal("This is a fatal message.");
    }
}

Logging in Python projects

A Python package containing a variety of utilities can be found within the Agents/utils/python-utils directory. At the time of writing this only includes logging utilities.

Via the twa_logging script within the logs submodule, developers can acquire a logger configured for development (dev) or production (prod) environments. Calls to this logger can then follow the standard Python logging framework. After importing the module, initialisation of the logging library will automatically take place, so developers only need to import it then uses its loggers.

The configurations for each logger:

Development Production
Logs to Console Logs to Console
Logs to Rolling File Logs to Rolling File
Minimum level: DEBUG Minimum level: WARN

Log files will be written to ~/.jps/logs; the standard out stream will also be redirected to the logging system, developers can call the print() function as normal and the contents should get redirected to the logger (and to the console and log file in turn).

Setting up new code bases

If you're starting a new Python code base, it's highly suggested that you take a copy of the example Python agent. This has already been configured to include the twautils package, and contains example logging statements. Comments are provided within the source code, and Docker configuration of this example agent; it's worth taking the time to read and understand what's going on before adding your own changes.

Updating existing code bases

If adding logging to (or updating logging in) an existing code base, simply follow the below steps.

  1. Install the twautils package directly from the GitHub repository
    • This can be done with the following command (or by adding the URL to your project's requirements.txt file).
    • The branch the package is built from can also be changed by updating the URL.
    • Note: this does create a temporary clone of the repository before building and installing the package, so may take up to 3 minutes.
pip install git+https://github.com/cambridge-cares/TheWorldAvatar@develop#subdirectory=Agents/utils/python-utils
  1. Import the twa_logging script with from twautils.log import twa_logging.
  2. Create a logger using the twa_logging.get_logger() function, passing either dev or prod.
  3. Log statements as usual.

An example of a Python script using the logging is provided below.

from twautils.log import twa_logging

def demo():
    """
        Demo the logging functionality.
    """
    print("=== Development Logging ===")
    dev_logger = twa_logging.get_logger("dev")
    dev_logger.debug("This is a DEBUG statement")
    dev_logger.info("This is an INFO statement")
    dev_logger.warning("This is a WARNING statement.")
    dev_logger.error("This is an ERROR statement.")
    dev_logger.critical("This is a CRITICAL statement.")

    print("=== Production Logging ===")
    prod_logger = twa_logging.get_logger("prod")
    prod_logger.debug("This is a DEBUG statement")
    prod_logger.info("This is an INFO statement")
    prod_logger.warning("This is a WARNING statement.")
    prod_logger.error("This is an ERROR statement.")
    prod_logger.critical("This is a CRITICAL statement.")

    print("=== System Stream ===")
    print("This is a STANDARD OUT statement.")

demo()

Contents

Clone this wiki locally