Dummy Usage Generator for JVM Libraries

This tool generates JVM bytecode that calls all (non-private) methods of a library that runs on the JVM (e.g. Java). All reference type parameters that are passed to methods, return values, exceptions and instances are passed to sinks to assist with data flow analysis.

Usage

1. Compile and package the usage generator (run this in the root directory of the usagen project!):

$ sbt assembly
...
[success] Total time: ...

2. Copy the packaged usage generator jar wherever you want it, e.g. to the current working directory:

$ cp target/scala-2.12/usagegen.jar ./

3. Ask the tool how to use it:

$ java -jar usagegen.jar --help
Usage:  [options] libraryJarFile outputJarFile

  libraryJarFile           Path to the JAR file containing the tested library
  outputJarFile            Path to the output JAR file that contains the generated usage code
  -f, --force              Overwrite outputJarFile if it already exists
  -r, --run                Run generated bytecode after generation
  --runtime-jars <jar1>,<jar2>...
                           list of jars containing the runtime dependencies of the tested library
  -v, --verbose            Increase logging verbosity
  -h, --help               Print this usage text

Example 1: Using the tool to generate the usage bytecode without running the bytecode:

1. Replace slf4j-api-2.0.0-alpha7.jar with a comma separated list of runtime dependencies, slf4j-nop-2.0.0-alpha7.jar with the library that you want to test, output.jar with the path to the jar file containing the usage bytecode and run:

$ java -jar usagegen.jar --runtime-jars slf4j-api-2.0.0-alpha7.jar slf4j-nop-2.0.0-alpha7.jar output.jar

2. To execute the generate calls, add the generated bytecode, the runtime dependencies and the library itself to the classpath and call the main method on the entrypoint class like this:

$ java -cp output.jar:slf4j-nop-2.0.0-alpha7.jar:slf4j-api-2.0.0-alpha7.jar ___TEST_RUNNER_ENTRYPOINT___

Example 2: Using the tool to generate the usage bytecode and immediately run it afterwards:

$ java -jar usagegen.jar -rf --runtime-jars slf4j-api-2.0.0-alpha7.jar slf4j-nop-2.0.0-alpha7.jar output.jar

How to run the tests

Simply run

$ sbt test

If you have problems when running in the tests, check that the test resources (src/test/resources/*.jar) have been generated properly (and are up to date). You should either try running them with sbt test again, or manually generate the test resources by calling:

$ sbt Test/compile

Structure of the generated output

How does a generated jar file look?

First let's take a look at the library file, that is used in the examples. The jar only contains one class (org.slf4j.nop.NOPServiceProvider):

├── META-INF
│   ...
└── org
    └── slf4j
        └── nop
            └── NOPServiceProvider.class

Note: This class (or any other classes from the library/runtime jars) is not copied to the resulting jar.

Now, let's look at the generated jar that was produced in examples 1 and 2 (output.jar).

Excerpt of output.jar contents:

Mandatory jar manifest (specifies main class):

├── META-INF
│   └── MANIFEST.MF

Entry point class containing main method and performCalls. calling performCalls on all method caller classes (these start with ___METHOD_CALLER___$):

├── ___TEST_RUNNER_ENTRYPOINT___.class

Helper classes that generate parameters (and instances) for (instance) method calls at runtime:

├── ___parameter_generators___
│   ├── ___INSTANCE_PROVIDER_PARAMETER_GENERATOR___$c0.class
│   └── ___INSTANCE_PROVIDER_PARAMETER_GENERATOR___$c1.class

Concrete stub subclasses for abstract classes and interfaces that were found in runtime jars and project jar (and are used as parameters for methods / instances for instance methods):

└── org
    └── slf4j
        ├── ILoggerFactory___GENERATED_CONCRETE_SUBCLASS.class
        ├── Logger___GENERATED_CONCRETE_SUBCLASS.class

Helper classes for sourcing of instances at runtime (used in parameter generation). There might be multiple such classes per package. They only access classes in the respective package.

        ├── ___INSTANCE_PROVIDER___$c0.class

Method callers (the methods that call library methods). The class ___METHOD_CALLER___$NOPServiceProvider provides all caller methods for the class NOPServiceProvider. One such caller method is created for each class of the library and is placed in the same package as the class that is called.

        ├── nop
        │   ├── ___METHOD_CALLER___$NOPServiceProvider.class

The sink class for the NOPServiceProvider class. For each caller class, there is one sink class. Containing two sinks per method (normal sink and exception sink). The sink is placed in the same package as the caller (and therefore also in the same package as the called class).

        │   └── ___SINK___$NOPServiceProvider.class

How do the generated classes look like in more detail?

Short answer:

Use the javap tool to look at them yourself (For even more details use the -v flag instead of -c. If you don't care about the bytecode, only the methods, drop the -c flage entirely):

$ javap -c -classpath output.jar org.slf4j.nop.___METHOD_CALLER___\$NOPServiceProvider

Long anser:

The entry point class ___TEST_RUNNER_ENTRYPOINT___

public class ___TEST_RUNNER_ENTRYPOINT___ {
  public ___TEST_RUNNER_ENTRYPOINT___();
    Code:
       0: aload_0
       1: invokespecial #8                  // Method java/lang/Object."<init>":()V
       4: return

  public static void main(java.lang.String[]);
    Code:
       0: invokestatic  #12                 // Method run:()V
       3: return

For larger libraries, there would also be calls to the performCalls method on other method caller classes here:

  public static void run();
    Code:
       0: invokestatic  #19                 // Method org/slf4j/nop/___METHOD_CALLER___$NOPServiceProvider.performCalls:()V
       3: return
}

The method caller class org.slf4j.nop.___METHOD_CALLER___$NOPServiceProvider:

public class org.slf4j.nop.___METHOD_CALLER___$NOPServiceProvider {

has a performCalls method that calls the method callers that are specified in the method caller class itself:

  public static void performCalls();
    Code:
       0: invokestatic  #93                 // Method NOPServiceProvider___init___0:()V
       3: invokestatic  #95                 // Method NOPServiceProvider__getLoggerFactory__1:()V
       6: invokestatic  #97                 // Method NOPServiceProvider__getMDCAdapter__2:()V
       9: invokestatic  #99                 // Method NOPServiceProvider__getMarkerFactory__3:()V
      12: invokestatic  #101                // Method NOPServiceProvider__getRequestedApiVersion__4:()V
      15: invokestatic  #103                // Method NOPServiceProvider__initialize__5:()V
      18: return

Error handling (mostly handles exceptions that are thrown in static initializers):

      19: astore_0
      20: getstatic     #109                // Field java/lang/System.err:Ljava/io/PrintStream;
      23: ldc_w         #111                // String Caught Exception in org/slf4j/nop___METHOD_CALLER___$NOPServiceProvider}.performCalls
      26: invokevirtual #117                // Method java/io/PrintStream.println:(Ljava/lang/String;)V
      29: aload_0
      30: invokevirtual #120                // Method java/lang/Throwable.printStackTrace:()V
      33: return
    Exception table:
       from    to  target type
           0    19    19   any

Test method for parameterless constructor of NOPServiceProvider.

  public static void NOPServiceProvider___init___0();
    Code:
       0: new           #11                 // class org/slf4j/nop/NOPServiceProvider
       3: astore_0
       4: aload_0
       5: invokespecial #12                 // Method org/slf4j/nop/NOPServiceProvider."<init>":()V
       8: aload_0
       9: invokestatic  #18                 // Method org/slf4j/nop/___SINK___$NOPServiceProvider.org_slf4j_nop___NOPServiceProvider___init___0:(Lorg/slf4j/nop/NOPServiceProvider;)V
      12: return
      13: astore_2
      14: aload_2
      15: invokestatic  #22                 // Method org/slf4j/nop/___SINK___$NOPServiceProvider.org_slf4j_nop___NOPServiceProvider___init___0_exception:(Ljava/lang/Throwable;)V
      18: return
    Exception table:
       from    to  target type
           5    13    13   any

Test method for instance method NOPServiceProvider.getLoggerFactory()

  public static void NOPServiceProvider__getLoggerFactory__1();
    Code:

Get instance from parametergenerator

       0: invokestatic  #32                 // Method ___parameter_generators___/___INSTANCE_PROVIDER_PARAMETER_GENERATOR___$c0.org_slf4j_nop_NOPServiceProvider:()Lorg/slf4j/nop/NOPServiceProvider;

Check if parameter is null

       3: astore_0
       4: aload_0
       5: ifnonnull     9
       8: return

Don't generate any parameters (since the tested method doesn't take any), load instance and call instance method getLoggerFactory on instance.

       9: aload_0
      10: invokevirtual #36                 // Method org/slf4j/nop/NOPServiceProvider.getLoggerFactory:()Lorg/slf4j/ILoggerFactory;

Store return value and pass instance, return value to sink and return.

      13: astore_1
      14: aload_0
      15: aload_1
      16: invokestatic  #40                 // Method org/slf4j/nop/___SINK___$NOPServiceProvider.org_slf4j_nop___NOPServiceProvider__getLoggerFactory__1:(Lorg/slf4j/nop/NOPServiceProvider;Lorg/slf4j/ILoggerFactory;)V
      19: return

Exception handling: load instance and exception, pass to exception sink

      20: astore_2
      21: aload_0
      22: aload_2
      23: invokestatic  #44                 // Method org/slf4j/nop/___SINK___$NOPServiceProvider.org_slf4j_nop___NOPServiceProvider__getLoggerFactory__1_exception:(Lorg/slf4j/nop/NOPServiceProvider;Ljava/lang/Throwable;)V
      26: return
    Exception table:
       from    to  target type
          10    20    20   any

// Other methods...
}

The sink class org.slf4j.nop.___METHOD_CALLER___$NOPServiceProvider:

This is the sink class that is used by the method calls in the method caller class org.slf4j.nop.___METHOD_CALLER___$NOPServiceProvider.

public class org.slf4j.nop.___SINK___$NOPServiceProvider {

The regular sink method for the constructor caller method. It takes only the instance of the call (since the constructor is parameterless).

  public static void org_slf4j_nop___NOPServiceProvider___init___0(org.slf4j.nop.NOPServiceProvider);
    Code:
       0: return

The exception sink method for the constructor caller method. It takes only the exception of the call (since the constructor is parameterless and the initialization of the object failed, i.e., is not referencable).

  public static void org_slf4j_nop___NOPServiceProvider___init___0_exception(java.lang.Throwable);
    Code:
       0: return
}

The instance access helper class org.slf4j.___INSTANCE_PROVIDER___$c0:

This class has methods that generate instances using different sources. (i.e., static fields, methods, constructors and concrete stub subclasses). Here is one method that uses a concrete stub subclass to generate an instance at runtime:

public class org.slf4j.___INSTANCE_PROVIDER___$c0 {
// ...
  public static org.slf4j.ILoggerFactory using_stub_subclass__ILoggerFactory__2();
    Code:
       0: new           #11                 // class org/slf4j/ILoggerFactory___GENERATED_CONCRETE_SUBCLASS
       3: dup
       4: invokespecial #12                 // Method org/slf4j/ILoggerFactory___GENERATED_CONCRETE_SUBCLASS."<init>":()V
       7: areturn
       8: aconst_null
       9: areturn
    Exception table:
       from    to  target type
           0     8     8   any
// ...
}

The instance provider based parameter generator ___INSTANCE_PROVIDER_PARAMETER_GENERATOR___$c0:

This class has multiple methods that provide a parameter of a given type. This method provides an instance of type ILoggerFactory. It goes through (potentially) multiple instance providers and selects the first one that is not null. If no instance can be generated, null is returned.

public class ___parameter_generators___.___INSTANCE_PROVIDER_PARAMETER_GENERATOR___$c0 {
  public static org.slf4j.ILoggerFactory org_slf4j_ILoggerFactory();
    Code:
       0: aconst_null
       1: astore_0
       2: invokestatic  #15                 // Method org/slf4j/___INSTANCE_PROVIDER___$c0.using_stub_subclass__ILoggerFactory__2:()Lorg/slf4j/ILoggerFactory;
       5: astore_0
       6: aload_0
       7: ifnonnull     12
      // Tries other sources here, if there are any ...
      12: aload_0
      13: areturn
}