Handling Java Modifiers: Adjusting ANTLR Grammars for Code Generation

[volodya-lombrozo]

There is another important difference between program parsing and generation. Let's consider the following excerpt from the Java 8 ANTLR 4 grammar definition:

normalClassDeclaration
    : classModifier* 'class' Identifier classBody
    ;

classModifier
    : 'public'
    | 'protected'
    | 'private'
    | 'abstract'
    | 'static'
    | 'final'
    ;

This grammar perfectly parses classes like:

public static public final abstract class Noticed {}

But have you noticed something strange? Yes, there are several problems here:

1. Invalid Use of Modifiers. In Java, the static modifier for classes is only applicable to nested classes. Top-level classes cannot be static. Similarly, access modifiers like public, protected, and private are not allowed together on top-level classes.

class Top {
    static class Noticed{}
}

2. Modifier Repetition. Repeating the same modifier (e.g., public public) is syntactically incorrect in Java. Although the grammar allows classModifier*, Java's language specification doesn't permit such repetition.

class Top {
   public class Noticed {}
}

3. Conflicting Modifiers. The modifiers abstract and final cannot be used together because they are mutually exclusive. An abstract class is meant to be subclassed, whereas a final class cannot be subclassed.

final class Noticed {}
// or
abstract class Noticed {}

So, while the initial grammar is suitable for parsing (accepting a wide range of inputs and deferring semantic checks), it isn't ideal for program generation where we need to produce semantically correct code. To fix this, we need to adjust the grammar:

normalClassDeclaration
    : inheritanceModifier? 'class' Identifier classBody
    ;

innerClassDeclaration
    : inheritanceModifier? staticModifier? accessModifier? 'class' Identifier classBody
    ;

inheritanceModifier
    : 'final'
    | 'abstract'
    ;

accessModifier
    : 'public'
    | 'protected'
    | 'private'
    ;

staticModifier
    : 'static'
    ;

In the revised grammar, we've:

1. Separated Modifiers. Divided modifiers into inheritanceModifier, accessModifier, and staticModifier to enforce correct usage and prevent invalid combinations.
2. Adjusted Class Declarations. Created separate rules for normalClassDeclaration and innerClassDeclaration to reflect the different contexts (top-level vs. nested classes).
3. Restricted Repetition. Used ? (zero or one occurrence) instead of * to prevent modifier repetition and enforce the correct order.
   These changes help ensure that the generated code is semantically correct according to Java's language specification. While the grammar becomes more verbose, it provides the necessary strictness for code generation.

In conclusion, we can't just use a raw ANTLR grammar intended for parsing to generate code. Parsing aims to accept as much correct code as possible, often leaving semantic checks for later stages. In contrast, code generation requires a stricter grammar to prevent semantic errors in the output.