Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #740: Adding visitors for handling different types along with ClassType in Generic Type invariance check #806

Merged
merged 31 commits into from
Aug 30, 2023
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
a0b76ea
adding failing test case for issue 740
akulk022 Jun 24, 2023
b2c533a
adding visitors for handling ArrayTypes
akulk022 Aug 7, 2023
6ae742b
clean-up of redundant code in visitors
akulk022 Aug 14, 2023
15e1601
fixing indentation
akulk022 Aug 14, 2023
9fae10c
Merge branch 'master' into generic_type_issue_#740_ak
msridhar Aug 14, 2023
47dd926
updating visitor methods returning null
akulk022 Aug 16, 2023
f5b6c1d
making default Type method Nullable in visitor
akulk022 Aug 16, 2023
e9d969b
formatting
msridhar Aug 16, 2023
20d747a
changing TreeVisitor second argument type to Void since it is unused
akulk022 Aug 16, 2023
0d3c2f0
Update default return type for Compare Nullability Visitor
akulk022 Aug 16, 2023
4273601
removing cast from TYPE_ARG_VISITOR
akulk022 Aug 16, 2023
9e96103
removing unnecessary @Nullable annotation for method that always retu…
akulk022 Aug 17, 2023
d1e61bf
Merge branch 'generic_type_issue_#740_ak' of https://github.com/akulk…
akulk022 Aug 17, 2023
d2db3cc
Merge remote-tracking branch 'origin/master' into generic_type_issue_…
akulk022 Aug 17, 2023
d7c9812
After merging with changes made by PR#805 making GenericChecks method…
akulk022 Aug 17, 2023
aed09fd
Adding test cases
akulk022 Aug 23, 2023
1a12cef
Merge branch 'master' into generic_type_issue_#740_ak
msridhar Aug 25, 2023
a4bd9bc
Merge branch 'master' into generic_type_issue_#740_ak
msridhar Aug 25, 2023
680a17c
Removing redundant code from TypeWithPreservedAnnotations
akulk022 Aug 27, 2023
d314952
Adding more descriptive comments for visitors and fixing formatting o…
akulk022 Aug 27, 2023
b9eb8fc
fix NPE issue
msridhar Aug 27, 2023
989a755
fix other test failure
msridhar Aug 27, 2023
4e5adf0
copy type symbol for array type from original
msridhar Aug 27, 2023
657f38e
removing TYPE_ARG_VISITOR and the check to see if we have more nested…
akulk022 Aug 28, 2023
6556379
Update nullaway/src/main/java/com/uber/nullaway/GenericsChecks.java
akulk022 Aug 28, 2023
ce51560
using the existing visitor instead of allocating a new one
akulk022 Aug 28, 2023
2e14f16
fix position of annotations
msridhar Aug 28, 2023
6f5c722
add todo
msridhar Aug 28, 2023
b419103
restore
msridhar Aug 28, 2023
394ff52
more tests
msridhar Aug 29, 2023
1438bbd
Merge branch 'master' into generic_type_issue_#740_ak
msridhar Aug 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
244 changes: 186 additions & 58 deletions nullaway/src/main/java/com/uber/nullaway/GenericsChecks.java
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import com.google.errorprone.util.ASTHelpers;
import com.sun.source.tree.AnnotatedTypeTree;
import com.sun.source.tree.AnnotationTree;
import com.sun.source.tree.ArrayTypeTree;
import com.sun.source.tree.AssignmentTree;
import com.sun.source.tree.ConditionalExpressionTree;
import com.sun.source.tree.ExpressionTree;
Expand All @@ -20,6 +21,7 @@
import com.sun.source.tree.ParameterizedTypeTree;
import com.sun.source.tree.Tree;
import com.sun.source.tree.VariableTree;
import com.sun.source.util.SimpleTreeVisitor;
import com.sun.tools.javac.code.Attribute;
import com.sun.tools.javac.code.BoundKind;
import com.sun.tools.javac.code.Symbol;
Expand Down Expand Up @@ -351,60 +353,12 @@ public static void checkTypeParameterNullnessForFunctionReturnType(
*
* @param lhsType type for the lhs of the assignment
* @param rhsType type for the rhs of the assignment
* @param state the visitor state
*/
private static boolean compareNullabilityAnnotations(
Type.ClassType lhsType, Type.ClassType rhsType, VisitorState state) {
Types types = state.getTypes();
// The base type of rhsType may be a subtype of lhsType's base type. In such cases, we must
// compare lhsType against the supertype of rhsType with a matching base type.
rhsType = (Type.ClassType) types.asSuper(rhsType, lhsType.tsym);
// This is impossible, considering the fact that standard Java subtyping succeeds before running
// NullAway
if (rhsType == null) {
throw new RuntimeException("Did not find supertype of " + rhsType + " matching " + lhsType);
}
List<Type> lhsTypeArguments = lhsType.getTypeArguments();
List<Type> rhsTypeArguments = rhsType.getTypeArguments();
// This is impossible, considering the fact that standard Java subtyping succeeds before running
// NullAway
if (lhsTypeArguments.size() != rhsTypeArguments.size()) {
throw new RuntimeException(
"Number of types arguments in " + rhsType + " does not match " + lhsType);
}
for (int i = 0; i < lhsTypeArguments.size(); i++) {
Type lhsTypeArgument = lhsTypeArguments.get(i);
Type rhsTypeArgument = rhsTypeArguments.get(i);
boolean isLHSNullableAnnotated = false;
List<Attribute.TypeCompound> lhsAnnotations = lhsTypeArgument.getAnnotationMirrors();
// To ensure that we are checking only jspecify nullable annotations
for (Attribute.TypeCompound annotation : lhsAnnotations) {
if (annotation.getAnnotationType().toString().equals(NULLABLE_NAME)) {
isLHSNullableAnnotated = true;
break;
}
}
boolean isRHSNullableAnnotated = false;
List<Attribute.TypeCompound> rhsAnnotations = rhsTypeArgument.getAnnotationMirrors();
// To ensure that we are checking only jspecify nullable annotations
for (Attribute.TypeCompound annotation : rhsAnnotations) {
if (annotation.getAnnotationType().toString().equals(NULLABLE_NAME)) {
isRHSNullableAnnotated = true;
break;
}
}
if (isLHSNullableAnnotated != isRHSNullableAnnotated) {
return false;
}
// nested generics
if (lhsTypeArgument.getTypeArguments().length() > 0) {
if (!compareNullabilityAnnotations(
(Type.ClassType) lhsTypeArgument, (Type.ClassType) rhsTypeArgument, state)) {
return false;
}
}
}
return true;
Type lhsType, Type rhsType, VisitorState state) {
// it is fair to assume rhyType should be the same as lhsType as the Java compiler has passed
// before NullAway.
return lhsType.accept(new CompareNullabilityVisitor(state), rhsType);
}

/**
Expand Down Expand Up @@ -455,13 +409,11 @@ private static Type.ClassType typeWithPreservedAnnotations(
TypeMetadata typeMetadata =
new TypeMetadata(new TypeMetadata.Annotations(nullableAnnotationCompound));
Type currentTypeArgType = castToNonNull(ASTHelpers.getType(curTypeArg));
if (currentTypeArgType.getTypeArguments().size() > 0) {
// nested generic type; recursively preserve its nullability type argument annotations
currentTypeArgType =
typeWithPreservedAnnotations((ParameterizedTypeTree) curTypeArg, state);
List<Type> genericArgType = currentTypeArgType.accept(TYPE_ARG_VISITOR, null);
if (genericArgType.size() > 0) {
currentTypeArgType = curTypeArg.accept(new PreservedAnnotationTreeVisitor(state), null);
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this logic is checking, if there type arguments nested at any depth within currentTypeArgType, then we need to recurse and preserve type annotations there. It makes sense at a high level, but I'm not sure this is really an optimization (since we are adding another traversal of the type), and I'm not sure the logic in TYPE_ARG_VISITOR is correct (it doesn't seem to recursively apply the visitor). For now, how about we remove this optimization, and just unconditionally always do currentTypeArgType = curTypeArg.accept(new PreservedAnnotationTreeVisitor(state), null);?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the purpose of TYPE_ARG_VISITOR is just to be able to get the type arguments even if the Argument is not a ClassType, but the if condition checking the size is greater than 0 was already there as given below, do you want me to remove that logic?

if (currentTypeArgType.getTypeArguments().size() > 0) { // nested generic type; recursively preserve its nullability type argument annotations currentTypeArgType = typeWithPreservedAnnotations((ParameterizedTypeTree) curTypeArg, state); }

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mentioned you get an NPE without this check. What is the stack trace for that?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the test cases that failed is: A<A<@Nullable String>[]> var = new A<A<String>[]>();, when we traverse the ParametrizedTypeTree on the RHS (A<A[]>), when we get to the String, since we don't check if there are further nested arguments and accept the visitor, we get a null for currentTypeArgType, so we would certainly need some type of a check to make sure we don't run the logic again if we don't have further generic types.

Stack Trace:
java.lang.NullPointerException: Cannot invoke "com.sun.tools.javac.code.Type.cloneWithMetadata(com.sun.tools.javac.code.TypeMetadata)" because "currentTypeArgType" is null
at com.uber.nullaway.GenericsChecks$PreservedAnnotationTreeVisitor.visitParameterizedType(GenericsChecks.java:654)
at com.uber.nullaway.GenericsChecks$PreservedAnnotationTreeVisitor.visitParameterizedType(GenericsChecks.java:595)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed this by overriding the defaultAction method in b9eb8fc

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good idea, Thanks!

Type.ClassType newTypeArgType =
(Type.ClassType) currentTypeArgType.cloneWithMetadata(typeMetadata);
Type newTypeArgType = currentTypeArgType.cloneWithMetadata(typeMetadata);
newTypeArgs.add(newTypeArgType);
}
Type.ClassType finalType =
Expand Down Expand Up @@ -577,6 +529,182 @@ public static void compareGenericTypeParameterNullabilityForCall(
}
}

/** To dispatch the logic to obtain the generic type arguments for different Types */
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming we agree with my suggestion above, this visitor can be deleted

private static final Type.Visitor<List<Type>, Void> TYPE_ARG_VISITOR =
new Types.DefaultTypeVisitor<List<Type>, Void>() {
@Override
public List<Type> visitClassType(Type.ClassType t, Void s) {
return t.getTypeArguments();
}

@Override
public List<Type> visitArrayType(Type.ArrayType t, Void unused) {
return t.getComponentType().getTypeArguments();
}

@Override
public List<Type> visitCapturedType(Type.CapturedType t, Void s) {
return t.wildcard.accept(this, null);
}

@Override
public List<Type> visitType(Type t, Void unused) {
return Collections.emptyList();
}
};

/** To dispatch the logic to0 obtain the generic type arguments for different Types */
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment seems wrong

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it is wrong, it should be fixed in the newer commits, Thanks!

public static class CompareNullabilityVisitor extends Types.DefaultTypeVisitor<Boolean, Type> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine with these being inner classes for now, but a good follow up PR would be a simple refactor of this into a com.uber.nullaway.generics package with three top level classes (preferably in its own PR without any other changes, so it's easier to review).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created #817, agreed this is good for a follow up

private final VisitorState state;

CompareNullabilityVisitor(VisitorState state) {
this.state = state;
}

@Override
public Boolean visitClassType(Type.ClassType lhsType, Type rhsType) {
Types types = state.getTypes();
// The base type of rhsType may be a subtype of lhsType's base type. In such cases, we
// must
// compare lhsType against the supertype of rhsType with a matching base type.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: formatting

rhsType = (Type.ClassType) types.asSuper(rhsType, lhsType.tsym);
// This is impossible, considering the fact that standard Java subtyping succeeds before
// running
// NullAway
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: formatting

if (rhsType == null) {
throw new RuntimeException("Did not find supertype of " + rhsType + " matching " + lhsType);
}
List<Type> lhsTypeArguments = lhsType.getTypeArguments();
List<Type> rhsTypeArguments = rhsType.getTypeArguments();
// This is impossible, considering the fact that standard Java subtyping succeeds before
// running
// NullAway
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: formatting

if (lhsTypeArguments.size() != rhsTypeArguments.size()) {
throw new RuntimeException(
"Number of types arguments in " + rhsType + " does not match " + lhsType);
}
for (int i = 0; i < lhsTypeArguments.size(); i++) {
Type lhsTypeArgument = lhsTypeArguments.get(i);
Type rhsTypeArgument = rhsTypeArguments.get(i);
boolean isLHSNullableAnnotated = false;
List<Attribute.TypeCompound> lhsAnnotations = lhsTypeArgument.getAnnotationMirrors();
// To ensure that we are checking only jspecify nullable annotations
for (Attribute.TypeCompound annotation : lhsAnnotations) {
if (annotation.getAnnotationType().toString().equals(NULLABLE_NAME)) {
isLHSNullableAnnotated = true;
break;
}
}
boolean isRHSNullableAnnotated = false;
List<Attribute.TypeCompound> rhsAnnotations = rhsTypeArgument.getAnnotationMirrors();
// To ensure that we are checking only jspecify nullable annotations
for (Attribute.TypeCompound annotation : rhsAnnotations) {
if (annotation.getAnnotationType().toString().equals(NULLABLE_NAME)) {
isRHSNullableAnnotated = true;
break;
}
}
if (isLHSNullableAnnotated != isRHSNullableAnnotated) {
return false;
}
// nested generics
List<Type> genericArgs = lhsTypeArgument.accept(TYPE_ARG_VISITOR, null);
if (genericArgs.size() > 0) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again let's get rid of this optimization and just unconditionally do a recursive traversal

if (!lhsTypeArgument.accept(new CompareNullabilityVisitor(state), rhsTypeArgument)) {
return false;
}
}
}
return true;
}

@Override
public Boolean visitArrayType(Type.ArrayType lhsType, Type rhsType) {
Type.ArrayType arrRhsType = (Type.ArrayType) rhsType;
return lhsType
.getComponentType()
.accept(new CompareNullabilityVisitor(state), arrRhsType.getComponentType());
akulk022 marked this conversation as resolved.
Show resolved Hide resolved
}

@Override
public Boolean visitType(Type t, Type type) {
return true;
}
}

/** Visitor For Handling Different Tree Types */
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a more descriptive comment

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, I have added a more descriptive comment

public static class PreservedAnnotationTreeVisitor extends SimpleTreeVisitor<Type, Void> {

private final VisitorState state;

PreservedAnnotationTreeVisitor(VisitorState state) {
this.state = state;
}

@Override
public Type visitArrayType(ArrayTypeTree tree, Void p) {
ParameterizedTypeTree paramTree = (ParameterizedTypeTree) tree.getType();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we sure tree.getType() will always be a ParameterizedTypeTree?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure, I tried running a test with a regular ArrayType inside the generic like A<String[]> and that ran fine. Maybe its better to just do Type.ClassType classType = (Type.ClassType) tree.getType().accept(new PreservedAnnotationTreeVisitor(), null); return new Type.ArrayType(classType, classType.tsym); instead of assuming?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you try a type like A<int[]>? If we can't find a counterexample we can leave this code as is

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It gives a ClassCastException in the TYPE_ARG_VISITOR since it cannot cast PrimitiveType to ClassType

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok then we need a fix somewhere. Please add that as another test case as well

Copy link
Collaborator Author

@akulk022 akulk022 Aug 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was a simple fix to not cast it into ClassType on those lines, as far as this piece of code is concerned, I just realised that we do not accept this Visitor unless we have generic type arguments so we never even reach this method in those situations.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what the latest is here, but let's see what happens after we add the other tests I suggested

Type.ClassType classType =
(Type.ClassType) paramTree.accept(new PreservedAnnotationTreeVisitor(state), null);
return new Type.ArrayType(classType, classType.tsym);
}

@Override
public Type visitParameterizedType(ParameterizedTypeTree tree, Void p) {
Type.ClassType type = (Type.ClassType) ASTHelpers.getType(tree);
msridhar marked this conversation as resolved.
Show resolved Hide resolved
Preconditions.checkNotNull(type);
Type nullableType = NULLABLE_TYPE_SUPPLIER.get(state);
List<? extends Tree> typeArguments = tree.getTypeArguments();
List<Type> newTypeArgs = new ArrayList<>();
boolean hasNullableAnnotation = false;
for (int i = 0; i < typeArguments.size(); i++) {
AnnotatedTypeTree annotatedType = null;
Tree curTypeArg = typeArguments.get(i);
// If the type argument has an annotation, it will either be an AnnotatedTypeTree, or a
// ParameterizedTypeTree in the case of a nested generic type
if (curTypeArg instanceof AnnotatedTypeTree) {
annotatedType = (AnnotatedTypeTree) curTypeArg;
} else if (curTypeArg instanceof ParameterizedTypeTree
&& ((ParameterizedTypeTree) curTypeArg).getType() instanceof AnnotatedTypeTree) {
annotatedType = (AnnotatedTypeTree) ((ParameterizedTypeTree) curTypeArg).getType();
}
List<? extends AnnotationTree> annotations =
annotatedType != null ? annotatedType.getAnnotations() : Collections.emptyList();
for (AnnotationTree annotation : annotations) {
if (ASTHelpers.isSameType(
nullableType, ASTHelpers.getType(annotation.getAnnotationType()), state)) {
hasNullableAnnotation = true;
break;
}
}
// construct a TypeMetadata object containing a nullability annotation if needed
com.sun.tools.javac.util.List<Attribute.TypeCompound> nullableAnnotationCompound =
hasNullableAnnotation
? com.sun.tools.javac.util.List.from(
Collections.singletonList(
new Attribute.TypeCompound(
nullableType, com.sun.tools.javac.util.List.nil(), null)))
: com.sun.tools.javac.util.List.nil();
TypeMetadata typeMetadata =
new TypeMetadata(new TypeMetadata.Annotations(nullableAnnotationCompound));
Type currentTypeArgType = castToNonNull(ASTHelpers.getType(curTypeArg));
List<Type> genericArgType = currentTypeArgType.accept(TYPE_ARG_VISITOR, null);
if (genericArgType.size() > 0) {
// nested generic type; recursively preserve its nullability type argument annotations
currentTypeArgType = curTypeArg.accept(new PreservedAnnotationTreeVisitor(state), null);
}
// Type.ClassType newTypeArgType = (Type.ClassType)
// currentTypeArgType.cloneWithMetadata(typeMetadata);
Type newTypeArgType = currentTypeArgType.cloneWithMetadata(typeMetadata);
newTypeArgs.add(newTypeArgType);
}
Type.ClassType finalType =
new Type.ClassType(
type.getEnclosingType(), com.sun.tools.javac.util.List.from(newTypeArgs), type.tsym);
return finalType;
}
}

/**
* Checks that type parameter nullability is consistent between an overriding method and the
* corresponding overridden method.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -760,6 +760,26 @@ public void rawTypes() {
.doTest();
}

@Test
public void nestedGenericTypeAssignment() {
msridhar marked this conversation as resolved.
Show resolved Hide resolved
makeHelper()
.addSourceLines(
"Test.java",
"package com.uber;",
"import org.jspecify.annotations.Nullable;",
"class Test {",
" static class A<T extends @Nullable Object> { }",
" static void testPositive() {",
" // BUG: Diagnostic contains: Cannot assign from type",
" A<A<@Nullable String>[]> var = new A<A<String>[]>();",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about adding the:

A<A<String>[]> var = new A<A<@Nullable String>[]>();

and

A<A<String>[]> var = new A<A<String>[]>();

cases?

Copy link
Collaborator

@msridhar msridhar Aug 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

" }",
" static void testNegative() {",
" A<A<@Nullable String>[]> var = new A<A<@Nullable String>[]>();",
" }",
"}")
.doTest();
}

@Test
public void overrideReturnTypes() {
makeHelper()
Expand Down