Skip to content

Commit

Permalink
handle explicit generic routine instantiations in sigmatch (#24010)
Browse files Browse the repository at this point in the history 
fixes #16376

The way the compiler handled generic proc instantiations in calls (like
`foo[int](...)`) up to this point was to instantiate `foo[int]`, create
a symbol for the instantiated proc (or a symchoice for multiple procs
excluding ones with mismatching generic param counts), then perform
overload resolution on this symbol/symchoice. The exception to this was
when the called symbol was already a symchoice node, in which case it
wasn't instantiated and overloading was called directly ([these
lines](https://github.com/nim-lang/Nim/blob/b7b1313d21deb687adab2b4a162e716ba561a26b/compiler/semexprs.nim#L3366-L3371)).

This has several problems:

* Templates and macros can't create instantiated symbols, so they
couldn't participate in overloaded explicit generic instantiations,
causing the issue #16376.
* Every single proc that can be instantiated with the given generic
params is fully instantiated including the body. #9997 is about this but
isn't fixed here since the instantiation isn't in a call.

The way overload resolution handles explicit instantiations by itself is
also buggy:

* It doesn't check constraints.
* It allows only partially providing the generic parameters, which makes
sense for implicit generics, but can cause ambiguity in overloading.

Here is how this PR deals with these problems:

* Overload resolution now always handles explicit generic instantiations
in calls, in `initCandidate`, as long as the symbol resolves to a
routine symbol.
* Overload resolution now checks the generic params for constraints and
correct parameter count (ignoring implicit params). If these don't
match, the entire overload is considered as not matching and not
instantiated.
* Special error messages are added for mismatching/missing/extra generic
params. This is almost all of the diff in `semcall`.
* Procs with matching generic parameters now instantiate only the type
of the signature in overload resolution, not the proc itself, which also
works for templates and macros.

Unfortunately we can't entirely remove instantiations because overload
resolution can't handle some cases with uninstantiated types even though
it's resolved in the binding (see the last 2 blocks in
`texplicitgenerics`). There are also some instantiation issues with
default params that #24005 didn't fix but I didn't want this to become
the 3rd huge generics PR in a row so I didn't dive too deep into trying
to fix them. There is still a minor instantiation fix in `semtypinst`
though for subscripts in calls.

Additional changes:

* Overloading of `[]` wasn't documented properly, it somewhat is now
because we need to mention the limitation that it can't be done for
generic procs/types.
* Tests can now enable the new type mismatch errors with just
`-d:testsConciseTypeMismatch` in the command.

Package PRs:

- using fork for now:
[combparser](PMunch/combparser#7) (partial
generic instantiation)
- merged: [cligen](c-blake/cligen#233) (partial
generic instantiation but non-overloaded + template)
- merged: [neo](andreaferretti/neo#56) (trying
to instantiate template with no generic param)
  • Loading branch information
@metagn
metagn authored Sep 2, 2024
1 parent c8af099 commit 71de7fc
Show file tree
Hide file tree
Showing 22 changed files with 436 additions and 118 deletions.
1 change: 1 addition & 0 deletions compiler/ast.nim
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1511,6 +1511,7 @@ proc newType*(kind: TTypeKind; idgen: IdGenerator; owner: PSym; son: sink PType

proc setSons*(dest: PType; sons: sink seq[PType]) {.inline.} = dest.sons = sons
proc setSon*(dest: PType; son: sink PType) {.inline.} = dest.sons = @[son]
proc setSonsLen*(dest: PType; len: int) {.inline.} = setLen(dest.sons, len)

proc mergeLoc(a: var TLoc, b: TLoc) =
if a.k == low(typeof(a.k)): a.k = b.k
Expand Down
1 change: 1 addition & 0 deletions compiler/sem.nim
View file
Original file line number Diff line number Diff line change
Expand Up @@ -728,6 +728,7 @@ proc preparePContext*(graph: ModuleGraph; module: PSym; idgen: IdGenerator): PCo
result.semOverloadedCall = semOverloadedCall
result.semInferredLambda = semInferredLambda
result.semGenerateInstance = generateInstance
result.instantiateOnlyProcType = instantiateOnlyProcType
result.semTypeNode = semTypeNode
result.instTypeBoundOp = sigmatch.instTypeBoundOp
result.hasUnresolvedArgs = hasUnresolvedArgs
Expand Down
86 changes: 65 additions & 21 deletions compiler/semcall.nim
View file
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,7 @@ proc presentFailedCandidates(c: PContext, n: PNode, errors: CandidateErrors):
let nArg = if err.firstMismatch.arg < n.len: n[err.firstMismatch.arg] else: nil
let nameParam = if err.firstMismatch.formal != nil: err.firstMismatch.formal.name.s else: ""
if n.len > 1:
const genericParamMismatches = {kGenericParamTypeMismatch, kExtraGenericParam, kMissingGenericParam}
if verboseTypeMismatch notin c.config.legacyFeatures:
case err.firstMismatch.kind
of kUnknownNamedParam:
Expand All @@ -269,6 +270,12 @@ proc presentFailedCandidates(c: PContext, n: PNode, errors: CandidateErrors):
of kMissingParam:
candidates.add(" missing parameter: " & nameParam)
candidates.add "\n"
of kExtraGenericParam:
candidates.add(" extra generic param given")
candidates.add "\n"
of kMissingGenericParam:
candidates.add(" missing generic parameter: " & nameParam)
candidates.add "\n"
of kVarNeeded:
doAssert nArg != nil
doAssert err.firstMismatch.formal != nil
Expand All @@ -292,9 +299,36 @@ proc presentFailedCandidates(c: PContext, n: PNode, errors: CandidateErrors):
candidates.addPragmaAndCallConvMismatch(wanted, got, c.config)
effectProblem(wanted, got, candidates, c)
candidates.add "\n"
of kGenericParamTypeMismatch:
let pos = err.firstMismatch.arg
doAssert n[0].kind == nkBracketExpr and pos < n[0].len
let arg = n[0][pos]
doAssert arg != nil
var wanted = err.firstMismatch.formal.typ
if wanted.kind == tyGenericParam and wanted.genericParamHasConstraints:
wanted = wanted.genericConstraint
let got = arg.typ
doAssert err.firstMismatch.formal != nil
doAssert wanted != nil
doAssert got != nil
candidates.add " generic parameter mismatch, expected "
candidates.addTypeDeclVerboseMaybe(c.config, wanted)
candidates.add " but got '"
candidates.add renderTree(arg)
candidates.add "' of type: "
candidates.addTypeDeclVerboseMaybe(c.config, got)
if got != nil and got.kind == tyProc and wanted.kind == tyProc:
# These are proc mismatches so,
# add the extra explict detail of the mismatch
candidates.addPragmaAndCallConvMismatch(wanted, got, c.config)
if got != nil:
effectProblem(wanted, got, candidates, c)
candidates.add "\n"
of kUnknown: discard "do not break 'nim check'"
else:
candidates.add(" first type mismatch at position: " & $err.firstMismatch.arg)
if err.firstMismatch.kind in genericParamMismatches:
candidates.add(" in generic parameters")
# candidates.add "\n reason: " & $err.firstMismatch.kind # for debugging
case err.firstMismatch.kind
of kUnknownNamedParam:
Expand All @@ -306,20 +340,35 @@ proc presentFailedCandidates(c: PContext, n: PNode, errors: CandidateErrors):
of kPositionalAlreadyGiven: candidates.add("\n positional param was already given as named param")
of kExtraArg: candidates.add("\n extra argument given")
of kMissingParam: candidates.add("\n missing parameter: " & nameParam)
of kTypeMismatch, kVarNeeded:
doAssert nArg != nil
let wanted = err.firstMismatch.formal.typ
of kExtraGenericParam:
candidates.add("\n extra generic param given")
of kMissingGenericParam:
candidates.add("\n missing generic parameter: " & nameParam)
of kTypeMismatch, kGenericParamTypeMismatch, kVarNeeded:
var arg: PNode = nArg
let genericMismatch = err.firstMismatch.kind == kGenericParamTypeMismatch
if genericMismatch:
let pos = err.firstMismatch.arg
doAssert n[0].kind == nkBracketExpr and pos < n[0].len
arg = n[0][pos]
else:
arg = nArg
doAssert arg != nil
var wanted = err.firstMismatch.formal.typ
if genericMismatch and wanted.kind == tyGenericParam and
wanted.genericParamHasConstraints:
wanted = wanted.genericConstraint
doAssert err.firstMismatch.formal != nil
candidates.add("\n required type for " & nameParam & ": ")
candidates.addTypeDeclVerboseMaybe(c.config, wanted)
candidates.add "\n but expression '"
if err.firstMismatch.kind == kVarNeeded:
candidates.add renderNotLValue(nArg)
candidates.add renderNotLValue(arg)
candidates.add "' is immutable, not 'var'"
else:
candidates.add renderTree(nArg)
candidates.add renderTree(arg)
candidates.add "' is of type: "
let got = nArg.typ
let got = arg.typ
candidates.addTypeDeclVerboseMaybe(c.config, got)
doAssert wanted != nil
if got != nil:
Expand All @@ -331,8 +380,8 @@ proc presentFailedCandidates(c: PContext, n: PNode, errors: CandidateErrors):

of kUnknown: discard "do not break 'nim check'"
candidates.add "\n"
if err.firstMismatch.arg == 1 and nArg.kind == nkTupleConstr and
n.kind == nkCommand:
if err.firstMismatch.arg == 1 and nArg != nil and
nArg.kind == nkTupleConstr and n.kind == nkCommand:
maybeWrongSpace = true
for diag in err.diagnostics:
candidates.add(diag & "\n")
Expand Down Expand Up @@ -780,21 +829,16 @@ proc explicitGenericInstError(c: PContext; n: PNode): PNode =
result = n

proc explicitGenericSym(c: PContext, n: PNode, s: PSym): PNode =
if s.kind in {skTemplate, skMacro}:
internalError c.config, n.info, "cannot get explicitly instantiated symbol of " &
(if s.kind == skTemplate: "template" else: "macro")
# binding has to stay 'nil' for this to work!
var m = newCandidate(c, s, nil)

for i in 1..<n.len:
let formal = s.ast[genericParamsPos][i-1].typ
var arg = n[i].typ
# try transforming the argument into a static one before feeding it into
# typeRel
if formal.kind == tyStatic and arg.kind != tyStatic:
let evaluated = c.semTryConstExpr(c, n[i], n[i].typ)
if evaluated != nil:
arg = newTypeS(tyStatic, c, son = evaluated.typ)
arg.n = evaluated
let tm = typeRel(m, formal, arg)
if tm in {isNone, isConvertible}: return nil
matchGenericParams(m, n, s)
if m.state != csMatch:
# state is csMatch only if *all* generic params were matched,
# including implicit parameters
return nil
var newInst = generateInstance(c, s, m.bindings, n.info)
newInst.typ.flags.excl tfUnresolved
let info = getCallLineInfo(n)
Expand Down
3 changes: 3 additions & 0 deletions compiler/semdata.nim
View file
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,9 @@ type
semInferredLambda*: proc(c: PContext, pt: Table[ItemId, PType], n: PNode): PNode
semGenerateInstance*: proc (c: PContext, fn: PSym, pt: Table[ItemId, PType],
info: TLineInfo): PSym
instantiateOnlyProcType*: proc (c: PContext, pt: TypeMapping,
prc: PSym, info: TLineInfo): PType
# used by sigmatch for explicit generic instantiations
includedFiles*: IntSet # used to detect recursive include files
pureEnumFields*: TStrTable # pure enum fields that can be used unambiguously
userPragmas*: TStrTable
Expand Down
58 changes: 31 additions & 27 deletions compiler/semexprs.nim
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1070,14 +1070,6 @@ proc resolveIndirectCall(c: PContext; n, nOrig: PNode;
result = initCandidate(c, t)
matches(c, n, nOrig, result)

proc bracketedMacro(n: PNode): PSym =
if n.len >= 1 and n[0].kind == nkSym:
result = n[0].sym
if result.kind notin {skMacro, skTemplate}:
result = nil
else:
result = nil

proc finishOperand(c: PContext, a: PNode): PNode =
if a.typ.isNil:
result = c.semOperand(c, a, {efDetermineType})
Expand Down Expand Up @@ -1167,11 +1159,6 @@ proc semIndirectOp(c: PContext, n: PNode, flags: TExprFlags; expectedType: PType
let t = n[0].typ
if t != nil and t.kind in {tyVar, tyLent}:
n[0] = newDeref(n[0])
elif n[0].kind == nkBracketExpr:
let s = bracketedMacro(n[0])
if s != nil:
setGenericParams(c, n[0], s.ast[genericParamsPos])
return semDirectOp(c, n, flags, expectedType)
elif isSymChoice(n[0]) and nfDotField notin n.flags:
# overloaded generic procs e.g. newSeq[int] can end up here
return semDirectOp(c, n, flags, expectedType)
Expand Down Expand Up @@ -1721,8 +1708,6 @@ proc maybeInstantiateGeneric(c: PContext, n: PNode, s: PSym): PNode =
result = explicitGenericInstantiation(c, n, s)
if result == n:
n[0] = copyTree(result[0])
else:
n[0] = result

proc semSubscript(c: PContext, n: PNode, flags: TExprFlags): PNode =
## returns nil if not a built-in subscript operator; also called for the
Expand Down Expand Up @@ -3013,19 +2998,42 @@ proc semTupleConstr(c: PContext, n: PNode, flags: TExprFlags; expectedType: PTyp
else:
result = tupexp

proc shouldBeBracketExpr(n: PNode): bool =
result = false
proc isExplicitGenericCall(c: PContext, n: PNode): bool =
## checks if a call node `n` is a routine call with explicit generic params
##
## the callee node needs to be either an nkBracketExpr or a call to a
## symchoice of `[]` in which case it will be transformed into nkBracketExpr
##
## the LHS of the bracket expr has to either be a symchoice or resolve to
## a routine symbol
template checkCallee(n: PNode) =
# check subscript LHS, `n` must be mutable
if isSymChoice(n):
result = true
else:
let s = qualifiedLookUp(c, n, {})
if s != nil and s.kind in routineKinds:
result = true
n = semSymGenericInstantiation(c, n, s)
assert n.kind in nkCallKinds
result = false
let a = n[0]
if a.kind in nkCallKinds:
case a.kind
of nkBracketExpr:
checkCallee(a[0])
of nkCallKinds:
let b = a[0]
if b.kind in nkSymChoices:
for i in 0..<b.len:
if b[i].kind == nkSym and b[i].sym.magic == mArrGet:
let be = newNodeI(nkBracketExpr, n.info)
let name = b.getPIdent
if name != nil and name.s == "[]":
checkCallee(a[1])
if result:
# transform callee into normal bracket expr, only on success
let be = newNodeI(nkBracketExpr, a.info)
for i in 1..<a.len: be.add(a[i])
n[0] = be
return true
else:
result = false

proc asBracketExpr(c: PContext; n: PNode): PNode =
proc isGeneric(c: PContext; n: PNode): bool =
Expand Down Expand Up @@ -3365,11 +3373,7 @@ proc semExpr(c: PContext, n: PNode, flags: TExprFlags = {}, expectedType: PType
else:
#liMessage(n.info, warnUser, renderTree(n));
result = semIndirectOp(c, n, flags, expectedType)
elif (n[0].kind == nkBracketExpr or shouldBeBracketExpr(n)) and
isSymChoice(n[0][0]):
# indirectOp can deal with explicit instantiations; the fixes
# the 'newSeq[T](x)' bug
setGenericParams(c, n[0], nil)
elif isExplicitGenericCall(c, n): # this modifies `n` if true
result = semDirectOp(c, n, flags, expectedType)
elif nfDotField in n.flags:
result = semDirectOp(c, n, flags, expectedType)
Expand Down
16 changes: 16 additions & 0 deletions compiler/seminst.nim
View file
Original file line number Diff line number Diff line change
Expand Up @@ -321,6 +321,22 @@ proc instantiateProcType(c: PContext, pt: TypeMapping,
prc.typ = result
popInfoContext(c.config)

proc instantiateOnlyProcType(c: PContext, pt: TypeMapping, prc: PSym, info: TLineInfo): PType =
# instantiates only the type of a given proc symbol
# used by sigmatch for explicit generics
# wouldn't be needed if sigmatch could handle complex cases,
# examples are in texplicitgenerics
# might be buggy, see rest of generateInstance if problems occur
let fakeSym = copySym(prc, c.idgen)
incl(fakeSym.flags, sfFromGeneric)
fakeSym.instantiatedFrom = prc
openScope(c)
for s in instantiateGenericParamList(c, prc.ast[genericParamsPos], pt):
addDecl(c, s)
instantiateProcType(c, pt, fakeSym, info)
closeScope(c)
result = fakeSym.typ

proc fillMixinScope(c: PContext) =
var p = c.p
while p != nil:
Expand Down
13 changes: 10 additions & 3 deletions compiler/semtypinst.nim
View file
Original file line number Diff line number Diff line change
Expand Up @@ -147,10 +147,14 @@ proc prepareNode(cl: var TReplTypeVars, n: PNode): PNode =
# exception exists for the call name being a dot expression since
# dot expressions need their LHS instantiated
assert n.len != 0
let ignoreFirst = n[0].kind != nkDotExpr
# avoid instantiating generic proc symbols, refine condition if needed:
let ignoreFirst = n[0].kind notin {nkDotExpr, nkBracketExpr} + nkCallKinds
let name = n[0].getPIdent
let ignoreSecond = name != nil and name.s == "[]" and n.len > 1 and
(n[1].typ != nil and n[1].typ.kind == tyTypeDesc)
# generic type instantiation:
((n[1].typ != nil and n[1].typ.kind == tyTypeDesc) or
# generic proc instantiation:
(n[1].kind == nkSym and n[1].sym.isGenericRoutineStrict))
if ignoreFirst:
result.add(n[0])
else:
Expand All @@ -168,7 +172,10 @@ proc prepareNode(cl: var TReplTypeVars, n: PNode): PNode =
# dot expressions need their LHS instantiated
assert n.len != 0
let ignoreFirst = n[0].kind != nkDotExpr and
n[0].typ != nil and n[0].typ.kind == tyTypeDesc
# generic type instantiation:
((n[0].typ != nil and n[0].typ.kind == tyTypeDesc) or
# generic proc instantiation:
(n[0].kind == nkSym and n[0].sym.isGenericRoutineStrict))
if ignoreFirst:
result.add(n[0])
else:
Expand Down
Loading

0 comments on commit 71de7fc

Please sign in to comment.