@overload return union unexpected behaviour

[AgatZan]

How are you using the lua-language-server?

NeoVim

Which OS are you using?

Windows

What is the issue affecting?

Type Checking, Diagnostics/Syntax Checking

Expected Behaviour

Type of r1 = EA|0
Not warning: probably nil

Actual Behaviour

Type of r1 = integer|EA|0
Warning: probably nil

Reproduction steps

1. Create file test.lua
2. Fill

---@enum EA
local EA = {
  a = 1,
  b = 2,
}
local ab = 0
---@overload fun(): 0
---@overload fun(): EA, [string]
local function fua()
  if ab == 0 then
    return 0
  else
    return EA[ab] or EA.a, { "aa" }
  end
end

local r1, r2 = fua()

if r1 == 0 then
  print(0)
else
  print(r2[1])
end

4. See type of r1
5. See Need check nil at line 22

Additional Notes

Maybe duplicate of #1456

Log File

Log file

[tomlau10]

I believe this is a duplicate of #2933 and #1583

Currently there is no support for narrowing based on return tuple type.
Server will just merge all possible types for a given positional return value.

In your example:

• since both overload param list is just fun() => no param based narrowing can be performed
• then your 1st one is returning 0 => 0, nil, 2nd one is returning EA, [string]
• thus when combined, the result variables type will be 0|EA, [string]|nil
  => causing line22 to warn Need check nil

additional notes

• @overload means an additional signature, this will not change the auto infer of the function body.
  if you don't annotate a function with @param / @return first, there will still be an auto inferred signature for that function
• the function body itself has an auto inferred signature as:

function fua()
  -> integer
  2. table|nil

(you can see from its hover)
=> that's why you are seeing r1: integer|EA|0, but not r1: EA|0

• this has been reported in Overload intellise prototype issue with meta functions #2709
  and I have tried to fix it in Fix incorrect function params' type infer when there is only @overload #2838
  which in short ignore baseline function signature when there is only @overload
  but seems my fix only works when the function has >= 1 params 😕
• maybe extra handling needs to be added in the if not args case, to remove the baseline function
  

  lua-language-server/script/vm/function.lua

  Lines 389 to 392 in 807ae52

  	function vm.getExactMatchedFunctions(func, args)
  	local funcs = vm.getMatchedFunctions(func, args)
  	if not args or not funcs then
  	return funcs

    if not funcs or #funcs == 1 then
        return funcs
    elseif not args then
        -- remove the base function if it is annotated with only @overload
        for i, n in ipairs(funcs) do
            if vm.isFunctionWithOnlyOverloads(n) then
                table.remove(funcs, i)
                break
            end
        end
        return funcs
    end

[tomlau10]

Seems my fix above works for your example

• now r1: 0|EA and r2: [string]
  
• however since there is no type narrowing based on tuple return type
  therefore if you try to access r2[1] in the if r1 == 0 case
  => there will not be need-nil-check warning 😅
  => because r2 is just [string] now

---

if you like this change, you may test the patch and PR it

[AgatZan]

@tomlau10 First of all, thanks. Then I found another issue with that example, which may also be duplicated.

---@type string
local ab = ''
-- or
---@param abc string
---@return EA, [string]
---@overload fun(): 0
local function lua(abc) --- fun(string):EA|0, [string]
  if abc == 0 then
    return 0
  else
    return EA[ab] or EA.a, { "aa" }
  end
end

-- or more
local function nua() -- fun():integer|string
  if true then
    return 0
  else
    return "not allowed"
  end
end

lua, like fua, cannot return 0
nua always return integer(0).

What about it's

=> there will not be need-nil-check warning 😅

It's about false positives and false negatives. In my opinion, false positives is more correct, because it forces you to explicit think about why it is there. But all of that is not allowed.

[tomlau10]

seems you are posting multiple issues in a single example 😂

lua, like fua, cannot return 0

• IIUC, in this example you expect that a language server would trace the if branch logic to infer the return value of a function ?
  i.e., since abc: string | nil (nil from the overload), it can never fulfill abc == 0 => should not return 0
• however I doubt if a language server can perform such perfect infer logic 😕
  I tried to write this code in typescript, and the ts language server also cannot do such perfect infer:

function lua(abc: string | undefined) { // function lua(abc: string | undefined): 0 | ""
    if (abc == 0) { // though this line will have warning:
        // This comparison appears to be unintentional because the types 'string | undefined' and 'number' have no overlap. ts(2367)
        return 0;
    } else {
        return "";
    }
}

• typescript infer the return value as 0 | "", not just 0
  => how the function signature is inferred, is a different logic from what diagnostics are performed.
• so in this new example, maybe what you are requesting is a new diagnostic 🤔
  => check if param type and its comparison type has overlap
  https://typescript.tv/errors/#ts2367

---

nua always return integer(0).

similarly, I also converted it into typescript and here is the result:

function nua() { // function nua(): 0 | "not allowed"
    if (true) {
        return 0;
    } else {
        return "not allowed"    // Unreachable code detected. ts(7027)
    }
}

• typescript language server will still infer the return as 0 | "not allowed", not just 0
• however it has another warning TS7027: unreachable code detected
  https://typescript.tv/errors/#ts7027
• => seems you are asking another new diagnostic: unreachable code 🤔 🤔

---

language server can only perform static analysis at its best, and it will not execute lua logic.
I think it will be too much if we ask it to do perfect infer.

Let me give a very extreme example:

local function rua() -- luals: function rua() -> string|integer
    local result = true
    for i = 1, 100000 do
        result = math.random(2) % 2 == 9
    end
    if result then
        return 0
    else
        return ""
    end
end

• default value for result is true
• however in the middle forloop math.random(2) % 2 will never be 9, it can only be 0 / 1
  => after the forloop, result will always be false
  => thus this function will never return 0
• but do you think a language server can be able to, and efficiently detect it 😂 ?