NativeCallback unable to leave thread's last error unchanged

[HexKitchen]

Calling a JavaScript function wrapped as NativeCallback unexpectedly forces the current thread's error status to 0.

JavaScript code can make a change to the thread's error status by assigning to this.errno / this.lastError, and that works correctly. But if the script wishes instead to preserve the original value, it does not work as expected. Instead, the error status is forced to 0. Even if the script expressly sets this.errno / this.lastError to the original value, the error status gets set to 0 instead.

Example:

import frida

pid = frida.spawn("test.exe")
session = frida.attach(pid)

script = session.create_script("""
function myCallback() {
    //console.log("Entering myCallback. this.lastError: " + this.lastError);
    //this.lastError = 1337;   // [1]
}

Interceptor.replace(DebugSymbol.fromName("func1").address, new NativeCallback(myCallback, 'void', []))
""")
script.load()

frida.resume(pid)

input()

#include <iostream>
#include <Windows.h>

#pragma auto_inline(off)

extern "C" {
    void func1()
    {
        std::cout << "In func1" << std::endl;
    }
}


int main()
{
    SetLastError(1337);

    std::cout << "Last error: " << GetLastError() << std::endl;
    func1();
    std::cout << "Last error: " << GetLastError() << std::endl;
    func1();
    std::cout << "Last error: " << GetLastError() << std::endl;
}

Expected output:

Last error: 1337
Last error: 1337
Last error: 1337

Actual output:

Last error: 1337
Last error: 0
Last error: 0

Uncommenting the line marked [1] produces these results:

Last error: 1337
Last error: 0
Last error: 1337

Related issue: #405 .