SafeHandle with other handles than IntPtr

[msedi]

I'm using SafeHandles in my pinvokes for CUDA API calls. I have created many SafeHandles that work pretty well since most of the API calls work with IntPtr for which purpose the SafeHandle was made for.

Now I'm encountering pinvoke signatures. As an example:

public static extern cufftResult cufftCreate(out CufftHandle plan);

where

/// <summary>
/// cufftHandle is a handle type used to store and access CUFFT plans
/// </summary>
public readonly struct CufftHandle
{
  public readonly int Handle;
}

I wanted to use a derived SafeHandle but since the handle is in the end an int I have no idea how to put it in there if it's even possible.

[gfoidl]

but since the handle is in the end an int

(Hope that I understood the problem), but beginning with C# 9 you can use nint (native integer) which has the same size as IntPtr.

[msedi]

@gfoidl: Thanks for your feedback. I think I didn't describe the problem completely. I'm not sure if you are aware of the SafeHandle concept, so I will describe it a bit. If you already know it you can just skip it.

The problem is (as far as I know) that the SafeHandle only accepts IntPtr which are platform dependent (which is OK) for most scenarios because many methods in windows return handles also platform dependent. As a example HFILE or HWND are defined as void* since those are opaque structures where for you as an "end-user" it's ok to only know the handle. But since the handles (HFILE,HWND and others) are void* which can be expressed by IntPtr most old C#/.NET implementations based on the IntPtr they have been used until the SafeHandle was introduced.

The runtime marshaller creates a SafeHandle structures and ensures there's the handle and the attached resources are freed if the handle is not used anymore either by Dispose or GC finalizer.

So for all methods that return an IntPtr as a handle can be exchanged by SafeHandle or its derivatives. BUT - and here is my problem - only if the method returns an IntPtr (meaning an platform dependent pointer).

Here's an example of CreateFile and CloseHandle to close the file again:

HANDLE CreateFileA(
  LPCSTR                lpFileName,
  DWORD                 dwDesiredAccess,
  DWORD                 dwShareMode,
  LPSECURITY_ATTRIBUTES lpSecurityAttributes,
  DWORD                 dwCreationDisposition,
  DWORD                 dwFlagsAndAttributes,
  HANDLE                hTemplateFile
);

BOOL CloseHandle(
  HANDLE hObject
);

when doing pinvoke in C# the HANDLE can be expressed as a SafeFileHandle, here some excerpt from the source how the finalizer works where simply a CloseHandle is called.

  protected override bool ReleaseHandle()
        {
            // When the SafeFileHandle was opened, we likely issued an flock on the created descriptor in order to add
            // an advisory lock.  This lock should be removed via closing the file descriptor, but close can be
            // interrupted, and we don't retry closes.  As such, we could end up leaving the file locked,
            // which could prevent subsequent usage of the file until this process dies.  To avoid that, we proactively
            // try to release the lock before we close the handle. (If it's not locked, there's no behavioral
            // problem trying to unlock it.)
            Interop.Sys.FLock(handle, Interop.Sys.LockOperations.LOCK_UN); // ignore any errors
 
            // Close the descriptor. Although close is documented to potentially fail with EINTR, we never want
            // to retry, as the descriptor could actually have been closed, been subsequently reassigned, and
            // be in use elsewhere in the process.  Instead, we simply check whether the call was successful.
            int result = Interop.Sys.Close(handle);
            if (result != 0)
            {
                t_lastCloseErrorInfo = Interop.Sys.GetLastErrorInfo();
            }
            return result == 0;
        }

In my current situation the function always returns int which is not a pointer but an index to an internal table. But in the end this special int handle is similar to IntPtr but not platform dependent. But the SafeHandle only works with IntPtr. As I said above HANDLE is defined as void*, here is an example of a method that has no IntPtr handle but an int:

public static extern cufftResult cufftPlan1d(out int plan, int nx, cufftType type, int batch);
public static extern cufftResult cufftDestroy(int plan);

I wanted to rewrite this with SafeHandles

public static extern cufftResult cufftPlan1d(out SafeCuFFTHandle plan, int nx, cufftType type, int batch);
public static extern cufftResult cufftDestroy(SafeCuFFTHandle plan);

So I'm not sure if the pinvoke marshaller is aware of this problem and knows that he needs to marshal an int instead of a IntPtr.

[CoffeeFlux]

I think the marshaling code for SafeHandle assumes pointer-sized handles, unfortunately. Given that, I'm not actually sure what our recommendation is for a fixed-sized handle - it might be possible to replicate SafeHandle functionality from only managed code but that seems fairly high-effort. @AaronRobinsonMSFT do you know?

[AaronRobinsonMSFT]

I could be missing something here so please forgive the ignorance, but the IntPtr type shouldn't really matter in most cases. The SafeHandle is mapped to various handle types on Windows (e.g. HANDLE) which is typically of type void* or now in C# nint/nuint - 64-bit on x86_64 and 32-bit on x86. I assume we can all agree that a 32-bit machine here isn't interesting.

This would mean that on an x86_64 machine one would marshal more data than is needed but typically that would just be a full register (64-bit) as opposed to just the lower 32-bits. When the SafeHandle argument spills onto the stack things get more complicated and I appreciate that but lets stick with the "handles are in registers" assumption. One should be able to ignore the upper bits and use the lower in the cleanup code. The mapping from IntPtr to a "int handle" is common enough that even in the documentation it is mentioned. For stack based arguments I would be more intentional about verifying given the assumption on sizes and where the marshaller assumes one argument ends and the next begins.

@msedi Have you hit something specific or is this a preliminary query to avoid future issues?

[msedi]

@AaronRobinsonMSFT: Honestly I didn't try because I was assuming that with the signature

public static extern cufftResult cufftPlan1d(out SafeCuFFTHandle plan, int nx, cufftType type, int batch);

the marshaller tries to fit a 64bit pointer (in x64 mode) or a 32bit pointer (in x86 mode) into the SafeHandle.
Since the value in the original method signature

public static extern cufftResult cufftPlan1d(out int plan, int nx, cufftType type, int batch);

is always int (32bit), isn't this mismatch to the SafeHandle?
Ignoring the x86 case I thought the marshaller assumes that it needs to read an 64bit IntPtr from the methods' stack(?)
and copies the result into the Handle (IntPtr) of the SafeHandle. But since the method only "returns" an 32bit isn't this a problem.
How does the marshaller know what type (bit size) it should marshal ?

Since I don't know the details another assumption was that it work work without any problem when we are in x86 mode.

[AaronRobinsonMSFT]

is always int (32bit), isn't this mismatch to the SafeHandle?

You are giving the marshaller a bit too much credit here. The marshaller only knows what is expressed in the P/Invoke signature. In this case both those signatures are completely acceptable from my perspective. The question here is what is the native signature.

Based on the P/Invoke signatures I am going to assume the C declaration of the function is the following. I am assuming for ease of illustration that cufftType and cufftResult are simple enum or int based values, but that doesn't matter too much in this case. Also, we are targeting a compiler that defines a C int as 32-bit and not 16-bit -, .NET defines int as 32-bit.

cufftResult cufftPlan1d(int* plan, int nx, cufftType type, int batch);

This means that plan is an out and pointing at some supplied memory location likely provided by the caller. So let's go through three scenarios int, IntPtr, and SafeCuFFTHandle (inherited from SafeHandle).

The int is simple because the marshaller sees the out int and knows that it must provide memory for an int and passes a reference (pointer) to that memory to the function. The IntPtr is similar but the marshaller is told to pass a pointer to a pointer sized amount of memory (i.e. sizeof(void*). The C function only asked for a pointer to sizeof(int), and doesn't care if we supply more as long as we give it at least what it needs (i.e. sizeof(int)). Which means in this case the marshaller is wasting 4 bytes of stack on a 64-bit machine.

Hopefully at this point it is clear that specifying an int or an IntPtr are okay with the above signature. Now we come to the SafeCuFFTHandle version. This one is the same as the IntPtr actually. The SafeHandle type has a special marshaller that assumes a handle is no bigger than a pointer but is permitted to be less. It is up to the implementer of the subclass of SafeHandle to properly interpret and handle the filled out IntPtr value in the SafeHandle base field.

In parameters are similar in that we are providing more memory than is read by the native function. The marshaller is definitely wasting stack space and it could be reduced if one wanted to write a custom SafeHandle implementation.

[msedi]

@AaronRobinsonMSFT: Thanks a lot for your detailed explanation. That helps a lot.