DllNotFoundException from Android emulator when calling into a native dll

[AArnott]

Android application type

.NET Android (net7.0-android, net8.0-android, etc.)

Affected platform version

VS 2022 Update 9, net8.0-android

Description

When my new Avalonia Android project consumes a native library, at runtime it throws DllNotFoundException.
These native libraries come by way of NuGet. I've tried Nerdbank.Zcash and libgit2sharp. Both of these carry a runtimes folder. The Nerdbank.Zcash package in particular carries a runtimes/android-x64/native/libnerdbank_zcash_rust.so file.
I can see that the file is included in the apk. But the android emulator still fails.

More details and diagnostic efforts are described in AArnott/Avalonia-WithNativeDependency#1

Steps to Reproduce

A minimal repro is available here: https://github.com/AArnott/Avalonia-WithNativeDependency
Be sure to switch the solution platform to x64, since in AnyCPU, no native binary is included in the deployed app.

In effect:

1. Create a new Avalonia Android app. Bump its target framework from net7.0-android to net8.0-android.
2. Add a PackageReference to Nerdbank.Zcash
3. Add code that exercises the native library, for example these two lines.

Run the app in the android emulator. (I haven't tried a real android device).

Did you find any workaround?

Nothing yet.

Relevant log output

No response

[AArnott]

I just tested this on a physical android device. It works fine. Yay.
So... why is the emulator broken?

More trivia: the emulator is running x86_64, and that's the ABI of the binary selected for inclusion in the app as well.

[maxkatz6]

Would be useful to know what's a typical approach diagnosing DllNotFoundException issues on Android. Supposedly it can happen when ABI is wrong or there is other problem with .so file.

[grendello]

@AArnott I don't think Avalonia use much of our infrastructure, so this issue might probably be better filed in Avalonia, but can you provide the APK/AAB archives you get? I'd like to examine what's in them as produced on your machine.

[dellis1972]

Looking at the Nuget package for Nerdbank.Zcash 0.2.68-beta it does not contain any andorid runtimes. only linux-x64?

[dellis1972]

see https://nuget.info/packages/Nerdbank.Zcash/0.2.68-beta

[akoeplinger]

@dellis1972 you need to get the latest version from the nuget feed: https://github.com/AArnott/Avalonia-WithNativeDependency/blob/master/nuget.config#L7

[dellis1972]

ok. My guess then is its trying to use an x64 binary on an x86 or arm image on the emulator (as it works on a device).

[grendello]

@AArnott also, a detailed exception message and stack trace would be necessary to see what might be going on. Try these commands to get the logcat output (including our runtime logging, not sure if it works with Avalonia):

$ adb shell setprop debug.mono.log default,assembly,timing=bare,mono_log_level=debug,mono_log_mask=all
$ adb logcat -G 64M
$ adb logcat -c
# Start the app here, wait for it to crash, then wait 15s and
$ adb logcat -d > log.txt

Attach the resulting log.txt to this issue please.

[jonpryor]

Running the sample in an emulator, two issues stand out:

@AArnott: First, the emulator doesn't like libnerdbank_zcash_rust.so:

I monodroid-assembly: Failed to load shared library '/data/app/com.CompanyName.AvaloniaTest-D9SFv_VrlfJEnJE_eJbnFQ==/lib/x86_64/libnerdbank_zcash_rust.so'. dlopen
 failed: cannot locate symbol "__extenddftf2" referenced by "/data/app/com.CompanyName.AvaloniaTest-D9SFv_VrlfJEnJE_eJbnFQ==/lib/x86_64/libnerdbank_zcash_rust.so"...

i.e. "something is wrong" with libnerdbank_zcash_rust.so. Some cursory searches find:

• NDK version compatibility mozilla/rust-android-gradle#105
• [bug] cargo tauri android dev -> cannot locate symbol __extenddftf2 tauri-apps/tauri#7413
• flutter run fails on x86_64 emulator breez/c-breez#553 (comment)

@grendello: The other issue is that commit 1aa0ea7 appears to have a bug, as my adb logcat contains:

I DOTNET  : JavaProxyThrowable: translation threw an exception: Java.Lang.NullPointerException: Declaring class is null
I DOTNET  :    at Java.Interop.JniEnvironment.InstanceMethods.CallNonvirtualVoidMethod(JniObjectReference instance, JniObjectReference type, JniMethodInfo method, JniArgumentValue* args)
I DOTNET  :    at Java.Interop.JniPeerMembers.JniInstanceMethods.FinishCreateInstance(String constructorSignature, IJavaPeerable self, JniArgumentValue* parameter s)
I DOTNET  :    at Java.Lang.StackTraceElement..ctor(String declaringClass, String methodName, String fileName, Int32 lineNumber)
I DOTNET  :    at Android.Runtime.JavaProxyThrowable.TranslateStackTrace()
I DOTNET  :    at Android.Runtime.JavaProxyThrowable.Create(Exception innerException)
I DOTNET  :   --- End of managed Java.Lang.NullPointerException stack trace ---
I DOTNET  : java.lang.NullPointerException: Declaring class is null
I DOTNET  :      at java.util.Objects.requireNonNull(Objects.java:228)
I DOTNET  :      at java.lang.StackTraceElement.<init>(StackTraceElement.java:71)
I DOTNET  :      at crc6431345fe65afe8d98.AvaloniaMainActivity_1.n_onCreate(Native Method)

Looks like this line (and others?) cannot be null:

https://github.com/xamarin/xamarin-android/blob/e41a31b9520dd1a4dbfe8162cf49fed7052dc69f/src/Mono.Android/Android.Runtime/JavaProxyThrowable.cs#L74

[grendello]

@jonpryor looks like that NREX is something really unexpected, will open a PR today.

[maxkatz6]

@grendello classic Avalonia mobile backends are built on top of .NET workloads (i.e. new Xamarin.Native), xamarin-android and xamarin-ios.

[AArnott]

Thanks folks. I've tried applying the workaround, but it requires the ANDROID_NDK_HOME env var to be set, which cross build doesn't set. I've tried setting it to the path where the ndk is present inside the docker image that cross build uses, but it lacks the toolchains subdirectory, so it ultimately fails anyway.

Here is my attempt:
https://github.com/nerdcash/Nerdbank.Cryptocurrencies/pull/262/files

[AArnott]

I got the NDK issues worked out and a successful build. However it still fails on the emulator.
@jonpryor how did you get the error about the missing external? I never saw that in my error log, and I'm wondering if the workaround that you linked to and I applied did work, but perhaps there's yet another problem to be seen from those logs.

[AArnott]

I'll try the logcat commands... if I can find where to enter them...

[AArnott]

It looks like it's still failing with the same missing external, despite having applied this workaround.

log.txt

03-11 08:47:11.418  4742  4742 D monodroid-assembly: monodroid_dlopen: hash for name 'nerdbank_zcash_rust' is 0xd0f2277af2079d08
03-11 08:47:11.419  4742  4742 D monodroid-assembly: monodroid_dlopen: hash match found, DSO name is 'libnerdbank_zcash_rust.so'
03-11 08:47:11.419  4742  4742 I monodroid-assembly: Trying to load shared library '/data/user/0/com.CompanyName.AvaloniaTest/files/.__override__/libnerdbank_zcash_rust.so'
03-11 08:47:11.419  4742  4742 I monodroid-assembly: Shared library '/data/user/0/com.CompanyName.AvaloniaTest/files/.__override__/libnerdbank_zcash_rust.so' not found
03-11 08:47:11.419  4742  4742 I monodroid-assembly: Trying to load shared library '/data/app/~~w-RdbcHnRRxxJPEf2jUm5w==/com.CompanyName.AvaloniaTest-YCtUhwt-dHGlFQ876tmu7A==/lib/x86_64/libnerdbank_zcash_rust.so'
03-11 08:47:11.435  4742  4742 I monodroid-assembly: Failed to load shared library '/data/app/~~w-RdbcHnRRxxJPEf2jUm5w==/com.CompanyName.AvaloniaTest-YCtUhwt-dHGlFQ876tmu7A==/lib/x86_64/libnerdbank_zcash_rust.so'. dlopen failed: cannot locate symbol "__extenddftf2" referenced by "/data/app/~~w-RdbcHnRRxxJPEf2jUm5w==/com.CompanyName.AvaloniaTest-YCtUhwt-dHGlFQ876tmu7A==/lib/x86_64/libnerdbank_zcash_rust.so"...
03-11 08:47:11.435  4742  4742 I monodroid-assembly: Trying to load shared library '/data/user/0/com.CompanyName.AvaloniaTest/files/.__override__/nerdbank_zcash_rust'
03-11 08:47:11.435  4742  4742 I monodroid-assembly: Shared library '/data/user/0/com.CompanyName.AvaloniaTest/files/.__override__/nerdbank_zcash_rust' not found
03-11 08:47:11.435  4742  4742 I monodroid-assembly: Trying to load shared library '/data/app/~~w-RdbcHnRRxxJPEf2jUm5w==/com.CompanyName.AvaloniaTest-YCtUhwt-dHGlFQ876tmu7A==/lib/x86_64/nerdbank_zcash_rust'
03-11 08:47:11.435  4742  4742 I monodroid-assembly: Shared library '/data/app/~~w-RdbcHnRRxxJPEf2jUm5w==/com.CompanyName.AvaloniaTest-YCtUhwt-dHGlFQ876tmu7A==/lib/x86_64/nerdbank_zcash_rust' not found
03-11 08:47:11.435  4742  4742 W monodroid-assembly: Shared library 'nerdbank_zcash_rust' not loaded, p/invoke 'get_orchard_fvk_bytes_from_sk_bytes' may fail

[jonpryor]

@AArnott: you can use llvm-nm from the NDK to see if the symbol is required without running the app on an emulator:

# update as appropriate for your host OS
% $ANDROID_NDK_HOME/toolchains/llvm/prebuilt/darwin-x86_64/bin/llvm-nm -D \
  …/bin/Debug/net8.0/runtimes/android-x64/native/libnerdbank_zcash_rust.so
…
                 U __extenddftf2

That won't tell you whether or not the binary is usable on the emulator -- running on the emulator is the best for that -- but it will be useful as part of the inner dev loop of "did this fix remove the use of this symbol?.

[AArnott]

Thanks. I ran that command on the binary before and after the popular workaround was applied. The before output included U __extenddftf2 and the after output did not include that symbol in the file at all (regardless of the single character prefix).
I'm not sure how to interpret this result. The workaround did something, but was it the right thing?
And given the symbol no longer appears in the after output, why is the emulator still failing with an error mentioning it?

[grendello]

@AArnott the symbol missing after applying the workaround is the desired outcome, so we know the problem is fixed on that end. To make sure the correct library is used in the application, you might want to remove the application completely from the device and then perform a clean build of your app (after removing all the bin and obj directories in your build tree). You might want to build the application from the command line with dotnet build -bl, then deploy the resulting .apk file to the emulator with adb install path/to/file.apk and see if that works fine. If it doesn't, examine the msbuild.binlog file that was created by the build command (you can either use https://msbuildlog.com/ or dotnet msbuild -v:diag msbuild.binlog > msbuild.txt) and look for mentions of your shared library's name, making sure that it is copied from the correct location.

[AArnott]

Thank you very much for those specific steps. In following them, I found some issues that may have led to the wrong binary being delivered to the emulator. Even after fixing those, the emulator crashed the first time, but then when debugging from VS, the crash disappeared, and then stayed away when not using the debugger. So I'm not sure what all was happening, but it appears to be resolved. Thank you!