run llama.cpp failed

[xvim]

Using cricket ,I can run nbody success. but failed to run llama.cpp in a virtual machine.

liucong@liucong:~$ ldd nbody 
	linux-vdso.so.1 (0x00007ffed65af000)
	libGL.so.1 => /lib/x86_64-linux-gnu/libGL.so.1 (0x000074b5e1d24000)
	libGLU.so.1 => /lib/x86_64-linux-gnu/libGLU.so.1 (0x000074b5e1cce000)
	libglut.so.3 => /lib/x86_64-linux-gnu/libglut.so.3 (0x000074b5e1c81000)
	libcudart.so.11.0 => /lib/x86_64-linux-gnu/libcudart.so.11.0 (0x000074b5e1800000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x000074b5e1400000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x000074b5e1b98000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x000074b5e1b78000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000074b5e1000000)
	libGLdispatch.so.0 => /lib/x86_64-linux-gnu/libGLdispatch.so.0 (0x000074b5e1ac0000)
	libGLX.so.0 => /lib/x86_64-linux-gnu/libGLX.so.0 (0x000074b5e17cc000)
	libOpenGL.so.0 => /lib/x86_64-linux-gnu/libOpenGL.so.0 (0x000074b5e17a0000)
	libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x000074b5e1660000)
	libXi.so.6 => /lib/x86_64-linux-gnu/libXi.so.6 (0x000074b5e1aaa000)
	libXxf86vm.so.1 => /lib/x86_64-linux-gnu/libXxf86vm.so.1 (0x000074b5e1aa3000)
	/lib64/ld-linux-x86-64.so.2 (0x000074b5e1e07000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x000074b5e165b000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x000074b5e1656000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x000074b5e1651000)
	libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x000074b5e13d6000)
	libXext.so.6 => /lib/x86_64-linux-gnu/libXext.so.6 (0x000074b5e163a000)
	libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x000074b5e1634000)
	libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x000074b5e162c000)
	libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x000074b5e13be000)
	libmd.so.0 => /lib/x86_64-linux-gnu/libmd.so.0 (0x000074b5e13af000)

liucong@liucong:~$ REMOTE_GPU_ADDRESS=10.42.143.229 LD_PRELOAD=/home/liucong/cricket-client.so ./nbody --benchmark
+08:00:00.000007 INFO:	connection to host "10.42.143.229"
+08:00:00.000239 INFO:	connecting via TCP...
Run "nbody -benchmark [-numbodies=<numBodies>]" to measure performance.
	-fullscreen       (run n-body simulation in fullscreen mode)
	-fp64             (use double precision floating point values for simulation)
	-hostmem          (stores simulation data in host memory)
	-benchmark        (run benchmark to measure performance) 
	-numbodies=<N>    (number of bodies (>= 1) to run in simulation) 
	-device=<d>       (where d=0,1,2.... for the CUDA device to use)
	-numdevices=<i>   (where i=(number of CUDA devices > 0) to use for simulation)
	-compare          (compares simulation results running once on the default GPU and once on the CPU)
	-cpu              (run n-body simulation on the CPU)
	-tipsy=<file.bin> (load a tipsy model file for simulation)

NOTE: The CUDA Samples are not meant for performance measurements. Results may vary when GPU Boost is enabled.

> Windowed mode
> Simulation data stored in video memory
> Single precision floating point simulation
> 1 Devices used for simulation
GPU Device 0: "Ada" with compute capability 8.9

> Compute 8.9 CUDA device: [NVIDIA GeForce RTX 4060 Ti]
34816 bodies, total time for 10 iterations: 5123.033 ms
= 2.366 billion interactions per second
= 47.322 single-precision GFLOP/s at 20 flops per interaction
+08:00:05.780127 INFO:	api-call-cnt: 76
+08:00:05.780172 INFO:	memcpy-cnt: 3342360

liucong@liucong:~$ cd llama.cpp_cuda/
liucong@liucong:~/llama.cpp_cuda$ ldd build/bin/llama-server 
	linux-vdso.so.1 (0x00007ffcaf1d4000)
	libllama.so => /home/liucong/llama.cpp_cuda/build/src/libllama.so (0x00007e91d900c000)
	libggml.so => /home/liucong/llama.cpp_cuda/build/ggml/src/libggml.so (0x00007e91cb000000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007e91cac00000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007e91d8f16000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007e91d8ef6000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007e91ca800000)
	/lib64/ld-linux-x86-64.so.2 (0x00007e91d931c000)
	libcudart.so.11.0 => /lib/x86_64-linux-gnu/libcudart.so.11.0 (0x00007e91ca400000)
	libcublas.so.11 => /lib/x86_64-linux-gnu/libcublas.so.11 (0x00007e91c0800000)
	libcuda.so.1 => /lib/x86_64-linux-gnu/libcuda.so.1 (0x00007e91bea00000)
	libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1 (0x00007e91d8eaa000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007e91d8ea3000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007e91d8e9e000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007e91d8e99000)
	libcublasLt.so.11 => /lib/x86_64-linux-gnu/libcublasLt.so.11 (0x00007e91a9400000)

liucong@liucong:~/llama.cpp_cuda$ REMOTE_GPU_ADDRESS=10.42.143.229 LD_PRELOAD=/home/liucong/cricket-client.so ./build/bin/llama-server -m /home/liucong/tinygrad/weights/gemma-1.1-7b-it.Q4_K_M.gguf -ngl 33 --host 0.0.0.0
+17:24:50.965811 ERROR: section .nv.info. not found	in cpu-elf2.c:687
+17:24:50.965964 ERROR: get_parm_for_kernel failed for kernel _Z7acc_f32PKfS0_Pfiiiiiii	in cpu-elf2.c:993
corrupted size vs. prev_size
Aborted (core dumped)
liucong@liucong:~/llama.cpp_cuda$ 

[MJChku]

Hi, I running into the same .nv.info. not found issue for ggml cuda support, after dig a bit, I think it's because void **__cudaRegisterFatBinary(void *fatCubin) runs ahead of cricket client constructor, calling init_rpc if has not been initialized solved this error.