Compile bug: How to compile llama.cpp with Vulkan for android device #11695

liuxiaotiao · 2025-02-06T04:02:27Z

Git commit

902368a

Operating systems

Linux

GGML backends

Vulkan

Problem description & steps to reproduce

I tried to compile llama.cpp(b4644) using NDK 27 and Vulkan-header(v1.4.307) and encountered the following compilation issues.

First Bad Commit

No response

Compile command

cmake \
 -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
 -DANDROID_ABI=arm64-v8a \
 -DANDROID_PLATFORM=latest \
 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod \
 -DGGML_VULKAN=ON \
 -DGGML_VULKAN_CHECK_RESULTS=OFF \
 -DGGML_VULKAN_DEBUG=ON \
 -DGGML_VULKAN_MEMORY_DEBUG=ON \
 -DGGML_VULKAN_SHADER_DEBUG_INFO=ON \
 -DGGML_VULKAN_PERF=OFF \
 -DGGML_VULKAN_VALIDATE=OFF \
 -DGGML_VULKAN_RUN_TESTS=OFF \
 -DVK_USE_PLATFORM_ANDROID_KHR=ON \
 -DVulkan_INCLUDE_DIR=$Vulkan-Headers/include 
 -DGGML_CCACHE=OFF \
 -B build-android

Relevant log output

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1378:2: warning: extra ';' outside of a function is incompatible with C++98 [-Wc++98-compat-extra-semi]
 1378 | };
      |  ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
 1607 |         CREATE_FA(GGML_TYPE_F16, f16)
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:163: note: expanded from macro 'CREATE_FA2'
 1590 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:46:1: note: expanded from here
   46 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:214: note: expanded from macro 'CREATE_FA2'
 1590 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:48:1: note: expanded from here
   48 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
 1591 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:51:1: note: expanded from here
   51 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:214: note: expanded from macro 'CREATE_FA2'
 1591 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:53:1: note: expanded from here
   53 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:163: note: expanded from macro 'CREATE_FA2'
 1592 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "_f32acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:56:1: note: expanded from here
   56 | flash_attn_f32_f16_f16_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:214: note: expanded from macro 'CREATE_FA2'
 1592 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "_f32acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:58:1: note: expanded from here
   58 | flash_attn_f32_f16_f16_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:163: note: expanded from macro 'CREATE_FA2'
 1593 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:61:1: note: expanded from here
   61 | flash_attn_f32_f16_f16_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:214: note: expanded from macro 'CREATE_FA2'
 1593 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:63:1: note: expanded from here
   63 | flash_attn_f32_f16_f16_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:163: note: expanded from macro 'CREATE_FA2'
 1594 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "_f16acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:66:1: note: expanded from here
   66 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:214: note: expanded from macro 'CREATE_FA2'
 1594 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "_f16acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:68:1: note: expanded from here
   68 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:163: note: expanded from macro 'CREATE_FA2'
 1595 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:71:1: note: expanded from here
   71 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:214: note: expanded from macro 'CREATE_FA2'
 1595 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:73:1: note: expanded from here
   73 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:163: note: expanded from macro 'CREATE_FA2'
 1596 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "_f32acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:76:1: note: expanded from here
   76 | flash_attn_f32_f16_f16_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:214: note: expanded from macro 'CREATE_FA2'
 1596 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "_f32acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:78:1: note: expanded from here
   78 | flash_attn_f32_f16_f16_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:163: note: expanded from macro 'CREATE_FA2'
 1597 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:81:1: note: expanded from here
   81 | flash_attn_f32_f16_f16_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
 1600 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:214: note: expanded from macro 'CREATE_FA2'
 1597 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:83:1: note: expanded from here
   83 | flash_attn_f32_f16_f16_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 80) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:163: note: expanded from macro 'CREATE_FA2'
 1590 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:102:1: note: expanded from here
  102 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 80) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:214: note: expanded from macro 'CREATE_FA2'
 1590 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:104:1: note: expanded from here
  104 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 80) \
      |         ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
 1591 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:107:1: note: expanded from here
  107 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
1 warning and 20 errors generated.
gmake[2]: *** [ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/build.make:160: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[1]: *** [CMakeFiles/Makefile2:1795: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2

The text was updated successfully, but these errors were encountered:

barzoj · 2025-02-06T13:24:15Z

the same issue on my end, building against Vulkan-Headers-1.3.302

jeffbolznv · 2025-02-06T14:20:25Z

Please attach the full log for a clean build. This looks like another "weird combination of tooling installed" issue.

barzoj · 2025-02-06T16:30:43Z

@jeffbolznv please take a look.

echo $ANDROID_NDK
/build/Android/Sdk/ndk/28.0.12674087/

b4651

git status
HEAD detached at b4651
nothing to commit, working tree clean

cmake   -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake   -DANDROID_ABI=arm64-v8a   -DANDROID_PLATFORM=android-33   -DCMAKE_C_FLAGS="-march=armv8.7a"   -DCMAKE_CXX_FLAGS="-march=armv8.7a"   -DGGML_VULKAN=ON -DVulkan_INCLUDE_DIR=$VULKAN/include  -B build-android 
-- The C compiler identification is Clang 19.0.0
-- The CXX compiler identification is Clang 19.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /build/Android/Sdk/ndk/28.0.12674087/toolchains/llvm/prebuilt/linux-x86_64/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /build/Android/Sdk/ndk/28.0.12674087/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.48.1") 
-- Setting GGML_NATIVE_DEFAULT to OFF
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- Including CPU backend
-- Found OpenMP_C: -fopenmp=libomp  
-- Found OpenMP_CXX: -fopenmp=libomp  
-- Found OpenMP: TRUE   
-- ARM detected
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- Adding CPU backend variant ggml-cpu:  
-- Found Vulkan: /build/Android/Sdk/ndk/28.0.12674087/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/33/libvulkan.so (found version "1.3.302") found components: glslc missing components: glslangValidator
-- Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 supported by glslc
-- Host compiler: /usr/bin/gcc /usr/bin/g++
-- vulkan-shaders-gen toolchain file: /build/prj/test/llama.cpp/build-android/host-toolchain.cmake
-- Including Vulkan backend
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Configuring done (1.3s)
-- Generating done (0.1s)
-- Build files have been written to: /build/prj/test/llama.cpp/build-android

cmake --build build-android/ --config Release
[  1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[  1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[  2%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[  2%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[  3%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[  3%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
[  4%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
[  4%] Linking CXX shared library ../../bin/libggml-base.so
[  4%] Built target ggml-base
[  5%] Creating directories for 'vulkan-shaders-gen'
[  5%] No download step for 'vulkan-shaders-gen'
[  6%] No update step for 'vulkan-shaders-gen'
[  6%] No patch step for 'vulkan-shaders-gen'
[  7%] Performing configure step for 'vulkan-shaders-gen'
CMake Warning (dev) in CMakeLists.txt:
  No project() command is present.  The top-level CMakeLists.txt file must
  contain a literal, direct call to the project() command.  Add a line of
  code such as

    project(ProjectName)

  near the top of the file, but after cmake_minimum_required().

  CMake is pretending there is a "project(Project)" command on the first
  line.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
  cmake_minimum_required() should be called prior to this top-level project()
  call.  Please see the cmake-commands(7) manual for usage documentation of
  both commands.
This warning is for project developers.  Use -Wno-dev to suppress it.

-- The C compiler identification is GNU 14.2.1
-- The CXX compiler identification is GNU 14.2.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
CMake Warning (dev) in CMakeLists.txt:
  No cmake_minimum_required command is present.  A line of code such as

    cmake_minimum_required(VERSION 3.27)

  should be added at the top of the file.  The version specified may be lower
  if you wish to support older CMake versions for this project.  For more
  information run "cmake --help-policy CMP0000".
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Configuring done (0.4s)
-- Generating done (0.0s)
-- Build files have been written to: /build/prj/test/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build
[  7%] Performing build step for 'vulkan-shaders-gen'
[ 50%] Building CXX object CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.o
[100%] Linking CXX executable /build/prj/test/llama.cpp/build-android/bin/vulkan-shaders-gen
[100%] Built target vulkan-shaders-gen
[  7%] Built target vulkan-shaders-gen-build
[  7%] Performing install step for 'vulkan-shaders-gen'
-- Install configuration: "Release"
[  7%] Built target vulkan-shaders-gen-install
[  8%] Completed 'vulkan-shaders-gen'
[  8%] Built target vulkan-shaders-gen
[  8%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
[  8%] Building CXX object ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1379:2: warning: extra ';' outside of a function is incompatible with C++98 [-Wc++98-compat-extra-semi]
 1379 | };
      |  ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
 1608 |         CREATE_FA(GGML_TYPE_F16, f16)
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
 1591 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:212:1: note: expanded from here
  212 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:214: note: expanded from macro 'CREATE_FA2'
 1591 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:214:1: note: expanded from here
  214 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:163: note: expanded from macro 'CREATE_FA2'
 1592 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:217:1: note: expanded from here
  217 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:214: note: expanded from macro 'CREATE_FA2'
 1592 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:219:1: note: expanded from here
  219 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:163: note: expanded from macro 'CREATE_FA2'
 1593 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "_f32acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:222:1: note: expanded from here
  222 | flash_attn_f32_f16_f16_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:214: note: expanded from macro 'CREATE_FA2'
 1593 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "_f32acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:224:1: note: expanded from here
  224 | flash_attn_f32_f16_f16_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:163: note: expanded from macro 'CREATE_FA2'
 1594 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:227:1: note: expanded from here
  227 | flash_attn_f32_f16_f16_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:214: note: expanded from macro 'CREATE_FA2'
 1594 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:229:1: note: expanded from here
  229 | flash_attn_f32_f16_f16_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:163: note: expanded from macro 'CREATE_FA2'
 1595 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "_f16acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:232:1: note: expanded from here
  232 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:214: note: expanded from macro 'CREATE_FA2'
 1595 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "_f16acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:234:1: note: expanded from here
  234 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:163: note: expanded from macro 'CREATE_FA2'
 1596 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:237:1: note: expanded from here
  237 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:214: note: expanded from macro 'CREATE_FA2'
 1596 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:239:1: note: expanded from here
  239 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:163: note: expanded from macro 'CREATE_FA2'
 1597 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "_f32acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:242:1: note: expanded from here
  242 | flash_attn_f32_f16_f16_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:214: note: expanded from macro 'CREATE_FA2'
 1597 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "_f32acc_smallrows"         #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:244:1: note: expanded from here
  244 | flash_attn_f32_f16_f16_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1598:163: note: expanded from macro 'CREATE_FA2'
 1598 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:247:1: note: expanded from here
  247 | flash_attn_f32_f16_f16_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
 1601 |         CREATE_FA2(TYPE, NAMELC, 64) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1598:214: note: expanded from macro 'CREATE_FA2'
 1598 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "_aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16_ ## NAMELC ## _cm2_len,         flash_attn_f32_f16_ ## NAMELC ## _cm2_data,         "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:2:1: note: expanded from here
    2 | flash_attn_f32_f16_f16_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1602:9: note: expanded from macro 'CREATE_FA'
 1602 |         CREATE_FA2(TYPE, NAMELC, 80) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
 1591 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                   ^
<scratch space>:21:1: note: expanded from here
   21 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1602:9: note: expanded from macro 'CREATE_FA'
 1602 |         CREATE_FA2(TYPE, NAMELC, 80) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:214: note: expanded from macro 'CREATE_FA2'
 1591 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "_f16acc"         #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);     \
      |                                                                                                                                                                                                                      ^
<scratch space>:23:1: note: expanded from here
   23 | flash_attn_f32_f16_f16_f16acc_cm2_data
      | ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1608:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1602:9: note: expanded from macro 'CREATE_FA'
 1602 |         CREATE_FA2(TYPE, NAMELC, 80) \
      |         ^
/build/prj/test/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:163: note: expanded from macro 'CREATE_FA2'
 1592 |         ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "_aligned_f16acc" #NAMELC,           flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_len,  flash_attn_f32_f16_ ## NAMELC ## _f16acc_cm2_data,  "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);     \
      |                                                                                                                                                                   ^
<scratch space>:26:1: note: expanded from here
   26 | flash_attn_f32_f16_f16_f16acc_cm2_len
      | ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
1 warning and 20 errors generated.
gmake[2]: *** [ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/build.make:161: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:1795: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2

liuxiaotiao · 2025-02-06T16:45:58Z

Please attach the full log for a clean build. This looks like another "weird combination of tooling installed" issue.

Hi, here are all the logs from when I try to compile the Vulkan and llama.cpp for Android.

~/test/llama.cpp-b4644$ cmake
-DCMAKE_TOOLCHAIN_FILE=/home/ubuntu/android-ndk-r27c/build/cmake/android.toolchain.cmake
-DANDROID_ABI=arm64-v8a
-DANDROID_PLATFORM=android-28
-DCMAKE_C_FLAGS="-march=armv8.7a"
-DCMAKE_CXX_FLAGS="-march=armv8.7a"
-DGGML_OPENMP=OFF
-DGGML_LLAMAFILE=OFF
-DGGML_VULKAN=1
-DVulkan_INCLUDE_DIR=/home/ubuntu/Vulkan-Headers/include
-DVulkan_LIBRARY=/home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/35/libvulkan.so
-B build-android
-- The C compiler identification is Clang 18.0.3
-- The CXX compiler identification is Clang 18.0.3
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.34.1")
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
-- Setting GGML_NATIVE_DEFAULT to OFF
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- Including CPU backend
-- ARM detected
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- Adding CPU backend variant ggml-cpu:
-- Found Vulkan: /home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/35/libvulkan.so
-- Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 supported by glslc
-- Host compiler: /usr/bin/gcc /usr/bin/g++
-- vulkan-shaders-gen toolchain file: /home/ubuntu/test/llama.cpp-b4644/build-android/host-toolchain.cmake
-- Including Vulkan backend
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
CMake Warning at common/CMakeLists.txt:32 (message):
Git repository not found; to enable automatic generation of build info,
make sure Git is installed and the project is a Git repository.

-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/test/llama.cpp-b4644/build-android

~/test/llama.cpp-b4644$ cmake --build build-android --config Release -j8
[ 0%] Building CXX object common/CMakeFiles/build_info.dir/build-info.cpp.o
[ 0%] Building C object examples/gguf-hash/CMakeFiles/sha256.dir/deps/sha256/sha256.c.o
[ 1%] Building C object examples/gguf-hash/CMakeFiles/sha1.dir/deps/sha1/sha1.c.o
[ 2%] Building C object examples/gguf-hash/CMakeFiles/xxhash.dir/deps/xxhash/xxhash.c.o
[ 3%] Creating directories for 'vulkan-shaders-gen'
[ 4%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[ 5%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[ 5%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[ 5%] Built target build_info
[ 5%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[ 5%] No download step for 'vulkan-shaders-gen'
[ 6%] No update step for 'vulkan-shaders-gen'
[ 6%] Built target sha1
[ 7%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[ 7%] Built target sha256
[ 7%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
[ 7%] No patch step for 'vulkan-shaders-gen'
[ 8%] Performing configure step for 'vulkan-shaders-gen'
CMake Warning (dev) in CMakeLists.txt:
No project() command is present. The top-level CMakeLists.txt file must
contain a literal, direct call to the project() command. Add a line of
code such as

project(ProjectName)

near the top of the file, but after cmake_minimum_required().

CMake is pretending there is a "project(Project)" command on the first
line.
This warning is for project developers. Use -Wno-dev to suppress it.

-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
[ 9%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
CMake Warning (dev) in CMakeLists.txt:
No cmake_minimum_required command is present. A line of code such as

cmake_minimum_required(VERSION 3.22)

should be added at the top of the file. The version specified may be lower
if you wish to support older CMake versions for this project. For more
information run "cmake --help-policy CMP0000".
This warning is for project developers. Use -Wno-dev to suppress it.

-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build
[ 9%] Performing build step for 'vulkan-shaders-gen'
gmake[3]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
[ 50%] Building CXX object CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.o
[ 9%] Built target xxhash
[100%] Linking CXX executable /home/ubuntu/test/llama.cpp-b4644/build-android/bin/vulkan-shaders-gen
[100%] Built target vulkan-shaders-gen
[ 9%] Built target vulkan-shaders-gen-build
[ 9%] Performing install step for 'vulkan-shaders-gen'
-- Install configuration: "Release"
[ 9%] Built target vulkan-shaders-gen-install
[ 10%] Completed 'vulkan-shaders-gen'
[ 10%] Built target vulkan-shaders-gen
[ 10%] Linking CXX shared library ../../bin/libggml-base.so
[ 10%] Built target ggml-base
[ 11%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
[ 13%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.c.o
[ 13%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-traits.cpp.o
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-aarch64.cpp.o
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.cpp.o
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-hbm.cpp.o
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/amx.cpp.o
[ 14%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-quants.c.o
[ 15%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/mmq.cpp.o
[ 15%] Linking CXX shared library ../../bin/libggml-cpu.so
[ 15%] Built target ggml-cpu
[ 16%] Building CXX object ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o
[ 16%] Building CXX object ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan-shaders.cpp.o
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1378:2: warning: extra ';' outside of a function is incompatible with C++98 [-Wc++98-compat-extra-semi]
1378 | };
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
1607 | CREATE_FA(GGML_TYPE_F16, f16)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:163: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
:44:1: note: expanded from here
44 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:214: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
:46:1: note: expanded from here
46 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "aligned_f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
:49:1: note: expanded from here
49 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:214: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "aligned_f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
:51:1: note: expanded from here
51 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:163: note: expanded from macro 'CREATE_FA2'
1592 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "f32acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
:54:1: note: expanded from here
54 | flash_attn_f32_f16_f16_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1592:214: note: expanded from macro 'CREATE_FA2'
1592 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][0], "flash_attn_f32_f16_D" #D "f32acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
:56:1: note: expanded from here
56 | flash_attn_f32_f16_f16_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:163: note: expanded from macro 'CREATE_FA2'
1593 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "aligned_f32acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
:59:1: note: expanded from here
59 | flash_attn_f32_f16_f16_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1593:214: note: expanded from macro 'CREATE_FA2'
1593 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][0][1], "flash_attn_f32_f16_D" #D "aligned_f32acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
:61:1: note: expanded from here
61 | flash_attn_f32_f16_f16_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:163: note: expanded from macro 'CREATE_FA2'
1594 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "f16acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);
| ^
:64:1: note: expanded from here
64 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1594:214: note: expanded from macro 'CREATE_FA2'
1594 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][0], "flash_attn_f32_f16_D" #D "f16acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);
| ^
:66:1: note: expanded from here
66 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:163: note: expanded from macro 'CREATE_FA2'
1595 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);
| ^
:69:1: note: expanded from here
69 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1595:214: note: expanded from macro 'CREATE_FA2'
1595 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][1][1], "flash_attn_f32_f16_D" #D "aligned_f16acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);
| ^
:71:1: note: expanded from here
71 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:163: note: expanded from macro 'CREATE_FA2'
1596 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "f32acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);
| ^
:74:1: note: expanded from here
74 | flash_attn_f32_f16_f16_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1596:214: note: expanded from macro 'CREATE_FA2'
1596 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][0], "flash_attn_f32_f16_D" #D "f32acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,true), fa_spec_constants(D,1,TYPE,true), 1);
| ^
:76:1: note: expanded from here
76 | flash_attn_f32_f16_f16_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:163: note: expanded from macro 'CREATE_FA2'
1597 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);
| ^
:79:1: note: expanded from here
79 | flash_attn_f32_f16_f16_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1600:9: note: expanded from macro 'CREATE_FA'
1600 | CREATE_FA2(TYPE, NAMELC, 64)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1597:214: note: expanded from macro 'CREATE_FA2'
1597 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][1][1][1], "flash_attn_f32_f16_D" #D "aligned_f32acc_smallrows" #NAMELC, flash_attn_f32_f16 ## NAMELC ## cm2_len, flash_attn_f32_f16 ## NAMELC ## _cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,true), fa_spec_constants(D,0,TYPE,true), fa_rows_cols(D,0,TYPE,true)[1]);
| ^
:81:1: note: expanded from here
81 | flash_attn_f32_f16_f16_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 80)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:163: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
💯1: note: expanded from here
100 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_data'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 80)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1590:214: note: expanded from macro 'CREATE_FA2'
1590 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][0], "flash_attn_f32_f16_D" #D "f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,1,TYPE,false), fa_spec_constants(D,1,TYPE,false), 1);
| ^
:102:1: note: expanded from here
102 | flash_attn_f32_f16_f16_f16acc_cm2_data
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1607:9: error: use of undeclared identifier 'flash_attn_f32_f16_f16_f16acc_cm2_len'
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1601:9: note: expanded from macro 'CREATE_FA'
1601 | CREATE_FA2(TYPE, NAMELC, 80)
| ^
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/ggml-vulkan.cpp:1591:163: note: expanded from macro 'CREATE_FA2'
1591 | ggml_vk_create_pipeline(device, device->pipeline_flash_attn_f32_f16_D ## D[TYPE][0][0][1], "flash_attn_f32_f16_D" #D "aligned_f16acc" #NAMELC, flash_attn_f32_f16 ## NAMELC ## f16acc_cm2_len, flash_attn_f32_f16 ## NAMELC ## _f16acc_cm2_data, "main", 5, sizeof(vk_flash_attn_push_constants), fa_wg_denoms(D,0,TYPE,false), fa_spec_constants(D,0,TYPE,false), fa_rows_cols(D,0,TYPE,false)[1]);
| ^
:105:1: note: expanded from here
105 | flash_attn_f32_f16_f16_f16acc_cm2_len
| ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
1 warning and 20 errors generated.
gmake[2]: *** [ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/build.make:160: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[1]: *** [CMakeFiles/Makefile2:1795: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2

jeffbolznv · 2025-02-06T17:00:54Z

OK, so cmake detects coopmat2 support, but those shader are either not being generated at all, or the generation is silently crashing (seems less likely). Please check whether GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT is defined when vulkan-shaders-gen.cpp is compiled. I'm thinking this could be an issue with cross compiles not picking up the cmake state.

liuxiaotiao · 2025-02-06T17:53:09Z

OK, so cmake detects coopmat2 support, but those shader are either not being generated at all, or the generation is silently crashing (seems less likely). Please check whether GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT is defined when vulkan-shaders-gen.cpp is compiled. I'm thinking this could be an issue with cross compiles not picking up the cmake state.

Hi, GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT was not defined when compiling vulkan-shaders-gen.cpp. However, when I add
#ifdef GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT
#pragma message("GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT is defined")
#else
#pragma message("GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT is NOT defined")
#define GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT
#endif
at the beginning of vulkan-shaders-gen.cpp, it still has the same compilation error. Can you help check again?

jeffbolznv · 2025-02-06T18:01:12Z

at the beginning of vulkan-shaders-gen.cpp, it still has the same compilation error. Can you help check again?

That's surprising. Was it a clean build? I don't fully trust the dependencies (they're manually hacked together in _ggml_vk_shader_deps).

GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT gets set in ggml-vulkan/CMakeLists.txt and in normal builds that setting gets inherited into the vulkan-shaders subproject. It sounds like this cross-compile may not be getting that behavior.

Either that (or in addition to that!) the cross-compile might be picking up a different glslc that does not support coopmat2 (seems very possible if the ndk hasn't updated). Is it possible to check what version of glslc is in the NDK and also try using it to compile -o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp

barzoj · 2025-02-06T18:59:20Z

I updated the nkd to the latest version

/build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc -o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp
vulkan-shaders/test_coopmat_support.comp:3: error: '#extension' : extension not supported: GL_KHR_cooperative_matrix
1 error generated.

hardcoding glslc executable to the one provided by ndk solves the issue.

diff --git a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
index 07403108..4e7271ff 100644
--- a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
+++ b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
@@ -1,5 +1,5 @@
 find_package (Threads REQUIRED)
-find_program(GLSLC_EXECUTABLE glslc)
+find_program(GLSLC_EXECUTABLE /build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc)
 if(NOT GLSLC_EXECUTABLE)
     message(FATAL_ERROR "glslc not found.")
 endif()

liuxiaotiao · 2025-02-06T19:45:33Z

I updated the nkd to the latest version

/build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc -o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp
vulkan-shaders/test_coopmat_support.comp:3: error: '#extension' : extension not supported: GL_KHR_cooperative_matrix
1 error generated.

hardcoding glslc executable to the one provided by ndk solves the issue.

diff --git a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
index 07403108..4e7271ff 100644
--- a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
+++ b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
@@ -1,5 +1,5 @@
 find_package (Threads REQUIRED)
-find_program(GLSLC_EXECUTABLE glslc)
+find_program(GLSLC_EXECUTABLE /build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc)
 if(NOT GLSLC_EXECUTABLE)
     message(FATAL_ERROR "glslc not found.")
 endif()

Hi, which version of llama.cpp did you compile, when I use this way and the latest NDK, the problem still exists.

liuxiaotiao · 2025-02-06T19:50:23Z

at the beginning of vulkan-shaders-gen.cpp, it still has the same compilation error. Can you help check again?

That's surprising. Was it a clean build? I don't fully trust the dependencies (they're manually hacked together in _ggml_vk_shader_deps).

GGML_VULKAN_COOPMAT2_GLSLC_SUPPORT gets set in ggml-vulkan/CMakeLists.txt and in normal builds that setting gets inherited into the vulkan-shaders subproject. It sounds like this cross-compile may not be getting that behavior.

Either that (or in addition to that!) the cross-compile might be picking up a different glslc that does not support coopmat2 (seems very possible if the ndk hasn't updated). Is it possible to check what version of glslc is in the NDK and also try using it to compile -o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp

Hi, I use clean build every time. When downgrading vulkan from 1.4 to 1.3. The compilation log is as follows:
cmake --build build-android/ --target clean
~/test/llama.cpp-b4644$ cmake -DCMAKE_TOOLCHAIN_FILE=/home/ubuntu/android-ndk-r27c/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-28 -DCMAKE_C_FLAGS="-march=armv8.7a" -DCMAKE_CXX_FLAGS="-march=armv8.7a" -DGGML_OPENMP=OFF -DGGML_LLAMAFILE=OFF -DGGML_VULKAN=1 -DVulkan_INCLUDE_DIR=/home/ubuntu/test/Vulkan-Headers/include -DVulkan_LIBRARY=/home/ubuntu/android-ndk-r27c/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/35/libvulkan.so -B build-android
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
-- Setting GGML_NATIVE_DEFAULT to OFF
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- Including CPU backend
-- ARM detected
-- Adding CPU backend variant ggml-cpu:
-- Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 not supported by glslc
-- Host compiler: /usr/bin/gcc /usr/bin/g++
-- vulkan-shaders-gen toolchain file: /home/ubuntu/test/llama.cpp-b4644/build-android/host-toolchain.cmake
-- Including Vulkan backend
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
CMake Warning at common/CMakeLists.txt:32 (message):
Git repository not found; to enable automatic generation of build info,
make sure Git is installed and the project is a Git repository.

-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/test/llama.cpp-b4644/build-android
~/test/llama.cpp-b4644$ cmake --build build-android --config Release -j8
[ 0%] Generating build details from Git
[ 1%] Building C object examples/gguf-hash/CMakeFiles/xxhash.dir/deps/xxhash/xxhash.c.o
[ 2%] Creating directories for 'vulkan-shaders-gen'
[ 3%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
-- Found Git: /usr/bin/git (found version "2.34.1")
[ 4%] Building C object examples/gguf-hash/CMakeFiles/sha1.dir/deps/sha1/sha1.c.o
[ 5%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[ 5%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
fatal: not a git repository (or any of the parent directories): .git
[ 5%] Building C object examples/gguf-hash/CMakeFiles/sha256.dir/deps/sha256/sha256.c.o
fatal: not a git repository (or any of the parent directories): .git
[ 5%] Building CXX object common/CMakeFiles/build_info.dir/build-info.cpp.o
[ 5%] No download step for 'vulkan-shaders-gen'
[ 5%] Built target build_info
[ 5%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[ 6%] No update step for 'vulkan-shaders-gen'
[ 6%] No patch step for 'vulkan-shaders-gen'
[ 6%] Built target sha1
[ 7%] Performing configure step for 'vulkan-shaders-gen'
[ 8%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
CMake Warning (dev) in CMakeLists.txt:
No project() command is present. The top-level CMakeLists.txt file must
contain a literal, direct call to the project() command. Add a line of
code such as

project(ProjectName)

near the top of the file, but after cmake_minimum_required().

CMake is pretending there is a "project(Project)" command on the first
line.
This warning is for project developers. Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
No cmake_minimum_required command is present. A line of code such as

cmake_minimum_required(VERSION 3.22)

should be added at the top of the file. The version specified may be lower
if you wish to support older CMake versions for this project. For more
information run "cmake --help-policy CMP0000".
This warning is for project developers. Use -Wno-dev to suppress it.

-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build
[ 8%] Built target sha256
[ 8%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
[ 8%] Performing build step for 'vulkan-shaders-gen'
gmake[3]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
Consolidate compiler generated dependencies of target vulkan-shaders-gen
[100%] Built target vulkan-shaders-gen
[ 8%] Built target vulkan-shaders-gen-build
[ 9%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
[ 9%] Performing install step for 'vulkan-shaders-gen'
-- Install configuration: "Release"
[ 9%] Built target vulkan-shaders-gen-install
[ 10%] Completed 'vulkan-shaders-gen'
[ 10%] Built target vulkan-shaders-gen
[ 10%] Built target xxhash
[ 10%] Linking CXX shared library ../../bin/libggml-base.so
[ 10%] Built target ggml-base
[ 11%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
[ 11%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.cpp.o
[ 12%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu.c.o
[ 12%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/amx.cpp.o
[ 12%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-aarch64.cpp.o
[ 13%] Building C object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-quants.c.o
[ 13%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-hbm.cpp.o
[ 14%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/ggml-cpu-traits.cpp.o
[ 15%] Building CXX object ggml/src/CMakeFiles/ggml-cpu.dir/ggml-cpu/amx/mmq.cpp.o
cannot compile matmul_f32_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_cm2.spv -DACC_TYPE=float -DB_TYPE=float16_t -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_f32_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_TYPE=float16_t -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_cm2.spv -DACC_TYPE=float -DB_TYPE=float16_t -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q4_1_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q4_0_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q4_1_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_1_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q4_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_TYPE=float16_t -DDATA_A_F16=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q5_0_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q5_0_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q5_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q5_1_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q5_1_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q5_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q4_0_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q4_0_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q4_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q5_0_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q5_0_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q5_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q5_1_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q5_1_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q5_1=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q8_0_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q8_0_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q8_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q8_0_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q8_0_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q8_0=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q2_k_f16_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q2_k_f16_cm2.spv -DACC_TYPE=float -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q2_K=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

cannot compile matmul_q2_k_f16_aligned_cm2

/home/ubuntu/VulkanSDK/1.3.261.1/x86_64/bin/glslc -fshader-stage=compute --target-env=vulkan1.3 -O /home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp -o /home/ubuntu/test/llama.cpp-b4644/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_q2_k_f16_aligned_cm2.spv -DACC_TYPE=float -DALIGNED=1 -DB_IS_FLOAT=1 -DB_TYPE=float16_t -DDATA_A_Q2_K=1 -DD_TYPE=float -DFLOAT16=1 -DFLOAT_TYPE=float16_t -DLOAD_VEC_A=1 -DLOAD_VEC_B=1

/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:13: warning: '#extension' : extension not supported: GL_NV_cooperative_matrix2
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutNV' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: 'tensorLayoutA' : undeclared identifier
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '>' : wrong operand types: no operation '>' exists that takes a left-hand operand of type ' temp bool' and a right operand of type ' temp float' (or there is no acceptable conversion)
/home/ubuntu/test/llama.cpp-b4644/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm_cm2.comp:193: error: '' : syntax error, unexpected EQUAL, expecting COMMA or SEMICOLON
1 warning and 4 errors generated.

[ 15%] Linking CXX shared library ../../bin/libggml-cpu.so
[ 15%] Built target ggml-cpu

And the compilation process will be blocked at [ 15%] Built target ggml-cpu.

jeffbolznv · 2025-02-06T20:04:18Z

/build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc -o - -fshader-stage=compute --target-env=vulkan1.3 vulkan-shaders/test_coopmat_support.comp
vulkan-shaders/test_coopmat_support.comp:3: error: '#extension' : extension not supported: GL_KHR_cooperative_matrix
1 error generated.

OK, sounds like the ndk glslc is very old (GL_KHR_cooperative_matrix was release in June 2023). And the issue seems to be related to vulkan-shaders/CMakeLists.txt picking up a different glslc than ggml-vulkan/CMakeLists.txt.

hardcoding glslc executable to the one provided by ndk solves the issue.

diff --git a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
index 07403108..4e7271ff 100644
--- a/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
+++ b/ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt
@@ -1,5 +1,5 @@
 find_package (Threads REQUIRED)
-find_program(GLSLC_EXECUTABLE glslc)
+find_program(GLSLC_EXECUTABLE /build/Android/Sdk/ndk/28.0.13004108//shader-tools/linux-x86_64/glslc)
 if(NOT GLSLC_EXECUTABLE)
     message(FATAL_ERROR "glslc not found.")
 endif()

This is surprising. If this glslc doesn't support KHR_coopmat, I'd expect those compiles to fail. Does this somehow affect the logic in ggml-vulkan/CMakeLists.txt? I'm not an expert on cmake.

But it seems like there are two glslcs in the system, the one from the ndk is old, the two CMakeLists.txt files are picking up different versions, and I suspect this is somehow related to the cross-compiling or overriding the toolchain in ggml-vulkan/CMakeLists.txt.

jeffbolznv · 2025-02-06T20:06:32Z

Hi, I use clean build every time. When downgrading vulkan from 1.4 to 1.3. The compilation log is as follows:
...
-- GL_NV_cooperative_matrix2 not supported by glslc

This downgrade picked up a different glslc that doesn't support coopmat2 and this avoids the problem. We need the two CMakeLists.txt files to agree on which build of glslc is being used.

(Or, we need to stop relying on whatever random stuff is installed and fetch and build shaderc directly. This just keeps coming up).

barzoj · 2025-02-07T09:02:11Z

I spent some time debugging the issue

CMake Debug Log at ggml/src/ggml-vulkan/CMakeLists.txt:5 (find_package):
  find_package considered the following paths for FindVulkan.cmake:

    /buid/prj/test/llama.cpp/cmake/FindVulkan.cmake

  The file was found at

    /build/.local/lib/python3.10/site-packages/cmake/data/share/cmake-3.27/Modules/FindVulkan.cmake

added a debug message to check what gets resolved

iff --git a/ggml/src/ggml-vulkan/CMakeLists.txt b/ggml/src/ggml-vulkan/CMakeLists.txt
index d970f7e2..43c3c244 100644
--- a/ggml/src/ggml-vulkan/CMakeLists.txt
+++ b/ggml/src/ggml-vulkan/CMakeLists.txt
@@ -17,6 +26,7 @@ endfunction()
 
 if (Vulkan_FOUND)
     message(STATUS "Vulkan found")
+    message(STATUS Vulkan::Vulkan: ${Vulkan_INCLUDE_DIRS} : ${Vulkan_LIBRARIES} : ${Vulkan_GLSLC_EXECUTABLE})

here is the result:

-- Vulkan::Vulkan:/build/prj/vulkan/Vulkan-Headers-1.3.302/include:/build/Android/Sdk/ndk/28.0.13004108/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/aarch64-linux-android/33/libvulkan.so:/bin/glslc

so cmake properly detects path to headers and the library, but fails to lookup the right glslc executable, it simply doesn't look inside $ANDROID_NDK/shader-tools/linux-x86_64/

as a potential workaround, we can have a patched copy of FindVulkan.cmake in the repo under cmake folder which is capable to find the right glslc in case we cross-compile with android ndk toolchain.

@jeffbolznv I can put together a quick PR, please lmk if the proposed approach makes sense

jeffbolznv · 2025-02-07T14:33:03Z

Having a copy of a core cmake module seems hacky and potentially has licensing issues.

I can think of a couple options for dealing with this:
(1) Have ggml-vulkan lookup glslc and put the executable name in a variable that vulkan-shaders uses. This way they use the same version and agree on what extensions are supported. Doesn't matter if this is the installed version or the NDK version (so I assume it would get the installed version).
(2) Let them continue to use different versions of GLSLC, but have the makefiles communicate to each other which extensions are supported, and only enable those supported by both.

Option (1) seems much easier, so that's what I think we should try.

liuxiaotiao · 2025-02-09T00:33:33Z

When I hardcode /home/ubuntu/android-ndk-r28/shader-tools/linux-x86_64/glslc in ggml/src/ggml-vulkan/vulkan-shaders/CMakeLists.txt and add -DVulkan_GLSLC_EXECUTABLE=/home/ubuntu/android-ndk-r28/shader-tools/linux-x86_64/glslc in cmake command, compilation succeeds.

New cmake command is
cmake
-DCMAKE_TOOLCHAIN_FILE=/home/ubuntu/android-ndk-r28/build/cmake/android.toolchain.cmake
-DANDROID_ABI=arm64-v8a
-DANDROID_PLATFORM=android-33
-DCMAKE_C_FLAGS="-march=armv8.7a"
-DCMAKE_CXX_FLAGS="-march=armv8.7a"
-DGGML_VULKAN=ON
-DVulkan_INCLUDE_DIR=/home/ubuntu/Vulkan-Headers/include
-DVulkan_GLSLC_EXECUTABLE=/home/ubuntu/android-ndk-r28/shader-tools/linux-x86_64/glslc
-B build-android

When I try to run LD_LIBRARY_PATH=/data/local/tmp/mobile-vulkan/lib ./bin/llama-bench -m ../model/Llama-3.2-1B-Instruct-Q4_K_M.gguf on Android device, the result shows "CANNOT LINK EXECUTABLE "./bin/llama-bench": library "libomp.so" not found: needed by /data/local/tmp/mobile-vulkan/lib/libggml-cpu.so in namespace (default)".

After moving libomp.so from ${NDK}/toolchains/llvm/prebuilt/linux-x86_64/lib/clang/19/lib/linux/aarch64/libomp.so to {install-dir}/lib, running "LD_LIBRARY_PATH=/data/local/tmp/mobile-vulkan/lib ./bin/llama-bench -m ../model/Llama-3.2-1B-Instruct-Q8_0.gguf", got:
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Adreno (TM) 750 (Qualcomm Technologies Inc. Adreno Vulkan Driver) | uma: 1 | fp16: 1 | warp size: 64 | shared memory: 32768 | matrix cores: none
libc++abi: terminating due to uncaught exception of type vk::DeviceLostError: vk::Queue::submit: ErrorDeviceLost
Aborted.

Run "LD_LIBRARY_PATH=/data/local/tmp/mobile-vulkan/lib ./bin/llama-bench -m ../model/Llama-3.2-1B-Instruct-Q4_K_M.gguf", got:
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = Adreno (TM) 750 (Qualcomm Technologies Inc. Adreno Vulkan Driver) | uma: 1 | fp16: 1 | warp size: 64 | shared memory: 32768 | matrix cores: none
ggml_vulkan: Compute pipeline creation failed for mul_mat_vec_q4_k_f32_f32_1
ggml_vulkan: vk::Device::createComputePipeline: ErrorUnknown
libc++abi: terminating due to uncaught exception of type std::out_of_range: unordered_map::at: key not found
Aborted.

Is there any way to solve this problem, or is there another compilation method required to solve this runtime error?

Having a copy of a core cmake module seems hacky and potentially has licensing issues.

I can think of a couple options for dealing with this: (1) Have ggml-vulkan lookup glslc and put the executable name in a variable that vulkan-shaders uses. This way they use the same version and agree on what extensions are supported. Doesn't matter if this is the installed version or the NDK version (so I assume it would get the installed version). (2) Let them continue to use different versions of GLSLC, but have the makefiles communicate to each other which extensions are supported, and only enable those supported by both.

Option (1) seems much easier, so that's what I think we should try.

jeffbolznv · 2025-02-15T17:45:07Z

Please check if this branch fixes the glslc mismatch: https://github.com/jeffbolznv/llama.cpp/tree/glslc_mismatch.

-DVulkan_GLSLC_EXECUTABLE=/home/ubuntu/android-ndk-r28/shader-tools/linux-x86_64/glslc

Given how old it is, I kind of think it's a bad idea to force it to use the glslc from the ndk.

liuxiaotiao · 2025-02-15T21:08:13Z

Please check if this branch fixes the glslc mismatch: https://github.com/jeffbolznv/llama.cpp/tree/glslc_mismatch.

This is so weird. If I remove -DVulkan_GLSLC_EXECUTABLE=, the compilation information is as follows:

/llama.cpp$ cmake --build build-android --config Release -j8
[ 1%] Building C object examples/gguf-hash/CMakeFiles/xxhash.dir/deps/xxhash/xxhash.c.o
[ 2%] Building C object examples/gguf-hash/CMakeFiles/sha1.dir/deps/sha1/sha1.c.o
[ 2%] Building CXX object common/CMakeFiles/build_info.dir/build-info.cpp.o
[ 3%] Creating directories for 'vulkan-shaders-gen'
[ 3%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[ 4%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[ 4%] Building C object examples/gguf-hash/CMakeFiles/sha256.dir/deps/sha256/sha256.c.o
[ 5%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[ 5%] Built target build_info
[ 5%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[ 5%] No download step for 'vulkan-shaders-gen'
[ 6%] No update step for 'vulkan-shaders-gen'
[ 6%] No patch step for 'vulkan-shaders-gen'
[ 6%] Built target sha1
[ 7%] Performing configure step for 'vulkan-shaders-gen'
[ 8%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
CMake Warning (dev) in CMakeLists.txt:
No project() command is present. The top-level CMakeLists.txt file must
contain a literal, direct call to the project() command. Add a line of
code such as

project(ProjectName)

near the top of the file, but after cmake_minimum_required().

CMake is pretending there is a "project(Project)" command on the first
line.
This warning is for project developers. Use -Wno-dev to suppress it.

[ 8%] Built target sha256
[ 8%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
[ 9%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
CMake Warning (dev) in CMakeLists.txt:
No cmake_minimum_required command is present. A line of code such as

cmake_minimum_required(VERSION 3.22)

should be added at the top of the file. The version specified may be lower
if you wish to support older CMake versions for this project. For more
information run "cmake --help-policy CMP0000".
This warning is for project developers. Use -Wno-dev to suppress it.

-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-build
[ 9%] Performing build step for 'vulkan-shaders-gen'
gmake[3]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
[ 50%] Building CXX object CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.o
[ 9%] Built target xxhash
[100%] Linking CXX executable /home/ubuntu/llama.cpp/build-android/bin/vulkan-shaders-gen
[100%] Built target vulkan-shaders-gen
[ 9%] Built target vulkan-shaders-gen-build
[ 9%] Performing install step for 'vulkan-shaders-gen'
-- Install configuration: "Release"
[ 9%] Built target vulkan-shaders-gen-install
[ 10%] Completed 'vulkan-shaders-gen'
[ 10%] Built target vulkan-shaders-gen
[ 10%] Linking CXX shared library ../../bin/libggml-base.so
[ 10%] Built target ggml-base
[ 11%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
cannot compile matmul_f32_f16_fp32

Vulkan_GLSLC_EXECUTABLE-NOTFOUND -fshader-stage=compute --target-env=vulkan1.2 -O /home/ubuntu/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/mul_mm.comp -o /home/ubuntu/llama.cpp/build-android/ggml/src/ggml-vulkan/vulkan-shaders.spv/matmul_f32_f16_fp32.spv -DACC_TYPE=float -DB_TYPE=float16_t -DDATA_A_F32=1 -DD_TYPE=float -DFLOAT_TYPE=float