ollama: 0.5.1 -> 0.5.4 #365718

GaetanLepage · 2024-12-16T22:51:03Z

Things done

Changelog: https://github.com/ollama/ollama/releases/tag/v0.5.2

cc @abysssol @dit7ya @elohmeier @RoyDubnium

Add a 👍 reaction to pull requests you find important.

GaetanLepage · 2024-12-16T22:51:15Z

go: go.mod requires go >= 1.23.4 (running go 1.23.3; GOTOOLCHAIN=local)

abysssol · 2024-12-17T11:40:14Z

This will need to be rebased once #361878 is merged, which contains #361606, which updated go to 1.23.4.

mastoca · 2024-12-18T00:27:52Z

Ollama are now at v0.5.4

mastoca · 2024-12-24T19:46:17Z

This will need to be rebased once #361878 is merged, which contains #361606, which updated go to 1.23.4.

@abysssol This merged yesterday, I'm building to test Cuda now.

GaetanLepage · 2024-12-25T15:27:31Z

pkgs/by-name/ol/ollama/package.nix

-  postInstall = lib.optionalString stdenv.hostPlatform.isLinux ''
-    # copy libggml_*.so and runners into lib
-    # https://github.com/ollama/ollama/blob/v0.4.4/llama/make/gpu.make#L90
-    mkdir -p $out/lib
-    cp -r dist/*/lib/* $out/lib/
-  '';
-


This was failing, so I removed it. It now builds fine.
Not 100% sure about the eventual consequences.

this copies built .so files which, if not present will not include cuda/rocm support. We will need to fix this as well.

Perhaps the condition stdenv.hostPlatform.isLinux should be replaced with (enableRocm || enableCuda)? Both enableRocm and enableCuda already check stdenv.hostPlatform.isLinux, and if this is unnecessary for CPU builds, then it shouldn't break anything (and may fix any issues from the cp failing due to the libraries being absent).

PostInstall is still needed and should not be removed.

I'm getting an error for rocm.

> [29/322] Building CXX object libc/src/__support/StringUtil/CMakeFiles/libc.src.__support.Stri… ┃ > [30/322] Building CXX object libc/src/__support/File/CMakeFiles/libc.src.__support.File.file.… ┃ > [31/322] Building CXX object libc/test/UnitTest/CMakeFiles/LibcUnitTest.dir/LibcTest.cpp.o ┃ > FAILED: libc/test/UnitTest/CMakeFiles/LibcUnitTest.dir/LibcTest.cpp.o ┃ > /nix/store/7sqn5142gh5asri8aqxvsvib9ii37fyq-rocm-llvm-clang-unwrapped-wrapper-6.0.2/bin/clang… ┃ > /build/source/libc/test/UnitTest/LibcTest.cpp:16:10: fatal error: 'cassert' file not found ┃ > #include <cassert> ┃ > ^~~~~~~~~ ┃ > 1 error generated. ┃ > [32/322] Building CXX object libc/test/UnitTest/CMakeFiles/TestLogger.dir/TestLogger.cpp.o ┃ > ninja: build stopped: subcommand failed. ┃

not sure if this ollama/ollama#7392 is related.

yes, why was it removed. It seems to have resulted in my CPU being used and not my GPU even when GPU is detected.

@rjpcasalino This is fixed in #373234.

yeah, thanks so much! Wondering how it could have been avoided. Anyway, thanks again

GaetanLepage · 2024-12-25T17:23:10Z

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 365718

`x86_64-linux`

❌ 1 package failed to build:

ollama-rocm

✅ 4 packages built:

alpaca
chatd
ollama
ollama-cuda

`aarch64-linux`

✅ 4 packages built:

alpaca
chatd
ollama
ollama-cuda

`x86_64-darwin`

❌ 1 package failed to build:

ollama

`aarch64-darwin`

❌ 1 package failed to build:

ollama

abysssol · 2024-12-25T17:33:52Z

ollama-rocm is failing to build on master, so it's not necessarily a problem with this PR, but likely something caused by a breaking change or bug in a package dependency.

I'm currently running git bisect to see if I can figure out the cause, but due to the way nixpkgs has a lot of large merges (like with staging and staging-next), I've found it difficult to bisect to the exact original faulty commit in the past, which could happen again.

mastoca · 2024-12-25T21:16:48Z

ollama-rocm is failing to build on master, so it's not necessarily a problem with this PR, but likely something caused by a breaking change or bug in a package dependency.

I'm currently running git bisect to see if I can figure out the cause, but due to the way nixpkgs has a lot of large merges (like with staging and staging-next), I've found it difficult to bisect to the exact original faulty commit in the past, which could happen again.

I'll also investigate, after family festivities end... I did notice the error and it looks like there may be some llvm dependencies we need to add for rocm.

GaetanLepage · 2024-12-25T22:35:30Z

I added __darwinAllowLocalNetworking = true, but darwin now fails with:

Running phase: unpackPhase
unpacking source archive /nix/store/41dhijnzvqnf5x5v5qx8wbx4h221gkrv-source
source root is source
Running phase: patchPhase
Running phase: updateAutotoolsGnuConfigScriptsPhase
Running phase: configurePhase
Running phase: buildPhase
fatal: not a git repository (or any of the parent directories): .git
GOARCH=arm64 go build -buildmode=pie "-ldflags=-w -s \"-X=github.com/ollama/ollama/version.Version=\"  " -trimpath  -o ollama .
Building subPackage .
Building subPackage ./api
Building subPackage ./app
Building subPackage ./app/assets
Building subPackage ./app/lifecycle
Building subPackage ./app/store
Building subPackage ./app/tray
Building subPackage ./app/tray/commontray
Building subPackage ./app/tray/wintray
package github.com/ollama/ollama/app/tray/wintray: build constraints exclude all Go files in /private/tmp/nix-build-ollama-0.5.4.drv-0/source/app/tray/wintray
Building subPackage ./auth
Building subPackage ./cmd
Building subPackage ./cmd/runner
Building subPackage ./convert
Building subPackage ./convert/sentencepiece
Building subPackage ./discover
Building subPackage ./envconfig
Building subPackage ./format
Building subPackage ./integration
package github.com/ollama/ollama/integration: build constraints exclude all Go files in /private/tmp/nix-build-ollama-0.5.4.drv-0/source/integration
Building subPackage ./llama
Building subPackage ./llama/runner
Building subPackage ./llm
Building subPackage ./model/imageproc
Building subPackage ./model/mllama
Building subPackage ./model/pixtral
Building subPackage ./model/qwen2vl
Building subPackage ./openai
Building subPackage ./parser
Building subPackage ./progress
Building subPackage ./readline
Building subPackage ./runners
Building subPackage ./server
Building subPackage ./template
Building subPackage ./types/errtypes
Building subPackage ./types/model
Building subPackage ./util/bufioutil
Building subPackage ./version
buildPhase completed in 49 seconds
Running phase: checkPhase
ok      github.com/ollama/ollama/api    0.167s
ok      github.com/ollama/ollama/app/lifecycle  0.171s
ok      github.com/ollama/ollama/cmd    0.297s
ok      github.com/ollama/ollama/convert        0.278s
SIGTRAP: trace trap
PC=0x1a04dced8 m=0 sigcode=0
signal arrived during cgo execution

goroutine 36 gp=0x14000186a80 m=0 mp=0x102694300 [syscall]:
runtime.cgocall(0x10246d474, 0x14000076e08)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/cgocall.go:167 +0x44 fp=0x14000076dd0 sp=0x14000076d90 pc=0x1023316b4
github.com/ollama/ollama/discover._Cfunc_getRecommendedMaxVRAM()
        _cgo_gotypes.go:78 +0x34 fp=0x14000076e00 sp=0x14000076dd0 pc=0x10246d144
github.com/ollama/ollama/discover.GetGPUInfo()
        /private/tmp/nix-build-ollama-0.5.4.drv-0/source/discover/gpu_darwin.go:40 +0x68 fp=0x14000076ef0 sp=0x14000076e00 pc=0x10246d1c8
github.com/ollama/ollama/discover.TestBasicGetGPUInfo(0x140001ac820)
        /private/tmp/nix-build-ollama-0.5.4.drv-0/source/discover/gpu_test.go:12 +0x20 fp=0x14000076f60 sp=0x14000076ef0 pc=0x10246c7a0
testing.tRunner(0x140001ac820, 0x102532800)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/testing/testing.go:1690 +0xe4 fp=0x14000076fb0 sp=0x14000076f60 pc=0x1023abb34
testing.(*T).Run.gowrap1()
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/testing/testing.go:1743 +0x2c fp=0x14000076fd0 sp=0x14000076fb0 pc=0x1023ac97c
runtime.goexit({})
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000076fd0 sp=0x14000076fd0 pc=0x10233f6c4
created by testing.(*T).Run in goroutine 1
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/testing/testing.go:1743 +0x314

goroutine 1 gp=0x140000021c0 m=nil [chan receive]:
runtime.gopark(0x140001f39d8?, 0x10?, 0xa8?, 0x7c?, 0x18?)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/proc.go:424 +0xc8 fp=0x140001f39a0 sp=0x140001f3980 pc=0x102337498
runtime.chanrecv(0x14000182690, 0x140001f3a9f, 0x1)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/chan.go:639 +0x414 fp=0x140001f3a20 sp=0x140001f39a0 pc=0x1022cd654
runtime.chanrecv1(0x1026932c0?, 0x1024ea880?)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/chan.go:489 +0x14 fp=0x140001f3a50 sp=0x140001f3a20 pc=0x1022cd234
testing.(*T).Run(0x140001ac680, {0x102472914?, 0x140001f3b48?}, 0x102532800)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/testing/testing.go:1751 +0x328 fp=0x140001f3b10 sp=0x140001f3a50 pc=0x1023ac848
testing.runTests.func1(0x140001ac680)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/testing/testing.go:2168 +0x40 fp=0x140001f3b50 sp=0x140001f3b10 pc=0x1023ae850
testing.tRunner(0x140001ac680, 0x140001f3c68)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/testing/testing.go:1690 +0xe4 fp=0x140001f3ba0 sp=0x140001f3b50 pc=0x1023abb34
testing.runTests(0x140001b62a0, {0x10268c7a0, 0x3, 0x3}, {0x2100000000000000?, 0x212d5a5b7a134476?, 0x1026937c0?})
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/testing/testing.go:2166 +0x3ac fp=0x140001f3c90 sp=0x140001f3ba0 pc=0x1023ae74c
testing.(*M).Run(0x1400019ed20)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/testing/testing.go:2034 +0x588 fp=0x140001f3ec0 sp=0x140001f3c90 pc=0x1023ad498
main.main()
        _testmain.go:49 +0x90 fp=0x140001f3f40 sp=0x140001f3ec0 pc=0x10246d3c0
runtime.main()
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/proc.go:272 +0x288 fp=0x140001f3fd0 sp=0x140001f3f40 pc=0x102300a68
runtime.goexit({})
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140001f3fd0 sp=0x140001f3fd0 pc=0x10233f6c4

goroutine 18 gp=0x14000102380 m=nil [force gc (idle)]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/proc.go:424 +0xc8 fp=0x14000064790 sp=0x14000064770 pc=0x102337498
runtime.goparkunlock(...)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/proc.go:430
runtime.forcegchelper()
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/proc.go:337 +0xb8 fp=0x140000647d0 sp=0x14000064790 pc=0x102300dc8
runtime.goexit({})
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000647d0 sp=0x140000647d0 pc=0x10233f6c4
created by runtime.init.7 in goroutine 1
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/proc.go:325 +0x24

goroutine 19 gp=0x14000102540 m=nil [GC sweep wait]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/proc.go:424 +0xc8 fp=0x14000064f60 sp=0x14000064f40 pc=0x102337498
runtime.goparkunlock(...)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/proc.go:430
runtime.bgsweep(0x14000112000)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/mgcsweep.go:277 +0xa0 fp=0x14000064fb0 sp=0x14000064f60 pc=0x1022e9310
runtime.gcenable.gowrap1()
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/mgc.go:204 +0x28 fp=0x14000064fd0 sp=0x14000064fb0 pc=0x1022dd268
runtime.goexit({})
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x14000064fd0 sp=0x14000064fd0 pc=0x10233f6c4
created by runtime.gcenable in goroutine 1
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/mgc.go:204 +0x6c

goroutine 20 gp=0x14000102700 m=nil [GC scavenge wait]:
runtime.gopark(0x14000112000?, 0x1024cfee8?, 0x1?, 0x0?, 0x14000102700?)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/proc.go:424 +0xc8 fp=0x14000065760 sp=0x14000065740 pc=0x102337498
runtime.goparkunlock(...)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/proc.go:430
runtime.(*scavengerState).park(0x102693840)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/mgcscavenge.go:425 +0x5c fp=0x14000065790 sp=0x14000065760 pc=0x1022e6d0c
runtime.bgscavenge(0x14000112000)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/mgcscavenge.go:653 +0x44 fp=0x140000657b0 sp=0x14000065790 pc=0x1022e7254
runtime.gcenable.gowrap2()
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/mgc.go:205 +0x28 fp=0x140000657d0 sp=0x140000657b0 pc=0x1022dd208
runtime.goexit({})
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000657d0 sp=0x140000657d0 pc=0x10233f6c4
created by runtime.gcenable in goroutine 1
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/mgc.go:205 +0xac

goroutine 34 gp=0x14000186380 m=nil [finalizer wait]:
runtime.gopark(0x0?, 0x102311c28?, 0xc8?, 0x85?, 0x10230b208?)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/proc.go:424 +0xc8 fp=0x14000068580 sp=0x14000068560 pc=0x102337498
runtime.runfinq()
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/mfinal.go:193 +0x108 fp=0x140000687d0 sp=0x14000068580 pc=0x1022dc368
runtime.goexit({})
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140000687d0 sp=0x140000687d0 pc=0x10233f6c4
created by runtime.createfing in goroutine 1
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/mfinal.go:163 +0x80

goroutine 35 gp=0x140001868c0 m=nil [chan receive]:
runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/proc.go:424 +0xc8 fp=0x140002046f0 sp=0x140002046d0 pc=0x102337498
runtime.chanrecv(0x14000182460, 0x0, 0x1)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/chan.go:639 +0x414 fp=0x14000204770 sp=0x140002046f0 pc=0x1022cd654
runtime.chanrecv1(0x0?, 0x0?)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/chan.go:489 +0x14 fp=0x140002047a0 sp=0x14000204770 pc=0x1022cd234
runtime.unique_runtime_registerUniqueMapCleanup.func1(...)
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/mgc.go:1781
runtime.unique_runtime_registerUniqueMapCleanup.gowrap1()
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/mgc.go:1784 +0x3c fp=0x140002047d0 sp=0x140002047a0 pc=0x1022e038c
runtime.goexit({})
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/asm_arm64.s:1223 +0x4 fp=0x140002047d0 sp=0x140002047d0 pc=0x10233f6c4
created by unique.runtime_registerUniqueMapCleanup in goroutine 1
        /nix/store/bva4mymi5b2lhvvbccx8ipws89rar1d8-go-1.23.4/share/go/src/runtime/mgc.go:1779 +0xa0

r0      0x0
r1      0x0
r2      0x1ab22d27c
r3      0x10
r4      0x20772c3b0
r5      0x10
r6      0x4000100
r7      0x0
r8      0x1a0690f0d
r9      0x20a004000
r10     0xffbffe0000000000
r11     0x1ffe2000000000
r12     0x6000031a80b8
r13     0x60002000000102
r14     0x100000209b801a1
r15     0x209b801a0
r16     0x1a02fd9e8
r17     0x212193550
r18     0x0
r19     0x0
r20     0x0
r21     0x3
r22     0x13
r23     0x3cb9d69707
r24     0x8e41dc0be9fcd734
r25     0xbe71521dac
r26     0x102532800
r27     0x820
r28     0x102693a60
r29     0x16db386b0
lr      0x1a02fda6c
sp      0x16db386a0
pc      0x1a04dced8
fault   0x1a04dced8
FAIL    github.com/ollama/ollama/discover       0.176s
FAIL

abysssol · 2024-12-26T10:50:18Z

It appears that the commit that broke ollama-rocm is 7047ba9 (merged to staging in 597a7ad) from PR #356162.

mastoca

looks good to me

don't have rocm device to test...but it does build for all three cases.

collares · 2025-01-04T23:16:45Z

At 8bf5d83 (auto-rebased onto 3667e0b by nixpkgs-review):

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 365718

`x86_64-linux`

✅ 5 packages built:

alpaca
chatd
ollama
ollama-cuda
ollama-rocm

GaetanLepage · 2025-01-04T23:32:59Z

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 365718

`x86_64-linux`

❌ 1 package failed to build:

ollama-cuda

✅ 4 packages built:

alpaca
chatd
ollama
ollama-rocm

GaetanLepage · 2025-01-04T23:33:58Z

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 365718

`x86_64-linux`

❌ 1 package failed to build:

ollama-cuda

✅ 4 packages built:

alpaca
chatd
ollama
ollama-rocm

collares · 2025-01-04T23:45:37Z

I only see the targets dist_cuda_v11 and dist_cuda_v12, no dist_cuda. But I can verify that, even on a rebased 8bf5d83 (always using make dist), ollama-rocm compiled a lot of ROCm stuff (and ollama-cuda did the same for CUDA).

mastoca · 2025-01-05T00:02:13Z

@collares in 0.5.4 they refactored the entire make structure. On the cuda side according to the comment in here https://github.com/ollama/ollama/blob/main/Makefile#L14 they try to build for both. in the package.nix we choose based on what's on the system for cuda (that's how it was)

Looking at it now, I think maybe some more re-thinking needs to be done to match what the new make structure is doing.

EDIT: see

nixpkgs/pkgs/by-name/ol/ollama/package.nix

Lines 90 to 104 in 6084a82

    
           # Extract the major version of CUDA. e.g. 11 12 
        
           cudaMajorVersion = lib.versions.major cudaPackages.cuda_cudart.version; 
        
           cudaToolkit = buildEnv { 
        
             # ollama hardcodes the major version in the Makefile to support different variants. 
        
             # - https://github.com/ollama/ollama/blob/v0.4.4/llama/Makefile#L17-L18 
        
             name = "cuda-merged-${cudaMajorVersion}"; 
        
             paths = map lib.getLib cudaLibs ++ [ 
        
               (lib.getOutput "static" cudaPackages.cuda_cudart) 
        
               (lib.getBin (cudaPackages.cuda_nvcc.__spliced.buildHost or cudaPackages.cuda_nvcc)) 
        
             ]; 
        
           }; 
        
           cudaPath = lib.removeSuffix "-${cudaMajorVersion}" cudaToolkit;

lucasew · 2025-01-05T03:32:39Z

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 365718

`x86_64-linux`

❌ 1 package failed to build:

ollama-cuda

✅ 4 packages built:

alpaca
chatd
ollama
ollama-rocm

GaetanLepage · 2025-01-05T09:48:37Z

I only see the targets dist_cuda_v11 and dist_cuda_v12, no dist_cuda.

Thanks, I tweaked the target name and now it works with cudaSupport.

GaetanLepage · 2025-01-05T12:55:50Z

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 365718

`x86_64-linux`

✅ 5 packages built:

alpaca
chatd
ollama
ollama-cuda
ollama-rocm

`x86_64-darwin`

✅ 1 package built:

ollama

`aarch64-darwin`

❌ 1 package failed to build:

ollama

GaetanLepage · 2025-01-05T12:57:52Z

Fails on aarch64-darwin:

SIGTRAP: trace trap

Full log: https://paste.glepage.com/raw/spider-goose-pug

Diff: ollama/ollama@v0.5.1...v0.5.4 Changelog: https://github.com/ollama/ollama/releases/tag/v0.5.4 Co-authored-by: Mastoca <[email protected]>

GaetanLepage · 2025-01-05T14:36:35Z

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 365718

`x86_64-linux`

✅ 5 packages built:

alpaca
chatd
ollama
ollama-cuda
ollama-rocm

`x86_64-darwin`

✅ 1 package built:

ollama

`aarch64-darwin`

❌ 1 package failed to build:

ollama

GaetanLepage · 2025-01-05T18:20:56Z

Looks like the aarch64-darwin is the only remaining blocker.

mastoca · 2025-01-05T19:38:35Z

perhaps a different version of go might work for darwin buildGo124Module?

GaetanLepage · 2025-01-05T21:02:16Z

perhaps a different version of go might work for darwin buildGo124Module?

Same error unfortunately.

toonn · 2025-01-12T16:04:01Z

Pinging @NixOS/darwin-maintainers.

niklaskorz · 2025-01-12T18:46:14Z

`nixpkgs-review` result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 365718

`aarch64-darwin`

✅ 2 packages built:

chatd
ollama

GaetanLepage · 2025-01-12T18:55:41Z

Ok merging since it's building fine outside the sandbox.

sarahec · 2025-01-12T18:57:58Z

And 0.5.5 released two days ago 🤷🏻

niklaskorz · 2025-01-12T19:09:17Z

And 0.5.5 released two days ago 🤷🏻

And there already is a PR, so no need to be grumpy :)

#373234

Edit: also, 0.5.5 is a pre-release, so nothing to merge just yet

sarahec · 2025-01-12T19:17:35Z

And 0.5.5 released two days ago 🤷🏻

And there already is a PR, so no need to be grumpy :)

Interesting, that renders as a shrug, not a grumpy person, on Darwin ;)

scd31 · 2025-01-15T01:55:08Z

FYI I think this may have broken GPU acceleration for me! Confirmed with a git bisect of nixpkgs. I'm running an Nvidia P100.

ollama/ollama#8349 (comment) @peperunas also seems to be running into this

scd31 · 2025-01-15T01:55:38Z

pkgs/by-name/ol/ollama/package.nix

-  postInstall = lib.optionalString stdenv.hostPlatform.isLinux ''
-    # copy libggml_*.so and runners into lib
-    # https://github.com/ollama/ollama/blob/v0.4.4/llama/make/gpu.make#L90
-    mkdir -p $out/lib
-    cp -r dist/*/lib/* $out/lib/
-  '';


I think this might be the problem? I'm going to some messing around

Got it working locally! PR coming shortly

yes. thanks!

github-actions bot added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Dec 16, 2024

GaetanLepage marked this pull request as draft December 17, 2024 11:46

GaetanLepage force-pushed the ollama branch from df09bc7 to aeea8a9 Compare December 18, 2024 06:47

drupol changed the title ~~ollama: 0.5.1 -> 0.5.2~~ ollama: 0.5.1 -> 0.5.4 Dec 18, 2024

GaetanLepage force-pushed the ollama branch from aeea8a9 to 08e2f29 Compare December 24, 2024 22:00

GaetanLepage marked this pull request as ready for review December 24, 2024 22:09

GaetanLepage force-pushed the ollama branch 2 times, most recently from f8b1bfb to cf779e7 Compare December 24, 2024 22:33

ofborg bot requested review from dit7ya, abysssol, RoyDubnium and elohmeier December 25, 2024 08:30

ofborg bot added the 10.rebuild-darwin: 1 label Dec 25, 2024

GaetanLepage force-pushed the ollama branch from cf779e7 to 3aae943 Compare December 25, 2024 15:26

GaetanLepage commented Dec 25, 2024

View reviewed changes

github-actions bot removed the 10.rebuild-darwin: 1 label Dec 25, 2024

ofborg bot added the 10.rebuild-darwin: 1 label Dec 25, 2024

GaetanLepage force-pushed the ollama branch from 3aae943 to ccf4865 Compare December 25, 2024 22:35

github-actions bot removed the 10.rebuild-darwin: 1 label Dec 25, 2024

ofborg bot added the 10.rebuild-darwin: 1 label Dec 26, 2024

mastoca approved these changes Jan 4, 2025

View reviewed changes

GaetanLepage force-pushed the ollama branch from 6084a82 to a75fdfd Compare January 5, 2025 09:48

GaetanLepage marked this pull request as ready for review January 5, 2025 09:49

ollama: 0.5.1 -> 0.5.4

be7b92e

Diff: ollama/ollama@v0.5.1...v0.5.4 Changelog: https://github.com/ollama/ollama/releases/tag/v0.5.4 Co-authored-by: Mastoca <[email protected]>

GaetanLepage force-pushed the ollama branch from a75fdfd to be7b92e Compare January 5, 2025 13:14

niklaskorz approved these changes Jan 12, 2025

View reviewed changes

GaetanLepage merged commit f23ec9f into NixOS:master Jan 12, 2025
24 of 27 checks passed

GaetanLepage deleted the ollama branch January 12, 2025 18:55

scd31 reviewed Jan 15, 2025

View reviewed changes

abysssol mentioned this pull request Jan 15, 2025

fix ollama gpu acceleration #373924

Closed

13 tasks

ollama: 0.5.1 -> 0.5.4 #365718

ollama: 0.5.1 -> 0.5.4 #365718

Conversation

GaetanLepage commented Dec 16, 2024

Things done

GaetanLepage commented Dec 16, 2024 • edited Loading

abysssol commented Dec 17, 2024

mastoca commented Dec 18, 2024

mastoca commented Dec 24, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abysssol Dec 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GaetanLepage commented Dec 25, 2024

nixpkgs-review result

x86_64-linux

aarch64-linux

x86_64-darwin

aarch64-darwin

abysssol commented Dec 25, 2024

mastoca commented Dec 25, 2024

GaetanLepage commented Dec 25, 2024

abysssol commented Dec 26, 2024

mastoca left a comment • edited Loading

Choose a reason for hiding this comment

collares commented Jan 4, 2025 • edited Loading

nixpkgs-review result

x86_64-linux

GaetanLepage commented Jan 4, 2025

nixpkgs-review result

x86_64-linux

GaetanLepage commented Jan 4, 2025

nixpkgs-review result

x86_64-linux

collares commented Jan 4, 2025

mastoca commented Jan 5, 2025 • edited Loading

lucasew commented Jan 5, 2025

nixpkgs-review result

x86_64-linux

GaetanLepage commented Jan 5, 2025

GaetanLepage commented Jan 5, 2025

nixpkgs-review result

x86_64-linux

x86_64-darwin

aarch64-darwin

GaetanLepage commented Jan 5, 2025

GaetanLepage commented Jan 5, 2025

nixpkgs-review result

x86_64-linux

x86_64-darwin

aarch64-darwin

GaetanLepage commented Jan 5, 2025

mastoca commented Jan 5, 2025

GaetanLepage commented Jan 5, 2025

toonn commented Jan 12, 2025

niklaskorz commented Jan 12, 2025

nixpkgs-review result

aarch64-darwin

GaetanLepage commented Jan 12, 2025

sarahec commented Jan 12, 2025

niklaskorz commented Jan 12, 2025 • edited Loading

sarahec commented Jan 12, 2025

scd31 commented Jan 15, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GaetanLepage commented Dec 16, 2024 •

edited

Loading

abysssol Dec 26, 2024 •

edited

Loading

`nixpkgs-review` result

`x86_64-linux`

`aarch64-linux`

`x86_64-darwin`

`aarch64-darwin`

mastoca left a comment •

edited

Loading

collares commented Jan 4, 2025 •

edited

Loading

`nixpkgs-review` result

`x86_64-linux`

`nixpkgs-review` result

`x86_64-linux`

`nixpkgs-review` result

`x86_64-linux`

mastoca commented Jan 5, 2025 •

edited

Loading

`nixpkgs-review` result

`x86_64-linux`

`nixpkgs-review` result

`x86_64-linux`

`x86_64-darwin`

`aarch64-darwin`

`nixpkgs-review` result

`x86_64-linux`

`x86_64-darwin`

`aarch64-darwin`

`nixpkgs-review` result

`aarch64-darwin`

niklaskorz commented Jan 12, 2025 •

edited

Loading