Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

successful update to Vulkan version 1.3.239, working very reliably. #62

Open
wants to merge 29 commits into
base: master
Choose a base branch
from

Conversation

rcoreilly
Copy link

I added an UPDATING.md file that details what I did.

It builds, but crashes on my mac..

@rcoreilly rcoreilly mentioned this pull request May 12, 2022
89 tasks
garymm and others added 6 commits August 25, 2022 23:18
For now the main benefit is that it specifies the dependency on the
Vulkan headers, so it can be built at least on some Linux machines
without installing the Vulkan SDK.

Currently this cannot be built a MacOS machine that doesn't
have the Vulkan SDK installed. I'll try to fix that in the future.
add Bazel build support
This makes this target usable in Bazel without requiring the user to
install a compatible Vulkan SDK.
Bazel: Fetch libMoltenVK rather than relying on system library
The previous attempt to use a hermetic copy of it didn't work in all
cases. The dylib was linked properly, but some Vulkan libraries at
runtime access it through the "loader driver interface", which requires
an `icd.d` directory to be present in a particular path.

See
https://vulkan.lunarg.com/doc/view/1.3.211.0/mac/LoaderDriverInterface.html#user-content-driver-discovery-on-macos

So for now we'll just rely on the user to install MoltenVK
(probably via the LunarG Vulkan SDK), but I hope that maybe this is
fixable once bazelbuild/bazel#13930 is fixed.
Rely on system installation of MoltenVK with Bazel
@xlab
Copy link
Member

xlab commented Sep 29, 2022

Hey! Thanks for the PR; I'll need some time to comprehend the changes :)

Few points:

  1. moltenVK seems to have the support of 1.2, as stated on their GH. Need to check if that's true, also, not sure how much of a difference is between 1.2 and 1.3. Needs investigation

  2. I am running macOS too, so will check the crashes

  3. Did you try to generate using the latest c-for-go? I saw that you had to revert to the version from 2018.. Ideally if we debug crashing, better debug it using newer c-for-go ;)

  4. Bazel seems like a good tool, but need to be sure that the package remains go-gettable. I'm fine if it's using tags (e.g. tags to switch between system-wide installation and provided artifacts).

Otherwise, good job on trying to keep this project up to date! Additional interest motivates me to return some day to these projects :D Spent too much time in the world of commercial development, time to revive hobby projects of the past, I guess...

@rcoreilly
Copy link
Author

Yes I did have to revert to c-for-go 2018 -- I didn't try anything in between once the old version worked. There is some relevant discussion from another user in the issue there or a related issue -- I believe she reported it might have been fixed, but I haven't tried anything.

The good news not reflected in these PR comments perhaps is that everything is working quite reliably on a daily basis for my intensive use of the vulkan code, and across various collaborators and students on the 3 major platforms -- so from an empirical perspective, we're "all good" here! Would be great to get this stuff pushed upstream and supported with the latest c-for-go. I'll certainly be ready to give it a good test.

Here's my attempt to provide a layer on top of vulkan in case you're interested in taking a look at that: https://github.com/goki/vgpu -- your example code in asche was very helpful!

The last thing that we really need to make go-vulkan easily usable on mac is some easy way of installing the vulkan and molten-vk dependencies so that people don't have to download the SDK -- the homebrew effort on that front seems to have hit a wall of some sort..

@tomas-mraz
Copy link
Contributor

Hi, Android mobile devices seem to be stuck on Vulkan 1.1 ... up to date SoC Snapdragon 695 (5G connectivity, 6nm lithography) https://www.qualcomm.com/products/application/smartphones/snapdragon-6-series-mobile-platforms/snapdragon-695-5g-mobile-platform
What does it mean for compiling?

I have a working Linux pipeline on a shared GitLab runner to produce macOS binary using clang, llvm, ...
https://github.com/tpoechtrager/osxcross
https://github.com/phracker/MacOSX-SDKs
and MoltenVK dylib library compiled before on macOS(friend's machine).
Making it simpler will be fine :)

@xlab
Copy link
Member

xlab commented Sep 30, 2022

my intensive use of the vulkan code, and across various collaborators and students on the 3 major platforms -- so from an empirical perspective, we're "all good" here!

Glad it helps! Tbh originally I planned this project to be a great learning tool because the C++ primers for Vulkan were unreadable in 2016. Slowly but surely gets there :D

I will do my investigation, and then we can merge everything because I prefer practicality there over anything else. Even if that means I have to lock the generator using the version from 2018.

your example code in asche was very helpful!

Thanks! My heart melts :)

@xlab
Copy link
Member

xlab commented Sep 30, 2022

Hi, Android mobile devices seem to be stuck on Vulkan 1.1 ... up to date SoC Snapdragon 695 (5G connectivity, 6nm lithography) https://www.qualcomm.com/products/application/smartphones/snapdragon-6-series-mobile-platforms/snapdragon-695-5g-mobile-platform What does it mean for compiling?

I have a working Linux pipeline on a shared GitLab runner to produce macOS binary using clang, llvm, ... https://github.com/tpoechtrager/osxcross https://github.com/phracker/MacOSX-SDKs and MoltenVK dylib library compiled before on macOS(friend's machine). Making it simpler will be fine :)

I'll test on older Android devices to make sure. In any way, we can maintain BOTH 1.1 and 1.3 (tip), same way they do for opengl there https://github.com/go-gl/gl (sub ideal, but totally manageable)

@johanhenriksson
Copy link

This looks super promising! Any updates?

I will try to give it a test since im also on mac

@rcoreilly rcoreilly changed the title successful update to Vulkan version 1.3.211.0 -- builds but crashes on mac though.. successful update to Vulkan version 1.3.239, working very reliably. Feb 10, 2023
@xlab xlab self-requested a review February 13, 2023 12:33
garymm and others added 3 commits March 10, 2023 13:07
No you can run `bazel build` or `bazel run` without installing the
Vulkan SDK!
make bazel build not rely on system Vulkan
@xlab
Copy link
Member

xlab commented Mar 12, 2023

Hey @rcoreilly I've added you as an owner to vulkan-go GitHub organisation, so you are actually free to create necessary repos to maintain Bazel platform artefacts there.

name = "com_github_goki_vulkan_mac_deps",
sha256 = "348bc84c0fc1f1e79fb28bcf83454faa2a84c4d4c2286225d4413830a3c0a29c",
strip_prefix = "vulkan_mac_deps-%s" % vulkan_mac_deps_version,
url = "https://github.com/goki/vulkan_mac_deps/archive/refs/tags/%s.tar.gz" % vulkan_mac_deps_version,
Copy link
Member

@xlab xlab Mar 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean this one for an instance, would like to contain all base Vulkan API bindings and platform deps within this organisation.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xlab thanks -- @garymm is the one who manages all the Bazel stuff, so if it is Ok with you, I can add him and he can do that?

What are the current prospects for your availability to dig into this stuff again and get a cleaner version of it working here? i.e., not using the old version of c-for-go, and automating the logic so the manual patching isn't required? I don't have time to dig into this level of issues at this point, and my hacky solution is working OK for now..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rcoreilly yes, feel free to add your contributors also as members ;)

i.e., not using the old version of c-for-go, and automating the logic so the manual patching isn't required? I don't have time to dig into this level of issues at this point, and my hacky solution is working OK for now..

Yes I still have hopes to dig into this, for this reason I don't want to just blindly merge the PR yet. Although I appreciate the fact that you and @garymm keep this alive and fresh, so I just want it to be available to everyone, even it's not "officially" cleaned up and merged.

For that reason I just ask about publishing build deps / artefacts under this org, so even if your personal focus changes, the repos are not gone and still usable by enthusiasts. I seen so many good repos on GH gone...
So yeah, this was my main motivation to add you into vulkan-go org, so all necessities could be stored there.

But in long term, I still want to clean this up. ✌️

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added vulkan-go/vulkan_mac_deps.
Just update the references to goki/vulkan_mac_deps and it should work.

https://github.com/vulkan-go/vulkan_mac_deps

@kkoreilly kkoreilly deleted the branch vulkan-go:master August 27, 2023 23:48
@kkoreilly kkoreilly deleted the master branch August 27, 2023 23:48
@nergdron
Copy link

any updates on what's going on with this PR or where it's headed? it'd be nice to get this stuff merged at some point, maybe with another update to match current vulkan releases, since it doesn't seem like the refactor xlabs was after is going to happen any time soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants