doc: revise Darwin SDK documentation #353439
Conversation
github-actions
bot
added
the
8.has: documentation
reckenrode
force-pushed
the
push-olxpouysuotk
branch
2 times, most recently
from
b0e615d to
dce84b0
Compare
doc/stdenv/platform-notes.chapter.md
Outdated
| - The Darwin `stdenv` uses clang instead of gcc. When referring to the compiler `$CC` or `cc` will work in both cases. Some builds hardcode gcc/g++ in their build scripts, that can usually be fixed with using something like `makeFlags = [ "CC=cc" ];` or by patching the build scripts. | ||
| - Darwin uses clang by default instead of GCC. Packages that refer to `$CC` or `cc` should just work in most cases. Some packages may hardcode `gcc` or `g++`. You can usually fix that by setting `makeFlags = [ "CC=cc" "CXX=C++" ]`. If that does not work, you will have to patch the build scripts yourself to use the correct compiler for Darwin. | ||
| - Darwin needs an SDK to build software. The SDK provides a default set of frameworks and libraries to build software, most of which are specific to Darwin. There are multiple versions of the SDK packages in nixpkgs, but one is included by default in the `stdenv`. Usually, you don’t have to change or pick a different SDK. When in doubt, use the default. | ||
| - The SDK used by your build can be found using the `DEVELOPER_DIR` environment variable. There are also versions of this variable available when cross-compiling depending on the SDK’s role. The `SDKROOT` variable is also set with the path to the SDK’s libraries and frameworks. `SDKROOT` is always a sub-folder of `DEVELOPER_DIR`. |
There was a problem hiding this comment.
This can go later in documentation of internals, most users won’t have to know about this and if they run into a situation where they do they’ll need more information than this.
There was a problem hiding this comment.
I want the opening section to explain what the SDK is and what it contains. If they run into a situation that’s not easily resolvable with basic knowledge, that’s what the rest of the documentation is for.
doc/stdenv/platform-notes.chapter.md
Outdated
| - On Darwin, libraries are linked using absolute paths, libraries are resolved by their `install_name` at link time. Sometimes packages won’t set this correctly causing the library lookups to fail at runtime. This can be fixed by adding extra linker flags or by running `install_name_tool -id` during the `fixupPhase`. | ||
| #### Library install name issues | ||
|
|
||
| Libraries on Darwin are usually linked with absolute paths. This is determiend by something called an “install name”, which is resolved at link time. Sometimes packages will not set this correctly, causing binaries linking to it not to find their libraries at runtime. This can be fixed by adding extra linker flags or by using `install_name_tool` to set it in `fixupPhase`. |
There was a problem hiding this comment.
Let’s recommend fixDarwinDylibNames over manual patching; would simplify this whole section. I also think it could go further down; having to add a newer SDK or xcbuild is much more common than this.
There was a problem hiding this comment.
I can move it down, but this was in the previous documentation. I’d also prefer not to recommend that hook because it normally shouldn’t be needed, and if it’s in the documentation, it might proliferate.
There was a problem hiding this comment.
I added a mention of fixDarwinDylibNames to the documentation in the install name section, which I moved to the end of the troubleshooting section.
doc/stdenv/platform-notes.chapter.md
Outdated
|
|
||
| The `DEVELOPER_DIR` variable in the build environment has the path to the SDK in the build environment. The `SDKROOT` variable there contains a sysroot with the framework, header, and library paths. You can reference an SDK’s sysroot from Nix using the `sdkroot` attribute on the SDK package. Note that it is preferable to use `SDKROOT` because the latter will be resolved to the highest SDK version of any available to your derivation. | ||
| To use a non-default SDK, add it to your derivation’s `buildInputs`. If your derivation needs a non-default SDK at build time (e.g., for a `depsBuildBuild` compiler), see the cross-compilation documentation for which input you should use. |
There was a problem hiding this comment.
We should at least give an example or mention a package name here, since currently people will have to scroll a lot further to find out what they actually add to buildInputs.
There was a problem hiding this comment.
I added an example for the normal SDK override.
doc/stdenv/platform-notes.chapter.md
Outdated
|
|
||
| - You should rely on the default SDK when possible. If a package specifies a required SDK version, use that version (e.g., libuv requires 11.0, so it should use `apple-sdk_11`). When a package supports multiple SDKs, determine which SDK package to use based on the following rules of thumb: | ||
| #### Picking an SDK version |
There was a problem hiding this comment.
I think this can go above to address my earlier concern and because it’s a lot more common than specifying a minimum version or propagating anything.
There was a problem hiding this comment.
It now comes after the SDK and minimum version sections. Propagation is down at the very end of troubleshooting now.
doc/stdenv/platform-notes.chapter.md
Outdated
| - libresolv | ||
| - libsbuf | ||
| #### How to use libiconv on Darwin | ||
|
|
||
| Other common libraries are available in Darwin-specific versions with modifications from Apple. Note that these packages may be made the default on Darwin in the future. | ||
| The libiconv package is included in the SDK by default along with libresolv and libsbuf. You do not need to do anything to use these packages. They are available automatically. If your derivation needs the `iconv` binary, add the `libiconv` package to your `nativeBuildInputs` (or `nativeCheckInputs` for tests). |
There was a problem hiding this comment.
Not sure this section is necessary any more, as it is the assumption from the other platforms we support that libiconv is necessary. It could go under a more general “migration from the old pattern” heading along with the apple_sdk stuff, perhaps.
We should maybe have an earlier mention of things like CUPS that are part of the upstream SDK but need specifying explicitly with Nixpkgs though?
There was a problem hiding this comment.
There’s a lot of if darwin, therefore libiconv in nixpkgs. I want to address any shared knowledge that it’s necessary to continue doing that. If someone wants to remove it in a package, they can point to this section in the documentation if it’s questioned in review.
There was a problem hiding this comment.
Also, I added libutil to the documentation. If it makes more sense, I can split it out into the separate libutil commit that symlinks the dylib, but I’d like to leave it here for review until this is fully iterated and ready.
doc/stdenv/platform-notes.chapter.md
Outdated
|
|
||
| When a package depended on the location of frameworks, references to those framework packages can usually be replaced with `${apple-sdk.sdkroot}/System` or `$SDKROOT/System`. For example, if you substituted `${darwin.apple_sdk.frameworks.OpenGL}/Library/Frameworks/OpenGL.framework` in your derivation, you should replace it with `${apple-sdk.sdkroot}/System/Library/Frameworks/OpenGL.framework` or `$SDKROOT/System/Library/Frameworks`. The latter is preferred because it supports using the SDK that is resolved when multiple SDKs are propagated (see above). | ||
| Some derivations may depend on the location of frameworks in those old packages. To update your derivation to find them in the new SDK, use `$SDKROOT` instead in `preConfigure`. For example, if you substitute `${darwin.apple_sdk.frameworks.OpenGL}/Library/Frameworks/OpenGL.framework` in `postPatch`, replace it with `$SDKROOT/System/Library/Frameworks` in `preConfigure`. |
There was a problem hiding this comment.
Even nicer is to just replace -F with -iframeworkwithsysroot, a la https://gitlab.freedesktop.org/xorg/lib/libapplewm/-/commit/be972ebc3a97292e7d2b2350eff55ae12df99a42. Or in many cases this could be removed entirely, I think.
There was a problem hiding this comment.
It’s not an exhaustive list. The goal isn’t to specify alternatives but to help people deal with their packages’ build systems. I did add a comment about possibly being able to remove some substitutions.
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/the-darwin-sdks-have-been-updated/55295/22 |
|
|
||
| The following is a list of Xcode versions, the SDK version in nixpkgs, and the attribute to use to add it. Generally, only the last SDK release for a major version is packaged (each _x_ in 10._x_ until 10.15 is considered a major version). | ||
| - Try building your derivation with the default SDK. If it works, you’re done. | ||
| - If the package specifies a specific version, use that. See below for how to map Xcode version to SDK version. |
There was a problem hiding this comment.
Maybe some hints for the most common ways/places a project defines the SDK version?
There was a problem hiding this comment.
I added some suggestions about looking in the supported platforms or installation documentation for the package.
There was a problem hiding this comment.
I’m a bit confused as to the overlap between this section and the later “Picking an SDK version”, now that this gives advice about… well… picking an SDK version – perhaps they could be merged?
There was a problem hiding this comment.
I kept the suggestion for where to find it but removed the wider commentary. I think it’s important to have the table be its own section, so I don’t want to combine it with the other one.
reckenrode
force-pushed
the
push-olxpouysuotk
branch
from
dce84b0 to
0ed9c51
Compare
|
I pushed my latest round of updates, which addresses some of the feedback. Time permitting, I’ll return to this later today or tomorrow depending. |
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/the-darwin-sdks-have-been-updated/55295/26 |
reckenrode
force-pushed
the
push-olxpouysuotk
branch
3 times, most recently
from
19014bd to
c1c560c
Compare
doc/stdenv/platform-notes.chapter.md
Outdated
|
|
||
| ##### Fixing absolute paths to `xcodebuild`, `xcrun`, and `PListBuddy` {#sec-darwin-troubleshooting-xcodebuild-absolute-paths} | ||
|
|
||
| Many build systems hardcode the absolute paths to `xcodebuild`, `xcrun`, and `PListBuddy` as `/usr/bin/xcodebuild`, `/usr/bin/xcrun`, and `/usr/bin/PListBuddy` respectively. |
There was a problem hiding this comment.
It’s PlistBuddy (lowercase l) and /usr/libexec/PlistBuddy (not /usr/bin).
| } | ||
| ``` | ||
|
|
||
| #### Package requires a non-default SDK or fails to build due to missing frameworks or symbols {#sec-darwin-troubleshooting-using-sdks} |
There was a problem hiding this comment.
I’d be partial to moving this first, since I think it’s more common than xcrun issues or packages that use Xcode build systems, especially since it often affects non‐Darwin‐specific software, but will leave this up to your judgement.
|
|
||
| The following is a list of Xcode versions, the SDK version in nixpkgs, and the attribute to use to add it. Generally, only the last SDK release for a major version is packaged (each _x_ in 10._x_ until 10.15 is considered a major version). | ||
| - Try building your derivation with the default SDK. If it works, you’re done. | ||
| - If the package specifies a specific version, use that. See below for how to map Xcode version to SDK version. |
There was a problem hiding this comment.
I’m a bit confused as to the overlap between this section and the later “Picking an SDK version”, now that this gives advice about… well… picking an SDK version – perhaps they could be merged?
doc/stdenv/platform-notes.chapter.md
Outdated
| #### Accessing libutil headers {#sec-darwin-troubleshooting-libutil} | ||
|
|
||
| - If a package specifies a range of supported SDK versions _and_ a minimum supported version, assume the package is using availability checks to support the indicated minimum version. Add the highest supported SDK and a `darwinMinVersionHook` set to the minimum version supported by the upstream package. | ||
| Some packages only need to link `libutil.dylib`. | ||
| A link to the dylib for it is included in the SDK. | ||
| If a package needs the libutil headers, include `darwin.libutil` in your `buildInputs`. | ||
| The `darwin.libutil` package is not propagated by default because the headers are not included by default in the upstream SDK. | ||
| Adding them to the SDK confuses some packages and breaks their builds (e.g., Python). | ||
|
|
There was a problem hiding this comment.
There’s not a single package in the tree that wants <libutil.h> on Darwin other than Apple source releases, and in fact it seems like giving build systems <libutil.h> sometimes confuses them. I think we can elide this as more likely to lead people down blind alleys than to help, now that the PR to propagate the darwin.libutil library has been merged. (True, there’ll be a lag of a few weeks, but it’s not worth revising the documentation again.)
doc/stdenv/platform-notes.chapter.md
Outdated
| } | ||
| ``` | ||
| Libraries on Darwin are usually linked with absolute paths. | ||
| This is determiend by something called an “install name”, which is resolved at link time. |
Based on experience with other maintainers trying to update their packages, the documentation has been updated for clarity. It is also no longer recommended to propagate an SDK in most circumstances.
reckenrode
force-pushed
the
push-olxpouysuotk
branch
from
c1c560c to
b8b4cdc
Compare
|
This is a great improvement 💜 We should add a release notes highlight about the new pattern. |
Based on experience with other maintainers trying to update their packages, the documentation has been updated for clarity. It is also no longer recommended to propagate an SDK in most circumstances.
Things done
nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)Add a 👍 reaction to pull requests you find important.