Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qt6: don't treat absolute paths without known suffix as library #367206

Merged
merged 1 commit into from
Dec 31, 2024

Conversation

pitkling
Copy link
Member

@pitkling pitkling commented Dec 21, 2024

Fixes issue #366069 (broken sioyek darwin build) by adapting an upstream change when Qt was updated from 6.7.2 to 6.7.3 in PR #344928. Might fix other similar issues (e.g., for packages depending on qt6.qtspeech).

I submitted the same fix upstream which has been accepted and is currently in the staging/integration phase.

Details

(Note: Take the following with a grain of salt, as I've not much experience with cmake/qmake/Qt.)

This upstream commit changed how Qt treats linker flags without a suffix (like .so) when generating .pri/.prl files. Until 6.7.2, a linker flag like ws2_32 was left untouched. The above commit changed it such that such flags are converted to -lws2_32 (seemingly in order to better support FFmpeg, according to the commit message).

Now, nixpkgs on darwin links some libraries, like QtMultimedia via framwork paths like /nix/store/wmm6s68wk9ygg84ibzdf95yp22lcg4ay-qtmultimedia-6.7.3/lib/QtMultimedia.framework/Versions/A/QtMultimedia. As long as these are part of the current build dir or of Qt's $prefix/lib dir, they are recognized as frameworks and handled accordingly. This works, e.g., for something like /nix/store/rdjd1nqlr1ncr4616dcyqdf6ymqy82xc-qtbase-6.7.3/lib/QtGui.framework/Versions/A/QtGui since it is part of qt6.qtbase.

However, QtMultimedia is not part of qt6.qtbase, so /nix/store/wmm6s68wk9ygg84ibzdf95yp22lcg4ay-qtmultimedia-6.7.3/lib/QtMultimedia.framework/Versions/A/QtMultimedia is not recognized as a framework path and, instead, treated like a non-Qt library. Now, before the above mentioned upstream commit, the linker flag was handed over to clang++ unchanged. After the upstream commit, it is transformed into -l/nix/store/wmm6s68wk9ygg84ibzdf95yp22lcg4ay-qtmultimedia-6.7.3/lib/QtMultimedia.framework/Versions/A/QtMultimedia, which causes a linker error since it's not a valid linker flag (the file is a dylib).

This commit adds a patch that slightly adapts the upstream commit. Namely, it only prepends flags without a suffix (like ws2_32) with an -l if they are not given as an absolute path.

Additional Remarks

WIth this fix, the generated prl files look again exactly as before the update from Qt 6.7.2 to 6.7.3 and linking of packages like sioyek works again. However, I'm not sure whether this is actually the correct way to fix the issue.

It seems to me that instead of handing /nix/store/wmm6s68wk9ygg84ibzdf95yp22lcg4ay-qtmultimedia-6.7.3/lib/QtMultimedia.framework/Versions/A/QtMultimedia through to clang++, it should be converted into -framework QtMultimedia (with a suitable -F … flag). In fact, somehow these flags do end up in the actual linker command, not sure how though (maybe due to the recent Apple SDK improvements?). Also, I am surprised that clang++ does not choke on getting simply a directory as an argument. I misinterpreted the output path, /nix/store/wmm6s68wk9ygg84ibzdf95yp22lcg4ay-qtmultimedia-6.7.3/lib/QtMultimedia.framework/Versions/A/QtMultimedia is actually a dylib file. So the absolute path makes sense for linking.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Pinging Maintainers etc.


Add a 👍 reaction to pull requests you find important.

@K900
Copy link
Contributor

K900 commented Dec 22, 2024

Please retarget this to staging. Also, is upstream aware of this at all?

@pitkling pitkling force-pushed the sioyek-fix-darwin-build branch from 0f7cfda to 2364dab Compare December 22, 2024 08:23
@github-actions github-actions bot added 6.topic: python 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: haskell 6.topic: qt/kde 6.topic: kernel The Linux kernel 8.has: documentation This PR adds or changes documentation 8.has: changelog 8.has: module (update) This PR changes an existing module in `nixos/` 6.topic: emacs Text editor 6.topic: rust 6.topic: policy discussion 6.topic: golang 6.topic: ruby 6.topic: vim 6.topic: ocaml 6.topic: stdenv Standard environment 6.topic: nodejs 6.topic: pantheon The Pantheon desktop environment 6.topic: TeX Issues regarding texlive and TeX in general 6.topic: lua 6.topic: testing Tooling for automated testing of packages and modules 6.topic: cinnamon Desktop environment 6.topic: systemd labels Dec 22, 2024
@pitkling
Copy link
Member Author

pitkling commented Dec 22, 2024

Please retarget this to staging.

Ah sorry, I wasn't aware of the mass-rebuild rule for contributions. I retargeted the PR, hope it's correct now. Note that I didn't finish building sioyek with this retargeted PR locally yet to test the fix, it's taking a while…

Also, is upstream aware of this at all?

Not yet, since I wanted to confirm first whether nixpkgs's current darwin behavior (as it was in Qt 6.7.2 and again after this PR) is intended/correct. That is, whether something like the following excerpt from QtTextToSpeech.prl is correct:

…
QMAKE_PRL_LIBS = /nix/store/zs4wfigl2b16vpxqq6q3bsfyxbn99rcv-qtmultimedia-6.7.2/lib/QtMultimedia.framework/Versions/A/QtMultimedia -F/nix/store/5kwgk3bpsz8x76d82dv148py7hc4cgrr-qtspeech-6.7.2/lib -framework QtGui -framework OpenGL -framework AGL -framework AppKit -framework ImageIO -framework Metal -framework QtNetwork -framework QtCore -framework IOKit -framework DiskArbitration
…

In particular, is the first entry the intended linker flag to be generated or should it be also a -framework … flag like the other linker flags in that line?

@ofborg ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Dec 22, 2024
@niklaskorz
Copy link
Contributor

niklaskorz commented Dec 22, 2024

In particular, is the first entry the intended linker flag to be generated or should it be also a -framework … flag like the other linker flags in that line?

Looking at the linker flags for sioyek (without this PR):

/nix/store/l8n4jk7l60yb7jag3i17bz57s7rvf0rd-clang-wrapper-16.0.6/bin/clang++ -stdlib=libc++ -headerpad_max_install_names  -arch arm64 -isysroot /nix/store/bbwmxj5wv6nh3cydiyijp80zn30q5svf-apple-sdk-11.3/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -mmacosx-version-min=11 -o sioyek.app/Contents/MacOS/sioyek  TouchSlider.o TouchCheckbox.o TouchListView.o TouchCopyOptions.o TouchRectangleSelectUI.o TouchRangeSelectUI.o TouchPageSelector.o TouchConfigMenu.o TouchTextEdit.o TouchSearchButtons.o TouchDeleteButton.o TouchHighlightButtons.o TouchSettings.o TouchAudioButtons.o TouchMarkSelector.o TouchDrawControls.o TouchMacroEditor.o TouchGenericButtons.o TouchMainMenu.o book.o config.o database.o document.o document_view.o input.o main.o main_widget.o pdf_renderer.o pdf_view_opengl_widget.o checksum.o new_file_checker.o coordinates.o sqlite3.o ui.o path.o utils.o mysortfilterproxymodel.o RunGuard.o OpenWithApplication.o fzf.o synctex_parser.o synctex_parser_utils.o macos_specific.o qrc_resources.o moc_TouchSlider.o moc_TouchCheckbox.o moc_TouchListView.o moc_TouchCopyOptions.o moc_TouchRectangleSelectUI.o moc_TouchRangeSelectUI.o moc_TouchPageSelector.o moc_TouchConfigMenu.o moc_TouchTextEdit.o moc_TouchSearchButtons.o moc_TouchDeleteButton.o moc_TouchHighlightButtons.o moc_TouchSettings.o moc_TouchAudioButtons.o moc_TouchMarkSelector.o moc_TouchDrawControls.o moc_TouchMacroEditor.o moc_TouchGenericButtons.o moc_TouchMainMenu.o moc_main_widget.o moc_pdf_renderer.o moc_ui.o moc_mysortfilterproxymodel.o moc_RunGuard.o moc_OpenWithApplication.o   -F/nix/store/y49q5fdmygz3jpwzxlrh0khqismcx89q-qtdeclarative-6.8.0/lib -F/nix/store/3la8b8fg6s9awj2adhqm1zvcq7v6jyyv-qtbase-6.8.0/lib -F/nix/store/sxnj63iybx2307i0mha8qm7yrn8n7kbm-qt3d-6.8.0/lib -F/nix/store/a2bdag3c252mn9ayny15c78991n2q26i-qtsvg-6.8.0/lib -F/nix/store/mb907xjzb0y16720qbj4a9aw4hsxpl8d-qtspeech-6.8.0/lib -F/nix/store/azzryfg5p4hfrpp2f33xr9zfa8lp4zcv-qtmultimedia-6.8.0/lib -ldl -L/private/tmp/nix-build-sioyek-2.0.0-unstable-2024-09-29.drv-0/source/mupdf/build/release -lmupdf -lgumbo -lharfbuzz -lfreetype -ljbig2dec -ljpeg -lopenjp2 -lz -framework QtQuickWidgets -framework QtQuick -framework QtOpenGLWidgets -framework QtOpenGL -framework QtWidgets -framework Qt3DInput -framework Qt3DCore -framework QtSvg -framework QtTextToSpeech -l/nix/store/azzryfg5p4hfrpp2f33xr9zfa8lp4zcv-qtmultimedia-6.8.0/lib/QtMultimedia.framework/Versions/A/QtMultimedia -framework QtMultimedia -framework QtGui -framework AppKit -framework ImageIO -framework Metal -framework QtQmlMeta -framework QtQmlModels -framework QtQmlWorkerScript -framework QtQml -framework QtNetwork -framework QtConcurrent -framework QtCore -framework IOKit -framework DiskArbitration -framework UniformTypeIdentifiers -framework AGL -framework OpenGL

That already includes the framework flag directly after the lib flag:

-l/nix/store/azzryfg5p4hfrpp2f33xr9zfa8lp4zcv-qtmultimedia-6.8.0/lib/QtMultimedia.framework/Versions/A/QtMultimedia -framework QtMultimedia

The -l/nix/store/.../QtMultimedia library flag should not be included at all, only the -framework is needed.

@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Dec 22, 2024
@ofborg ofborg bot requested review from NickCao and LunNova December 22, 2024 23:27
@storvik
Copy link
Member

storvik commented Dec 23, 2024

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 367206


aarch64-darwin

✅ 1 package built:
  • sioyek

@pitkling
Copy link
Member Author

Also, is upstream aware of this at all?

Not yet, […]

Just FYI @K900: I just posted a comment in the original upstream issue that was targeted by the mentioned upstream commit to ask whether prepending the -l also to absolute paths was intended. I'll update this thread when I hear back from them.

@pitkling
Copy link
Member Author

@K900: I submitted the change from this PR as an upstream fix which has been reviewed and accepted. It is currently in the staging/integration cycle.

I assume it's gonna take a while until an updated Qt version with the patch hits nixpkgs. Should we include the patch from this PR until it does? I converted this from draft PR to a normal PR and rebased onto current staging.

@pitkling pitkling marked this pull request as ready for review December 31, 2024 11:02
@K900
Copy link
Contributor

K900 commented Dec 31, 2024

Thanks! Let's backport this for the time being.

@K900 K900 merged commit d1a5190 into NixOS:staging Dec 31, 2024
26 of 27 checks passed
@pitkling
Copy link
Member Author

Great, thanks for the quick merge! Quick question: For this to eventually hit 24.11 instead of only unstable, do I have to create a back port PR?

@K900 K900 added the backport staging-24.11 Backport PR automatically label Dec 31, 2024
@K900
Copy link
Contributor

K900 commented Dec 31, 2024

Just adding the label should trigger the automation, assuming it applies cleanly.

@nix-backports
Copy link

nix-backports bot commented Dec 31, 2024

Successfully created backport PR for staging-24.11:

qtprojectorg pushed a commit to qt/qtbase that referenced this pull request Dec 31, 2024
In the update from 6.7.2 to 6.7.3, commit `ea0f00d5d` changed how linker
flags like `ws2_32` are treated when generating pri/prl files.
Specifically, before the commit a flag like `ws2_32` was left
untouched. The above commit changed it such that such flags are
converted to `-lws2_32` (seemingly in order to better support FFmpeg,
according to the commit message). However, this change also affects
absolute paths if the file has no extension. That is, after the above
mentioned commit, an absolute path linker flag to, say, a dylib on macOS
without a suffix will result in prepending the `-l` flag. This will
result in errors during link time.

An example where this caused problems can be found in the nixpkgs PR
draft #367206 (NixOS/nixpkgs#367206).

This adds a small check to ensure that `-l` is not prepended if the
linker flag is an absolute path without a suffix.

Change-Id: I2c7ce3aac6624e1a27c59af233e3da2c1ae7ba60
Reviewed-by: Alexey Edelev <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants