diff --git a/docs/guides/preinstall.md b/docs/guides/preinstall.md index f2d2c50df..ff5d66084 100644 --- a/docs/guides/preinstall.md +++ b/docs/guides/preinstall.md @@ -67,17 +67,6 @@ Or you can use `dd` in macOS Terminal: While the installation image is being written to the USB, you can skip to [Copy Wi-Fi firmware](#copy-wi-fi-firmware); but don't follow the steps after it, wait until the ISO has been written to disk. -## Copy Wi-Fi firmware - -!!! Warning "Arch/EndeavourOS" - If you're going to install Arch or EndeavourOS, you do not need to follow this step. - -Linux's Wi-Fi driver uses the same Wi-Fi firmware files as macOS, so we need to copy these files from macOS to Linux. - -[Follow here](https://wiki.t2linux.org/guides/wifi-bluetooth/#on-macos) the **first part in macOS** and come back to this page. - -The second part must be followed **on Linux** after you have completed the installation. - ## Disable Secure Boot Now that you have completed the first part of the Wi-Fi firmware in macOS, you can proceed with the disabling of the secure boot. diff --git a/docs/guides/wifi-bluetooth.md b/docs/guides/wifi-bluetooth.md index ef9fd4de0..2153f7dd6 100644 --- a/docs/guides/wifi-bluetooth.md +++ b/docs/guides/wifi-bluetooth.md @@ -20,15 +20,20 @@ Refer to the "Updating Kernel" section on your distro's FAQ for instructions if ## Setting up -We now use a script which can help you set up Wi-Fi and Bluetooth. Follow the instructions below to use this script :- +We now use a script which can help you set up Wi-Fi and Bluetooth. Click [here](../tools/firmware.sh) to download the script. -### On macOS +There are 4 methods supported by this script to get firmware for Linux, named as **Method 1-4** in this guide. Details of each method are given in the [On macOS](#on-macos) and [On Linux](#on-linux) section. + +**Method 1-3** require macOS installed on your Mac. The initial steps of these methods are to be followed on macOS, and later steps have to be followed on Linux. Thus, if you choose one of these methods, you first need to follow the [On macOS](#on-macos) section and then proceed to the [On Linux](#on-linux) section. + +**Method 4** does not require macOS, so you can directly follow the [On Linux](#on-linux) section if you choose it. + +!!! Tip "macOS Removed after installing Linux" + In case you have removed macOS after installing Linux, and need the firmware, **Method 4** is the only option for you. -1. Click [here](../tools/firmware.sh) to download the script. -2. Boot into macOS. -3. Run this script there. +### On macOS -When you run the script in macOS, it will ask you to choose between 3 methods to move firmware to Linux: +Run the script on the macOS terminal. After you run the script, it will ask you to choose between 3 methods to move firmware to Linux: === "Method 1" **Method 1: Run the same script on Linux** @@ -46,7 +51,7 @@ When you run the script in macOS, it will ask you to choose between 3 methods to 1. **python3** - Renames the firmware and creates the tarball. - The script shall automatically detect if any dependency is missing, and if required, will give you the option of installing it. So you need not worry about not having any dependency installed. + The script shall automatically detect if any dependency is missing, and if required, will give you the option of installing it. So you need not worry about having missing dependencies. Once the script confirms that you have the necessary dependencies installed, it shall create a tarball of the firmware by the name of `firmware.tar` in your **Downloads** folder. @@ -63,7 +68,7 @@ When you run the script in macOS, it will ask you to choose between 3 methods to 4. **makepkg** - Creates a package that can be installed on Linux using `pacman`. 5. **coreutils** - Additional requirement of **makepkg**. - The script shall automatically detect if any dependency is missing, and if required, will give you the option of installing it. So you need not worry about not having any dependency installed. + The script shall automatically detect if any dependency is missing, and if required, will give you the option of installing it. So you need not worry about having missing dependencies. Once the script confirms that you have the necessary dependencies installed, it shall create a package of the firmware which can be installed by `apt`, `dnf` or `pacman`, depending on the option you chose while running the script. The package shall be saved in your **Downloads** folder. @@ -73,9 +78,6 @@ When you run the script in macOS, it will ask you to choose between 3 methods to Once you have run the script on macOS, depending on the method you chose, the steps to be followed on Linux are described below: -!!! Warning "Running script directly on Linux" - We have noticed a lot of users directly running the script on Linux and without running it first on macOS. Please ensure that you have run the script on macOS first. If you have removed macOS, this script won't be very helpful. - === "Method 1" **Method 1: Run the same script on Linux** @@ -83,12 +85,14 @@ Once you have run the script on macOS, depending on the method you chose, the st === "Option 1" - In this option, you simply have to copy the same script to Linux, and run it with: + In this option, you have to run the firmware script on Linux. You run it with: ```bash bash /path/to/firmware.sh ``` + After you run the script, you have to choose the **"Retrieve the firmware from EFI"** option. After choosing that option, the script will install the firmware on Linux. + !!! note Replace `/path/to/firmware.sh` with the actual path of the script. For example, if the script is in the Downloads folder in Linux, command to be run would be `bash $HOME/Downloads/firmware.sh` @@ -104,6 +108,8 @@ Once you have run the script on macOS, depending on the method you chose, the st sudo umount /tmp/apple-wifi-efi ``` + After you run the above commands, you have to choose the **"Retrieve the firmware from EFI"** option. After choosing that option, the script will install the firmware on Linux. + This option shall be useful if you are unable to copy the script to Linux. === "Method 2" @@ -176,6 +182,24 @@ Once you have run the script on macOS, depending on the method you chose, the st Replace `/path/to/firmware_package.pkg.tar.zst` with the actual path of the package. For example, if `apple-firmware-14.5-1-any.pkg.tar.zst` was created in macOS and has been copied to the Downloads folder in Linux, command to be run would be `sudo pacman -U $HOME/Downloads/apple-firmware-14.5-1-any.pkg.tar.zst` +=== "Method 4" + **Method 4: Download a macOS Recovery Image from Apple and extract the firmware from there** + + !!! warning "Internet connection is required for **Method 4**" + + **Method 4** downloads a macOS Recovery image from Apple. So you need to have an active internet connection on Linux. You can use Ethernet, USB tethering or an external Wi-Fi adapter to get internet. + + This method does not have any steps to be followed on macOS. So, you have to run the script directly on Linux. After you run the script on Linux, you have to choose the **"Download a macOS Recovery Image from Apple and extract the firmware from there**" option. + + If you choose this method, the script will install the following dependencies, if missing, on Linux: + + 1. **curl** - Downloads a [python script](https://github.com/kholia/OSX-KVM/blob/master/fetch-macOS-v2.py) which is used to download the macOS Recovery image from Apple. + 2. **dmg2img** - Converts the downloaded macOS Recovery Image to a Linux readable format. + + The script shall automatically detect if any dependency is missing, and if required, will give you the option of installing it. So you need not worry about having missing dependencies. + + Once the script confirms that you have the necessary dependencies installed, it shall give you the option to choose which macOS version you wish to download. You must choose **macOS Monterey or later** in order to get complete firmware files. After you choose the desired macOS version, the script should do the rest of the work itself. + ## Testing Firmware You can check the logs to confirm correct loading of the firmware using `sudo journalctl -k --grep=brcmfmac`, the output should look similar to this: diff --git a/docs/tools/firmware.sh b/docs/tools/firmware.sh index 8f5ad551e..13922eb5e 100755 --- a/docs/tools/firmware.sh +++ b/docs/tools/firmware.sh @@ -199,7 +199,7 @@ create_arch_pkg () { brew install makepkg coreutils fi - echo -e "\nBuilding Arch package" + echo -e "\nBuilding pacman package" workarea=$(mktemp -d) python3 "$0" /usr/share/firmware ${workarea}/firmware.tar cd ${workarea} @@ -228,11 +228,13 @@ create_arch_pkg () { source=("firmware.tar") noextract=("firmware.tar") sha256sums=('SKIP') + EOF + cat <<- 'EOF' >> PKGBUILD package() { - mkdir -p \$pkgdir/usr/lib/firmware/brcm - cd \$pkgdir/usr/lib/firmware/brcm - tar xf \$srcdir/firmware.tar + mkdir -p $pkgdir/usr/lib/firmware/brcm + cd $pkgdir/usr/lib/firmware/brcm + tar xf $srcdir/firmware.tar } install=apple-firmware.install @@ -257,7 +259,7 @@ create_arch_pkg () { then PKGEXT='.pkg.tar.zst' makepkg else - PKGEXT='.pkg.tar.zst' makepkg >/dev/null 2>&1 || echo "Failed to make Arch package. Run the script with -v to get logs." + PKGEXT='.pkg.tar.zst' makepkg >/dev/null 2>&1 || echo "Failed to make pacman package. Run the script with -v to get logs." fi # Revert path to its original form @@ -268,23 +270,120 @@ create_arch_pkg () { echo -e "\nCleaning up" rm -r ${verbose} ${workarea} - echo -e "\nArch package apple-firmware-${ver}-1-any.pkg.tar.zst has been saved to Downloads!" + echo -e "\nPacman package apple-firmware-${ver}-1-any.pkg.tar.zst has been saved to Downloads!" echo "Copy it to Linux and install it by running the following in the Linux terminal:" echo -e "\nsudo pacman -U /path/to/apple-firmware-${ver}-1-any.pkg.tar.zst" } +detect_package_manager () { + if $(apt --help >/dev/null 2>&1) + then + PACKAGE_MANAGER=apt + elif $(dnf >/dev/null 2>&1) + then + PACKAGE_MANAGER=dnf + elif $(pacman -h >/dev/null 2>&1) + then + PACKAGE_MANAGER=pacman + else + PACKAGE_MANAGER=NONE + echo -e "\nUnable to detect the package manager your distro is using!" + fi +} + +curl_check () { + if ! $(curl --help >/dev/null 2>&1) + then + echo -e "\ncurl and/or its dependencies are missing!" + detect_package_manager + echo + if [[ ${PACKAGE_MANAGER} = "NONE" ]] + then + read -p "The script could not detect your package manager. Please install curl manually and press enter once you have it installed." + else + read -p "Press enter to install curl and its dependencies via ${PACKAGE_MANAGER}. Alternatively you can terminate this script by pressing Control+C and install curl yourself, if you want to install it via some alternate method." + echo -e "\nInstalling curl and its dependencies" + case ${PACKAGE_MANAGER} in + (apt) + sudo apt update + sudo apt install -y curl + ;; + (dnf) + sudo dnf install -y curl + ;; + (pacman) + sudo pacman -Sy --noconfirm curl + ;; + (*) + echo -e "\nUnknown error" + exit 1 + ;; + esac + + fi + fi +} + +dmg2img_check () { + if ! $(dmg2img >/dev/null 2>&1) + then + echo -e "\ndmg2img and/or its dependencies are missing!" + detect_package_manager + echo + if [[ ${PACKAGE_MANAGER} = "NONE" ]] + then + read -p "The script could not detect your package manager. Please install dmg2img manually and press enter once you have it installed." + else + read -p "Press enter to install dmg2img and its dependencies via ${PACKAGE_MANAGER}. Alternatively you can terminate this script by pressing Control+C and install dmg2img yourself, if you want to install it via some alternate method." + echo -e "\nInstalling dmg2img and its dependencies" + case ${PACKAGE_MANAGER} in + (apt) + sudo apt update + sudo apt install -y dmg2img + ;; + (dnf) + sudo dnf install -y dmg2img + ;; + (pacman) + dmg2imgdir=$(mktemp -d) + cd ${dmg2imgdir} + sudo pacman -Sy --noconfirm git base-devel + git clone https://aur.archlinux.org/dmg2img.git ${dmg2imgdir} + makepkg -si --noconfirm + cd - >/dev/null + sudo rm -r ${dmg2imgdir} + ;; + (*) + echo -e "\nUnknown error" + exit 1 + ;; + esac + + fi + fi +} + os=$(uname -s) case "$os" in (Darwin) echo "Detected macOS" ver=$(sw_vers -productVersion) ver_check=$(sw_vers -productVersion | cut -d "." -f 1) - if [[ ${ver_check} < 12 ]] + identifier=$(system_profiler SPHardwareDataType | grep "Model Identifier" | cut -d ":" -f 2 | xargs) + if [[ ${ver_check} < 12 ]] && [[ (${identifier} = MacBookPro15,4) || (${identifier} = MacBookPro16,3) || (${identifier} = MacBookAir9,1) ]] then - echo -e "\nThis script is compatible only with macOS Monterey or later. Please upgrade your macOS." - exit 1 + cat <<- EOF + + Warning: You are running a macOS version earlier than macOS Monterey. + Your Mac model needs Bluetooth firmware for in addition to Wi-Fi firmware, which is available only on macOS Monterey or later. + Only Wi-Fi firmware shall be copied to Linux. + For Bluetooth firmware, you can either: + + a) Upgrade macOS to Monterey or later. + b) Run this script directly in Linux and choose the option to Download a macOS Recovery Image from there. + + EOF fi - identifier=$(system_profiler SPHardwareDataType | grep "Model Identifier" | cut -d ":" -f 2 | xargs) echo -e "\nHow do you want to copy the firmware to Linux?" echo -e "\n1. Run the same script on Linux." echo "2. Create a tarball of the firmware and extract it to Linux." @@ -377,65 +476,118 @@ case "$os" in (Linux) echo "Detected Linux" - if [[ ! -e /lib/firmware ]]; then - echo "/lib/firmware does not seem to exist. This script requires that directory to function." + if [[ ! -e /lib/firmware/brcm ]]; then + echo "/lib/firmware/brcm does not seem to exist. This script requires that directory to function." echo "If you are on some exotic distro like NixOS, please check the wiki for more information:" echo " https://wiki.t2linux.org" echo "Exiting..." exit 1 fi - echo -e "\nIf you are running this script in Linux, make sure you ran it first in macOS and choose option 1 (Run the same script on Linux)." - echo -e "\nContinue? (Y/n)" - read input - if [[ ($input = n) || ($input = N) ]] - then - echo -e "\nRun the script again in Linux when you have ran it in macOS." - exit 1 - fi - echo -e "\nRe-mounting the EFI partition" - mountpoint=$(mktemp -d) - workdir=$(mktemp -d) - echo "Installing Wi-Fi and Bluetooth firmware" - sudo mount ${verbose} /dev/nvme0n1p1 $mountpoint - sudo tar --warning=no-unknown-keyword ${verbose} -xC $workdir -f $mountpoint/firmware.tar.gz - sudo python3 "$0" $workdir $workdir/firmware-renamed.tar ${verbose} - - sudo tar ${verbose} -xC /lib/firmware/brcm -f $workdir/firmware-renamed.tar - - for file in "$mountpoint/brcmfmac4364b2-pcie.txt" \ - "$mountpoint/brcmfmac4364b2-pcie.txcap_blob" - do - if [ -f "$file" ] - then - sudo cp ${verbose} $file /lib/firmware/brcm - fi - done - echo "Reloading Wi-Fi and Bluetooth drivers" - sudo modprobe -r brcmfmac_wcc || true - sudo modprobe -r brcmfmac || true - sudo modprobe brcmfmac || true - sudo modprobe -r hci_bcm4377 || true - sudo modprobe hci_bcm4377 || true - echo "Keeping a copy of the firmware and the script in the EFI partition shall allow you to set up Wi-Fi again in the future by running this script or the commands told in the macOS step in Linux only, without the macOS step." - read -p "Do you want to keep a copy? (y/N)" input - if [[ ($input != y) && ($input != Y) ]] - then - echo "Removing the copy from the EFI partition" - for file in "$mountpoint/brcmfmac4364b2-pcie.txt" \ - "$mountpoint/brcmfmac4364b2-pcie.txcap_blob" \ - "$mountpoint/firmware.tar.gz" \ - "$mountpoint/firmware.sh" - do - if [ -f "$file" ] + echo -e "\nHow do you want to copy the firmware to Linux?" + echo -e "\n1. Retrieve the firmware from EFI." + echo "2. Download a macOS Recovery Image from Apple and extract the firmware from there." + echo -e "\nNote: If you are choosing Option 1, then make sure you have run the same script on macOS before and chose Option 1 (Run the same script on Linux) there." + read choice + case ${choice} in + (1) + echo -e "\nRe-mounting the EFI partition" + mountpoint=$(mktemp -d) + workdir=$(mktemp -d) + echo "Installing Wi-Fi and Bluetooth firmware" + sudo mount ${verbose} /dev/nvme0n1p1 $mountpoint + sudo tar --warning=no-unknown-keyword ${verbose} -xC ${workdir} -f $mountpoint/firmware.tar.gz + sudo python3 "$0" ${workdir} ${workdir}/firmware-renamed.tar ${verbose} + + sudo tar ${verbose} -xC /lib/firmware/brcm -f ${workdir}/firmware-renamed.tar + + for file in "$mountpoint/brcmfmac4364b2-pcie.txt" \ + "$mountpoint/brcmfmac4364b2-pcie.txcap_blob" + do + if [ -f "$file" ] + then + sudo cp ${verbose} $file /lib/firmware/brcm + fi + done + echo "Reloading Wi-Fi and Bluetooth drivers" + sudo modprobe -r brcmfmac_wcc || true + sudo modprobe -r brcmfmac || true + sudo modprobe brcmfmac || true + sudo modprobe -r hci_bcm4377 || true + sudo modprobe hci_bcm4377 || true + echo -e "\nKeeping a copy of the firmware and the script in the EFI partition shall allow you to set up Wi-Fi again in the future by running this script or the commands told in the macOS step in Linux only, without the macOS step." + read -p "Do you want to keep a copy? (y/N)" input + if [[ ($input != y) && ($input != Y) ]] then - sudo rm ${verbose} $file + echo -e "\nRemoving the copy from the EFI partition" + for file in "$mountpoint/brcmfmac4364b2-pcie.txt" \ + "$mountpoint/brcmfmac4364b2-pcie.txcap_blob" \ + "$mountpoint/firmware.tar.gz" \ + "$mountpoint/firmware.sh" + do + if [ -f "$file" ] + then + sudo rm ${verbose} $file + fi + done fi - done - fi - sudo rm -r ${verbose} $workdir - sudo umount $mountpoint - sudo rmdir $mountpoint - echo "Done!" + sudo rm -r ${verbose} ${workdir} + sudo umount $mountpoint + sudo rmdir $mountpoint + echo -e "\nDone!" + ;; + (2) + # Detect whether curl and dmg2img are installed + curl_check + dmg2img_check + cleanup_dmg () { + sudo rm -r ${verbose} ${workdir} + sudo umount ${verbose} ${loopdevice} + sudo rm -r ${verbose} ${imgdir} + sudo losetup -d /dev/loop50 + } + + echo -e "\nDownloading macOS Recovery Image" + workdir=$(mktemp -d) + imgdir=$(mktemp -d) + cd ${workdir} + if [[ ${verbose} = -v ]] + then + curl -O https://raw.githubusercontent.com/kholia/OSX-KVM/master/fetch-macOS-v2.py + else + curl -s -O https://raw.githubusercontent.com/kholia/OSX-KVM/master/fetch-macOS-v2.py + fi + echo -e "\nNote: In order to get complete firmware files, download macOS Monterey or later.\n" + python3 fetch-macOS-v2.py + echo -e "\nConverting image from .dmg to .img" + if [[ ${verbose} = -v ]] + then + dmg2img -v BaseSystem.dmg fw.img + else + dmg2img -s BaseSystem.dmg fw.img + fi + echo "Mounting image" + sudo losetup -P loop50 fw.img + loopdevice=/dev/$(lsblk -o KNAME,TYPE,MOUNTPOINT -n | grep loop50 | tail -1 | awk '{print $1}') + sudo mount ${verbose} ${loopdevice} ${imgdir} + echo "Getting firmware" + cd - >/dev/null + python3 "$0" ${imgdir}/usr/share/firmware ${workdir}/firmware-renamed.tar ${verbose} || (echo -e "\nCouldn't extract firmware. Try choosing some other macOS version (should be Monterey or later). If error still persists, try restarting your Mac and then run the script again." && cleanup_dmg && exit 1) + sudo tar ${verbose} -xC /lib/firmware/brcm -f ${workdir}/firmware-renamed.tar + echo "Reloading Wi-Fi and Bluetooth drivers" + sudo modprobe -r brcmfmac_wcc || true + sudo modprobe -r brcmfmac || true + sudo modprobe brcmfmac || true + sudo modprobe -r hci_bcm4377 || true + sudo modprobe hci_bcm4377 || true + echo "Cleaning up" + cleanup_dmg + echo "Done!" + ;; + (*) + echo -e "\nError: Invalid option!" + exit 1 + ;; + esac ;; (*) echo "Error: unsupported platform" @@ -448,6 +600,7 @@ exit 0 import logging, os, os.path, re, sys, pprint, statistics, tarfile, io from collections import namedtuple, defaultdict from hashlib import sha256 +from pathlib import Path log = logging.getLogger("asahi_firmware.bluetooth") @@ -601,7 +754,7 @@ class WiFiFWCollection(object): dirnames.remove("perf") if "assert" in dirnames: dirnames.remove("assert") - subpath = dirpath.lstrip(source_path) + subpath = os.path.relpath(dirpath, source_path) for name in sorted(filenames): if not any(name.endswith("." + i) for i in self.EXTMAP): continue @@ -625,6 +778,10 @@ class WiFiFWCollection(object): for dim in self.DIMS: if dim in props: ident.append(props.pop(dim)) + + if props: + log.error(f"Unhandled properties found: {props} in file {relpath}") + assert not props node = self.root @@ -756,7 +913,10 @@ logging.getLogger().setLevel(logging.WARNING if (len(sys.argv) >= 4 and sys.argv pkg = FWPackage(sys.argv[2]) wifi_col = WiFiFWCollection(sys.argv[1]+"/wifi") pkg.add_files(sorted(wifi_col.files())) -bt_col = BluetoothFWCollection(sys.argv[1]+"/bluetooth") -pkg.add_files(sorted(bt_col.files())) +if not Path(sys.argv[1] + "/bluetooth").exists(): + log.warning("\nBluetooth firmware missing.\n\nThe source of the firmware is likely macOS Big Sur or earlier. Therefore, only Wi-Fi firmware shall be extracted.\nBluetooth firmware is needed only for MacBookPro15,4, MacBookPro16,3 and MacBookAir9,1. So, you can ignore this message if you do not have these Macs.\n") +else: + bt_col = BluetoothFWCollection(sys.argv[1]+"/bluetooth") + pkg.add_files(sorted(bt_col.files())) pkg.close()