Merge pull request #342 from OpenChemistry/release-docs

Add documentation for the release process

docs/release/README.md

@@ -0,0 +1,88 @@
+Update Tomviz Version in CMake
+==============================
+
+In the OpenChemistry/tomviz repository, update the hard-coded version
+numbers in the root `CMakeLists.txt` file labeled `tomviz_version_major`,
+`tomviz_version_minor`, and `tomviz_version_patch`. These are used by
+the source tarball to correctly identify the version.
+
+Create the Git Tag
+==================
+
+In the OpenChemistry/tomviz repository, create a signed git tag for the
+release/release candidate, and then push it. For instance:
+
+```bash
+git tag -s 2.0.0-rc1 -m "Tag 2.0.0 release candidate 1"
+git push --tags origin
+```
+
+Never delete a tag once pushed. If a mistake is made, make a new tag.
+
+
+Build the Binaries
+==================
+
+In the OpenChemistry/tomviz-superbuild repository, trigger a manual "build"
+workflow in order to generate the release binaries. To do so, follow these
+steps:
+
+1. Go to the [build workflow](https://github.com/OpenChemistry/tomviz-superbuild/actions/workflows/build.yml) on the tomviz-superbuild GitHub page.
+2. On the right side of the web page at the top of the table, there should be
+a `Run workflow` button. Click this, selecting the master branch, and then
+type in the Tomviz tag that should be used for the release. Start the workflow.
+3. Once the workflow finishes, a new tag and release should have automatically
+been created in the OpenChemistry/tomviz-superbuild repository that matches the
+Tomviz tag that was used for the build. The attached assets should be as follows:
+
+* Tomviz-<tag>.dmg (macOS DMG binary)
+* Tomviz-<tag>.msi (Windows installer binary)
+* tomviz-<tag>.tar.gz (the release source)
+* Tomviz-<tag>.zip (Windows zip binary)
+* Tomviz-<tag>-Linux.tar.gz (the Linux binary)
+
+Do some testing on each of these to verify that they work properly
+
+
+Windows Signing
+===============
+
+You will need to use [Microsoft's code signing](https://learn.microsoft.com/en-us/windows-hardware/drivers/install/authenticode) with the `.msi` file
+in order for it to be opened without warnings.
+
+
+macOS Notarization
+==================
+
+See the [notarization directory](notarization/readme.md).
+
+After notarization, double-check again that the binary works properly
+(sometimes, notarization can cause issues with the binary itself).
+
+
+Create the SHAs
+===============
+
+After the Windows and Mac binaries have been signed/notarized, replace their
+respective unsigned/unnotarized binaries located in the
+OpenChemistry/tomviz-superbuild release with the signed/notarized versions.
+
+A [helper script](generate_sha_sums.py) located in this directory can then
+be used to create the SHAs file, like so:
+
+```
+./generate_sha_sums.py <tag>
+```
+
+This will automatically download all of the release files and create the
+SHAs file from them. It will also print a command to make a GPG signature.
+Upload the SHAs file along with the GPG signature to the release directory.
+
+
+Final Steps
+===========
+
+With this, the binaries and source are done. We can now proceed to create
+release notes, create a copy of the `OpenChemistry/tomviz-superbuild` release
+on the `OpenChemistry/tomviz` repository (by uploading the same assets), and
+then editing the links on the tomviz website to point to the new release.

docs/release/generate_sha_sums.py

@@ -0,0 +1,61 @@
+#!/usr/bin/env python3
+
+from pathlib import Path
+import urllib.request
+import subprocess
+import sys
+
+if len(sys.argv) < 2:
+    sys.exit('Usage: <script> <release_tag>')
+
+release_tag = sys.argv[1]
+
+filenames = [
+    f'Tomviz-{release_tag}.dmg',
+    f'Tomviz-{release_tag}-Linux.tar.gz',
+    f'Tomviz-{release_tag}.msi',
+    f'tomviz-{release_tag}.tar.gz',
+    f'Tomviz-{release_tag}.zip',
+]
+
+print('Downloading files...')
+for filename in filenames:
+    repo_url = 'https://github.com/openchemistry/tomviz-superbuild'
+    release_url = f'{repo_url}/releases/download/{release_tag}/{filename}'
+
+    print(f'Downloading "{filename}"...')
+    urllib.request.urlretrieve(release_url, filename)
+
+output_file_name = f'Tomviz-{release_tag}-SHAs.txt'
+
+print('Generating shas...')
+with open(output_file_name, 'w') as wf:
+    sum_methods = {
+        'SHA-256': 'sha256sum',
+        'SHA-512': 'sha512sum',
+    }
+
+    last_label = list(sum_methods)[-1]
+
+    for sum_label, sum_exec in sum_methods.items():
+        wf.write(f'{sum_label}\n\n')
+        for filename in filenames:
+            cmd = [sum_exec, filename]
+            output = subprocess.check_output(cmd)
+            wf.write(output.decode())
+
+        # Write a newline if it isn't the last one
+        if sum_label != last_label:
+            wf.write('\n')
+
+print(f'SHAs written out to "{output_file_name}"')
+
+print('Cleaning up...')
+for filename in filenames:
+    Path(filename).unlink()
+
+print('All done. Now use GPG to create a signature, like so:')
+print(f'$ gpg --armor --output Tomviz-{release_tag}-SHAs.txt.asc --detach-sig Tomviz-{release_tag}-SHAs.txt')
+
+print(f'Then, upload both "Tomviz-{release_tag}-SHAs.txt"',
+      f'and "Tomviz-{release_tag}-SHAs.txt.asc" to GitHub!')

docs/release/notarization/README.md

@@ -0,0 +1,52 @@
+Intro
+=====
+
+Notarization on macOS requires several steps, and can be somewhat tedious to
+work through. The benefit, however, is that it guarantees a high level of
+security for the software.
+
+See [here](https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution) for more information.
+
+Steps
+=====
+
+1. Create an Apple ID if you don't already have one
+2. Get added to an authorized provider
+3. Create an app-specific password for your application from your Apple ID
+  a) Sign in at https://appleid.apple.com/
+  b) Scroll down to Security and generate an app-specific password
+4. Sign into the computer that is set up for notarization. Set the environment
+variables `MY_APPLE_ID` and `MY_APPLE_PASSWORD` (the app-specific password).
+You will likely want to remove `MY_APPLE_PASSWORD` from your `~/.bash_history`
+afterward.
+5. Run `bash sign.bash.tomviz` with an argument of the `.dmg` file to sign.
+This will sign the contents of the bundle, create a zip file, and upload it to
+Apple servers for Apple's blessing. In 15-20 minutes, you will receive an
+email indicating success/failure of the notarization process. If it fails,
+see the `Troubleshooting` section below.
+6. On success, type `done`, and the next phase of notarization begins. This
+"staples" the notarization results onto the `.dmg` file.
+
+If all of the above succeeds, you will now have a notarized `.dmg` file.
+You can test it by opening it and double-clicking the application icon.
+It should open without prompting or complaining about missing signatures.
+
+Troubleshooting
+===============
+
+Chances are, your notarization will fail the first few times. You'll need
+to use a command-line tool to get specifics to help debug why, but usually
+it's because you didn't sign a .dylib or other executables in the packaging
+executable needs to be signed.
+
+From the command line (the script will print the UUID once notarization has been requested):
+
+```bash
+xcrun altool --notarization-info <uuid> -u <username>
+```
+
+You may need to modify the entitlements.xml file generated by the script
+to allow the binaries to do certain things. Tomviz requires two. It can
+be hard to figure out which entitlements you need, but if your application
+fails with weird bugs involving Apples Guardian, you might need to add an
+entitlement.

docs/release/notarization/post-notarization.tomviz

@@ -0,0 +1,79 @@
+#!/usr/bin/env bash
+
+# Anything that starts with FIXME_ needs to be filled in
+
+if [ $# -ne 1 ]
+then
+  echo "Usage: $0 /path/to/file.dmg"
+  exit 1
+fi
+
+if [ -z "$MY_APPLE_ID" ]
+then
+  echo "Please set MY_APPLE_ID environment variable and run again."
+  exit 1
+fi
+
+if [ -z "$MY_APPLE_PASSWORD" ]
+then
+  echo "Please set MY_APPLE_PASSWORD environment variable and run again. It should be a Tomviz-specific password created for your account at appleid.apple.com"
+  exit 1
+fi
+
+umask 022
+
+pkg="$1"
+pkg_base="$(basename ${pkg} .dmg)"
+vers="$(echo ${pkg_base} | sed -E -n 's|tomviz-([^-]*(-RC[^-]*)?).*|\1|p')"
+app_name="tomviz.app"
+vol_name="/Volumes/${pkg_base}"
+app_dir="${vol_name}/${app_name}"
+cert_name_app='Developer ID Application: FIXME_DEVELOPER_ID_APPLICATION'
+cert_name_inst='Developer ID Installer: FIXME_DEVELOPER_ID_INSTALLER'
+readonly sla_xml="sla.xml"
+
+echo "Stapling"
+xcrun stapler staple -v "${app_dir}"
+
+echo "Verifying .app is signed and notarized"
+spctl -a -vvv -t install ${app_dir}
+if [ $? -ne 0 ]
+then
+  echo "app is not notarized"
+  exit 1
+fi
+
+# This section does not yet work
+#echo "Insert SLA into PKG with productbuild"
+#curl https://gitlab.kitware.com/paraview/paraview-superbuild/-/raw/master/projects/files/paraview.license.txt > paraview.license.txt
+
+#echo '<?xml version="1.0" encoding="utf-8"?>
+#<installer-gui-script minSpecVersion="1">
+#  <pkg-ref id="org.paraview.ParaView"/>
+#  <license file="paraview.license.txt" />
+#  <options customize="never" require-scripts="false"/>
+#  <choices-outline>
+#    <line choice="default">
+#      <line choice="org.paraview.ParaView"/>
+#    </line>
+#  </choices-outline>
+#  <choice id="default"/>
+#  <choice id="org.paraview.ParaView" visible="false">
+#    <pkg-ref id="org.paraview.ParaView"/>
+#  </choice>
+#  <pkg-ref id="org.paraview.ParaView" version="'${vers}'" onConclusion="none">'${pkg_base}'.pkg</pkg-ref>
+#</installer-gui-script>' > distribution.dist
+# NOTE - the following is needed to add a license, but signing the package with producebuild doesn't work (keychain access error)
+#productbuild --sign "${cert_name_inst}" --distribution distribution.dist --resources . --package-path ${pkg_base}.pkg tmp.pkg
+#mv tmp.pkg ${pkg_base}.pkg
+
+echo "Convert to intermediate format needed for rez tool."
+hdiutil detach "${vol_name}"
+hdiutil resize -size min tmp-udrw.dmg
+
+# Re-compress
+hdiutil convert tmp-udrw.dmg -format UDZO -imagekey zlib-level=9 -ov -o ${pkg_base}.dmg
+rm tmp-udrw.dmg
+
+echo "Inserting SLA into new image"
+hdiutil udifrez -xml "${sla_xml}" 'WHY_IS_THIS_ARGUMENT_NEEDED' "${pkg_base}.dmg"

docs/release/notarization/sign.bash.tomviz

@@ -0,0 +1,108 @@
+#!/usr/bin/env bash
+
+# Anything that starts with FIXME_ needs to be filled in
+
+if [ $# -ne 1 ]
+then
+  echo "Usage: $0 /path/to/file.dmg"
+  exit 1
+fi
+
+if [ -z "$MY_APPLE_ID" ]
+then
+  echo "Please set MY_APPLE_ID environment variable and run again."
+  exit 1
+fi
+
+if [ -z "$MY_APPLE_PASSWORD" ]
+then
+  echo "Please set MY_APPLE_PASSWORD environment variable and run again. It should be a Tomviz-specific password created for your account at appleid.apple.com"
+  exit 1
+fi
+
+umask 022
+rm -f tmp-*.dmg
+
+pkg="$1"
+pkg_base="$(basename ${pkg} .dmg)"
+vers="$(echo ${pkg_base} | sed -E -n 's|tomviz-([^-]*(-RC[^-]*)?).*|\1|p')"
+app_name="tomviz.app"
+vol_name="/Volumes/${pkg_base}"
+app_dir="${vol_name}/${app_name}"
+cert_name_app='Developer ID Application: FIXME_DEVELOPER_ID_APPLICATION'
+cert_name_inst='Developer ID Installer: FIXME_DEVELOPER_ID_INSTALLER'
+readonly sla_xml="sla.xml"
+
+echo "Backing up the original unsigned dmg"
+cp -v ${pkg} ${pkg/.dmg/.unsigned.dmg}
+
+echo '<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+   <key>com.apple.security.cs.allow-dyld-environment-variables</key>
+   <true/>
+   <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
+   <true/>
+</dict>
+</plist>' > entitlements.xml
+
+echo "Extract SLA"
+hdiutil udifderez -xml "$pkg" > "$sla_xml"
+# Prune unneeded sections
+plutil -remove 'blkx' "$sla_xml"
+plutil -remove 'plst' "$sla_xml"
+
+echo "Convert from read-only original image to read-write"
+hdiutil convert ${pkg} -format UDRW -o tmp-udrw.dmg
+
+echo "Making image bigger to avoid running out of space when signing"
+hdiutil resize -size 10G tmp-udrw.dmg
+
+echo "Mount"
+hdiutil attach tmp-udrw.dmg
+
+echo "Collect python executables"
+python_exes=`find "${app_dir}" -name "python*" -type f`
+
+echo "Collect tomviz executables"
+tomviz_exes=`find "${app_dir}" -name "tomviz*" -type f`
+
+echo "Cleanup frameworks"
+for D in ${app_dir}/Contents/Frameworks/*.framework
+do
+  echo "  $(basename ${D})"
+  rm -f ${D}/*.prl
+  mkdir -p ${D}/Versions/Current/Resources
+  if [ -e ${D}/Contents/Info.plist ]
+  then
+    mv ${D}/Contents/Info.plist ${D}/Versions/Current/Resources
+  fi
+  rm -rf ${D}/Contents
+done
+chmod -R ugo+rX ${app_dir}
+
+echo "Collecting binaries"
+so_files=`find "${app_dir}" -name "*.so" -type f`
+echo "Collected so files:"
+echo "${so_files}"
+dylib_files=`find "${app_dir}" -name "*.dylib" -type f`
+echo "Collected dylib files:"
+echo "${dylib_files}"
+
+echo "Signing App ${app_dir}"
+echo codesign --verify --force --timestamp --options=runtime --verbose --deep -s "${cert_name_app}" --entitlements entitlements.xml "${app_dir}"
+# Important: .app bundle needs to be signed last here
+codesign --verify --force --timestamp --options=runtime --verbose --deep -s "${cert_name_app}" --entitlements entitlements.xml \
+  ${so_files} ${dylib_files} ${python_exes} ${tomviz_exes} "${app_dir}"
+
+echo "Creating temporary App zip"
+ditto -c -k --keepParent "${app_dir}/" "tmp-app.zip"
+
+echo "Requesting notarization"
+xcrun altool --notarize-app --primary-bundle-id "org.tomviz.Tomviz" --username "${MY_APPLE_ID}" --password "${MY_APPLE_PASSWORD}" --asc-provider "FIXME_ASC_PROVIDER" --file "tmp-app.zip"
+
+echo "Notarization requested.  Type 'done' when notarization email arrives."
+while read line; do test "$line" == "done" && break; done
+
+bash post-notarization.tomviz $1