config: cache results of kernel checks

[robn]

Motivation and Context

Kernel checks are the heaviest part of the configure checks. This allows the results to be cached through the normal autoconf cache.

Since we don't want to reuse cached values for different kernels, but don't want to discard the entire cache on every kernel, we instead add a short checksum to kernel config cache keys, based on the version and path, so the cache can hold results for multiple different kernels.

Description

In ZFS_AC_KERNEL, once we've done the kernel detection, we create a simple checksum out of kernelsrc, kernelbuild and kernsrcver.

In ZFS_LINUX_TEST_SRC, if the output cache key exists (regardless of value), we skip creating the test source at all, since we don't want to build it.

Then the test module build runs as normal, if there's anything left that wasn't cached already.

In ZFS_LINUX_TEST_RESULT and ZFS_LINUX_TEST_RESULT_SYMBOL, if the cache key doesn't exist, we run the test and set the cache var from it. Then, based on the cached value, we run the "yes" or "no" commands.

How Has This Been Tested?

By hand, trying combinations of --config-cache and different kernels, then building.

Without --config-cache, works as normal.

With --config-cache, works as normal, and all sorts of interesting kernel things in the cache. Just one:

$ grep zfs_cv_kernel_type_intptr_t config.cache
zfs_cv_kernel_type_intptr_t_2133237685=${zfs_cv_kernel_type_intptr_t_2133237685=no}

Running again with --config-cache, same result just faster.

Building against a different kernel, again, slow on the first run, fast on the next. And then, two items in the cache:

$ grep zfs_cv_kernel_type_intptr_t config.cache
zfs_cv_kernel_type_intptr_t_2133237685=${zfs_cv_kernel_type_intptr_t_2133237685=no}
zfs_cv_kernel_type_intptr_t_3448135993=${zfs_cv_kernel_type_intptr_t_3448135993=no}

And finally, the real win for me is on a potato riscv board I've been messing with this weekend:

$ time ./configure --config-cache
configure: creating cache config.cache
checking for gawk... no
checking for mawk... mawk
checking metadata... git describe
checking build system type... riscv64-unknown-linux-gnu
checking host system type... riscv64-unknown-linux-gnu
checking target system type... riscv64-unknown-linux-gnu
...
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing po-directories commands

real	3m32.698s
user	9m45.988s
sys	0m59.956s

$ time ./configure --config-cache
configure: loading cache config.cache
checking for gawk... (cached) mawk
checking metadata... git describe
checking build system type... (cached) riscv64-unknown-linux-gnu
checking host system type... (cached) riscv64-unknown-linux-gnu
checking target system type... (cached) riscv64-unknown-linux-gnu
...
config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing po-directories commands

real	1m21.683s
user	2m18.199s
sys	0m19.924s

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[satmandu]

As a dkms user, any suggestions for getting this to work with the dkms PRE_BUILD command in scripts/dkms.mkconf ?

I assume one would want to add --config-cache along with some--cache-file= to a location that might be preserved between different zfs dkms builds for the same given kernel?

Or would that be dangerous if the logic behind the caching changes? Say one is updating from OpenZFS 2.3.1 to 2.3.2... it would be useful to reuse a config.cache for a given kernel version if the logic underlying generating every cached value hasn't changed.

PRE_BUILD="configure
  --disable-dependency-tracking
  --prefix=/usr
  --with-config=kernel
  --with-linux=\$(
    if [ -e "\${kernel_source_dir/%build/source}" ]
    then
      echo "\${kernel_source_dir/%build/source}"
    else
      echo "\${kernel_source_dir}"
    fi
  )
  --with-linux-obj="\${kernel_source_dir}"
  \$(
    [[ -n \"\${ICP_ROOT}\" ]] && \\
    {
      echo --with-qat=\"\${ICP_ROOT}\"
    }
  )
  \$(
    [[ -r \${PACKAGE_CONFIG} ]] \\
    && source \${PACKAGE_CONFIG} \\
    && shopt -q -s extglob \\
    && \\
    {
      if [[ \${ZFS_DKMS_ENABLE_DEBUG,,} == @(y|yes) ]]
      then
        echo --enable-debug
      fi
      if [[ \${ZFS_DKMS_ENABLE_DEBUGINFO,,} == @(y|yes) ]]
      then
        echo --enable-debuginfo
      fi
    }
  )
"

[robn]

@satmandu your suggestion --config-cache --cache-file=/some/distant/path is probably the way to do it, if you were going to.

I personally wouldn't though unless you're actually doing DKMS builds tens of times a day, because on any machine you're likely to run OpenZFS on this will save, what, 30 seconds? I'd be wondering more why you're doing that many DKMS builds!

It's not going to help on a kernel update (no cache for that version), and the cache also holds library test results, which might change in a library upgrade. It might save a little bit in an OpenZFS upgrade as we don't often change the meaning of existing tests (generally only when adding or removing support for kernel versions), but again, only likely to be a few seconds.

Of course, if you're just doing it for yourself, then go for it - it's probably fine if you know what you're doing. I'd be against it for end-users though because if it goes wrong, it's not at all obvious what to do. The answer is "delete config.cache, rerun from scratch", not something a casual upgrader should be expected to know.

[tonyhutter]

Works for me 👍

# First time
$ time ./configure --config-cache --enable-debug --enable-debuginfo
real	0m21.568s
user	1m25.633s
sys	0m15.948s

# Second time
$ time ./configure --config-cache --enable-debug --enable-debuginfo
real	0m12.478s
user	0m19.701s
sys	0m4.946s

[tonyhutter]

Could we get one more review on this one? I'd like to pull it in.

[tonyhutter]

@satmandu any objections to this PR?

[satmandu]

@satmandu any objections to this PR?

No objections!

[robn]

Mint, cheers!