Skip to content

Commit

Permalink
Merge branch '2.11' into latest
Browse files Browse the repository at this point in the history
  • Loading branch information
vvolkl committed Sep 6, 2023
2 parents 1ba7422 + c5ee8ba commit 99fc4ab
Show file tree
Hide file tree
Showing 4 changed files with 130 additions and 8 deletions.
3 changes: 3 additions & 0 deletions apx-parameters.rst
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ CVMFS_BACKOFF_MAX Maximum backoff in seconds when retrying to down
CVMFS_CATALOG_WATERMARK | Try to release pinned catalogs when their number surpasses the given watermark.
| Defaults to 1/4 CVMFS_NFILES; explicitly set by shrinkwrap.
CVMFS_CACHE_BASE Location (directory) of the CernVM-FS cache.
CVMFS_CACHE_REFCOUNT If set to *yes*, deduplicate open file descriptors by refcounting.
CVMFS_CACHE_SYMLINKS If set to *yes*, enables symlink caching in the kernel.
CVMFS_CHECK_PERMISSIONS If set to *no*, disable checking of file ownership and permissions (open all files).
CVMFS_CLAIM_OWNERSHIP If set to *yes*, allows CernVM-FS to claim ownership of files and directories.
Expand Down Expand Up @@ -116,6 +117,7 @@ CVMFS_SERVER_URL Semicolon-separated chain of Stratum~1 servers.
CVMFS_SHARED_CACHE If set to *no*, makes a repository use an exclusive cache.
CVMFS_STATFS_CACHE_TIMEOUT | Caching time of ``statfs()`` in seconds (no caching by default).
| Calling ``statfs()`` in high frequency can be expensive.
CVMFS_STREAMING_CACHE If set to *yes*, use a download manager to download regular files on read.
CVMFS_STRICT_MOUNT If set to *yes*, mount only repositories that are listed in ``CVMFS_REPOSITORIES``.
CVMFS_SUID If set to *yes*, enable suid magic on the mounted repository. Requires mounting as root.
CVMFS_SYSLOG_FACILITY | If set to a number between 0 and 7, uses the corresponding
Expand All @@ -139,6 +141,7 @@ CVMFS_USYSLOG | All messages that normally are logged to syslo
CVMFS_XATTR_PRIVILEGED_GIDS Comma-separated list of (main) group IDs that are allowed to access the extended attributes listed by ``CVMFS_XATTR_PROTECTED_XATTRS``.
CVMFS_XATTR_PROTECTED_XATTRS Comma-separated list of extended attributes (full name, e.g. ``user.fqrn``) that are only accessible by ``root`` and the group IDs listed by ``CVMFS_XATTR_PRIVILEGED_GIDS``.
CVMFS_WORKSPACE Set the local directory for storing special files (defaults to the cache directory).
CVMFS_WORLD_READABLE Override posix read permissions to make files in repository globally readable
=============================== ========================================================================================


Expand Down
43 changes: 39 additions & 4 deletions cpt-configure.rst
Original file line number Diff line number Diff line change
Expand Up @@ -811,6 +811,28 @@ command ``sudo cvmfs_talk cache instance`` can be used to show the currently
used cache manager instance.


Refcounted Cache Mode
^^^^^^^^^^^^^^^^^^^^^

The default posix cache manager has a "refcounted" mode, which uses additional
maps to count references to open file descriptors. Multiple processes reading
the same cached files will then no longer create new duplicated file descriptors
for the same opened file, which can be useful for highly parallelized workloads.
This functionality comes with a small memory overhead, which should however not
exceed a few MBs.

The refcount mode can be turned on by setting

::

CVMFS_CACHE_REFCOUNT=yes

and reloading the cvmfs configuration. To switch it off, the repositories have
to be remounted. Switching it off and doing ``cvmfs_config reload`` will not fail,
but silently ignore the option until the next remount in order to properly
work with already open file descriptors.


Tiered Cache
^^^^^^^^^^^^

Expand All @@ -829,7 +851,20 @@ lower instances. The parameter
``CVMFS_CACHE_$tieredInstanceName_LOWER_READONLY=[yes|no]`` controls whether the
lower layer can be populated by the client or not.

Streaming Cache Manager
^^^^^^^^^^^^^^^^^^^^^^^

This mode uses a download manager and a backing cache manager to deliver data.
Pinned files and catalogs use the backing cache manager. Regular data blocks
are downloaded on read, the required data window copied to the user. In order
to use the streaming cache manager, set:

::

CVMFS_STREAMING_CACHE=yes

Note: The streaming cache manager is not ideal when doing multiple small reads
of a large chunk, as each read will trigger a re-download of the entire chunk.

External Cache Plugin
^^^^^^^^^^^^^^^^^^^^^
Expand Down Expand Up @@ -1038,7 +1073,7 @@ files used for :ref:`bulk changes of ownership on release manager machines <sct_
Hotpatching and Reloading
-------------------------

Hotpatching a running CernVM-FS instance allows reloading most of the code
Hotpatching a running CernVM-FS instance allows reloading most of the code
without unmounting the file system. The current active code is
unloaded and the code from the currently (newly) installed binaries is loaded.
Hotpatching is logged to syslog. Since CernVM-FS is re-initialized
Expand All @@ -1047,7 +1082,7 @@ can be also seen as a "reload".

.. note::
During ``reload`` not all client config parameters can be changed,
some need a remount to take effect.
some need a remount to take effect.

Since CernVM-FS 2.11, reloading the client considers the status of ``CVMFS_DEBUGLOG``.
Independent of if the client runs in debug mode or not before the reload, after the reload
Expand Down Expand Up @@ -1273,7 +1308,7 @@ A checker plugin is available `on our website <https://cernvm.cern.ch/fs/#downlo
Since CernVM-FS 2.11 there are two more options: 1) :ref:`Telemetry Aggregator <cpt_telemetry>` that allows the remote
monitoring of all counters of ``cvmfs_talk internal affairs``, and 2) sending an extended CURL HTTP header for
each download request. For this, ``CVMFS_HTTP_TRACING`` must be set. It will then include ``uid``, ``gid``, and
``pid`` with each download request.
``pid`` with each download request.

.. note::
Depending on which CernVM-FS component sends the CURL request, ``uid``, ``gid`` or ``pid`` might not be set.
Expand All @@ -1289,7 +1324,7 @@ accepted and white space around the key is ignored. Invalid keys are ignored. An
CVMFS_HTTP_TRACING=on #(default off)
# illegal headers are: CVMFS-X-h2:ff and X-CVMFS-h3:12_ad
CVMFS_HTTP_TRACING_HEADERS='h1:test|CVMFS-X-h2:ff|X-CVMFS-h3:12_ad | h4 : 12fs_?'
# debug output
(download) CURL Header for URL: /data/81/7c882d4a2e9dd7f9c5c2bfb4e04ff316e436dfC is:
Connection: Keep-Alive
Expand Down
8 changes: 4 additions & 4 deletions cpt-quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ To install the CVMFS package run

::

sudo dnf install https://ecsft.cern.ch/dist/cvmfs/cvmfs-2.10.0/cvmfs-2.10.0-1.fc34.x86_64.rpm https://ecsft.cern.ch/dist/cvmfs/cvmfs-config/cvmfs-config-default-latest.noarch.rpm
sudo dnf install https://ecsft.cern.ch/dist/cvmfs/cvmfs-2.11.0/cvmfs-2.11.0-1.fc37.x86_64.rpm https://ecsft.cern.ch/dist/cvmfs/cvmfs-config/cvmfs-config-default-latest.noarch.rpm


Docker Container
Expand All @@ -59,13 +59,13 @@ Import the container with

::

docker pull cvmfs/service
docker pull registry.cern.ch/cvmfs/service:latest

or with

::

curl https://ecsft.cern.ch/dist/cvmfs/cvmfs-2.10.0/cvmfs-service-2.10.0-1.x86_64.docker.tar.gz | docker load
curl https://ecsft.cern.ch/dist/cvmfs/cvmfs-2.11.0/cvmfs-service-2.11.0-1.x86_64.docker.tar.gz | docker load

Run the container as a system service with

Expand Down Expand Up @@ -100,7 +100,7 @@ Download the CernVM-FS client package in the terminal in order to avoid signatur

::

curl -o ~/Downloads/cvmfs-2.10.0.pkg https://ecsft.cern.ch/dist/cvmfs/cvmfs-2.10.0/cvmfs-2.10.0.pkg
curl -o ~/Downloads/cvmfs-2.11.0.pkg https://ecsft.cern.ch/dist/cvmfs/cvmfs-2.11.0/cvmfs-2.11.0.pkg

Install the CernVM-FS package by opening the .pkg file and reboot.
Future releases will provide a signed and notarized package.
Expand Down
84 changes: 84 additions & 0 deletions cpt-releasenotes.rst
Original file line number Diff line number Diff line change
@@ -1,2 +1,86 @@
Release Notes for CernVM-FS 2.11.0
==================================

CernVM-FS 2.11.0 is a sizeable feature release, containing a number of new features, bug fixes and performance improvements, some of which have been presented at `CHEP 2023 <https://indico.jlab.org/event/459/contributions/11483/attachments/9475/13736/presentation.pdf>`_.

Highlights are:

* Support for symlink kernel caching through CVMFS_CACHE_SYMLINKS (requires libfuse >= 3.16 and kernel >= 6.2-rc1)

* A new reference-counted cache manager mode that reduces the number of open file descriptors with CVMFS_CACHE_REFCOUNT, and a streaming cache mode with CVMFS_STREAMING_CACHE

* A bugfix for an issue that would slow down client startup when the limit for open file descriptors gets very high.

* A new telemetry option to send client performance counters to influx


As with previous releases, upgrading clients should be seamless just by
installing the new package from the repository. As usual, we recommend updating only a few worker nodes first and gradually ramp up once the new version proves
to work correctly. Please take special care when upgrading a cvmfs client in NFS mode.

For Stratum 1 servers, there should be no running snapshots during the upgrade.
For publisher and gateway nodes, all transactions must be closed; no active leases must be present before upgrading.

Packages are available for both the x86_64 and aarch64 architectures, now also for Debian 12. Packages for Ubuntu 16.04 are no longer provided after the deprecation in 2.10.1.

.. note:: The base package, cvmfs-libs, introduced in 2.10, is now used more widely as a dependency, in particular by the cvmfs client package.



Bug fixes
---------

* [client] Fix closing of file descriptors for very large nfiles limit (`#3158 <https://github.com/cvmfs/cvmfs/issues/3158>`_)
* [client] Fix occasional crashes of the watchdog helper process (`#3089 <https://github.com/cvmfs/cvmfs/issues/3089>`_)
* [client] Gracefully handle proxies that prematurely close connections (`#2925 <https://github.com/cvmfs/cvmfs/issues/2925>`_)
* [client] Fix changing to/from debug mode during cvmfs_config reload (`#2897 <https://github.com/cvmfs/cvmfs/issues/2897>`_, `#3359 <https://github.com/cvmfs/cvmfs/issues/3359>`_)
* [client] Fix mount helper so that it works with libfuse3 auto_unmount option (`#3143 <https://github.com/cvmfs/cvmfs/issues/3143>`_)
* [client] Enable core file generation if watchdog is disabled (`#3142 <https://github.com/cvmfs/cvmfs/issues/3142>`_)
* [client] Error out early if certificate is invalid (`#3238 <https://github.com/cvmfs/cvmfs/issues/3238>`_)
* [client] Fix race in signal handling when authz helper is started (`#3211 <https://github.com/cvmfs/cvmfs/issues/3211>`_)
* [client] Use http client auth only if membership is set (`#3333 <https://github.com/cvmfs/cvmfs/issues/3333>`_)
* [client] Use logging settings in mount helper (`#2962 <https://github.com/cvmfs/cvmfs/issues/2962>`_)
* [client] Use dedicated log files for mount helper (`#3314 <https://github.com/cvmfs/cvmfs/issues/3314>`_)
* [client] Warn about potential incorrect use of cvmfs_talk (`#3303 <https://github.com/cvmfs/cvmfs/issues/3303>`_)
* [client] Fix tmpfs recognition to skip readahead (`#3316 <https://github.com/cvmfs/cvmfs/issues/3316>`_)
* [macOS] Fix xattr on symlinks (`#3170 <https://github.com/cvmfs/cvmfs/issues/3170>`_)
* [server] Fix rare deadlock in uploading pipeline (`#3195 <https://github.com/cvmfs/cvmfs/issues/3195>`_)
* [server] Verify meta-info object in cvmfs_server check (`#3139 <https://github.com/cvmfs/cvmfs/issues/3139>`_)
* [server] Fix spurious error message regarding readahead (`#3305 <https://github.com/cvmfs/cvmfs/issues/3305>`_)
* [S3] Gracefully handle HTTP 500 return codes during upload (`#2912 <https://github.com/cvmfs/cvmfs/issues/2912>`_)
* [gc] Fix race in parallel catalog traversal (`#3171 <https://github.com/cvmfs/cvmfs/issues/3171>`_)
* [gc] Extend grep filter to accept microsecond precision in tags (`#3301 <https://github.com/cvmfs/cvmfs/issues/3301>`_)
* [gw] Fix transaction abort after gateway restart (`#3128 <https://github.com/cvmfs/cvmfs/issues/3128>`_)
* [gw] Fix transaction abort after client crash (`#3283 <https://github.com/cvmfs/cvmfs/issues/3283>`_)
* [gw] Fix occasional lease contention errors (`#3259 <https://github.com/cvmfs/cvmfs/issues/3259>`_, `#3077 <https://github.com/cvmfs/cvmfs/issues/3077>`_, `#3272 <https://github.com/cvmfs/cvmfs/issues/3272>`_)
* [gw] Fix publication of uncompressed files through gateway (`#3338 <https://github.com/cvmfs/cvmfs/issues/3338>`_)


Improvements and changes
------------------------

* [client] Re-use the file descriptor for a file already open in the local cache (`#3067 <https://github.com/cvmfs/cvmfs/issues/3067>`_)
* [client] Add support for symlink kernel cache through CVMFS_CACHE_SYMLINKS (`#2949 <https://github.com/cvmfs/cvmfs/issues/2949>`_)
* [client] Add telemetry framework to send performance counters to influx (`#3096 <https://github.com/cvmfs/cvmfs/issues/3096>`_)
* [client] Add streaming cache mode through CVMFS_STREAMING_CACHE=yes (`#3263 <https://github.com/cvmfs/cvmfs/issues/3263>`_, `#2948 <https://github.com/cvmfs/cvmfs/issues/2948>`_)
* [client] Add CVMFS_STATFS_CACHE_TIMEOUT parameter to cache statfs results (`#3015 <https://github.com/cvmfs/cvmfs/issues/3015>`_)
* [client] Make CVMFS_ARCH env available for use in variant symlinks (`#3127 <https://github.com/cvmfs/cvmfs/issues/3127>`_, CVM-910)
* [client] Add CVMFS_WORLD_READABLE client option (`#3115 <https://github.com/cvmfs/cvmfs/issues/3115>`_)
* [client] Restrict ShortString overflow counters to debug mode (`#3081 <https://github.com/cvmfs/cvmfs/issues/3081>`_)
* [client] Improve logging of I/O errors (`#2941 <https://github.com/cvmfs/cvmfs/issues/2941>`_)
* [client] Check for writable log file directories in chkconfig (`#3310 <https://github.com/cvmfs/cvmfs/issues/3310>`_)
* [client] Allow CPU affinity setting through CVMFS_CPU_AFFINITY (`#3330 <https://github.com/cvmfs/cvmfs/issues/3330>`_)
* [client] Add proxy_list and proxy_list_external magic xattrs (`#3233 <https://github.com/cvmfs/cvmfs/issues/3233>`_)
* [client] Add external_url magic xattr (`#3101 <https://github.com/cvmfs/cvmfs/issues/3101>`_)
* [client] Add support for protected xattrs, new client parameters
CVMFS_XATTR_[PRIVILEGED_GIDS,PROTECTED_XATTRS] (`#3103 <https://github.com/cvmfs/cvmfs/issues/3103>`_)
* [client] Add support for custom http tracing headers (`#3094 <https://github.com/cvmfs/cvmfs/issues/3094>`_)
* [client] Add curl debug info to debug log (`#3329 <https://github.com/cvmfs/cvmfs/issues/3329>`_)
* [client] Add support for evicting chunked files through cvmfs_talk (`#3122 <https://github.com/cvmfs/cvmfs/issues/3122>`_)
* [S3] Add support for CVMFS_S3_X_AMZ_ACL server parameter (`#2970 <https://github.com/cvmfs/cvmfs/issues/2970>`_)
* [gc] Avoid duplicate delete requests (`#3117 <https://github.com/cvmfs/cvmfs/issues/3117>`_)
* [gw] Store publisher hostname in lease (`#3130 <https://github.com/cvmfs/cvmfs/issues/3130>`_)
* [gw] Add HTTPS support for connecting to gateway (`#3060 <https://github.com/cvmfs/cvmfs/issues/3060>`_)
* [container] Bump base for service container to EL9
* [packaging] Change libcvmfs static library to libcvmfs_client shared library (`#3113 <https://github.com/cvmfs/cvmfs/issues/3113>`_)
* [packaging] Let client depend on cvmfs-libs (`#3107 <https://github.com/cvmfs/cvmfs/issues/3107>`_)

0 comments on commit 99fc4ab

Please sign in to comment.