Release Notes for CernVM-FS 2.8.0¶
CernVM-FS 2.8 is a feature release that comes with performance improvements, new functionality, and bugfixes.
CernVM-FS 2.8 highlights include
- A new “service container” aimed at easier kubernetes deployments
- Support for macOS 11 Big Sur
- Support for Windows Services for Linux (WSL2)
- Parallelized garbage collection for greatly reduced GC durations
- Support for generating podman image store meta-data in DUCC
- Ability to show the diff of the current transaction using the
cvmfs_server diff --worktreecommand
- Two new experimental features: “template transactions” and ephemeral publish containers (see below)
Together with CernVM-FS 2.8.0, we release the CernVM-FS Repository Gateway version 1.2. For gateway deployments, both package updates should be installed in concert.
As with previous releases, upgrading should be seamless just by installing the new package from the repository. As usual, we recommend to update only a few worker nodes first and gradually ramp up once the new version proves to work correctly. Please take special care when upgrading a client in NFS mode.
For Stratum 0 servers, all transactions must be closed before upgrading.
For Stratum 1 servers, there should be no running snapshots during the upgrade.
After the software upgrade, publisher nodes require doing
cvmfs_server migrate for each repository.
CernVM-FS Client Service Container¶
As of this release, we provide the CernVM-FS client in addition to the regular distribution packages as a minimal Docker container. The container is available from Dockerhub and as a standalone tarball to be used with docker load.
The service container can be used to expose /cvmfs mountpoint to the container host. It is meant for fully containerized Linux distributions such as Fedora CoreOS. It can also be deployed as a Kubernetes DaemonSet to provide /cvmfs to pods in kubernetes clusters.
To make meta-data heavy transactions faster, CernVM-FS now provides publishing using “template transactions”. In a template transaction, an existing directory is used as a template for the changes to be published. Cloning the existing directory tree is a fast, meta-data only operation. This can be useful, for instance, when a new patch release differs only in a few files from an existing software stack. In this case, the transaction to publish the patch release could be started like
cvmfs_server transaction -T /available/release=/patch/release # Install patched file in /cvmfs/<repository name>/patch/release cvmfs_server publish
Template transactions are an experimental feature for the time being. Note that template transactions do not yet work for remote publishers connected to a gateway. This limitation will be lifted in a future release.
Ephemeral Publish Container¶
This release provides the new
cvmfs_server enter command that can open
an ephemeral, writable container for a repository. The ephemeral container
effectively promotes a regular, read-only client mountpoint under /cvmfs to
a writable mountpoint. Starting an ephemeral publish container does not require
a full installation of a publisher node; availability of the
fuse-overlayfs packages is sufficient.
Opening an ephemeral publish container is an unprivileged operation. It does require, however, the relatively recent “user namespaces” and “unprivileged fuse mounts” features from the Linux kernel. CentOS 8, for instance, provides a recent enough kernel.
Ephemeral publish containers are an experimental feature for the time being.
The changes written to /cvmfs within the container are discarded upon closing
the container. As long as the container is active, however, new and changed
files can be extracted, e.g. into tarballs, or shown using the new
cvmfs_server diff --worktree command. In particular, build machines that
are not publisher nodes at the same time can make use of the new capability in
order to install software directly in the final location.
Publishing from the ephemeral container to the gateway will be implemented in a future release.
Publisher Statistics Plots¶
An automatically generated web page can present key figures of publish and
garbage collections operations, such as number of files and processed volume.
To activate, set
CVMFS_UPLOAD_STATS_DB=true in the repository server.conf
file. The raw data and the page are published to the stratum 0 /stats location.
- Client: fix reload if only the config repository is mounted
- Client: fix reload on macOS >= 10.15
- Client: fix
cvmfs_config statusoutput for broken mountpoints (CVM-1959)
- Server: fix reflog repair when there is a zombie hash in the manifest (CVM-1919)
- Server: fix
cvmfs_server ingestinto root directory
- Server: fix ingestion of hardlinked catalog markers (CVM-1931)
- Server: refuse non-regular .cvmfscatalog files during publish (CVM-1868)
- Server: fix double counting in
- Server, stratum 1: fix stuck Apache processes with disabled geo API (CVM-1956)
- Gateway: fix corrupted catalog when a nested catalog is replaced by a symlink (CVM-1930)
- Gateway: fix accidental creation of undeletable content caused by improper handling of the reflog
- DUCC: preserve timestamp of extracted files (CVM-1950)
- DUCC: improve robustness against intermittent registry failures (CVM-1829)
Improvements and Changes¶
- Client, macOS: update from osxfuse 3 to macFUSE 4 (CVM-1960)
- Client: several performance improvements on newer kernel and fuse versions
- Client: add
chunk_listmagic extended attribute (CVM-1875)
- Client: add
catalog_countersextended attribute (CVM-1824)
- Client: log when geosort ends up switching a proxy (CVM-1920)
- Client: add POSIX external cache plugin (CVM-1823)
- Client: add
cvmfs_talk chroot <hash>command
- Server: make overlayfs the default union file system for new repositories (CVM-1909)
- Server: make
CVMFS_IGNORE_XDIR_HARDLINKS=yesa default for new repositories
- Server, S3: improve performance of uploading small objects
- Server, S3: Add support for Azure blob storage
- Server: indicate error type by return value in
- Server: add support for wait & retry on opening transactions (CVM-1937)
- Server: show progress during garbage collection sweep phase (CVM-1929)
- Server: improve network error handling during garbage collection (CVM-1957)
- Server: Add
CVMFS_STATS_DB_DAYS_TO_KEEPparameter to prune publish statistics database, defaults to 356 days (CVM-1841)
- Server: spawn watchdog for
- Gateway: multi-threaded, faster processing of incoming data (CVM-1739)
- DUCC: add support for wildcards in image tags (CVM-1715)
- DUCC: add support for converting from private registries
- DUCC: add support for pulling Docker images with an authenticated user using
- DUCC: add
- DUCC: parallel check of image up-to-dateness
- DUCC: add
-toption to set location of temporary files (CVM-1826)
- DUCC: add systemd service unit
- Debian packaging: change apache2 dependency from required to recommended
- Removed perl as a package dependency
Manual Migration from CernVM-FS 2.7.5 Publishers¶
If you do not want to use
cvmfs_server migrate to automatically upgrade,
publisher nodes that maintain Stratum 0 repositories can be migrated from
version 2.7.5 with the following steps:
- Ensure that there are no open transactions and garbage collection processes before updating the server software and during the repository layout migration.
- Install the
- If you use the gateway, install the
cvmfs-gateway-1.2.0package on the gateway node.
- For each repository: adjust /etc/cvmfs/repositories.d/<REPOSITORY>/client.conf and add the
CVMFS_TALK_SOCKET=/var/spool/cvmfs/<REPOSITORY>/cvmfs_ioparameter and the
CVMFS_TALK_OWNER=<user name of repository owner>parameter
- For each repository: adjust /etc/cvmfs/repositories.d/<REPOSITORY>/server.conf and add the
CVMFS_IGNORE_XDIR_HARDLINKS=trueparameter if it is not already set.
- Update /etc/cvmfs/repositories.d/<REPOSITORY>/server.conf and set
In agreement with the repository owner it’s recommended to make a test publish
cvmfs_server transaction <REPOSITORY> cvmfs_server publish <REPOSITORY>
before resuming normal operation.