Release Notes for CernVM-FS 2.5.0¶
CernVM-FS 2.5 is a feature release that comes with performance improvements, new functionality, and bugfixes. We would like to thank Dave Dykstra (FNAL), Brian Bockelman (U. Nebraska) and Ben Tovar (U. Notre Dame) for their contributions to this release!
This release comes with the new Repository Gateway Services that allow for multiple release managers operating concurrently on different subtrees of a repository.
This release also comes with rewritten code for the processing of new files. This was necessary to address several lurking deadlocks. This change should be transparent to users.
Other notable changes include
- Support for AWSv4 authorization protocol in the S3 backend
- Removal of the “multi-bucket” support in the S3 backend (this feature was aimed at a specific, now outdated hardware product)
- Allow for automatic but infrequent garbage collection
- Support for publishing special files (named pipes, sockets, device files)
- Client can adjust itself to a change of the DNS servers
- New platforms: Fedora 26 and 27 on x86_64, macOS 10.11+
As with previous releases, upgrading should be seamless just by installing the new package from the repository. As usual, we recommend to update only a few worker nodes first and gradually ramp up once the new version proves to work correctly. Please take special care when upgrading a client in NFS mode.
For Stratum 0 servers, all transactions must be closed before upgrading.
For Stratum 1 servers, there should be no running snapshots during the upgrade.
After the software upgrade, both stratum 0 and 1 servers require doing
cvmfs_server migrate for each repository.
Note: if the configuration of the Stratum 0/1 server is handled by a configuration management system (Puppet, Chef, …), please see Section Manual Migration from 2.4.4 Release Manager Machines.
The new CernVM-FS Gateway Services allow for distributed server deployments. This can be used for multi-tenant repositories, where every tenant takes ownership of a specific repository subtree. It can also be used to parallelize publishing of content if the different change sets are limited to a specific subtree.
The gateway services come as separate packages. They control the access to the storage and they need to be installed on a central machine. Multiple release manager machines can then be installed that use the gateway service to operate on the same repository.
Detailed documentation is available in Chapter CernVM-FS Repository Gateway and Release Managers.
Automatic, Infrequent Garbage Collection¶
The new parameter
CVMFS_AUTO_GC_LAPSE can be used on stratum 0 and stratum 1
to specify how often the garbage collection should run
It works like the existing
CVMFS_..._TIMESPAN parameters with a string that
is parsed by the
date utility. The default setting is
1 day ago,
meaning that garbage collection runs on publish if the last garbage collection
(manual or automatic) was more that one day ago.
- Client: fix crash in
cvmfs_talk remountwith fixed repository snapshot
- Client: fix retry of repository manifest download in “offline mode”
- Client: fix statvfs for cache size >4G on macOS (CVM-1474)
- Client: use lazy unmount as a last resort in
- Client: Fix storage location of the catalog checksum destination in certain rare cache configurations (CVM-962)
- Client: fix error message when trying to mount an already mounted repo (CVM-1477)
- Server: fix garbage collection of idle repositories (CVM-1460)
- Server: use
systemd start <mount unit>in suid helper if applicable (CVM-1398)
- Server: fix transaction abort with many temporary files (CVM-1390)
- Server: place bootstrapping symlinks on replica storage (CVM-1366)
- Server: sanitize repository names in cvmfs_server (CVM-1389)
- Server: check for autofs in
cvmfs_server rmfsonly for stratum 0s (CVM-1490)
- Server: fix warnings with bash >= 4.4 (CVM-1401)
- Client: don’t enforce
user_allow_otherfuse option (CVM-1379)
- Client: use /etc/auto.master.d/cvmfs.autofs if applicable (CVM-675)
- Client: improve CPU utilization when downloading with limited bandwidth (CVM-1480)
- Client: send “offline mode” enter/recover events to syslog (CVM-1497)
- Client: implement
CVMFS_DNS_ROAMINGon Linux (CVM-496)
- Client: increase default cache limit to 20G on macOS
- Client: use
CVMFS_MAX_IPADDR_PER_PROXY=2by default on macOS
- Client: automatically restart failed authz helper after cool-off period
- Client: create libcvmfs.a and libcvmfs_cache.a on macOS (CVM-1489)
- Server: use AWSv4 S3 authorization if
CVMFS_S3_REGIONis set (CVM-988)
- Server: add
CAP_DAC_READ_SEARCHto swissknife to publish locked-down files
- Server: add support for diff snapshots based on root hash (CVM-1452)
- Server: add
cvmfs_server tag -bto print the hierarchy of branches (CVM-1392)
- Server: make
CVMFS_GENERATE_LEGACY_BULK_CHUNKS=falsethe default (CVM-1429)
- Server: add CloudFlare support to GeoAPI (CVM-1468)
- Server: set httpd selinux label for GeoIP database (CVM-1454)
- Server: new server parameter
Manual Migration from 2.4.4 Release Manager Machines¶
If you do not want to use
cvmfs_server migrate to automatically upgrade,
release manager machines that maintain Stratum 0 repositories as well as web
servers serving stratum 0/1 repositories can be migrated from version 2.4.4 with
the following steps:
- Ensure that there are no open transactions and no active replication or garbage collection processes before updating the server software and during the repository layout migration.
- Install the
- Only on release manager machines: Adjust the /etc/fstab entries for union file system mount (/cvmfs/…) of the repositories: add the
nodevmount option after the
- Only on systemd managed release manager machines: Ensure that the mount units for all the repositories exist by running
/usr/lib/systemd/system-generators/systemd-fstab-generator \ /run/systemd/generator '' '' 2>/dev/null systemctl daemon-reload
On both stratum 0 and stratum 1 servers
- Update /etc/cvmfs/repositories.d/<REPOSITORY>/server.conf and set
On release manager machines, in agreement with the repository owner it’s recommended to make a test publish
cvmfs_server transaction <REPOSITORY> cvmfs_server publish <REPOSITORY>
before resuming normal operation.