CernVM-FS Repository Gateway and Release Managers¶
This page details the installation and configuration of a repository setup involving a gateway machine and separate release manager machines.
Glossary¶
- Gateway (GW)
- The machine running an instance of the CVMFS repository gateway which has access to the authoritative storage of the managed repositories. This storage is made accessible either as a locally mounted partition or through an S3 API. The purpose of the GW is to manage access to a set of repositories by assigning exclusive leases to specific repository sub-paths to different release manager (RM) machines. The RM can publish files to the sub-path for which it currently holds a lease and send object packs to the GW. Having received the published payload from the RM, the final task of the GW in the publication lifecycle is to rebuild the catalogs and repository manifests for the modified repositories.
- Release manager (RM)
A machine running the CVMFS server tools which can request leases from a GW and publish changes to different repositories where it currently holds valid leases.
The computationally heavy parts of the publication operation take place on the RM: compressing and hashing the files which are to be added or modified. The processed files are then packed together and sent to the GW to be inserted into the repository and made available to clients.
Repository Gateway Configuration¶
As a prerequisite, we need to install the CVMFS client and server packages on the gateway. This means that the gateway machine can be used as a “master” release manager to perform some repository transformations before a separate release manager machine is set up.
The repository gateway application is currently packaged for Ubuntu 16.04, Ubuntu 18.04, SLC 6 and Cern CentOS 7. On Ubuntu, an additional step is required after the package is installed, the setup script needs to be run as root:
# /usr/libexec/cvmfs-gateway/scripts/setup.sh
Create the repository for the following section of this guide:
# cvmfs_server mkfs -o root test.cern.ch
Create an API key file for the new repo (replace <KEY_ID>
and <SECRET>
with actual values):
# cat <<EOF > /etc/cvmfs/keys/test.cern.ch.gw
plain_text <KEY_ID> <SECRET>
EOF
# chmod 600 /etc/cvmfs/keys/test.cern.ch.gw
Add the API key file to the repository configuration in the gateway application:
# cat <<EOF > /etc/cvmfs/gateway/repo.json
{
"repos": [
{
"domain": "test.cern.ch",
"keys": ["<KEY_ID>"]
}
],
"keys": [
{
"type": "file",
"file_name": "/etc/cvmfs/keys/test.cern.ch.gw",
"repo_subpath": "/"
}
]
}
EOF
If Systemd is available, the gateway application can be started with systemctl
:
# systemctl start cvmfs-gateway.service
otherwise it can be manually started:
# /usr/libexec/cvmfs-gateway/scripts/run_cvmfs_gateway.sh start
The ports 80/TCP and 4929/TCP need to be opened in the firewall, to allow access to the repository contents and to the gateway service API.
Alongside the repo.json
file, there is another configuration file
for the repository gateway - user.json
. The most important options
in this file are:
max_lease_time
- the maximum duration, in seconds, of an acquired leasefe_tcp_port
- the port on which the gateway application listens, 4929 by default
By default, the gateway application only spawns a single
cvmfs_receiver
worker process. It is possible to run multiple
worker processes by increasing the value of the size
entry in the
receiver_config
map, found in user.json
. This value should not
be increased beyond the number of available CPU cores.
Release Manager Configuration¶
This section describes the steps needed to set up a release manager for a specific CVMFS repository. The precondition is a working gateway machine where the repository has been created as a Stratum 0.
Example:¶
- The gateway machine is
gateway.cern.ch
. - The release manager is
rm.cern.ch
. - The new repository’s fully qualified name is
test.cern.ch
. - The repository’s public key is
test.cern.ch.pub
. - The GW API key is
test.cern.ch.gw
. - The GW gateway application is running on port 4929 at the URL
http:://gateway.cern.ch:4929/api/v1
. - The repository keys have been copied from the gateway machine onto
the release manager machine, in
/tmp/test.cern.ch_keys
.
To create the repository in the release manager configuration, run the following
command on rm.cern.ch
as an unprivileged user with sudo access:
$ sudo cvmfs_server mkfs -w http://gateway.cern.ch/cvmfs/test.cern.ch \
-u gw,/srv/cvmfs/test.cern.ch/data/txn,http://gateway.cern.ch:4929/api/v1 \
-k /tmp/test.cern.ch_keys -o `whoami` test.cern.ch
At this point, from the RM we can publish to the repository:
$ cvmfs_server transaction test.cern.ch
… make changes to the repository …
$ cvmfs_server publish test.cern.ch
Updating from cvmfs-gateway-0.2.5¶
In the first published version, cvmfs-gateway-0.2.5
, the
application files were installed under /opt/cvmfs-gateway
and the
database files under /opt/cvmfs-mnesia
. Starting with version 0.2.6,
the application is installed under /usr/libexec/cvmfs-gateway
, while
the database files are under /var/lib/cvmfs-gateway
.
When updating from 0.2.5, please make sure that the application is stopped:
# systemctl stop cvmfs-gateway
and rerun the setup script:
# /usr/libexec/cvmfs-gateway/scripts/setup.sh
At this point, the new version of the application can be started. If the old directories are still present, they can be deleted:
# rm -r /opt/cvmfs-{gateway,mnesia}