CernVM-FS Server Infrastructure¶
This section provides technical details on the CernVM-FS server setup including the infrastructure necessary for an individual repository. It is highly recommended to first consult “Notable CernVM-FS Server Locations and Files” for a more general overview of the involved directory structure.
A CernVM-FS server installation depends on the following environment setup and tools to be in place:
- kernel 4.2.x or later (for OverlayFS) or aufs support built into the kernel (see Section Installing the AUFS-enabled Kernel on Scientific Linux 6)
- Backend storage location available through HTTP
- Backend storage accessible at
/srv/cvmfs/...(unless stored on S3)
- cvmfs and cvmfs-server packages installed
Local Backend Storage Infrastructure¶
CernVM-FS stores the entire repository content (file content and
meta-data catalogs) into a content addressable storage (CAS). This
storage can either be a file system at
/srv/cvmfs or an S3
compatible object storage system (see “Confidential Repositories” for
details). In the former case the contents of
/srv/cvmfs are as
||Central repository storage location
Can be mounted or symlinked to another location before creating the first repository.
||Storage location of a specific repository
Can be symlinked to another location before creating the repository
||Manifest file of the repository
The manifest provides the entry point into the repository. It is the only file that needs to be signed by the repository’s private key.
||List of trusted repository certificates
Contains a list of certificate fingerprints that should be allowed to sign a repository manifest (see .cvmfspublished). The whitelist needs to be signed by a globally trusted private key.
||CAS location of the repository
Data storage of the repository. Contains catalogs, files, file chunks, certificates and history databases in a content addressable file format. This directory and all its contents need to be writable by the repository owner.
||Second CAS level directories
Splits the flat CAS namespace into multiple directories. First two digits of the file content hash defines the directory the remainder is used as file name inside the corresponding directory.
||CAS transaction directory
Stores partial files during creation. Once writing has completed, the file is committed into the CAS using an atomic rename operation.
Server Spool Area of a Repository (Stratum0)¶
The spool area of a repository contains transaction infrastructure and
scratch area of a Stratum0 or specifically a release manager machine
installation. It is always located inside
directories for individual repositories. Note that the data volume of
the spool area can grow very large for massive repository updates since
it contains the writable union file system branch and a CernVM-FS client
||CernVM-FS server spool area
Contains administrative and scratch space for CernVM-FS repositories. This directory should only contain directories corresponding to individual CernVM-FS repositories.
||Individual repository spool area
Contains the spool area of an individual repository and might temporarily contain large data volumes during massive repository updates. This location can be mounted or symlinked to other locations. Furthermore it must be writable by the repository owner.
||CernVM-FS client cache directory
Contains the cache of the CernVM-FS client mounting the r/o branch (i.e.
||CernVM-FS client mount point
Serves as the mount point of the CernVM-FS client exposing the latest published state of the CernVM-FS repository. It needs to be owned by the repository owner and should be empty if CernVM-FS is not mounted to it.
||Writable union file system scratch area
All file system changes applied to
||Temporary scratch location
Some CernVM-FS server operations like publishing store temporary data files here, hence it needs to be writable by the repository owner. If the repository is idle this directory should be empty.
||CernVM-FS client configuration
This contains client configuration variables for the CernVM-FS client mounted to
Repository Configuration Directory¶
The authoritative configuration of a CernVM-FS repository is located in
/etc/cvmfs/repositories.d and should only be writable by the
administrator. Furthermore the repository’s keychain is located in
/etc/cvmfs/keys and follows the naming convention
<fqrn>.key for the repository’s private key and
<fqrn>.pub for the public key. All of those files can be symlinked
somewhere else if necessary.
||CernVM-FS server config directory
This contains the configuration directories for individual CernVM-FS repositories. Note that this path is shortened using
||Config directory for specific repo
This contains the configuration files for one specific CernVM-FS repository server.
||Server configuration file
Authoriative configuration file for the CernVM-FS server tools. This file should only contain valid server configuration variables as it controls the behaviour of the CernVM-FS server operations like publishing, pulling and so forth.
||Client configuration file
Authoriative configuration file for the CernVM-FS client used to mount the latest revision of a Stratum 0 release manager machine. This file should only contain valid client configuration variables. This file must not exist for Stratum 1 repositories.
||Replication configuration file
Contains configuration variables for Stratum 1 specific repositories. This file must not exist for Stratum 0 repositories.
Apart from file and directory locations a CernVM-FS server installation
depends on a few environment configurations. Most notably the
possibility to access the backend storage through HTTP and to allow for
mounting of both the CernVM-FS client at
/var/spool/cvmfs/<fqrn>/rdonly and a union file system on
Granting HTTP access can happen in various ways and depends on the chosen backend storage type. For an S3 hosted backend storage, the CernVM-FS client can usually be directly pointed to the S3 bucket used for storage (see “Confidential Repositories” for details). In case of a local file system backend any web server can be used for this purpose. By default CernVM-FS assumes Apache and uses that automatically.
Internally the CernVM-FS server uses a SUID binary (i.e.
cvmfs_suid_helper) to manipulate its mount points. This is necessary
since transactional CernVM-FS commands must be accessible to the
repository owner that is usually different from root. Both the mount
must be placed into
/etc/fstab for this reason. By default
CernVM-FS uses the following entries for these mount points:
cvmfs2#<fqrn> /var/spool/cvmfs/<fqrn>/rdonly fuse \ allow_other,config=/etc/cvmfs/repositories.d/<fqrn>/client.conf: \ /var/spool/cvmfs/<fqrn>/client.local,cvmfs_suid 0 0 aufs_<fqrn> /cvmfs/<fqrn> aufs br=/var/spool/cvmfs/<fqrn>/scratch=rw: \ /var/spool/cvmfs/<fqrn>/rdonly=rr,udba=none,ro 0 0