Skip to content

CentOS Backups

CentOS Backups

Overview

We don't have a lot of real data to backup in the CentOS Infra, so we started with just something simple and easy that was just extended over the years. We'll describe below the various methods/levels of backup that are complementary and used in serial.

Basically just using following tools:

  • tar/gzip for quick compressed archives
  • mysqldump/pg_dump/mongodump for various DB backup operations
  • rsync
  • over ssh for remote nodes
  • over plain rsyncd for nodes in same DC and through specific module/target
  • restic for rotation and multiple backup storage pools

Let's have a look at involved steps :

Node/server level

We have a dedicated ansible role centos-backup that is used for items to backup/archive on each node that needs a backup. That role can be declared directly on a node/group in ansible inventory and have centos_backup_folders ansible variable/list filled with some folders to locally compress. All local backups will be saved under dedicated unprivileged home folder on same node, with some local retention (so works for very small disk space usage)

Also worth noticing that other roles can import dynamically the centos-backup role (like for example the mysql one, etc) and can also drop a snippet under /etc/centos-backup.d/ so that it would be added to existing backup plan.

Having local backup isn't enough though (of course !), but that's where next step kicks in :

Central server level

A dedicated node having the same centos-backup role applied but with the centos_backup_server boolean set to True would be dedicated to centrally backup nodes from previous steps. It's basically the node having the needed key for unprivileged backup user that will be use to rsync locally the archives generated at the node/server level (see previous step). It can also use plain rsync:// url if there is a valid rsyncd target/module at the server level.

All the hosts to backup are declared in the centos_backup_server_nodes_list ansible list Other settings also permit to use or not local restic tool for local encryption/rotation/dedup.

The central node will also report backup status to zabbix and all logs are available in centos_backup_server_logdir

Offsite level

For some nodes (including but not limited to central server node) we can also export encrypted backup on a remote storage space. That's what the ansible restic role will do and that permits to also have remote (encrypted) backup using rotation on (for example) dedicated and private S3 bucket on AWS