Enterprise Bacula S3, Swift, CEPH and Cloud Storage Driver Quick Guide

This Quick Guide is intended to provide insight into the processes required to implement S3, Swift, CEPH and other Cloud Service Providers solutions stored backups, using Bacula Enterprise Edition Cloud Driver.

The Amazon Web Services (AWS), Oracle Cloud, Azure, Google Cloud and BackBlaze are some of the tested Object Storage providers, but most of S3, Swift or CEPH (with S3 Gateway) technologies are compatible with Bacula Cloud Driver.

The Driver provides one more Bacula storage device type, allowing the Bacula administrator to manage local disk, magnetic tapes, and now multiple cloud devices stored backups, using any desired architecture or distribution.

Bacula Enterprise only provides the software. The object storage service must be hired from your favorite service provider.

Cloud Transmission and Cache

The Bacula Cloud Driver transmission to the cloud is only limited by, backup clients disk read, local and to the cloud network capabilities.

In order to avoid Internet connection bottlenecks, the Driver supports advanced local cache and data splitting techniques. The cache may be kept for faster local restores or automatically discarded (truncated), according to the Bacula Administrator will and available local disk storage.

It is also possible to limit transmission bandwidth.

Installation

The Driver is installed on the same machine that hosts the Bacula Storage Daemon. To install both, add a new repository to your RedHat/CentOS distribution, replacing the @customer-string@ to your exclusive customer code, and @version@ to the desired Bacula Enterprise version:

cat /etc/yum.repos.d/bacula.repo

[Bacula]
name=Bacula Enterprise Edition
baseurl=https://www.baculasystems.com/dl/@customer-string@/rpms/bin/@version@/rhel7-64/
enabled=1
protect=0
gpgcheck=0

[BEECloudPlugin]
name=BEE Cloud Plugin
baseurl=https://www.baculasystems.com/dl/@customer-string@/rpms/cloud/@version@/rhel7-64/
enabled=1
protect=0
gpgcheck=0

The Cloud Driver is also available for Debian. E.g.:

cat /etc/apt/sources.list.d/bacula.list

#Bacula Enterprise
deb https://www.baculasystems.com/dl/@customer-string@/debs/bin/@version@/jessie-64/ jessie main
deb https://www.baculasystems.com/dl/@customer-string@/debs/cloud/@version@/jessie-64/ jessie cloud

Use the package manager to update and install the Driver. E.g.:

yum install bacula-enterprise-cloud-storage

# or

apt-get update && apt-get install bacula-enterprise-cloud-storage

Configuration

As shown in Figure 1, create a new Cloud connection configuration. It is possible to use bweb to perform that.

Enterprise Bacula S3, Swift, CEPH and Cloud Storage Driver Quick Guide 1

Figure 1. New Cloud Connection Storage Daemon Configuration

Alternative CEPH and Azure Cloud Service Providers configurations examples are listed as follows:

Cloud {
  Name = CEPH_S3
  Driver = "S3"
  HostName = ceph.mydomain.lan
  BucketName = "CEPHBucket"
  AccessKey = "xxxXXXxxxx"
  SecretKey = "xxheeg7iTe0Gaexee7aedie4aWohfuewohxx0"
  Protocol = HTTPS
  Upload = EachPart
  UriStyle = Path  # Must be set for CEPH
}


Cloud {
  Name = Azure
  Driver = "Azure"
  HostName = "Azure" #not used but needs to be specified
  BucketName = "AzureContainerName"
  AccessKey = "AzureStorareAccountName"
  SecretKey = ""gTL8YA6yyyVLZqrUt1HxHasdfTM5Vcloc5CVY+g+PcS3cF5FpDyNNwrjXwz78gvZj2K4mh0mTvV5TchQ=="
  Protocol = HTTPS
  UriStyle = Path
}

After the Cloud configuration, as viewed in Figure 2, add a Bacula Storage Daemon Device that will write to the configured bucket. It should have the Cloud DeviceType, and use the same Cloud configuration previously created. ArchiveDevice is the local disk cache path, typically with a large available space. MediaType should be a unique name in relation to all other existent Bacula Devices, and MaximumPartSize is the size of split data that Bacula will transfer to the bucket.

Enterprise Bacula S3, Swift, CEPH and Cloud Storage Driver Quick Guide 2

Figure 2. New Cloud Storage Daemon Device

Finally, as exhibited in Figure 3, it is time to attach the new Storage Daemon Device to your Bacula Director. Create a new Director Storage. The Address and Password are usually the same already used by the same Storage Daemon devices. The DeviceName and MediaType must match the values defined in the recently created Storage Daemon Device configuration.

Enterprise Bacula S3, Swift, CEPH and Cloud Storage Driver Quick Guide 3

Figure 3. Bacula Director New Storage Addition

It is advisable to create one or more new Pools to host the Cloud backups, using the newly created Storage. A new CacheRetention Pool directive time value (x days, months etc.) be specified, in order to expire cached volumes.

Modify Bacula Jobs and Schedules to use the new Cloud Pools, at will.

Apply changes and run a test backup job.

Test and Troubleshooting

The S3 CLI can be used to test AWS and other Cloud Storage connections. It is possible to install it using pip:

yum -y install python-pip
pip install awscli

The following command might be used to set up the bucket connection:

aws configure

Bucket creation and a list can be performed as follows (e.g.):

aws s3api --endpoint-url='https://s3.amazonaws.com' 
create-bucket --bucket bsyssdsync
aws s3 ls

More commands, details and general information about the Bacula Enterprise Cloud Driver can be found in the referenced whitepaper.

Reference

Bacula Cloud Backup – Bacula Enterprise Edition. http://baculasystems.com

 

Disponível em: pt-brPortuguês (Portuguese (Brazil)) enEnglish esEspañol (Spanish)

Leave a Reply

Close Menu