This quick guide is intended to provide information on the processes required to implement stored backups of S3, CEPH and other cloud service providers using the Bacula Community or Enterprise Cloud Driver.
Amazon Web Services (AWS), Wasabi, Huawei Cloud and BackBlaze are some of the tested object storage providers, but most S3 or CEPH technologies are compatible with Bacula Cloud Driver. Oracle Cloud, Azure, Google Cloud, Swift (etc.) have specific Bacula Enterprise exclusive Drivers.
The Driver provides one more type of Bacula storage device, allowing the Bacula administrator to manage local disks, tapes and now multiple backups stored on cloud devices using any desired architecture or distribution.
Bacula is all about software. The object storage service should be hired from your favorite service provider.
Cloud Transmission and Cache
The Bacula Cloud Driver transmission to the cloud is only limited by, backup clients disk read, local and to the cloud network capabilities.
In order to avoid Internet connection bottlenecks, the Driver supports advanced local cache and data splitting techniques. The cache may be kept for faster local restores or automatically discarded (truncated), according to the Bacula Administrator will and available local disk storage.
These are the new bacula-sd.conf Cloud resource Directives:
TruncateCache=AfterUpload # Each cache Volume part will be removed just after it is uploaded. Note, if this option is specified, all restores will require a download from the Cloud. # TruncateCache=AtEndOfJob # At the end of the Job, every part that has been successfully uploaded to the Cloud will be removed (truncated). Note, if this option is specified, all restores will require a download from the Cloud. # TruncateCache=No # In this case, the administrator must specify or manually truncate or by a console command script admin (e.g. cloud truncate). Upload=EachPart # Uploads a part (volume subdivision) when it reaches its define maximum size, even if during a Job execution. # Upload=AfterJob # Upload the volume parts only when the backup job is terminated. # Upload=No # Do not upload automatically. The administrator must perform the upload manually or by a command console do script admin (e.g. cloud upload allpools allfrompool storage=<storage_name>).
It is also possible to limit transmission bandwidth.
Installation
The Driver is installed on the same machine that hosts the Bacula Storage Daemon. To install both, add a new repository to your RedHat/CentOS distribution, replacing the @customer-string@ to your exclusive customer code, and @version@ to the desired Bacula Enterprise version:
cat /etc/yum.repos.d/bacula.repo [Bacula] name=Bacula Enterprise Edition baseurl=https://www.baculasystems.com/dl/@customer-string@/rpms/bin/@version@/rhel7-64/ enabled=1 protect=0 gpgcheck=0 [BEECloudPlugin] name=BEE Cloud Plugin baseurl=https://www.baculasystems.com/dl/@customer-string@/rpms/cloud/@version@/rhel7-64/ enabled=1 protect=0 gpgcheck=0
The Cloud Driver is also available for Debian. E.g.:
cat /etc/apt/sources.list.d/bacula.list #Bacula Enterprise deb https://www.baculasystems.com/dl/@customer-string@/debs/bin/@version@/jessie-64/ jessie main deb https://www.baculasystems.com/dl/@customer-string@/debs/cloud/@version@/jessie-64/ jessie cloud
Use the package manager to update and install the Driver. E.g.:
yum install bacula-enterprise-cloud-storage # or apt-get update && apt-get install bacula-enterprise-cloud-storage
Configuration
As shown in Figure 1, create a new Cloud connection configuration. It is possible to use bweb to perform that.
Figure 1. New Cloud Connection Storage Daemon Configuration
Alternative text AWS and CEPH Cloud Service Providers configurations examples are listed as follows:
Cloud {
Name = AWS
Driver = "S3"
HostName = "s3.amazonaws.com"
BucketName = "your_bucket"
AccessKey = "xyz"
SecretKey = "abc"
Protocol = HTTPS
UriStyle = VirtualHost
Truncate Cache = AfterUpload
Upload = EachPart
Region = "us-east-1"
MaximumConcurrentUploads=20
MaximumConcurrentDownloads=20
# MaximumUploadBandwidth = 5MB/s # Optional
}
Cloud {
Name = CEPH_S3
Driver = "S3"
HostName = ceph.mydomain.lan
BucketName = "CEPHBucket"
AccessKey = "xxxXXXxxxx"
SecretKey = "xxheeg7iTe0Gaexee7aedie4aWohfuewohxx0"
Protocol = HTTPS
Upload = EachPart
Truncate Cache = AfterUpload
UriStyle = Path # Must be set for CEPH
MaximumConcurrentUploads=20
MaximumConcurrentDownloads=20
}
After the Cloud configuration, as viewed in Figure 2, add a Bacula Storage Daemon Device that will write to the configured bucket. It should have the Cloud DeviceType, and use the same Cloud configuration previously created. ArchiveDevice is the local disk cache path, typically with a large available space. MediaType should be a unique name in relation to all other existent Bacula Devices, and MaximumPartSize is the size of split data that Bacula will transfer to the bucket.
Figure 2. New Cloud Storage Daemon Device
Here is an text configuration example of a single Storage Daemon Device (note that the Cloud plugin is also compatible with the Autochanger, which are Bacula Device groups):
Device {
Name = MyCloudAWS
Device Type = Cloud
Cloud = AWS
Archive Device = /mnt/backups # local volume cache
Maximum Part Size = 500 MB # important: it is the object upload granularity
Media Type = Cloud01
LabelMedia = yes
Random Access = Yes;
AutomaticMount = yes
RemovableMedia = no
AlwaysOpen = no
}
Finally, as exhibited in Figure 3, it is time to attach the new Storage Daemon Device to your Bacula Director. Create a new Director Storage. The Address and Password are usually the same already used by the same Storage Daemon devices. The DeviceName and MediaType must match the values defined in the recently created Storage Daemon Device configuration.
Figure 3. Bacula Director New Storage Addition
It is advisable to create one or more new Pools to host the Cloud backups, using the newly created Storage. A new CacheRetention Pool directive time value (x days, months etc.) be specified, in order to expire cached volumes.
Modify Bacula Jobs and Schedules to use the new Cloud Pools, at will.
Apply changes and run a test backup job.
Test and Troubleshooting
The S3 CLI can be used to test AWS and other Cloud Storage connections. It is possible to install it using pip:
yum -y install python-pip pip install awscli
The following command might be used to set up the bucket connection:
aws configure
Bucket creation and a list can be performed as follows (e.g.):
aws s3api --endpoint-url='https://s3.amazonaws.com' create-bucket --bucket bsyssdsync aws s3 ls
More commands, details and general information about the Bacula Enterprise Cloud Driver can be found in the referenced whitepaper.
Reference
Bacula Cloud Backup – Bacula Enterprise Edition. http://baculasystems.com
Disponível em: 
Português (Portuguese (Brazil))
English
Español (Spanish)
