This script allows you to dynamically list subdirectories and files in a backup directory, dividing lists of files for backup classified by size.
In the command example:
/opt/bacula/scripts/bsplitter 3 1 /var/lib/pgsql
- “3” is the total number of Jobs and FileSets. These must be created manually by the administrator, and must be configured to run in parallel. Jobs must contain the option Accurate = yes.
- “1” is the portion of the backup that will be performed. Then for each FileSet, you will modify the portion that will be backed up. Example:
FileSet1: bsplitter 3 1 /var/lib/pgsql
FileSet2: bsplitter 3 2 /var/lib/pgsql
FileSet3: bsplitter 3 3 /var/lib/pgsql
- “/var/lib/pgsql” is the directory whose content of the backup will be divided into parcels.
If you are using BWeb, as in Figure 1, just call the following script in the Include field of FileSet:
Figure 1. In BWeb, just insert in the Include field: \|/opt/bacula/scripts/bsplitter 3 1 /var/lib/pgsql
Follow the script, ideally saved in the path “/opt/bacula/scripts/bsplitter” of your host Backup client.
#!/bin/bash
#
# Autorship: Heitor Faria (Copyleft: all rights reversed).
# Advisor: Prof. Julio Neves (http://www.livrate.com.br/).
# Tester: xxxx
#
# It must be called in the INCLUDE sub-resource of the FileSet of bacula-dir.conf, referring to the backup of the installed client that wants to split the number of subdirectories:
#
# File = "\\|/opt/bacula/scripts/bsplitter 3 1 /var/lib/pgsql"
# <parallelism><parcel><dir_to_split>
#
#
bwdir="/opt/bacula/working"
sname=$(sed 's|/|.|g' <<< $3)
# minimum time between new files scan, in minutes. Default 22 hours
minupdate=1320
if [ $(find $bwdir/$sname.part* -mmin -$minupdate 2>/dev/null | wc -l) -eq 0 ]
then
rm -f $bwdir/$sname.part*
ListaOrdenada=$(find -L $3 \( -type f -o -type l \) -printf "%s\t%p\n" | sort -nr | cut -f2 | sed 's/ /^/') # replace the space with ^ not to break
ApontaVet=o
Sinal=1
while read Arq
do
let ApontaVet+=Sinal
((ApontaVet > $1)) && {
let ApontaVet--
let Sinal*=-1
}
((ApontaVet < 1)) && {
ApontaVet=1
let Sinal*=-1
}
eval V$ApontaVet+='($Arq)'
done < <(echo "$ListaOrdenada")
for ((N=1; N<=$1; N++))
do
eval TodosArqs=\${V$N[@]} # Generates a line with all files for each vector
for Nome in $TodosArqs # Break apart the list one by one to create the file stack
do
eval Nome=\${Nome//^/ }
echo $Nome >> $bwdir/$sname.part$N
done
done
fi
cat $bwdir/$sname.part$2
Note: Dynamic FileSets are not compatible with bacula’s Exclude. Exclusions must be handled in the script itself.
In the specific PostgreSQL example, to guarantee integrity, this backup is being made from an LVM snapshot mounted on another machine. Otherwise the bank would have to be stopped during the backup or put into backup mode (eg, with the Bacula Enterprise pgsql plugin).
Either way, the bsplitter can be used for any large set of files from the same host, for which you want parallelism.
Restore
As for the restore, you can concatenate the n sets of jobs made for a single restore operation, informing the jobids of the parallelized jobs. Example bconsole:
restore jobid=x,y,z
In BWeb, as shown in Figure 2, it is also possible to concatenate several Jobs in the file selection screen for restores, deselecting the option “Only Selected Fileset”
Figure 2. Restore multiple BWeb Jobs by deselecting “Only Selected Fileset”
Important! For the simultaneous restoration of multiple backups, they must have been written to different volumes/tapes/devices. One way to ensure this is to submit backups of plots to different Pools. Another way (more automatic and more recommended for disk backups), is to deselect the “PreferMountedVolumes = no” directive in the Devices of bacula-sd.conf.
Disponível em: 
Português (Portuguese (Brazil))
English
Español (Spanish)
