~~ODT~~
====== Prinzipien ======
* Generelle Punkte unabhängig von der eingesetzten Lösung
* Diese Fragen sind essentiell für eine funktionierende Backup/Restore Lösung
* Hast du eine Backup Lösung ?
* Wurde definiert welche Daten das Backup beinhalten soll ?
* Werden bestimmte Zustände deiner Daten zu bestimmten Zeitpunkten festgehalten ?
* Wurde definiert wie lange bestimmte Zustände für ein Restore verfügbar sein müssen ?
* Gibt es gesetzliche Verpflichtungen für die Dauer der Aufbewahrung und Qualität der Backups ?
* Wo liegen die Backup Daten physikalisch ?
* Wie wird der Raum in dem sich die Daten physikalisch befinden gesichert ?
* Sind die Daten gegen Diebstahl / Blitzschlag / Überschwemmungen gesichert ?
* Was passiert bei Ereignissen **"höherer Gewalt"** ?
* Gibt es zumindest einen 2. Standort deiner Backup Daten ?
* Ist es möglich die Integrität des Backups zu überprüfen ?
* Wurde schon einmal ein Restore durchgeführt ?
* Kannst du selbstständig einen Restore durchführen ?
* Wie lange hat der Restore gedauert ?
* Welche Softwarelösungen zB: Betriebssystem / Basis werden für den Restore benötigt ?
* Welche Lizenzen werden benötigt um einen Restore durchzuführen ?
====== Windows ======
* Windows Sicherung muss installiert sein
* Generiert VHDX Datein die gemountet werden können
===== Externer SAMBA Server rotierend =====
* **backup-generic.bat**
* Bare Metal Backup falls die Maschine selbst etwas hat
* **Achtung** unter BACKUP_TARGET müssen sich dann wie in diesem Fall die Verzeichnisse 0,1 befinden (wegen Module Operation der Anzahl der Wochen die gespeichert werden sollen)
setlocal
set LOGFILE=F:\Backup-Logging.txt
set G=%temp%\getWeek.vbs
set WEEKS=2
set USERNAME=USERNAME_SAMBA
set PASSWORD=PASSWORD_SAMBA
set BACKUP_TARGET=\\backup\backup_serverxy\bare_metal
>%G% echo WScript.Echo Datepart("ww",WScript.Arguments(0),2)
for /f %%i in ('cscript //nologo %G% %date%') do set calWeek=%%i
set /a DIRECTORY=%calweek%%%WEEKS%
date /t >> %LOGFILE%
wbadmin start backup -quiet -vssCopy -user:"%USERNAME%" -password:"%PASSWORD%" -allCritical -backupTarget:%BACKUP_TARGET%%DIRECTORY% >> %LOGFILE% 2>&1
===== "Externes" Laufwerk rotierend =====
* **backup-storagexy.bat**
* **Achtung** Hier wird das Laufwerk G:\ auf F:\ gesichert, wieder müssen die Verzeichnisse 0,1 unter BACKUP_TARGET existieren
setlocal
set LOGFILE=F:\Backup-Logging-StorageXY.txt
set G=%temp%\getWeek.vbs
set WEEKS=2
set BACKUP_TARGET=\\localhost\F$\Backups\STORAGEXY\
>%G% echo WScript.Echo Datepart("ww",WScript.Arguments(0),2)
for /f %%i in ('cscript //nologo %G% %date%') do set calWeek=%%i
set /a DIRECTORY=%calweek%%%WEEKS%
echo "----BEGIN BACKUP----" >> %LOGFILE% 2>&1
date /t >> %LOGFILE%
wbadmin start backup -vssCopy -quiet -include:G:\ -backupTarget:%BACKUP_TARGET%%DIRECTORY% >> %LOGFILE% 2>&1
echo "----END BACKUP----" >> %LOGFILE%
===== Externe Festplatten =====
* Verschiedene Bat Dateien mit Laufwerksangaben für verschiedene externe Festplatten
* gelöscht werden muss **manuell**
* zB: G:\Backups\RECHNERNAME\DATUM
* **backup-extern-g.bat**
@echo off & setlocal
set TARGET=G$
set NAME=%COMPUTERNAME%
for /f %%i in ('date /T') do Set DIRECTORY=%%i
if not exist "\\localhost\%TARGET%\Backups" goto fehler
mkdir \\localhost\%TARGET%\Backups\%NAME%\%DIRECTORY%\
wbadmin start backup -quiet -vssFull -allCritical -backupTarget:\\localhost\%TARGET%\Backups\%NAME%\%DIRECTORY%\
pause
exit
:fehler
echo Konnte Festplatte %TARGET% nicht finden ggf. \\localhost\%TARGET%\Backups Verzeichnis anlegen
pause
exit
===== Hyper-V Guests auf Festplatte rotierend =====
* **backup-hyper-v.bat**
setlocal
set LOGFILE=PFAD\Backup-Logging-Guests.txt
set G=%temp%\getWeek.vbs
set WEEKS=4
set BACKUP_TARGET=\\localhost\F$\Guests\
>%G% echo WScript.Echo Datepart("ww",WScript.Arguments(0),2)
for /f %%i in ('cscript //nologo %G% %date%') do set calWeek=%%i
set /a DIRECTORY=%calweek%%%WEEKS%
echo "----BEGIN BACKUP----" >> %LOGFILE% 2>&1
date /t >> %LOGFILE%
wbadmin start backup -quiet -hyperv:"GUESTNAME1,GUESTNAME2,GUESTNAME3" -backupTarget:%BACKUP_TARGET%%DIRECTORY% >> %LOGFILE% 2>&1
echo "----END BACKUP----" >> %LOGFILE%
===== Restore mit Windows Boardmitteln =====
* Über zB: Windows 10 iso / reparieren / System Image wiederherstellen / Im Netzwerk suchen
* Windows kann grundsätzlich vhdx Dateien als virtuelle Datenträger einbinden zB: über die "Computerverwaltung" -> "Datenträger" -> rechts anfügen vhd(x) - auch UNC Pfade auf Samba Server sind möglich (getestet mit Debian 10 / Windows 10 FIXME
* **Achtung**
* BIOS/UEFI beachten
* Ziel Datenträger muss mindestens gleich groß /größer sein (think thin Provisioning für restore)
* Format für die Eingabe von Benutzername: IP\BENUTZERNAME (wenn Daten auf samba Share liegen)
===== Duplicati 2 - verschlüsseltes Cloud Backup =====
* **Anfoderungen**
* Deduplizierendes, verschlüsseltes Backup , das in der "Cloud" abgelegt werden kann in diesem Fall ins One-Drive Business von einem User Account der als "Backup" User angelegt wurde
* **Lösungsmöglichkeit**
* Duplicati 2: https://www.duplicati.com/download
* **Achtung** **BETA**
* One-Drive Einstellungen
* **Achtung v2 auswählen und mit Graph API Key erstellen** - bei >5000 Dateien in einem Ordner kommt es sonst zu "Limits" die schwer zu debuggen sind
* Getestet auf **Windows 2016 Standard** und **Duplicati 2.0.4.23 - 64bit ** , **Backup enthält > 1 Million Dateien**
{{:know-how:duplicati_one_drive_v2.png?400|}}
{{:know-how:duplicati_overview_backup_entry.png?400|}}
{{:know-how:duplicati_success_restore.png?400|}}
* Wiederherstellung von einzelnen Dateien hat beim Testen funktioniert war jedoch **sehr** langsam
===== Urbackup =====
* https://www.urbackup.org/download.html
* Fehler Referenzmaschine war "Hyper-V" Client und wurde auf physikalische Maschine migriert - Fehlermeldung am Client ** "Hyper-V IC Software Shadow Copy Provider ist noch installiert **
2019-11-06 06:31:52: ERROR: Not found
2019-11-06 06:31:56: ERROR: backupcom->AddToSnapshotSet(&(Server->ConvertToWchar(selected_vols[i])[0]), GUID_NULL, &additional_refs[i].volid) failed. VSS error code VSS_E_UNEXPECTED_PROVIDER_ERROR
2019-11-06 06:31:56: ERROR: VSS provider information:
2019-11-06 06:31:56: ERROR: Anbietername: "Hyper-V IC Software Shadow Copy Provider"
2019-11-06 06:31:56: ERROR: Anbietertyp: Software
2019-11-06 06:31:56: ERROR: Anbieterkennung: {74600e39-7dc5-4567-a03b-f091d6c7b092}
2019-11-06 06:31:56: ERROR: Version: 1.0.0.0
2019-11-06 06:31:56: ERROR: Anbietername: "Microsoft Software Shadow Copy provider 1.0"
2019-11-06 06:31:56: ERROR: Anbietertyp: System
2019-11-06 06:31:56: ERROR: Anbieterkennung: {b5946137-7b9f-4925-af80-51abd60b20d5}
2019-11-06 06:31:56: ERROR: Version: 1.0.0.7
2019-11-06 06:31:56: ERROR: Creating shadowcopy of "C:" failed.
2019-11-06 06:31:56: ERROR: Creating shadow copy failed. See client log file for details.
2019-11-06 07:12:01: ERROR: Not found
2019-11-06 07:12:03: ERROR: backupcom->AddToSnapshotSet(&(Server->ConvertToWchar(selected_vols[i])[0]), GUID_NULL, &additional_refs[i].volid) failed. VSS error code VSS_E_UNEXPECTED_PROVIDER_ERROR
2019-11-06 07:12:03: ERROR: VSS provider information:
2019-11-06 07:12:03: ERROR: Anbietername: "Hyper-V IC Software Shadow Copy Provider"
2019-11-06 07:12:03: ERROR: Anbietertyp: Software
2019-11-06 07:12:03: ERROR: Anbieterkennung: {74600e39-7dc5-4567-a03b-f091d6c7b092}
2019-11-06 07:12:03: ERROR: Version: 1.0.0.0
2019-11-06 07:12:03: ERROR: Anbietername: "Microsoft Software Shadow Copy provider 1.0"
2019-11-06 07:12:03: ERROR: Anbietertyp: System
2019-11-06 07:12:03: ERROR: Anbieterkennung: {b5946137-7b9f-4925-af80-51abd60b20d5}
2019-11-06 07:12:03: ERROR: Version: 1.0.0.7
2019-11-06 07:12:03: ERROR: Creating shadowcopy of "C:" failed.
2019-11-06 07:12:03: ERROR: Creating shadow copy failed. See client log file for details.
* **Lösung**: Provider löschen (https://social.technet.microsoft.com/Forums/ie/en-US/ba9e020e-bbed-446a-be37-40cb00cd908d/manually-uninstall-integration-services-without-hyperv?forum=winserverhyperv )
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VSS\Providers\{74600e39-7dc5-4567-a03b-f091d6c7b092}]
"vssadmin list providers" sollte nach einem Reboot den Hyper V Provider nicht mehr anzeigen
* Bei der ISO Datei für den **Restore wird Kernel 4.19 (Debian)** verwendet
* Eintrag für PXE BIOS Boot über TFTP (https://urbackup.atlassian.net/wiki/spaces/US/pages/1441795/Restoring+Images+from+PXE) - dann könnte auch anderer Kernel verwendet werden hinsichtlich **Treiberunterstützung**
LABEL UrBackup Restore
menu label Urbackup Restore Backup
kernel urbackup/live/vmlinuz
append initrd=urbackup/live/initrd.img boot=live config username=urbackup toram noswap fetch=tftp://IP_ADRESSE_TFTP_SERVER/urbackup/live/filesystem.squashfs
* **Migration** von Urbackup Server Installation **ohne Backupdaten** jedoch mit Einstellungen von altem System (2016 Standard) auf neues System (2019 Standard)
* Installation gleiche Serverversion wie auf "altem System"
* Deaktivieren und beenden von Urbackup Server Dienst auf altem und neuem System
* Kopieren von C:\Programme..\urbackup Server\*.* von altem System auf neues System (damit auch die Keys für die Authentifizierung bleiben, alle Dateien überschreiben auf neuem System)
* Verzeichnis / Laufwerk für Backup wie auf altem System auf neuem System erstellen
* Urbackup Server Dienst wieder starten -> sobald die Clients im Web Menü sichtbar werden - vollständige Abbildsicherung auslösen / IP vom alten System muss **nicht** übernommen werden
===== Altaro =====
* Altaro 8 - **nw.js** Screen / Windows 2019 Standard (https://help.altaro.com/support/solutions/articles/43000467363-getting-nw-js-on-startup-here-s-how-to-fix-it-)
* **Ausloggen** und neu einloggen / beim Check von **%Temp%** existiert das Verzeichnis nicht mehr
* Bei den Einstellungen VSS Copy beachten bei virtuellen Domain Controllern !
* Scheduled Test Restores einrichten und die Daten des Backups regelmäßig überprüfen lassen
* Notifications per E-Mail konfigurieren / für den Fall dass sie nicht funktioniert haben
====== HP Aruba ======
* Für Cronjob siehe "HP Pro Curve"
* SSH Server Keys müssen bereits "accepted" worden sein / sonst hängt er bei der Accept Key Frage
* **backup-aruba.expect**
#!/usr/bin/expect -f
set timeout 60
log_user 0
spawn ssh USERNAME@[lindex $argv 0]
expect "password:"
send "PASSWORD\r"
#2021-02-08 cc: think wisely if you want to send password as argument to call
#send "[lindex $argv 1]\r"
expect "continue"
send "\r"
expect "#"
send "terminal length 1000\r"
expect "#"
log_user 1
send "show config\r"
expect "#"
send "exit\r"
expect ">"
send "exit\r"
expect "?"
send "y\r"
expect eof
====== HPE Comware ======
* Für Cronjob siehe "HP Pro Curve"
* SSH Server Keys müssen bereits "accepted" worden sein / sonst hängt er bei der Accept Key Frage
* **backup-hp-comware-switches.expect**
#!/usr/bin/expect -f
set timeout 60
spawn ssh admin@[lindex $argv 0]
expect "password:"
send "PASSWORD\r"
#2021-02-08 cc: think wisely if you want to send password as argument to call
#send "[lindex $argv 1]\r"
expect ">"
send "xtd-cli-mode\r"
expect "]:"
send "Y\r"
expect "Password:"
send "foes-bent-pile-atom-ship\r"
expect ">"
send "screen-length disable\r"
expect ">"
send "show current-configuration\r"
expect ">"
send "quit\r"
====== HP Pro Curve ======
* **Achtung** Sicherheitsimplikationen bei **telnet** beachten (Übertragung des passworts über plaintext)
* **/etc/cron.d/backup-switches**
# Every day @02:00 a.m.
SHELL=/bin/bash
00 2 * * * root switches="IP_SWITCH_1 IP_SWITCH_2 IP_SWITCH_3 IP_SWITCH_4"; for i in $switches ; do /usr/local/bin/get-switch-config $i > /var/backups/switches-configs/$i.conf ; done
* Dependency: expect
* **/usr/local/bin/get-switch-config**
#!/usr/bin/expect -f
# To avoid empty lines, 'nonewline' flag is used
set ip_address [lindex $argv 0];
set prompt "#"
set username "USERNAME\r"
set password "PASSWORD\r"
log_user 0
spawn telnet $ip_address
expect "Username:"
send $username
expect "Password:"
send $password
expect $prompt
send "terminal length 1000\r"
expect $prompt
set timeout 120
log_user 1
send "show config\r"
expect $prompt
set timeout 20
send "exit\r"
expect ">"
send "exit\r"
expect "?"
send "y\r"
expect eof
====== Linux ======
===== rsnapshot =====
* Erstellung von Prüfsummen zu den Dateien die gebackuped wurden
* rsnapshot.conf - Beispiele für die Verwendung von **integ.sh**
...
cmd_postexec /usr/local/sbin/integ.sh /mnt/external-backup/ebackup1/weekly.0 c ; /bin/mount -o remount,ro /mnt/external-backup
...
* rsnapshot.conf - Beispiele für die Verwendung von **backup-lvm-drive.sh**
...
backup_script /usr/local/sbin/backup-lvm-drive.sh /dev/vgroup/lvmname imagename.img.lzo ./pfad/
...
* **backup-lvm-drive.sh**
* Kann als **backup_script** ausgeführt werden zB: wenn auf einem Hostsystem virtualisiert über kvm eine "docker" blackbox läuft und lvm
#!/bin/bash
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
function bailout ()
{
echo -e "$1"
exit 2
}
function usage ()
{
echo -e "$1"
exit 1
}
# LVM PATH / generate LVM snapshot name suffix snapshot / compress devices / output PATH
ARGUMENTS=$#
SIZE_SNAPSHOT="20G"
BUFFER="10M"
which lvcreate > /dev/null || bailout "lvcreate not found"
which lvremove > /dev/null || bailout "lvremove not found"
which lzop > /dev/null || bailout "lzop not found - apt-get install lzop"
[ $ARGUMENTS != 2 ] && bailout "Usage: $0 PATH_TO_LVM_VOLUME BACKUP_FILE"
LVM_PATH="$1"
BACKUP_FILE="$2"
[ ! -r $LVM_PATH ] && bailout "FAIL: Cannot read PATH_TO_LVM VOLUME"
[ -r $BACKUP_FILE ] && bailout "FAIL: $BACKUP_FILE is already here aborting"
LVM_NAME=$(echo ${LVM_PATH##*/})
LVM_NAME_SNAPSHOT="$LVM_NAME""_snapshot"
LVM_PATH_SNAPSHOT="${LVM_PATH%%$LVM_NAME}$LVM_NAME_SNAPSHOT"
[ -r $LVM_PATH_SNAPSHOT ] && bailout "FAIL: Snapshot: $LVM_PATH_SNAPSHOT already here aborting"
lvcreate -L $SIZE_SNAPSHOT -s -n $LVM_NAME_SNAPSHOT $LVM_PATH > /dev/null || bailout "FAIL: Could not create Snapshot: $LVM_NAME_SNAPSHOT"
if [ $BACKUP_FILE == '-' ]
then
dd if=$LVM_PATH_SNAPSHOT bs=$BUFFER 2>/dev/null | lzop -
else
dd if=$LVM_PATH_SNAPSHOT bs=$BUFFER 2>/dev/null | lzop > $BACKUP_FILE
fi
lvremove -q -f $LVM_PATH_SNAPSHOT > /dev/null
exit 0
* **backup-lvm-files.sh**
* Um einen LVM Snapshot read only zu mounten und dann per RSYNC mit einem Verzeichnis zu syncen / kann mit rsnapshot backup_exec kombiniert werden
#!/bin/bash
function bailout ()
{
echo -e "$1"
[ -z "$2" ] || lvremove -q -f "$2"
exit 2
}
function usage ()
{
echo -e "$1"
exit 1
}
RSYNC_DEFAULT_OPTIONS="--dry-run -a --delete --numeric-ids "
RSYNC_DEFAULT_OPTIONS_ARRAY=($RSYNC_DEFAULT_OPTIONS)
ARGUMENTS=$#
SIZE_SNAPSHOT="10G"
BUFFER="10M"
TMP_MOUNT_POINT="/mnt/tmp"
which lvcreate > /dev/null || bailout "lvcreate not found"
which lvremove > /dev/null || bailout "lvremove not found"
which rsync > /dev/null || bailout "rsync not found"
which kpartx > /dev/null || bailout "kpartx not found"
which mount > /dev/null || bailout "mount not found"
which mountpoint > /dev/null || bailout "mountpoint not found"
if [ $ARGUMENTS != 2 ] && [ $ARGUMENTS != 3 ]
then
bailout "Usage: $0 PATH_TO_LVM_VOLUME BACKUP_DIRECTORY [ \""RSYNC_OPTIONS\"" ]"
fi
LVM_PATH="$1"
BACKUP_DIRECTORY="$2"
RSYNC_CUSTOM_OPTIONS="$3"
[ -n "$3" ] && RSYNC_DEFAULT_OPTIONS_ARRAY+=($RSYNC_CUSTOM_OPTIONS)
[ ! -r $LVM_PATH ] && bailout "FAIL: Cannot read $LVM_PATH VOLUME"
[ ! -d $BACKUP_DIRECTORY ] && bailout "FAIL: $BACKUP_DIRECTORY is already here aborting"
[ ! -d $TMP_MOUNT_POINT ] && bailout "FAIL: $TMP_MOUNT_POINT directory not found"
mountpoint -q $TMP_MOUNT_POINT && bailout "FAIL: $TMP_MOUNT_POINT is already mounted FAIL"
LVM_NAME=$(echo ${LVM_PATH##*/})
LVM_NAME_SNAPSHOT="$LVM_NAME""_snapshot"
LVM_PATH_SNAPSHOT="${LVM_PATH%%$LVM_NAME}$LVM_NAME_SNAPSHOT"
[ -r $LVM_PATH_SNAPSHOT ] && bailout "FAIL: Snapshot: $LVM_PATH_SNAPSHOT already here aborting"
lvcreate -L $SIZE_SNAPSHOT -s -n $LVM_NAME_SNAPSHOT $LVM_PATH > /dev/null || bailout "FAIL: Could not create Snapshot: $LVM_NAME_SNAPSHOT"
[ $? == "0" ] || bailout "FAIL: $LVM_PATH_SNAPSHOT could not be created"
MOUNT_PATH="$LVM_PATH_SNAPSHOT"
DEVICE_MAPPER_ENTRY=$(kpartx -av $LVM_PATH_SNAPSHOT)
RETURN_SUCCESS=$?
[ $? != "0" ] && bailout "FAIL: Kpartx did not return 0 - error" "$LVM_PATH_SNAPSHOT"
DEVICE_MAPPER_ENTRY=$(echo $DEVICE_MAPPER_ENTRY | cut -f 3 -d" ")
[ -z $DEVICE_MAPPER_ENTRY ] || MOUNT_PATH="/dev/mapper/""$DEVICE_MAPPER_ENTRY"
mount -o ro "$MOUNT_PATH" "$TMP_MOUNT_POINT" || bailout "FAIL: could not mount $MOUNT_PATH to $TMP_MOUNT_POINT" "$LVM_PATH_SNAPSHOT"
rsync "${RSYNC_DEFAULT_OPTIONS_ARRAY[@]}" "$TMP_MOUNT_POINT/" "$BACKUP_DIRECTORY/"
RETURN_RSYNC=$?
umount $TMP_MOUNT_POINT
kpartx -dv $LVM_PATH_SNAPSHOT > /dev/null
lvremove -q -f $LVM_PATH_SNAPSHOT > /dev/null
exit $RETURN_RSYNC
* **integ.sh**
* Um nach dem Backup Prüfsummen aller Dateien zu erstellen
#!/bin/bash
MACHINE="MEIN-SERVER-NAME"
function usage {
echo "Usage: $0 directory_root [c|v]"
echo "c...create hashes"
echo "v...veriy hashes"
exit 0
}
echo $MACHINE
function verifyDirectoryHashes {
echo "verify"
cd $DIRECTORY_ROOT
[[ ! -f hashes.sha1 ]] && echo "Hashes: $DIRECTORY_ROOT/hashes.sha1 not found" && exit 2
date1=$(date -u +"%s")
sha1sum --quiet -c hashes.sha1
retCode=$?
date2=$(date -u +"%s")
diff=$(($date2-$date1))
amount=$(wc -l hashes.sha1 | cut -d " " -f 1)
echo "$(($diff / 60)) minutes and $(($diff % 60)) seconds elapsed."
echo "Hashes verified: $amount"
echo "done"
exit $retCode
}
function createDirectoryHashes {
echo "create hashes"
cd $DIRECTORY_ROOT
echo -n > hashes.sha1
date1=$(date -u +"%s")
find ./ ! -name "*.sha1" -type f -exec sha1sum {} >> hashes.sha1 \;
date2=$(date -u +"%s")
diff=$(($date2-$date1))
amount=$(wc -l hashes.sha1 | cut -d " " -f 1)
echo "$(($diff / 60)) minutes and $(($diff % 60)) seconds elapsed."
echo "Hashes created: $amount"
echo "done"
exit 0
}
#Check parameters
DIRECTORY_ROOT="$1"
ACTION="$2"
[[ ! -d $DIRECTORY_ROOT ]] && echo "Cannot read directory: $DIRECTORY_ROOT" && usage
[[ $ACTION != "c" && $ACTION != "v" ]] && echo "Either verify or create" && usage
[[ $ACTION == "c" ]] && createDirectoryHashes
[[ $ACTION == "v" ]] && verifyDirectoryHashes
* rsnapshot mit **btrfs**
rsnapshot.conf
...
cmd_cp /usr/local/sbin/rsnapshot_btrfs_cp.sh
# uncomment this to use the rm program instead of the built-in perl routine.
#
cmd_rm /usr/local/sbin/rsnapshot_btrfs_rm.sh
...
* **rsnapshot_btrfs_cp.sh**
#!/bin/bash
# Arg 1: -al
# Arg 2: /path/daily.0
# Arg 3: /path/daily.1
btrfs subvolume snapshot -r $2 $3 >/dev/null
* **rsnapshot_btrfs_rm.sh**
#!/bin/bash
# Arg 1: -rf
# Arg 2: /path/daily.5/
# echo 1: $1 2: $@
# Try to delete the given path with btrfs subvolume delete first
# if this fails fall back to normal rm
if [ "$1" = "-rf" -a "$3" = "" ]; then
# "trying to delete with btrfs"
btrfs subvolume show "$2" &>/dev/null && btrfs subvolume delete "$2" &>/dev/null && exit 0
[[ -d "$2" ]] && rm -rf "$2" && exit 0
exit 2
fi
exit 2
* rsnapshot mit **borg** **cron job**
* In der Konfiguration der rsnapshot Backups existiert nur mehr daily.0 d.h. **retain daily 1** und beide Backup Jobs (customers und pannoniait) müssen bereits **abgeschlossen** sein
30 06 * * 1-5 root ( [[ ! -f /var/run/rsnapshot-customers.pid ]] && [[ ! -f /var/run/rsnapshot-pannoniait.pid ]] && borg create -x --numeric-owner --compression lz4 /mnt/storage/backups/borg/::d.$(date +"\%F") /mnt/storage/backups/customers/daily.0/ /mnt/storage/backups/pannoniait/daily.0/ && borg prune -P d -d 5 /mnt/storage/backups/borg ) || echo "Could not perform daily borg backup"
30 06 * * 6 root ( [[ ! -f /var/run/rsnapshot-customers.pid ]] && [[ ! -f /var/run/rsnapshot-pannoniait.pid ]] && borg create -x --numeric-owner --compression lz4 /mnt/storage/backups/borg/::w.$(date +"\%F") /mnt/storage/backups/customers/daily.0/ /mnt/storage/backups/pannoniait/daily.0/ && borg prune -P w -w 4 /mnt/storage/backups/borg ) || echo "Could not perform weekly borg backup"
30 07 1 * * root ( [[ ! -f /var/run/rsnapshot-customers.pid ]] && [[ ! -f /var/run/rsnapshot-pannoniait.pid ]] && borg create -x --numeric-owner --compression lz4 /mnt/storage/backups/borg/::m.$(date +"\%F") /mnt/storage/backups/customers/daily.0/ /mnt/storage/backups/pannoniait/daily.0/ && borg prune -P m -m 2 /mnt/stroage/backups/borg ) || echo "Could not perform monthly borg backup
===== borg - generell =====
* Backup Tests mit borg - Achtung **unverschlüsselt** da Testgerät ohnehin **vollverschlüsselt** ist
* Ganze LVM Devices basierend auf Snapshot backupppen - sollte konsistent sein **backup-lvm-drive-borg.sh**
#!/bin/bash
function bailout ()
{
echo -e "$1" 1>&2
exit 2
}
function usage ()
{
echo -e "$1"
exit 1
}
# LVM PATH / generate LVM snapshot name suffix snapshot / compress devices / output PATH
ARGUMENTS=$#
SIZE_SNAPSHOT="50G"
BORG_NR_TO_KEEP="4"
which lvcreate > /dev/null || bailout "lvcreate not found"
which borg > /dev/null || bailout "borg cannot be found"
which lvremove > /dev/null || bailout "lvremove not found"
[ $ARGUMENTS -lt 2 ] && bailout "Usage: $0 PATH_TO_BORG_REPOSITORY PATH_TO_LVM_VOLUME1 PATH_TO_LVM_VOLUME2 "
BACKUP_REPOSITORY=$(echo $1 | cut -d: -f 1)
[ -r $BACKUP_REPOSITORY ] || bailout "FAIL: $BACKUP_REPOSITORY cannot be read"
[ -w $BACKUP_REPOSITORY ] || bailout "FAIL: $BACKUP_REPOSITORY cannot be written to"
for i in "${@:2}" ; do
LVM_PATH="$i"
LVM_PATH_SNAPSHOT="$LVM_PATH""_snapshot"
[ ! -r $LVM_PATH ] && bailout "FAIL: Cannot read \"$LVM_PATH\" VOLUME"
[ -r $LVM_PATH_SNAPSHOT ] && bailout "FAIL: Attention snapshot still there: $LVM_PATH_SNAPSHOT"
done
lvm_snapshots=""
for i in "${@:2}" ; do
LVM_PATH="$i"
LVM_PATH_SNAPSHOT="$LVM_PATH""_snapshot"
lvcreate -L $SIZE_SNAPSHOT -s -n $LVM_PATH_SNAPSHOT $LVM_PATH > /dev/null || bailout "FAIL: Could not create Snapshot: $LVM_NAME_SNAPSHOT"
lvm_snapshots="$lvm_snapshots $LVM_PATH_SNAPSHOT"
done
borg create --stats --compression=lz4 --read-special "$1" $lvm_snapshots
RET_CODE_BORG="$?"
for i in "${@:2}" ; do
LVM_PATH="$i"
LVM_PATH_SNAPSHOT="$LVM_PATH""_snapshot"
lvremove -q -f $LVM_PATH_SNAPSHOT >/dev/null
done
borg prune --keep-last $BORG_NR_TO_KEEP $BACKUP_REPOSITORY
exit $RET_CODE_BORG
* Durchgeführt auf Kali Linux - Debian Testing
Borg Befehle:
1) Repository initialisieren d.h. Verzeichnis initialisieren wo das Backup landen soll
e.g.
Encryption modes
Hash/MAC Not encrypted no auth Not encrypted, but authenticated Encrypted (AEAD w/ AES) and authenticated
SHA-256 none authenticated repokey keyfile
BLAKE2b n/a authenticated-blake2 repokey-blake2 keyfile-blake2
( encryption -> 'none', 'keyfile', 'repokey', 'authenticated', 'keyfile-blake2', 'repokey-blake2', 'authenticated-blake2' )
root@mrChief:/home# borg init --encryption none /mnt/backup/mrChief/
root@mrChief:/home# echo $?
0
2) Backup erstellen
Exclude mir alle ISO Dateien unter /home/urnilxfgbez/
Nicht über das Dateisystem rausschießen deshalb / und /boot einzeln
Nummerische Ids der Ownership speichern
borg create --stats --progress --one-file-system --numeric-owner --exclude /swap.img --exclude '/home/*/*.iso' --exclude '/home/*/*.ISO' --compression lz4 /mnt/backup/mrChief/::$(date +"%T.%F") / /boot
3) Verfügbare Backups anzeigen (borg list)
root@mrChief:/home# borg list /mnt/backup/mrChief/
12:13:01.2019-10-16 Wed, 2019-10-16 12:13:02 [6e550028349bdde6f22ab513a83169ebf6ef87026af80e5598f7e9c82dff4229]
4) Verfügbares Backup mounten und zugänglich machen
root@mrChief:/mnt# borg mount /mnt/backup/mrChief::12:13:01.2019-10-16 /mnt/tmp
root@mrChief:/mnt# borg umount /mnt/tmp
root@mrChief:/mnt# ls -al /mnt/tmp
total 8
drwxr-xr-x 2 root root 4096 Jan 23 2019 .
drwxr-xr-x 14 root root 4096 Oct 16 09:47 ..
5) Backup Policy definieren f. Aufbewahrungsfristen (borg prune)
Letzte 4 Versionen behalten
root@mrChief:/mnt# borg prune --keep-last 4 /mnt/backup/mrChief/
===== borg - hetzner storage =====
1. key verteilen
https://wiki.hetzner.de/index.php/Backup_Space_SSH_Keys
root@mrStorage:~# echo -e "mkdir .ssh \n chmod 700 .ssh \n put /root/.ssh/id_rsa.pub .ssh/authorized_keys \n chmod 600 .ssh/authorized_keys" | sftp u12345678@u12345678.your-storagebox.de
The authenticity of host 'u12345678.your-storagebox.de (176.9.161.59)' can't be established.
RSA key fingerprint is SHA256:EMlfI8GsRIfpVkoW1H2u0zYVpFGKkIMKHFZIRkf2ioI.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'u12345678.your-storagebox.de,176.9.161.59' (RSA) to the list of known hosts.
u12345678@u12345678.your-storagebox.de's password:
Connected to u12345678@u12345678.your-storagebox.de.
sftp> mkdir .ssh
sftp> chmod 700 .ssh
Changing mode on /.ssh
sftp> put /root/.ssh/id_rsa.pub .ssh/authorized_keys
Uploading /root/.ssh/id_rsa.pub to /.ssh/authorized_keys
/root/.ssh/id_rsa.pub 100% 740 27.0KB/s 00:00
sftp> chmod 600 .ssh/authorized_keys
Changing mode on /.ssh/authorized_keys
2. Repo initialisieren
borg init --encryption=keyfile ssh://u12345678@u12345678.your-storagebox.de:23/./borg/
Keymaterial befindet sich ausschließlich auf "mrStorage" dem Server der das Backup in die Hetzner CLoud durchführt
3. Backup Keys exportieren - es wird Keyfile und Passphrase benötigt - backup des keyfiles
borg key export ssh://u12345678@u12345678.your-storagebox.de:23/./borg/ /tmp/borg_keyfile
3. Common Commands
root@mrStorage:~# borg create --compression lz4 --list --dry-run ssh://u12345678@u12345678.your-storagebox.de:23/./borg/::{now} /root/fw.log.2.gz
- /root/fw.log.2.gz
root@mrStorage:~# borg create --compression lz4 --list ssh://u12345678@u12345678.your-storagebox.de:23/./borg/::{now} /root/fw.log.2.gz
Enter passphrase for key /root/.config/borg/keys/u12345678_your_storagebox_de__borg:
A /root/fw.log.2.gz
root@mrStorage:~# borg create --compression lz4 --list --dry-run ssh://u12345678@u12345678.your-storagebox.de:23/./borg/::{now} /root/fw.log.2.gz
- /root/fw.log.2.gz
root@mrStorage:~# borg info ssh://u12345678@u12345678.your-storagebox.de:23/./borg/
Enter passphrase for key /root/.config/borg/keys/u12345678_your_storagebox_de__borg:
Location: ssh://u12345678@u12345678.your-storagebox.de:23/./borg
Encrypted: Yes (key file)
Key file: /root/.config/borg/keys/u12345678_your_storagebox_de__borg
------------------------------------------------------------------------------
Original size Compressed size Deduplicated size
All archives: 4.48 MB 4.50 MB 4.50 MB
Unique chunks Total chunks
Chunk index: 5 5
root@mrStorage:~# borg list ssh://u12345678@u12345678.your-storagebox.de:23/./borg/
Enter passphrase for key /root/.config/borg/keys/u12345678_your_storagebox_de__borg:
2020-03-05T23:35:02 Thu, 2020-03-05 23:35:11 [0fe96fe6a40b0eb5db5955affad70a570e3a37cb3d3dd9f6e9a3ba0a538eafc2]
root@mrStorage:~# borg mount ssh://u12345678@u12345678.your-storagebox.de:23/./borg/ /mnt/tmp
Enter passphrase for key /root/.config/borg/keys/u12345678_your_storagebox_de__borg:
* **/usr/local/bin/hetzner_borg_backup.sh**
#!/bin/bash
export BORG_PASSPHRASE="PASSPHRASE"
REPOSITORY="ssh://u12345678@u12345678.your-storagebox.de:23/./borg/"
DIRECTORIES="/mnt/storage/"
HOST=$(hostname --fqdn)
START_TIME=$(date +%Y.%m.%d-%H.%M.%S)
date1=$(date -u +"%s")
echo -e "Program: $0 \nstarted at: $START_TIME\nBacking Up: $DIRECTORIES\nHost: $HOST"
#2020-04-08 cc: Achtung sonst konsumiert er den gesamten Upload und die Leitung steht daher: Rate limit ~6 Mbit upload / bei 10Mbit verfügbarem Upload
borg create --compression lz4 --stats --remote-ratelimit 700 --exclude '*/.snapshots/*' $REPOSITORY::{now} $DIRECTORIES
[[ $? == "0" ]] && borg prune --stats --keep-last 6 $REPOSITORY
END_TIME=$(date +%Y.%m.%d-%H.%M.%S)
date2=$(date -u +"%s")
diff=$(($date2-$date1))
echo "$(($diff / 60)) minutes and $(($diff % 60)) seconds elapsed."
echo "$(($diff / 86400 )) days elapsed. "
echo -e "Program ended successfully : $0 \nended at: $END_TIME\n"
===== triggered =====
* Anforderungen:
* Es sollen Backups durchgeführt werden "Event" basierend d.h. wenn eine bestimmte Festplatte angesteckt wird
* Getestet auf Debian Buster
* Die Seagate Platte wird sporadisch angesteckt (siehe lsusb - ID) :
root@mrGodfather:/mnt/ebackup/ebackup1# lsusb
....
....
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 008: ID 0bc2:61b7 Seagate RSS LLC Maxtor M3 Portable
.....
* Udev Trigger für diese spezielle Festplatte
root@mrGodfather:/mnt/ebackup/ebackup1# cat /etc/udev/rules.d/30-ebackup.rules
ACTION=="add",KERNEL=="sd?[0-9]",ATTRS{idVendor}=="0bc2", ATTRS{idProduct}=="61b7", RUN+="/usr/local/sbin/backup_wrapper_systemd_ebackup1.sh"
* Workaround damit "Backup" Skript systemd entzogen wird (dort gibts ein definiertes timeout für Skript) und at übergeben
root@mrGodfather:/mnt/ebackup/ebackup1# cat /usr/local/sbin/backup_wrapper_systemd_ebackup1.sh
#!/bin/bash
echo "/usr/local/sbin/backup_ebackup1.sh" | at now
exit 0
* Backup Skript , das crypsetup Device erstellt und Backups durchführt (Achtung btrfs Dateisystem)
root@mrGodfather:/mnt/ebackup/ebackup1# cat /usr/local/sbin/backup_ebackup1.sh
#!/bin/bash
#1st step check if device is here
#2nd step check if already mounted
#3rd step mount using cryptsetup key
#4th step remember time backup of all hosts using rsync
#5th step take snapshot
#6th sync data umount drive
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
KEY_LOCATION="LOCATION_TO_KEYFILE"
MAPPER_NAME="ebackup1"
SUBVOLUME_NAME="ebackup1"
MOUNT_POINT="/mnt/ebackup"
HARDDISK_ID="/dev/disk/by-id/usb-Seagate_M3_Portable_NA844AER-0:0-part1"
MAIL_NOTIFICATION="EMAIL_ADDRESS"
RSYNC_STANDARD="rsync -a --timeout=300 --acls --delete --numeric-ids --exclude mnt/backup/ --exclude proc/ --exclude sys/ --exclude dev/ --exclude backup/ --exclude mnt/storage/backups/pannoniait/ --exclude mnt/storage/backups/customers/ --exclude .snapshots/ "
MACHINES="server1 server2 server3 server4"
BACKUP_ID=$(date +%s)
START_TIME=$(date +%Y.%m.%d-%H.%M.%S)
function bailout ()
{
echo -e "$1"
exit 2
}
function notify ()
{
echo -e "$1"
}
notify "Program: $0 \nstarted at: $START_TIME\nMachines to backup: $MACHINES mrGodfather\nDestination Hardware: $HARDDISK_ID\nBackup ID: $BACKUP_ID\nDestination MountPoint: $MOUNT_POINT\nKeyfile: $KEY_LOCATION"
which cryptsetup > /dev/null || bailout "FAIL: cryptsetup not found!"
which mail > /dev/null || ( echo "FAIL: mail not found!" && exit 2 )
which rsync > /dev/null || bailout "FAIL: rsync not found!"
which btrfs > /dev/null || bailout "FAIL: btrfs not found!"
[ ! -r $HARDDISK_ID ] && bailout "FAIL: Physical Harddisk: $HARDDISK_ID not found\nBackup ID: $BACKUP_ID"
[ ! -r $KEY_LOCATION ] && bailout "FAIL: Key not found: $KEY_LOCATION\nBackup ID: $BACKUP_ID"
[ ! -d $MOUNT_POINT ] && bailout "FAIL: Mount point not found: $MOUNT_POINT\nBackup ID: $BACKUP_ID"
[ -r "/dev/mapper/$MAPPER_NAME" ] && bailout "FAIL: Crypto Mapper: $MAPPER_NAME found close manualy\nBackup ID: $BACKUP_ID"
mountpoint -q $MOUNT_POINT && bailout "FAIL: Already mounted: $MOUNT_POINT\nBackup ID: $BACKUP_ID"
cryptsetup luksOpen $HARDDISK_ID $MAPPER_NAME --key-file $KEY_LOCATION || bailout "FAIL: Could not open cryptsetup device $HARDDISK_ID with $KEY_LOCATION\nBackup ID: $BACKUP_ID"
mount -o rw,acl,noexec /dev/mapper/$MAPPER_NAME $MOUNT_POINT || bailout "FAIL: could not mount: /dev/mapper/$MAPPER_NAME on $MOUNT_POINT\nBackup ID: $BACKUP_ID"
[ ! -d "$MOUNT_POINT/$SUBVOLUME_NAME" ] && bailout "FAIL: Directory not found on Mountpoint: $MOUNT_POINT/$SUBVOLUME_NAME\nBackup ID: $BACKUP_ID"
for machine in $MACHINES
do
$RSYNC_STANDARD root@$machine:/ $MOUNT_POINT/$SUBVOLUME_NAME/$machine/
done
btrfs subvolume snapshot -r $MOUNT_POINT/$SUBVOLUME_NAME $MOUNT_POINT/$SUBVOLUME_NAME/.snapshots/@GMT_$START_TIME
END_TIME=$(date +%Y.%m.%d-%H.%M.%S)
btrfs filesystem sync $MOUNT_POINT || bailout "FAIL: could not sync btrfs filesystem\nBackup ID: $BACKUP_ID"
/usr/local/sbin/clearLastSnapshot.sh $MOUNT_POINT ebackup1 5 || bailout "FAIL: could not clearLastSnapshot\nCheck Manualyy!\mBackup ID: $BACKUP_ID"
sync
sleep 60
umount $MOUNT_POINT || bailout "FAIL: Could not unmount filesystem\nBackup ID: $BACKUP_ID"
cryptsetup luksClose $MAPPER_NAME || bailout "FAIL: Could not close cryptsetup Handle $MAPPER_NAME\nBackup ID: $BACKUP_ID"
notify "Program ended successfully : $0 \nended at: $END_TIME\nBackup ID: $BACKUP_ID"
exit 0
* Skript um nur eine bestimmte Anzahl an Snapshots auf dem Device zu halten
root@mrGodfather:/mnt/ebackup/ebackup1# cat /usr/local/sbin/clearLastSnapshot.sh
#!/bin/bash
function usage
{
echo "Usage Keep this Nr of Snapshots: $0 LocalMountPoint LocalSubvolumeName DesiredSnapshotCount"
echo "Usage Show Nr of Snapshots: $0 LocalMountPoint LocalSubvolumeName"
echo "Usage: e.g. $0 /mnt/storage daten 3"
exit 1
}
LOCAL_MOUNT_POINT=$1
LOCAL_SUBVOLUME=$2
DESIRED_SNAPSHOTS=$3
[[ $# != 3 && $# != 2 ]] && usage
[[ ! -d $LOCAL_MOUNT_POINT ]] && echo "Couldn't validate local btrfs subvolume mountpoint: $LOCAL_MOUNT_POINT" && exit 2
CURRENT_NR_SNAPSHOTS=$(btrfs subvolume list $LOCAL_MOUNT_POINT/$LOCAL_SUBVOLUME/.snapshots -r -o --sort=+gen | wc -l )
[[ "$CURRENT_NR_SNAPSHOTS" == 0 ]] && echo "Couldn't aquire number of snapshots from $LOCAL_MOUNT_POINT/$LOCAL_SUBVOLUME/.snapshots" && exit 2
[[ $# == 2 ]] && echo -e "Mount Point: $LOCAL_MOUNT_POINT\nSubvolume: $LOCAL_SUBVOLUME\nCurrent Snapshots: $CURRENT_NR_SNAPSHOTS" && exit 0
REGEX_NUMBER='^[0-9]+$'
[[ ! $DESIRED_SNAPSHOTS =~ $REGEX_NUMBER ]] && echo "That's not a valid number: $NR_SNAPSHOTS" && exit 2
[[ $(($CURRENT_NR_SNAPSHOTS-$DESIRED_SNAPSHOTS)) -le 0 ]] && echo -e "Deletion not needed\nMount Point: $LOCAL_MOUNT_POINT\nSubvolume: $LOCAL_SUBVOLUME\nCurrent Snapshots: $CURRENT_NR_SNAPSHOTS\nDesired: $DESIRED_SNAPSHOTS" && exit 0
NR_SNAPSHOTS_REMOVE=$(($CURRENT_NR_SNAPSHOTS-$DESIRED_SNAPSHOTS))
CURRENT_SNAPSHOTS=$(btrfs subvolume list $LOCAL_MOUNT_POINT/$LOCAL_SUBVOLUME/.snapshots -r -o --sort=+gen | head -n $NR_SNAPSHOTS_REMOVE | cut -d' ' -f 9 )
for snap in $CURRENT_SNAPSHOTS
do
btrfs subvolume delete $LOCAL_MOUNT_POINT/$snap
done
btrfs filesystem sync $LOCAL_MOUNT_POINT
===== etc =====
* rsync für blockdevices Kopieren "--copy-devices"
* **Update**
rsync (3.2.0-1) unstable; urgency=low
This latest release changed two parameters which used to be present on the
Debian packaging of rsync as upstream now integrated the patches.
Previous parameter:
--copy-devices: write to devices as files (implies --inplace)
Is now called: --write-devices
root@mrChief:/home/urnilxfgbez# rsync -v --progress --block-size=131072 --no-whole-file --checksum --copy-devices /dev/mmcblk0 /tmp/foo
* Interessantes zu der Option und beim Kopieren von/auf lokalem Dateisystem: https://superuser.com/questions/234273/why-doest-rsync-use-delta-transfer-for-local-files
* Er scheint zuerst die Hashes vom Ziel zu berechnen , danach von der Quelle (debugged mit iostat auf dem jeweiligen Device)
root@mrChief:/home/urnilxfgbez# rsync -vv --no-whole-file --copy-devices /dev/mmcblk0 /tmp/foo
delta-transmission enabled
mmcblk0
total: matches=126225 hash_hits=126225 false_alarms=0 data=0
sent 505,008 bytes received 1,009,926 bytes 5,439.62 bytes/sec
total size is 0 speedup is 0.00
root@mrChief:/home/urnilxfgbez# rsync -vv --no-whole-file --copy-devices root@localhost:/dev/mmcblk0 /tmp/foo
opening connection using: ssh -l root localhost rsync --server --sender -vve.LsfxC --copy-devices . /dev/mmcblk0 (11 args)
delta-transmission enabled
mmcblk0
total: matches=126225 hash_hits=126225 false_alarms=0 data=0
sent 1,009,903 bytes received 505,075 bytes 5,439.78 bytes/sec
* rsync - Ausgabe ob etwas geändert werden würde - dry run und count
* **rsync -iaun --delete foo1/ foo2/ | wc -l**
0