Enhancement #2043
Backup: implement and document full restore
| Status: | CLOSED | Start date: | ||
|---|---|---|---|---|
| Priority: | Normal | Due date: | ||
| Assignee: | - | % Done: | 100% | |
| Category: | nethserver-backup-data | |||
| Target version: | v6.4-beta2 | |||
| Resolution: | NEEDINFO: | No |
Description
Implement and document a full restore for a disaster recovery scenario.
The restore should:- restore all data
- restore all configuration
- restore users from LDAP dump
- restore network configuration
- restore custom SSL certificates
Related issues
Associated revisions
restore-data-duplicity: search for a directory as restore point. Refs #2043
restore-config-duplicity: restore config file from data backup. Refs #2043
backup: save and restore maildir structure along with sieve filters. Refs #2043
Implement full restore scenario. Refs #2043
mysql-load-tables: do not empty mysql client configuration. Refs #2043
backup: implement backup and restore for ldap. Refs #2043
nethserver-mysql-conf: correctly start mysqld on first install. Refs #2043
post-restore-config event: reset network. Refs #2043
backup: save and restore ibay structure. Refs #2043
History
#1
Updated by Giacomo Sanchietti about 8 years ago
- Description updated (diff)
#2
Updated by Giacomo Sanchietti about 8 years ago
- Subject changed from Backup data: implement and document full restore to Backup: implement and document full restore
- Status changed from NEW to TRIAGED
- Assignee set to Giacomo Sanchietti
- % Done changed from 0 to 20
- The system is restored in two phases: first configuration, then all data
- restore-config will search for an existing configuration backup file, if not present the configuration backup can be retrieved from data backup
- configuration backup will also contains:
- shadow, passwd and group
- all maildir directories
- sieve filters
- first directory level of ibay
- ssl certificates
- At the end of restore-config all update will be launched and the network will be reconfigured
The restore can be performed before or after addon installation.
#3
Updated by Giacomo Sanchietti about 8 years ago
- Status changed from TRIAGED to ON_DEV
- % Done changed from 20 to 30
#4
Updated by Giacomo Sanchietti about 8 years ago
- Status changed from ON_DEV to MODIFIED
- % Done changed from 30 to 70
Implemented all wished features.
#5
Updated by Giacomo Sanchietti about 8 years ago
- Status changed from MODIFIED to ON_QA
- % Done changed from 70 to 80
- nethserver-directory-1.2.2-1
- nethserver-mail-server-1.4.3-1
- nethserver-base-1.2.4-1
- nethserver-backup-config-1.0.3-1
- nethserver-backup-data-1.0.4-1
- nethserver-mysql-1.0.3-1
Prepare system for test case
- Update the system from nethserver-testing
- Install nethserver-directory nethserver-mail-server nethserver-mysql
- Create users, groups, databases. Send some mail to users and groups.
- Configure a backup
- Execute the backup with:
backup-config && backup-data
- Install a new machine with the same hostname as the old one
- Configure a backup
- Restore configuration backup executing:
restore-config - Execute
bootstrap-console - Restore data backup executing:
restore-data - Install nethserver-directory nethserver-mail-server nethserver-mysql from nethserver-testing repository
- Check if mails, databases and users are present
- Install a new machine with the same hostname as the old one
- Configure a backup
- Install nethserver-directory nethserver-mail-server nethserver-mysql from nethserver-testing repository
- Restore configuration backup executing:
restore-config - Execute
bootstrap-console - Check imap server is working and users can authenticate themselves
- Restore data backup executing:
restore-data - Check if mails, databases and users are present
#6
Updated by Davide Principi about 8 years ago
- Assignee changed from Giacomo Sanchietti to Davide Principi
#7
Updated by Davide Principi about 8 years ago
- Status changed from ON_QA to ON_DEV
- Assignee deleted (
Davide Principi) - % Done changed from 80 to 30
Test case 1 OK
No problems with case 1. Tested also with nethserver-sogo-1.2.1-1.ns6.
Test case 2 FAILED
In test case 2 I restored configuration and data on the same VM with changed interface MAC address: this causes interface-update event to fail and host becomes unreachable. I guess the same thing would happen in case 1.
Note: after restore nethserver-sogo-update event must be triggered to reset mysql password for sogo user. We must backup /etc/openldap/*.pw files (or more appropriately, move *.pw files under /var/lib/nethserver/...) see #2063.
#8
Updated by Davide Principi about 8 years ago
Also note that nslcd must be stopped before stopping slapd to avoid log messages like this:
Jul 16 14:27:49 davidep2 nslcd[3666]: [fff902] ldap_search_ext() failed: Can't contact LDAP server Jul 16 14:27:49 davidep2 nslcd[3666]: [fff902] no available LDAP server found
#9
Updated by Giacomo Sanchietti about 8 years ago
Modifications to nethserver-mysql package break package installation process: nethserver-mysql-conf do not start mysql daemon on first install.
Fix:
--- nethserver-mysql-conf.ori 2013-07-04 14:49:58.147409157 +0000
+++ nethserver-mysql-conf 2013-07-04 14:50:15.614393501 +0000
@@ -42,7 +42,7 @@
if ( -f "$backup_dir/mysql.dump" ) {
system("/etc/e-smith/events/actions/mysql-load-tables");
} else { # or initialize the server
- if ( system("/sbin/service mysqld status >/dev/null") == 0 ) {
+ if ( system("/sbin/service mysqld status >/dev/null") != 0 ) {
system("/sbin/service mysqld start");
}
my $exp = new Expect;
#10
Updated by Giacomo Sanchietti about 8 years ago
- nethserver-mysql-1.0.4
- correctly start mysqld on first install
- remove unused Runlevels prop, requires:
- nethserver-base-1.4.0-1.ns6.noarch.rpm
- nethserver-lib-1.3.0-1.ns6.noarch.rpm
- nethserver-yum-1.1.1-1.ns6.noarch.rpm
- nethserver-nethgui-1.2.2-1.ns6.noarch.rpm
#11
Updated by Giacomo Sanchietti about 8 years ago
- Status changed from ON_DEV to MODIFIED
- Assignee set to Giacomo Sanchietti
- % Done changed from 30 to 60
- reset network interface configuration
- document execution of bootstrap-console after configuration restore (see: Nethserver-backup)
- Accept '*' character in include and exclude files
#12
Updated by Giacomo Sanchietti about 8 years ago
- Status changed from MODIFIED to ON_QA
- Assignee deleted (
Giacomo Sanchietti) - % Done changed from 60 to 70
- nethserver-backup-data-1.0.5
- nethserver-backup-config-1.0.4
- Add an exclusion to /etc/backup-data.d/custom.exclude
- Execute:
backup-data - Check the excluded file is not in backup
Example of exclusion:
/var/lib/nethserver/vmail/giacomo/ **giacomo**
List files in backup (saved on a NFS share):
/etc/e-smith/events/actions/mount-nfs /usr/bin/duplicity list-current-files --archive-dir /var/lib/nethserver/backup/duplicity/ file:///mnt/backup/test /etc/e-smith/events/actions/umount-nfs
#13
Updated by Davide Principi almost 8 years ago
- Assignee set to Davide Principi
#14
Updated by Davide Principi almost 8 years ago
- Status changed from ON_QA to TRIAGED
- Assignee deleted (
Davide Principi) - % Done changed from 70 to 20
Test case 1, PASSED NOTE
During restore-config the backup-data key configuration DB is overwritten with values from the backup.
Test case 2, FAILED
- See note above
- After
restore-configldap server is down and passwd backend is not available:# ldapsearch -Y EXTERNAL ldap_sasl_interactive_bind_s: Can't contact LDAP server (-1) # id nsstest id: nsstest: No such user
Test case 3, PASSED
NOTE
In /etc/backup-data.d/custom.exclude empty lines are considered invalid. Can we skip them? Added a note to nethserver-backup.
Extract from log file /var/log/last-backup.log:
Import of duplicity.backends.giobackend Failed: No module named gio
Import of duplicity.backends.sshbackend Failed: No module named paramiko
Reading globbing filelist /tmp/ql4gqwEmxq
Fatal Error: The file specification
cannot match any files in the base directory
/
Useful file specifications begin with the base directory or some
pattern (such as '**') which matches the base directory.#15
Updated by Davide Principi almost 8 years ago
- Status changed from TRIAGED to MODIFIED
- % Done changed from 20 to 60
The test case 2 was missing an important step:
before executing restore-data the bootstrap-console command must be issued.
#16
Updated by Davide Principi almost 8 years ago
- Status changed from MODIFIED to ON_QA
- % Done changed from 60 to 70
#17
Updated by Davide Principi almost 8 years ago
- NEEDINFO changed from No to Yes
Should backup-data always execute backup-config automatically?
#18
Updated by Giacomo Sanchietti almost 8 years ago
Should
backup-dataalways executebackup-configautomatically?
It's a nice enhancement but it's not blocking. I propose to open another issue (#2118).
#19
Updated by Davide Principi almost 8 years ago
- Status changed from ON_QA to VERIFIED
- % Done changed from 70 to 90
- NEEDINFO changed from Yes to No
#20
Updated by Davide Principi almost 8 years ago
- Status changed from VERIFIED to CLOSED
- % Done changed from 90 to 100
In nethserver-updates:
nethserver-backup-config-1.0.4-1.ns6.noarch.rpm
nethserver-backup-data-1.0.6-1.ns6.noarch.rpm