Feature #2739

Backup config: add web interface

Added by Giacomo Sanchietti over 7 years ago. Updated almost 7 years ago.

Status:CLOSEDStart date:
Priority:NormalDue date:
Assignee:-% Done:

100%

Category:nethserver-backup-config
Target version:v6.5
Resolution: NEEDINFO:No

Description

Create a simple web interface to manage the configuration backup.

The web interface should:
  • show the last backup
  • allow to force a new backup
  • restore the current backup

Related issues

Related to NethServer 6 - Enhancement #2699: Backup config: minimize creation of new backup CLOSED
Related to Nethgui - Bug #2901: Starting task state lost on double NethPlatform::exec calls CLOSED

Associated revisions

Revision 6e72550d
Added by Giacomo Sanchietti over 7 years ago

Add web interface. Refs #2739

Revision 16baa4b3
Added by Giacomo Sanchietti over 7 years ago

Web interface: add restore function. Refs #2739

Revision fe87d433
Added by Giacomo Sanchietti about 7 years ago

Web interface: refactor in 3 tabs. Refs #2739

Revision 904e9c2f
Added by Giacomo Sanchietti about 7 years ago

Web UI: update labels. Refs #2739

Revision 77f517ed
Added by Giacomo Sanchietti about 7 years ago

Backup algorithm: use sha1sum. Refs #2739

Revision 305330a1
Added by Giacomo Sanchietti about 7 years ago

Web interface, scripts: add support for ptrack. Refs #2739

Revision 33a460c1
Added by Davide Principi almost 7 years ago

restore-config: set tasks weights. Refs #2739

Revision 14c16ed0
Added by Davide Principi almost 7 years ago

BackupConfig UI module: workaround to show Tracker window. Refs #2739

Starting task state lost on double NethPlatform::exec calls
Refs #2901 [Nethgui]

Revision fe66e39e
Added by Davide Principi almost 7 years ago

Merge branch 'b2739' Refs #2739

Conflicts:
nethserver-backup-config.spec.in
root/etc/e-smith/events/actions/backup-config-execute

Revision 85365a56
Added by Davide Principi almost 7 years ago

backup-config include: ignore group and passwd unix DB. Refs #2739

The numeric identifiers of system users and groups must not be
overwritten.

Revision 9e217a3d
Added by Giacomo Sanchietti almost 7 years ago

backup-config include: list only relevant files. Refs #2739

Revision 8ca9a8f5
Added by Giacomo Sanchietti almost 7 years ago

backup-config include: list only relevant files. Refs #2739

Revision ba643b0e
Added by Giacomo Sanchietti almost 7 years ago

New implementation of backup algorithm. Refs #2739

Configuration backup is executed if:
  • backup file does not exist
  • file list is changed (modification to include/exclude files)
  • file contents is changed (md5sum doesn't match)

Revision bee5b90e
Added by Giacomo Sanchietti almost 7 years ago

Config backup: add secrets dir. Refs #2739

Revision 305a0b96
Added by Giacomo Sanchietti almost 7 years ago

restore-config: fix missing perl import. Refs #2739

Revision 1d577352
Added by Giacomo Sanchietti almost 7 years ago

Backup config: remove non-relevant files. Refs #2739

Revision 06f5aa44
Added by Giacomo Sanchietti almost 7 years ago

backup-config: remove configuration. Refs #2739

Revision 072907a5
Added by Giacomo Sanchietti almost 7 years ago

spec: remove unused dep perl-Unix-ConfigFile. Refs #2739

Revision 2dd1922e
Added by Giacomo Sanchietti almost 7 years ago

backup: move seqf to data backup. Refs #2739

Revision 72b0c3a7
Added by Giacomo Sanchietti almost 7 years ago

execute action: check for md5 file presence. Refs #2739

History

#1 Updated by Giacomo Sanchietti over 7 years ago

  • Status changed from NEW to TRIAGED
  • Target version set to v6.5
  • % Done changed from 0 to 20

#2 Updated by Giacomo Sanchietti over 7 years ago

  • Status changed from TRIAGED to ON_DEV
  • % Done changed from 20 to 30

#3 Updated by Giacomo Sanchietti over 7 years ago

  • Assignee set to Giacomo Sanchietti

#4 Updated by Giacomo Sanchietti over 7 years ago

  • Status changed from ON_DEV to MODIFIED
  • Assignee deleted (Giacomo Sanchietti)
  • % Done changed from 30 to 60

Implementation on branch 2739.
Actually the backup is restored in background to avoid errors during httpd-admin restart.

TODO: Add a process tracker to the web interface for the restore process.

#5 Updated by Filippo Carletti about 7 years ago

  • Status changed from MODIFIED to ON_DEV
  • % Done changed from 60 to 30

#6 Updated by Giacomo Sanchietti about 7 years ago

Web interface has been refactored.

Add inline help and manual sections before release.

#7 Updated by Davide Principi almost 7 years ago

  • Assignee set to Davide Principi

#8 Updated by Davide Principi almost 7 years ago

  • Related to Bug #2901: Starting task state lost on double NethPlatform::exec calls added

#9 Updated by Davide Principi almost 7 years ago

  • Assignee deleted (Davide Principi)

Pushed on branch b2739 (renames 2739).

#10 Updated by Davide Principi almost 7 years ago

  • Status changed from ON_DEV to MODIFIED
  • % Done changed from 30 to 60

Merged b2739 with master

Test case 1

  • Install nethserver-directory
  • Upgrade the modified package version
  • Create an account user1, set its props, enable shell access, set password
  • Verify user1 can login by ssh
  • From Backup (configuration) page create the configuration backup
  • Check the backup archive exists:
       #find /var/lib/nethserver/backup/ -type f -printf '%-42f %Tc\n'
    backup-config.tar.xz                       Wed 08 Oct 2014 05:52:26 PM CEST
    backup-config.tar.xz.md5                   Wed 08 Oct 2014 05:52:27 PM CEST
    backup-config.tar.xz-content.md5           Wed 08 Oct 2014 05:52:27 PM CEST
    

Test case2

[Continues from case 1]

  • Delete user1
  • Verify user1 cannot login
  • Restore the configuration backup
  • Verify user1 can login again
  • Verify user1 entry exists in accounts DB

Test case 3

[Continues from case 2]
  • Create a new account user2, give it shell access and password
  • Verify user2 can login

#11 Updated by Davide Principi almost 7 years ago

  • Status changed from MODIFIED to ON_QA
  • % Done changed from 60 to 70

In nethserver-testing:
nethserver-backup-config-1.2.0-11.0gitfe66e39e.ns6.noarch.rpm

#12 Updated by Davide Principi almost 7 years ago

  • Status changed from ON_QA to TRIAGED
  • % Done changed from 70 to 20

Add UI success/failure notifications.

#13 Updated by Davide Principi almost 7 years ago

  • Status changed from TRIAGED to ON_DEV
  • Assignee set to Davide Principi
  • % Done changed from 20 to 30

#14 Updated by Davide Principi almost 7 years ago

  • Status changed from ON_DEV to MODIFIED
  • Assignee deleted (Davide Principi)
  • % Done changed from 30 to 60

Not really MODIFIED: failure notifications work out-of-the-box.

#15 Updated by Davide Principi almost 7 years ago

  • Status changed from MODIFIED to ON_QA
  • % Done changed from 60 to 70

In nethserver-testing:
nethserver-backup-config-1.2.0-11.0gitfe66e39e.ns6.noarch.rpm (unmodified, see above)
nethserver-base-2.4.0-8.0git85365a56.ns6.noarch.rpm (was missing)

#16 Updated by Giacomo Sanchietti almost 7 years ago

  • Status changed from ON_QA to TRIAGED
  • % Done changed from 70 to 20

Actual implementation does not rebuild the backup archive when a file is added (or removed) to the set.

#17 Updated by Giacomo Sanchietti almost 7 years ago

  • Status changed from TRIAGED to ON_DEV
  • Assignee set to Giacomo Sanchietti
  • % Done changed from 20 to 30

#18 Updated by Giacomo Sanchietti almost 7 years ago

  • Status changed from ON_DEV to MODIFIED
  • % Done changed from 30 to 60
Configuration backup is executed if:
  • backup file does not exist
  • file list is changed (modification to include/exclude files)
  • file contents is changed (md5sum doesn't match)
Modified packages:
  • nethserver-backup-config
  • nethserver-base
  • nethserver-openssh
  • nethserver-mail-server
  • nethserver-hylafax

#19 Updated by Giacomo Sanchietti almost 7 years ago

  • Status changed from MODIFIED to ON_QA
  • Assignee deleted (Giacomo Sanchietti)
  • % Done changed from 60 to 70
Packages in nethserver-testing:
  • nethserver-backup-config-1.2.0-13.0git305a0b96.ns6.noarch.rpm nethserver-backup-config-1.2.0-14.0git072907a5.ns6.noarch.rpm
  • nethserver-base-2.4.0-12.0gitbee5b90e.ns6.noarch.rpm
  • nethserver-openssh-1.0.5-2.0git9e217a3d.ns6.noarch.rpm
  • nethserver-mail-server-1.8.0-1.1git06f5aa4.ns6.noarch.rpm
  • nethserver-hylafax-1.0.8-2.0git1d577352.ns6.noarch.rpm

Merged b2739 with master

Test case 1

  • Install nethserver-directory
  • Upgrade the modified package version
  • Create an account user1, set its props, enable shell access, set password
  • Verify user1 can login by ssh
  • From Backup (configuration) page create the configuration backup
  • Check the backup archive exists:
       #find /var/lib/nethserver/backup/ -type f -printf '%-42f %Tc\n'
    backup-config.tar.xz                       Wed 08 Oct 2014 05:52:26 PM CEST
    backup-config.tar.xz.md5                   Wed 08 Oct 2014 05:52:27 PM CEST
    backup-config.tar.xz-content.md5           Wed 08 Oct 2014 05:52:27 PM CEST
    

Test case2

[Continues from case 1]

  • Delete user1
  • Verify user1 cannot login
  • Restore the configuration backup
  • Verify user1 can login again
  • Verify user1 entry exists in accounts DB

Test case 3

[Continues from case 2]
  • Create a new account user2, give it shell access and password
  • Verify user2 can login
Test case 4
  • Install on a clean machine
  • Execute backup: backup-config
  • Execute backup again
  • Check the backup file is not changed

Test case 5

[Continues from case 4]

  • Add a file or directory to /etc/backup-config.d/custom.include
    echo "/etc/motd" >> /etc/backup-config.d/custom.include
    
  • Execute backup
  • Check the file is in the backup

Test case 6

[Continues from case 5]

  • Change the date of the added file:
    touch /etc/motd
    
  • Execute the backup
  • Check the backup is not changed

Test case 7

[Continues from case 6]

  • Modify /etc/motd
    echo "test" >  /etc/motd
    
  • Execute the backup
  • Check the backup is updated
Test case 8
  • Install a new module:
    yum install nethserver-nut
    
  • Execute the backup
  • Check the backup is updated

NOTE: Hylafax configuration forces a new configuration backup every time a fax is sent or received.

#20 Updated by Davide Principi almost 7 years ago

  • Assignee set to Davide Principi

#21 Updated by Davide Principi almost 7 years ago

  • Assignee changed from Davide Principi to Filippo Carletti

#22 Updated by Giacomo Sanchietti almost 7 years ago

Hylafax seqf files moved to backup-data:
  • nethserver-hylafax-1.0.8-3.0git2dd1922e.ns6.noarch.rpm

#23 Updated by Filippo Carletti almost 7 years ago

  • Status changed from ON_QA to VERIFIED
  • % Done changed from 70 to 90

I restored the config backup on a different server, network config has been correctly kept intact.
After I installed some modules (mail server), all accounts appeared.
I also restored on the same system, reverting config.

Notes:

# ssh -p 2233 filippo@localhost
filippo@localhost's password: 
[filippo@localhost ~]$ id
uid=5000(filippo) gid=5000(filippo) groups=5000(filippo),502(locals) context=unconfined_u:unconfined_r:unconfined_t:s0

Oct 14 11:09:36 localhost /sbin/e-smith/db[3703]: /var/lib/nethserver/db/accounts: DELETE filippo=user-deleted|City||Company||Department||FirstName|Filippo|LastName|Carletti|MailForwardStatus|disabled|MailSpamRetentionStatus|disabled|MailStatus|enabled|PhoneNumber||Samba|enabled|Shell|/bin/bash|Street||Uid|5000|__state|active

Oct 14 11:10:21 localhost sshd[3715]: Invalid user filippo from 127.0.0.1

# tar -atvf /var/lib/nethserver/backup/backup-config.tar.xz | grep motd
-rw-r--r-- root/root         0 2010-01-12 13:28 etc/motd

# tar -atvf /var/lib/nethserver/backup/backup-config.tar.xz | grep motd
-rw-r--r-- root/root         5 2014-10-14 14:09 etc/motd

#24 Updated by Filippo Carletti almost 7 years ago

Verified also nethserver-hylafax.

# tar -atvf /var/lib/nethserver/backup/backup-config.tar.xz | grep seqf
-rw------- uucp/60002        1 2014-10-14 14:51 var/spool/hylafax/docq/seqf
-rw------- uucp/60002        1 2014-10-14 14:51 var/spool/hylafax/sendq/seqf

# yum --disablerepo=\* --enablerepo=nethserver-testing update nethserver-hylafax

# rpm -q nethserver-hylafax
nethserver-hylafax-1.0.8-3.0git2dd1922e.ns6.noarch

# tar -atvf /var/lib/nethserver/backup/backup-config.tar.xz | grep seqf

#25 Updated by Giacomo Sanchietti almost 7 years ago

  • Assignee deleted (Filippo Carletti)

#26 Updated by Giacomo Sanchietti almost 7 years ago

  • Status changed from VERIFIED to CLOSED
  • % Done changed from 90 to 100
Packages released in nethserver-updates:
  • nethserver-openssh-1.0.6-1.ns6.noarch.rpm
  • nethserver-mail-server-1.8.1-1.ns6.noarch.rpm
  • nethserver-hylafax-1.0.9-1.ns6.noarch.rpm
  • nethserver-base-2.5.0-1.ns6.noarch.rpm
  • nethserver-backup-config-1.3.0-1.ns6.noarch.rpm

Also available in: Atom PDF