Enhancement #2460

Give wings to server-manager

Added by Davide Principi over 7 years ago. Updated over 7 years ago.

Status:CLOSEDStart date:12/12/2013
Priority:NormalDue date:12/13/2013
Assignee:-% Done:

100%

Category:nethserver-httpd-admin
Target version:v6.5-beta3
Resolution: NEEDINFO:No

Description

Esmith DB accesses from server-manager are very slow.
  • Reading any key or prop costs two exec() calls: sudo and db.
  • The server-manager is implemented in PHP, while db and Esmith DB libraries are implemented in Perl thus are forced to run in separate processes.

The server-manager speed can be improved implementing a long-running Perl server process that returns Esmith DB values to the PHP side, talking through a Unix domain socket.

An experiment with a light-fast prototype confirms and validates this new architecture.

Related issues

Related to NethServer 6 - Feature #2014: Service supervision with Upstart CLOSED 11/15/2013 11/15/2013

Associated revisions

Revision fac0880a
Added by Davide Principi over 7 years ago

Use smwingsd to speedup esmith DB reads. Refs #2460

Revision dfc4709a
Added by Davide Principi over 7 years ago

Fallback to exec() if smwingsd is not available. Refs #2460

Revision d65160c7
Added by Davide Principi over 7 years ago

PhpWrapper: added fsockopen wrapper. Refs #2460

The magic call had problems with return value. Why?

Revision d62735f8
Added by Davide Principi over 7 years ago

PhpWrapper (fsockopen): port argument is optional. Refs #2460

Revision 1089d4be
Added by Davide Principi over 7 years ago

smwingsd esmith DB fast access daemon. Refs #2460

Revision d47f5667
Added by Davide Principi over 7 years ago

smwingsd Upstart configuration. Refs #2460

Revision 2ed11cd2
Added by Davide Principi over 7 years ago

Fix fallback to dbExec() if socket open fails. Refs #2460

Revision a51983a8
Added by Davide Principi over 7 years ago

EsmithDatabase: added safeRead() method to read from buffered socket stream. Refs #2460

Fixes IO error when reading large (>8KB) database files from smwingsd.

History

#1 Updated by Davide Principi over 7 years ago

  • Status changed from TRIAGED to ON_DEV
  • % Done changed from 20 to 30

#2 Updated by Davide Principi over 7 years ago

  • Estimated time set to 12.00

#3 Updated by Davide Principi over 7 years ago

  • Due date set to 12/13/2013
  • Status changed from ON_DEV to MODIFIED
  • Start date set to 12/12/2013
  • % Done changed from 30 to 60

MODIFIED

A new daemon controlled by Upstart was added: smwingsd

To increase debug daemon messages exchanged on local Unix socket /var/run/smwingsd.sock set log_level=4 in /etc/smwingsd.conf and restart the daemon. Also rsyslogd might be restarted if it has still not reloaded its configuration (see /etc/rsyslog.d/smwingsd.conf).

The daemon configuration file /etc/smwingsd.conf follows the Net::Server Perl library conventions.

It can run chroot-ed by adding the following line to the configuration:

chroot=/var/lib/nethserver/db/

Test case

After upgrading to the modified version check smwingsd is running and server-manager runs faster without any error messages in /var/log/messages:

   # ps -C smwingsd -o user,group,cmd,rss,vsz,size
USER     GROUP    CMD                           RSS    VSZ    SZ
admin    admin    /usr/bin/perl /usr/libexec/  9852  56200  8184

#4 Updated by Davide Principi over 7 years ago

  • Status changed from MODIFIED to ON_QA
  • % Done changed from 60 to 70

In nethserver-testing:
nethserver-httpd-admin-1.0.6-14.0gitd47f5667.ns6.noarch.rpm
nethserver-nethgui-1.2.4-17.0gitd65160c7.ns6.noarch.rpm
nethserver-nethgui-1.2.4-18.0gitd62735f8.ns6.noarch.rpm

#5 Updated by Giacomo Sanchietti over 7 years ago

  • Assignee set to Giacomo Sanchietti

#6 Updated by Giacomo Sanchietti over 7 years ago

  • Status changed from ON_QA to VERIFIED
  • Assignee deleted (Giacomo Sanchietti)
  • % Done changed from 70 to 90

Changed log verbosity to 4. Extract from /var/log/smwingsd

...
Jan 15 14:45:16 test smwingsd[25969]: << ["networks","getjson"]
Jan 15 14:45:16 test smwingsd[25969]: >> [{"props":{"bootproto":"static","onboot":"yes","gateway":"192.168.5.253","netmask":"255.255.255.0","device":"br0","role":"green","ipaddr":"192.168.5.246"},"name":"br0","type":"bridge"},{"props":{"device":"eth0","bridge":"br0","role":"bridged","hwaddr":"08:00:27:0b:58:c8"},"name":"eth0","type":"ethernet"},{"props":{"role":"","hwaddr":"de:bc:ed:b5:76:d3"},"name":"tap0","type":"ethernet"}]
Jan 15 14:45:16 test smwingsd[25969]: << ["configuration","getprop","OrganizationContact","Company"]
Jan 15 14:45:16 test smwingsd[25969]: >> "XYZ Corporation" 
Jan 15 14:45:16 test smwingsd[25969]: << ["configuration","getprop","OrganizationContact","Street"]
Jan 15 14:45:16 test smwingsd[25969]: >> "123 Main Street" 
Jan 15 14:45:16 test smwingsd[25969]: << ["configuration","getprop","OrganizationContact","City"]
Jan 15 14:45:16 test smwingsd[25969]: >> "Ottawa" 
...

All changes to db are correctly re-read by the daemon:

...
Jan 15 14:47:51 test smwingsd[25969]: << ["configuration","getprop","OrganizationContact","Company"]
Jan 15 14:47:51 test smwingsd[25969]: >> "Nethesis" 
Jan 15 14:47:51 test smwingsd[25969]: << ["configuration","getprop","OrganizationContact","Street"]
Jan 15 14:47:51 test smwingsd[25969]: >> "Via degl Olmi" 
Jan 15 14:47:51 test smwingsd[25969]: << ["configuration","getprop","OrganizationContact","City"]
Jan 15 14:47:51 test smwingsd[25969]: >> "Ottawa" 
...

Now the web GUI is amazing fast.

Upstart correctly respawn the process:

[root@test yum.repos.d]# ps aux | grep wing
srvmgr   25969  0.0  1.9  56192  9864 ?        Ss   14:44   0:00 /usr/bin/perl /usr/libexec/nethserver/smwingsd
root     26147  0.0  0.1 103236   852 pts/1    S+   14:49   0:00 grep wing
[root@test yum.repos.d]# kill 25969
[root@test yum.repos.d]# ps aux | grep wing
srvmgr   26149  0.0  1.9  56064  9692 ?        Ss   14:49   0:00 /usr/bin/perl /usr/libexec/nethserver/smwingsd
root     26151  0.0  0.1 103236   852 pts/1    S+   14:49   0:00 grep wing

Marking as VERIFIED.

#7 Updated by Davide Principi over 7 years ago

  • Status changed from VERIFIED to ON_QA
  • % Done changed from 90 to 70

Found and fixed a problem with large ( > 8KB ) database files.

In 6.4/nethserver-testing:
nethserver-nethgui-1.3.1-2.0gita51983a8.ns6.noarch.rpm

#8 Updated by Giacomo Sanchietti over 7 years ago

  • Status changed from ON_QA to VERIFIED
  • % Done changed from 70 to 90

Marking as VERIFIED.

#9 Updated by Davide Principi over 7 years ago

  • Status changed from VERIFIED to CLOSED
  • % Done changed from 90 to 100

In nethserver/6.5/base
nethserver-nethgui-1.3.2-1.ns6.noarch.rpm

Also available in: Atom PDF