Enhancement #2506

Update ntopng to version 1.1

Added by Filippo Carletti over 7 years ago. Updated almost 7 years ago.

Status:CLOSEDStart date:
Priority:NormalDue date:
Assignee:-% Done:

100%

Category:nethserver-ntopng
Target version:v6.5
Resolution: NEEDINFO:No

Description

ntopng fixes some bugs we fixed in our nethserver-ntopng.

Associated revisions

Revision 0ff878ad
Added by Giacomo Sanchietti about 7 years ago

Web UI, templates: allow access without authentication. Refs #2506

Revision 9bf4e15e
Added by Giacomo Sanchietti about 7 years ago

db defaults: change access to private. Refs #2506

Revision 896f3cc4
Added by Giacomo Sanchietti about 7 years ago

Removed cronjob for geo data download. Refs #2506

Revision 579ea9ac
Added by Giacomo Sanchietti about 7 years ago

events: force password change on redis. Refs #2506

Revision 3d933fac
Added by Giacomo Sanchietti about 7 years ago

spec: add dependency to ntopng >= 1.1 Refs #2506

Revision f98e1876
Added by Giacomo Sanchietti about 7 years ago

Dashboard: add widget into applications tab. Refs #2506

Revision 64595554
Added by Davide Principi almost 7 years ago

Ntopng: build Ntopng URL from HTTP Host header. Refs #2506

Original patch from Filippo Carletti <>
http://dev.nethserver.org/issues/2506#note-23

Revision 6f36e596
Added by Davide Principi almost 7 years ago

Ntopng Dashboard plugin: show service status. Refs #2506

History

#1 Updated by Filippo Carletti over 7 years ago

I think that we could remove authentication, leaving port 3000 open only to internal network or to the same network enabled to access the server-manager.

#2 Updated by Giacomo Sanchietti over 7 years ago

  • Target version set to ~FUTURE

#4 Updated by Filippo Carletti over 7 years ago

ntopng 1.1.2 has a working startup script (/etc/init/ntopng.conf), we should remove our startup script from nethserver-ntopng.
ntopng 1.1.2 requires an updated zeromq library, available from http://zeromq.org/distro:centos
ntopng new deps:

Updating:
 ntopng                               x86_64           1.1.2_7303-7303               /ntopng-1.1.2_7303-7303.x86_64           4.5 M
Installing for dependencies:
 cairo                                x86_64           1.8.8-3.1.el6                 centos-base                              309 k
 dejavu-fonts-common                  noarch           2.30-2.el6                    centos-base                               59 k
 dejavu-lgc-sans-mono-fonts           noarch           2.30-2.el6                    centos-base                              393 k
 dejavu-sans-mono-fonts               noarch           2.30-2.el6                    centos-base                              450 k
 fontconfig                           x86_64           2.8.0-3.el6                   centos-base                              186 k
 fontpackages-filesystem              noarch           1.41-1.1.el6                  centos-base                              8.8 k
 freetype                             x86_64           2.3.11-14.el6_3.1             centos-base                              359 k
 libX11                               x86_64           1.5.0-4.el6                   centos-base                              584 k
 libX11-common                        noarch           1.5.0-4.el6                   centos-base                              192 k
 libXau                               x86_64           1.0.6-4.el6                   centos-base                               24 k
 libXft                               x86_64           2.3.1-2.el6                   centos-base                               55 k
 libXrender                           x86_64           0.9.7-2.el6                   centos-base                               30 k
 libpng                               x86_64           2:1.2.49-1.el6_2              centos-base                              182 k
 libthai                              x86_64           0.1.12-3.el6                  centos-base                              183 k
 libxcb                               x86_64           1.8.1-1.el6                   centos-base                              110 k
 pango                                x86_64           1.28.1-7.el6_3                centos-base                              350 k
 pixman                               x86_64           0.26.2-5.el6_4                centos-base                              200 k
 rrdtool                              x86_64           1.3.8-6.el6                   centos-base                              293 k

#5 Updated by Filippo Carletti over 7 years ago

geoip database is available in rpm:
http://www.nmon.net/packages/rpm/x86_64/ntopng-data/

#6 Updated by Filippo Carletti about 7 years ago

I Also found in logs:

/etc/cron.monthly/ntop-update-geodb: line 10: /etc/e-smith/events/actions/nethserver-ntopng-restart: No such file or directory

We could use a proxypass to access the UI (thanks Giacomo).

I experienced some startup errors that seems to be related to redis not started.
I tried "start on redis" in the ntopng startup file and it fixed the error (observed at boot-up).

#7 Updated by Giacomo Sanchietti about 7 years ago

  • Status changed from NEW to TRIAGED
  • Target version changed from ~FUTURE to v6.5
  • % Done changed from 0 to 20

#8 Updated by Giacomo Sanchietti about 7 years ago

  • Status changed from TRIAGED to ON_DEV
  • Assignee set to Giacomo Sanchietti
  • % Done changed from 20 to 30

#9 Updated by Giacomo Sanchietti about 7 years ago

  • Status changed from ON_DEV to MODIFIED
  • % Done changed from 30 to 60

Ntopng doesn't work well with Apache proxy pass, so a new Authentication parameter has been added.
The authentication can be enabled or disabled from the web interface.

Removed cronjob for geo data download. Geo data will be installed using ntopng-data rpm.

Changed default access from prublic to private.

#10 Updated by Giacomo Sanchietti about 7 years ago

  • Status changed from MODIFIED to ON_QA
  • Assignee deleted (Giacomo Sanchietti)
  • % Done changed from 60 to 70
Packages in nethserver-testing:
  • nethserver-ntopng/nethserver-ntopng-1.1.2-3.0git896f3cc4.ns6.noarch.rpm
  • ntopng-1.1.2_7640-7640.x86_64.rpm
  • ntopng-data-1.1.2_7640-7640.x86_64.rpm
  • openpgm-5.1.118-3.el6.x86_64.rpm
  • zeromq3-3.2.4-1.el6.x86_64.rpm
Test case 1
  • Install the package
  • Open the web interface, enable the service and leave authentication disabled
  • Apply modification and check the server is working
Test case 2
  • Install the package
  • Open the web interface, enable the authentication and set a password
  • Apply modification and check the server is working and required authentication

Note
Before releasing, update yum group file and add ntopng-data as optional package to nethserver-bandwidth-monitor group.

#11 Updated by Massimo Palazzetti about 7 years ago

  • Assignee set to Massimo Palazzetti

#12 Updated by Massimo Palazzetti about 7 years ago

  • Status changed from ON_QA to TRIAGED
  • Assignee deleted (Massimo Palazzetti)
  • % Done changed from 70 to 20

Test case 1: OK

Test case 2: once enabled the autentication wrote a password and hit the "save" button the web interface seem to load something anche nothing appened.

In the log file:

Jun  4 14:23:07 server /sbin/e-smith/db[23764]: /var/lib/nethserver/db/configuration: OLD ntopng=service|Authentication|disabled|Password|admin|TCPPort|3000|access|public|status|enabled
Jun  4 14:23:07 server /sbin/e-smith/db[23764]: /var/lib/nethserver/db/configuration: NEW ntopng=service|Authentication|enabled|Password|admin|TCPPort|3000|access|public|status|enabled
Jun  4 14:23:07 server esmith::event[23766]: Event: nethserver-ntopng-save
Jun  4 14:23:07 server esmith::event[23766]: expanding /etc/ntopng/ntopng.conf
Jun  4 14:23:07 server esmith::event[23766]: expanding /var/tmp/ntopng-users.conf
Jun  4 14:23:07 server esmith::event[23766]: Action: /etc/e-smith/events/actions/generic_template_expand SUCCESS [0.06331]
Jun  4 14:23:07 server esmith::event[23768]: Event: lokkit-save nethserver-ntopng-save
Jun  4 14:23:07 server esmith::event[23768]: expanding /etc/sysconfig/system-config-firewall
Jun  4 14:23:07 server esmith::event[23768]: Action: /etc/e-smith/events/actions/generic_template_expand SUCCESS [0.062935]
Jun  4 14:23:08 server kernel: ip_tables: (C) 2000-2006 Netfilter Core Team
Jun  4 14:23:08 server kernel: nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
Jun  4 14:23:08 server kernel: ip6_tables: (C) 2000-2006 Netfilter Core Team
Jun  4 14:23:08 server esmith::event[23768]: Action: /etc/e-smith/events/lokkit-save/S20lokkit-apply SUCCESS [0.574111]
Jun  4 14:23:08 server esmith::event[23768]: Event: lokkit-save SUCCESS
Jun  4 14:23:08 server esmith::event[23766]: Action: /etc/e-smith/events/nethserver-ntopng-save/S70firewall-adjust SUCCESS [0.686579]
Jun  4 14:23:08 server esmith::event[23766]: [INFO] service ntopng restart

I wait for some minutes and nothing happened:

# ps ax | grep ntop
23744 ?        Sl     0:04 /usr/local/bin/ntopng /etc/ntopng/ntopng.conf
23765 ?        S      0:00 /usr/bin/sudo /sbin/e-smith/signal-event nethserver-ntopng-save
23766 ?        S      0:00 /usr/bin/perl -w /sbin/e-smith/signal-event nethserver-ntopng-save
23954 ?        S      0:00 /usr/bin/perl -w /etc/e-smith/events/actions/adjust-services nethserver-ntopng-save
23956 ?        S      0:00 /sbin/restart ntopng

and the ntop web access is still visible without authentication.

#13 Updated by Giacomo Sanchietti about 7 years ago

I can't reproduce the problem. Can you try it on a clean machine without installing the old nethserver-ntopng package?

After enabling auth:

Jun  6 15:57:21 localhost esmith::event[14916]: Event: nethserver-ntopng-save 
Jun  6 15:57:22 localhost esmith::event[14916]: expanding /etc/ntopng/ntopng.conf
Jun  6 15:57:22 localhost esmith::event[14916]: expanding /var/tmp/ntopng-users.conf
Jun  6 15:57:22 localhost esmith::event[14916]: Action: /etc/e-smith/events/actions/generic_template_expand SUCCESS [0.075117]
Jun  6 15:57:22 localhost esmith::event[14918]: Event: lokkit-save nethserver-ntopng-save
Jun  6 15:57:22 localhost esmith::event[14918]: expanding /etc/sysconfig/system-config-firewall
Jun  6 15:57:22 localhost esmith::event[14918]: Action: /etc/e-smith/events/actions/generic_template_expand SUCCESS [0.082216]
Jun  6 15:57:22 localhost kernel: ip_tables: (C) 2000-2006 Netfilter Core Team
Jun  6 15:57:22 localhost kernel: nf_conntrack version 0.5.0 (3923 buckets, 15692 max)
Jun  6 15:57:22 localhost kernel: ip6_tables: (C) 2000-2006 Netfilter Core Team
Jun  6 15:57:22 localhost esmith::event[14918]: Action: /etc/e-smith/events/lokkit-save/S20lokkit-apply SUCCESS [0.267979]
Jun  6 15:57:22 localhost esmith::event[14918]: Event: lokkit-save SUCCESS
Jun  6 15:57:22 localhost esmith::event[14916]: Action: /etc/e-smith/events/nethserver-ntopng-save/S70firewall-adjust SUCCESS [0.407681]
Jun  6 15:57:22 localhost esmith::event[14916]: [INFO] service ntopng restart
Jun  6 15:57:24 localhost kernel: device eth0 left promiscuous mode
Jun  6 15:57:24 localhost esmith::event[14916]: ntopng start/running, process 15109
Jun  6 15:57:24 localhost esmith::event[14916]: Action: /etc/e-smith/events/actions/adjust-services SUCCESS [2.35722]
Jun  6 15:57:24 localhost esmith::event[14916]: Event: nethserver-ntopng-save SUCCESS

And the web page displays the login form.

#14 Updated by Filippo Carletti about 7 years ago

  • Status changed from TRIAGED to ON_DEV
  • % Done changed from 20 to 30
  1. nethserver-ntopng should require ntopng-1.1
  2. once a password for the admin user is set, it stays forever, i.e. the first pass you set will remain valid even if you modify it and the new pass will not be valid. If you modify the pass using ntopng web ui the new password is set.

Note: I updated a system where ntopng was previously installed.

#15 Updated by Filippo Carletti about 7 years ago

Opening the server-manager ntopng web page for the first time, no option was selected in the radio button for auth.
A missing default? No:

# rpm -q nethserver-ntopng
nethserver-ntopng-1.1.2-3.0git896f3cc4.ns6.noarch
# cat /etc/e-smith/db/configuration/defaults/ntopng/Authentication 
disabled

#16 Updated by Giacomo Sanchietti about 7 years ago

  • Status changed from ON_DEV to MODIFIED
  • % Done changed from 30 to 60

#17 Updated by Giacomo Sanchietti about 7 years ago

  • Status changed from MODIFIED to ON_QA
  • % Done changed from 60 to 70
Modifications:
  • fix password change
  • add dependency to ntopng >= 1.1
Package in nethserver-testing:
  • nethserver-ntopng-1.1.2-5.0git3d933fac.ns6.noarch.rpm

#18 Updated by Giovanni Bezicheri about 7 years ago

  • Assignee set to Giovanni Bezicheri

#19 Updated by Giovanni Bezicheri about 7 years ago

  • Status changed from ON_QA to TRIAGED
  • Assignee deleted (Giovanni Bezicheri)
  • % Done changed from 70 to 20

Test case 1: passed.

Test case 2: the authentication is requested if enabled but it keeps the default password after changing it!

#20 Updated by Giacomo Sanchietti about 7 years ago

  • Status changed from TRIAGED to MODIFIED
  • % Done changed from 20 to 60

My bad, the rpm on nethserver-testing was not updated.

#21 Updated by Giacomo Sanchietti about 7 years ago

  • Status changed from MODIFIED to ON_QA
  • % Done changed from 60 to 70
New release uploaded to nethserver-testing:
*- nethserver-ntopng-1.1.2-5.0git3d933fac.ns6.noarch.rpm-
  • nethserver-ntopng-1.1.2-6.0gitf98e1876.ns6.noarch.rpm

Test case 3

  • Access the Dashboard under the Applications tab and verify ntopng widget is displayed.

#22 Updated by Filippo Carletti almost 7 years ago

  • Status changed from ON_QA to TRIAGED
  • % Done changed from 70 to 20

Test case 2 (password protected access): OK
New password required to access after change (but logout needed, i.e. you're not kicked out).
Test case 3 (link in dashboard app tab): OK

The URL shown in dashboard uses the browser url, while the url on the ntopng page uses the FQDN. In my test system, the FQDN was not working.

We could:
1. remove url from ntopng page
2. use the same url on both pages

#23 Updated by Filippo Carletti almost 7 years ago

2. use the same url on both pages

A patch could be:

--- Ntopng.php    2014-07-30 10:18:54.000000000 +0000
+++ /usr/share/nethesis/NethServer/Template/Ntopng.php    2014-08-07 13:00:57.456424042 +0000
@@ -13,7 +13,9 @@
     ->insert($view->fieldsetSwitch('Authentication', 'disabled'));

-$url = "http://".$view['FQDN'].":".$view['TCPPort'];
+$host = explode(':',$_SERVER['HTTP_HOST']);
+$url = "http://".$host[0].":".$view['TCPPort'];
+
 echo "<p style='margin-bottom: 5px'>URL: <a href='$url' target='_blank'>$url</a></p>";

 echo $view->buttonList($view::BUTTON_SUBMIT | $view::BUTTON_HELP);

#24 Updated by Davide Principi almost 7 years ago

  • Status changed from TRIAGED to ON_DEV
  • Assignee set to Davide Principi
  • % Done changed from 20 to 30

#25 Updated by Davide Principi almost 7 years ago

  • Status changed from ON_DEV to MODIFIED
  • Assignee deleted (Davide Principi)
  • % Done changed from 30 to 60
  • Applied Filippo's patch with a minor visual enhancement
  • Show service status in Dashboard panel: the user can expect a connection error if service is disabled

#26 Updated by Davide Principi almost 7 years ago

  • Status changed from MODIFIED to ON_QA
  • % Done changed from 60 to 70

In nethserver-testing:
nethserver-ntopng-1.1.2-8.0git6f36e596.ns6.noarch.rpm

#27 Updated by Filippo Carletti almost 7 years ago

  • Status changed from ON_QA to VERIFIED
  • % Done changed from 70 to 90
Updated:
  nethserver-ntopng.noarch 0:1.1.2-8.0git6f36e596.ns6 

URL uses browser hostname.
Dashboard show a status: item reflecting ntopng status.

#28 Updated by Davide Principi almost 7 years ago

  • Status changed from VERIFIED to CLOSED
  • % Done changed from 90 to 100

In nethserver-updates:
nethserver-ntopng-1.2.0-1.ns6.noarch.rpm

#29 Updated by Davide Principi almost 7 years ago

In nethserver-updates (also):
ntopng-1.1.2_7640-7640.x86_64.rpm
ntopng-data-1.1.2_7640-7640.x86_64.rpm
openpgm-5.1.118-3.el6.x86_64.rpm
zeromq3-3.2.4-1.el6.x86_64.rpm

Also available in: Atom PDF