Enhancement #3319

« Previous | Next »

Ntopng: optimize redis memory usage

Added by Giacomo Sanchietti over 5 years ago. Updated over 5 years ago.

Status:CLOSEDStart date:
Priority:NormalDue date:
Assignee:-% Done:

100%

Category:nethserver-ntopng
Target version:v6.7
Resolution: NEEDINFO:No

Description

Ntopng heavily uses redis to save temporary data about host network traffic.
In a scenario where there are many hosts involved, ntopng will highly increase redis memory usage.

Proposed enhancement:
  • use a separated instance of redis only for ntopng
  • remove dependency from nethserver-redis
  • limit redis memory usage
  • avoid useless db dumps on disk

Related issues

Related to NethServer 6 - Task #3304: Update redis from remi CLOSED

Associated revisions

Revision 04cc5ae4
Added by Giacomo Sanchietti over 5 years ago

spec: remove nethserver-redis dependency. Refs #3319

Revision bc55db54
Added by Giacomo Sanchietti over 5 years ago

Add ntopng-redis instance. Refs #3319

Revision f31b3f25
Added by Giacomo Sanchietti over 5 years ago

ntopng-redis.conf: change maxmemory-policy. Refs #3319

History

#1 Updated by Giacomo Sanchietti over 5 years ago

  • Status changed from NEW to TRIAGED
  • Target version set to v6.7
  • % Done changed from 0 to 20

#2 Updated by Giacomo Sanchietti over 5 years ago

  • Status changed from TRIAGED to ON_DEV
  • Assignee set to Giacomo Sanchietti
  • % Done changed from 20 to 30

#3 Updated by Filippo Carletti over 5 years ago

ntopng sources now contain this readme file:

$ cat doc/README.redis
In order to avoid redis fillup all the available memory please set some memory limit in /etc/redis.conf

It is a good idea to setup some upper limits such as:

maxmemory 300mb
maxmemory-policy volatile-lru

#4 Updated by Giacomo Sanchietti over 5 years ago

  • Status changed from ON_DEV to MODIFIED
  • Assignee deleted (Giacomo Sanchietti)
  • % Done changed from 30 to 60

#5 Updated by Giacomo Sanchietti over 5 years ago

  • Related to Task #3304: Update redis from remi added

#6 Updated by Giacomo Sanchietti over 5 years ago

  • Status changed from MODIFIED to ON_QA
  • % Done changed from 60 to 70
Package in nethserver-testing:
  • nethserver-ntopng-1.2.3-1.2.gbc55db5.ns6.noarch.rpm
Test case 1
  • Update an existing installation where ntopng is enabled
  • Ntopng should continue running without problems
  • Two instances of redis should be running
Test case2
  • Install on a new machine and enable ntopng
  • Ntopng must be running
  • Only one instance of redis should be running

Note
The default redis instance is no longer required.
To cleanup the system after an upgrade execute:

service redis stop
yum remove nethserver-redis
rm -f /var/lib/redis/*

#7 Updated by Filippo Carletti over 5 years ago

  • Status changed from ON_QA to VERIFIED
  • % Done changed from 70 to 90

Test case 1 - update
Before update:

[root@nscom ~]# ps -ef | egrep "redis|ntopng" 
redis      592     1  0 17:44 ?        00:00:00 /usr/sbin/redis-server /etc/redis.conf
nobody     663     1  2 17:45 ?        00:00:01 /usr/bin/ntopng /etc/ntopng/ntopng.conf

After update:
[root@nscom ~]# ps -ef | egrep "redis|ntopng" 
redis      592     1  0 17:44 ?        00:00:00 /usr/sbin/redis-server /etc/redis.conf
root       714     1  0 17:47 ?        00:00:00 /usr/sbin/redis-server /etc/redis-ntopng.conf
nobody     721     1  3 17:47 ?        00:00:00 /usr/bin/ntopng /etc/ntopng/ntopng.conf

Test case 2 - new install
Before install:

[root@ns67-ent ~]# ps -ef | grep redis
root      7470  6757  0 19:33 pts/0    00:00:00 grep redis

After:
[root@ns67-ent ~]# ps -ef | grep redis
root      7514     1  0 09:47 ?        00:00:00 /usr/sbin/redis-server /etc/redis-ntopng.conf
[root@ns67-ent ~]# ps -ef | grep ntop
root      7514     1  0 09:47 ?        00:00:00 /usr/sbin/redis-server /etc/redis-ntopng.conf
nobody   14258     1  3 09:55 ?        00:00:00 /usr/bin/ntopng /etc/ntopng/ntopng.conf

#8 Updated by Giacomo Sanchietti over 5 years ago

  • Status changed from VERIFIED to CLOSED
  • % Done changed from 90 to 100
Released in nethserver-updates:
  • nethserver-ntopng-1.3.0-1.ns6.noarch.rpm

Also available in: Atom PDF