NethServer 6

Note

• Read the test cases. I suggest executing 2 before 1.
• Test Virus and Spam messages are available from amavisd-new package. Execute the following commands:

  mkdir amavis-test
  cd amavis-test
  perl -pe 's/./chr(ord($&)^255)/sge' </usr/share/doc/amavisd-new-2.8.0/test-messages/sample.tar.gz.compl | zcat | tar xvf -
  

Test case 1 - fresh install

In a clean NethServer 6.5 update base packages.

• yum --enablerepo=nethserver-testing install nethserver-fetchmail nethserver-mail-filter
• Create an user account first.user
• In POP3 connector page configure an IMAP or POP account where to fetch messages for first.user
• If a message is delivered to first.user you must find in /var/log/fetchmail.log

  Nov 18 15:31:05 [info]  12 messages (11 seen) for davidep2 at nethservice.nethesis.it (15855 octets).
  Nov 18 15:31:06 [info]  reading message davidep2@nethservice.nethesis.it:12 of 12 (7391 octets) not flushed (...or flushed)
  

  In /var/log/maillog:

  Nov 18 15:31:05 davidep2 dovecot: lmtp(25995): Connect from local
  Nov 18 15:31:06 davidep2 dovecot: lmtp(25995, first.user): X3IAMClYa1SLZQAA4KLHlg: sieve: msgid=<546b581b.l+j9LuDuzaR2ebOZ%davide.principi@nethesis.it>: stored mail into mailbox 'INBOX'
  Nov 18 15:31:06 davidep2 dovecot: lmtp(25995): Disconnect from local: Client quit (in reset)
  Nov 18 15:31:06 davidep2 amavis[25182]: (25182-01) Passed CLEAN {RelayedOpenRelay}, FETCHMAIL [192.168.5.252] <davide.principi@nethesis.it> -> <first.user@localhost>, Message-ID: <546b581b.l+j9LuDuzaR2ebOZ%davide.principi@nethesis.it>, mail_id: qz3X6CmDCqIo, Hits: -, size: 7543, queued_as: 250 2.0.0 <first.user@localhost> X3IAMClYa1SLZQAA4KLHlg Saved, 806 ms
  

  You must not see any Postfix message.
• If a message is SPAM it is discarded by amavisd:

  In /var/log/maillog:
  Nov 18 16:20:44 davidep2 amavis[26857]: (26857-01) Blocked SPAM {DiscardedOpenRelay,Quarantined}, FETCHMAIL [192.168.5.252] <davide.principi@nethesis.it> -> <first.user@localhost>, Message-ID: <546b639f.2JA24Xqk2Rj1qm4d%davide.principi@nethesis.it>, mail_id: d0jN0QqJoVwN, Hits: 999.999, size: 1707, 5128 ms
  
  In /var/log/fetchmail.log:
  Nov 18 16:20:44 [info]  reading message davidep2@nethservice.nethesis.it:16 of 16 (1555 octets) not flushed
  

  Amavis accepts the message then discards it silently.
• If a message is tagged SPAMMY it must be delivered to junkmail folder (if enabled).

  Dec  3 11:39:18 davidep2 dovecot: lmtp(17334, first.user): EJf3GiToflS2QwAAoK0gTQ: sieve: msgid=<N1msdrbJXNPfV4wg9>: stored mail into mailbox 'junkmail'
  Dec  3 11:39:18 davidep2 dovecot: lmtp(17334): Disconnect from local: Client quit (in reset)
  Dec  3 11:39:18 davidep2 amavis[17306]: (17306-01) Passed SPAMMY {RelayedTaggedInbound}, FETCHMAIL [63.10.249.142] <testx@spammers.gov> -> <first.user@vboxnet0.tld>, Message-ID: <N1msdrbJXNPfV4wg9>, mail_id: 23ntI-0s95qC, Hits: 8.361, size: 4800, queued_as: 250 2.0.0 <first.user+spam@vboxnet0.tld> EJf3GiToflS2QwAAoK0gTQ Saved, 6314 ms

• If a message is VIRUS it is rejected by amavisd:

  In /var/log/maillog:
  Nov 18 16:23:23 davidep2 amavis[26860]: (26860-01) Blocked INFECTED (Eicar-Test-Signature.UNOFFICIAL) {RejectedOpenRelay,Quarantined}, FETCHMAIL [192.168.5.252] <davide.principi@nethesis.it> -> <first.user@localhost>, Message-ID: <546b6463.3KQWNOuvtic9jPJ0%davide.principi@nethesis.it>, mail_id: v8g99-gBhtiF, Hits: -, size: 1083, 463 ms
  
  In /var/log/fetchmail.log:
  Nov 18 16:23:23 [err]  reading message davidep2@nethservice.nethesis.it:17 of 17 (931 octets) (log message incomplete)
  Nov 18 16:23:23 [err]  SMTP listener refused delivery
  Nov 18 16:23:23 [info]   not flushed
  

Test case 2 - fresh install w/o nethserver-mail-filter

Execute test case 1 without installing nethserver-mail-filter. Virus and SPAM messages must be delivered regularly.

Test case 3 - update

Repeat test case 1, updating an existing installation.

• The yum update nethserver-fetchmail command must update also mail-common and mail-server

In nethserver-testing:
nethserver-mail-server-1.8.2-1.1gitdd2412c.ns6.noarch.rpm
nethserver-mail-common-1.4.1-1.1git09556a6.ns6.noarch.rpm
nethserver-fetchmail-1.0.6-1.0git664a2e1d.ns6.noarch.rpm

Packager note

When set to CLOSED merge https://github.com/nethesis/nethserver-docs/pull/44

Stefano Fancello wrote:

I'm not able to have a spam hit > 6.x using fetchmail nor with spamtest and messages are delivered anyway
[...]

• connects on port 25
• Postfix acts as SMTP proxy with Amavis
• Amavis tags the message SPAMMY (Hits: 6.235)
• Amavis re-inject the message into the queue
• The message is delivered to dovecot and stores the messages into 'junkmail'

When I fetch an email that I expect to be marked as spam(using sample-spam.txt text):
When I try to enable junkmail, I expect that "SPAMMY" mails is delivered in there, but what I have is:
[...]

This sounds strange: dovecot stores the SPAMMY message into INBOX and needs a deeper test.

Could you attach the spam thresholds values? Please paste here the output of config show amavisd.

Those are packages that I've installed:
[...]

I assume they were the latest from nethserver-testing.

Finally, when I hit "download now" button on interface, I get an error message that tel me to look at /var/log/fetchmail.log for more information, but log file doesn't show anything interesting

I agree, it's ugly. How would you enhance it? Let's open an enhancement!

I also would like to point out that test cases aren't really clear on how to use amavis test mails to test fetchmail. As an email newbee, it took me a lot of time to test this.

I'm sorry, the mail system is complex and I wrote the test cases with a lot of assumptions. Thanks for your time and effort: I understand your frustration as I used to be a newbie too :) To find help ask here, on IRC or ML: the QA step always offers the chance to learn something new!

Test case

• If a message is tagged SPAMMY it must be delivered to junkmail folder (if enabled).

  Dec  3 11:39:18 davidep2 dovecot: lmtp(17334, first.user): EJf3GiToflS2QwAAoK0gTQ: sieve: msgid=<N1msdrbJXNPfV4wg9>: stored mail into mailbox 'junkmail'
  Dec  3 11:39:18 davidep2 dovecot: lmtp(17334): Disconnect from local: Client quit (in reset)
  Dec  3 11:39:18 davidep2 amavis[17306]: (17306-01) Passed SPAMMY {RelayedTaggedInbound}, FETCHMAIL [63.10.249.142] <testx@spammers.gov> -> <first.user@vboxnet0.tld>, Message-ID: <N1msdrbJXNPfV4wg9>, mail_id: 23ntI-0s95qC, Hits: 8.361, size: 4800, queued_as: 250 2.0.0 <first.user+spam@vboxnet0.tld> EJf3GiToflS2QwAAoK0gTQ Saved, 6314 ms
  

In nethserver-testing (6.5):
[...]
nethserver-mail-common-1.4.1-1.5gitf0c3400.ns6.noarch.rpm

• nethserver-mail-server-1.8.3-1.ns6.noarch.rpm
• nethserver-fetchmail-1.1.0-1.ns6.noarch.rpm

Release of nethserver-fetchmail-1.1.0-1.ns6.noarch.rpm has been delayed: a non-existent user error (SMTP code 550) from Dovecot is ignored by fetchmail that flushes the message from the origin server.

• Validate local recipient in fetchmailrc template expansion, to disable invalid fetchmail DB records
• Restart fetchmail on user-delete and group-delete events, and remove obsolete restarts on pseudonym-modify, domain-delete and domain-modify events.
• AD user/group deletion can't be detected: add documentation. Any fetchmail DB record pointing to a deleted AD account must be disabled manually.

Test case

• fetchmail must be restarted on the following conditions
  • one of user-modify, user-delete, group-modify, group-delete events, and
  • the involved account for the above events is referenced by any of fetchmail DB records
• if an external (AD) account has been removed, the fetchmail service must be restarted manually. Check the removed account (nx here) is skipped by fetchmailrc template:

  perl -Mesmith::templates -e 'print esmith::templates::processTemplate({TEMPLATE_PATH=>"fetchmailrc", OUTPUT_TYPE=>"string"});'
  WARNING in /etc/e-smith/templates/fetchmailrc/10base: [WARNING] non available account `nx` has been skipped! Check your configuration.
  [...]
  

Updated https://github.com/nethesis/nethserver-docs/pull/44
Must be merged on release.

In nethserver-testing (6.5):
nethserver-fetchmail-1.1.0-3.0gitecfd13cc.ns6.noarch.rpm

• user-modify
• group-modify
• user-delete
• group-delete

With a non-existing user:

[root@localhost amavis-test]# db fetchmail show
admin@example.tld=fetchmail
    account=tt
    active=YES
    nokeep=YES
    password=Nethesis,1234
    popserver=localhost
    proto=pop3
    ssl=NO
    username=admin
[root@localhost amavis-test]# service fetchmail restart
Shutting down fetchmail:                                   [  OK  ]
WARNING in /etc/e-smith/templates/fetchmailrc/10base: [WARNING] non available account `tt` has been skipped! Check your configuration.
WARNING: Template processing succeeded for /fetchmailrc: 1 fragment generated warnings
 at -e line 1
No active fetchmail accounts. Exiting gracefully.

• nethserver-fetchmail-1.1.1-1.ns6.noarch.rpm

Documentation has been updated.