Enhancement #2958

squidGuard: support multiple profiles

Added by Giacomo Sanchietti over 6 years ago. Updated over 6 years ago.

Status:CLOSEDStart date:
Priority:NormalDue date:
Assignee:-% Done:

100%

Category:nethserver-squidguard
Target version:v6.5
Resolution: NEEDINFO:Yes

Description

Add support for multiple profiles inside squidGuard.

Following features should be implemented:
  • creation of multiple profile: each profile can contain a list of allowed/blocked categories
  • a profile can be associated to an object; permitted objects are:
    • user
    • user group
    • host
    • host group
  • a profile can also be associated to a specific timeframe

Related issues

Related to NethServer 6 - Enhancement #2967: Transparent proxy: switch iplementation from TPROXY to RE... CLOSED

Associated revisions

Revision 908e289d
Added by Giacomo Sanchietti over 6 years ago

BlockIpAccess: default to enabled. Refs #2958

Revision b63e1597
Added by Giacomo Sanchietti over 6 years ago

Web interface: refactor for profile support. Refs #2958

Revision c1b4783b
Added by Giacomo Sanchietti over 6 years ago

Web UI: multiple-profile stub implemention. Refs #2958

Revision 197bed32
Added by Giacomo Sanchietti over 6 years ago

Web interface: refactor for profile support. Refs #2958

Revision 4292ed27
Added by Giacomo Sanchietti over 6 years ago

Web UI: add roles. Refs #2958

Revision 386895c3
Added by Giacomo Sanchietti over 6 years ago

Web UI: rename classes. Refs #2958

Revision 4ae3584f
Added by Giacomo Sanchietti over 6 years ago

Web UI: move BlockIpAccess inside filters. Refs #2958

Revision 623c8aaf
Added by Giacomo Sanchietti over 6 years ago

Web UI: correctly handle comma-separated fields. Refs #1984 #2958

Revision ca976795
Added by Giacomo Sanchietti over 6 years ago

Web UI: cosmetic changes. Refs #2958

Revision a37c5eba
Added by Giacomo Sanchietti over 6 years ago

squidGuard.conf: add support for times and multi profile. Refs #1984 #2958

Revision 1bcaa119
Added by Giacomo Sanchietti over 6 years ago

Web UI: refactor black and white lists. Refs #2958

Revision af425d58
Added by Giacomo Sanchietti over 6 years ago

Web UI: add stronger validator. Refs #2958 #1984

Revision 7ea81df5
Added by Giacomo Sanchietti over 6 years ago

DB defaults: remove old properties. Refs #2958

Revision 5b38d5a3
Added by Giacomo Sanchietti over 6 years ago

Web UI: clean up code. Refs #2958 #1984

Revision 052afe0b
Added by Giacomo Sanchietti over 6 years ago

Web UI: add custom categories. Refs #2958"

Revision 6cb8f117
Added by Giacomo Sanchietti over 6 years ago

scripts and templates: implement custom categories. Refs #2958

Revision 13c5da5b
Added by Giacomo Sanchietti over 6 years ago

Web UI: execute events only when saving profiles. Refs #1984 #2958

Revision 88db61e0
Added by Giacomo Sanchietti over 6 years ago

Web UI: handle categories with empty translation. Refs #2958

Revision 11767a33
Added by Giacomo Sanchietti over 6 years ago

Fix log permission. Refs #2958

Revision 98734363
Added by Giacomo Sanchietti over 6 years ago

custom-list: fix permission. Refs #2958

Revision d34a25c1
Added by Giacomo Sanchietti over 6 years ago

squidGuard.conf template: use getent for group. Refs #2958

Revision 64994e7d
Added by Giacomo Sanchietti over 6 years ago

custom list: fix permission. Refs #2958

Revision 76460525
Added by Giacomo Sanchietti over 6 years ago

squidGuard.conf: remove 302 redirect. Refs #2958

Revision f03dc018
Added by Giacomo Sanchietti over 6 years ago

spec: fix genfile syntax. Refs #2958

Revision 094540df
Added by Giacomo Sanchietti over 6 years ago

squidGuard.conf template: fix syntax. Refs #2958

Revision 7615041c
Added by Giacomo Sanchietti over 6 years ago

migratation fragment: correctly handle multiple runs. Refs #2958

Revision 28c3b3d0
Added by Giacomo Sanchietti over 6 years ago

templates: block and log blacklist per-profile. Refs #2958

Revision 3a1d7aef
Added by Giacomo Sanchietti over 6 years ago

squid.conf: skip url rewriter for localhost. Refs #2958

Revision 82d159f6
Added by Giacomo Sanchietti over 6 years ago

Web UI, db defaults: add default filter and profile. Refs #2958

Revision 9541fa5d
Added by Giacomo Sanchietti over 6 years ago

Web UI: add italian translation. Refs #2958 1984

Revision d098e5e1
Added by Giacomo Sanchietti over 6 years ago

Web UI: always show black and white lists. Refs #2958

Revision ef146c2d
Added by Giacomo Sanchietti over 6 years ago

db: migrate old configuration. Refs #2958

Revision 3daaa68c
Added by Giacomo Sanchietti over 6 years ago

default profile: add missing logic. Refs #2958

Revision 096bb642
Added by Giacomo Sanchietti over 6 years ago

Web UI: default profile can't be edited. Refs #2958

Revision 520ed721
Added by Giacomo Sanchietti over 6 years ago

Web UI: enable BlockIpAccess by default for new filters. Refs #2958

Revision 80ce1e29
Added by Giacomo Sanchietti over 6 years ago

createlinks: apply squidGuard config group-modify Refs #2958

If proxy is in authenticated mode,
squidGuard needs to be notified if a group composition
has been changed.

Revision 28a19ff6
Added by Giacomo Sanchietti over 6 years ago

Web UI: handle recursive caegories. Refs #2958

Revision 5cbea353
Added by Giacomo Sanchietti over 6 years ago

update custom list: avoid unnecessary expand-template. Refs #2958

Revision c2e2a03f
Added by Giacomo Sanchietti over 6 years ago

createlinks: configure custom lists before invoking squidGuard. Refs #2958

Revision e1092fe0
Added by Giacomo Sanchietti over 6 years ago

squidGuard.conf: handle recursive categories. Refs #2958

Revision ff42b5cc
Added by Giacomo Sanchietti over 6 years ago

squidGuard.conf: fix default profile. Refs #2958

Revision 8fceeac4
Added by Giacomo Sanchietti over 6 years ago

Web UI: fire event on delete. Refs #2958 #1984

Revision e8f0a29b
Added by Giacomo Sanchietti over 6 years ago

Web UI: fix whitelist handling. Refs #2958

Revision 6ce3d4f6
Added by Giacomo Sanchietti over 6 years ago

Logorate: rotate urlfilter.log. Refs #2958

Revision d57bd9f0
Added by Giacomo Sanchietti over 6 years ago

Inline help: add English and Italian. Refs #2958

Revision f285a787
Added by Giacomo Sanchietti over 6 years ago

Inline help: add English and Italian. Refs #2958

Revision 06af159f
Added by Giacomo Sanchietti over 6 years ago

Web UI: always show hosts and users in profile tab. Refs #2958

Revision 54b31a71
Added by Giacomo Sanchietti over 6 years ago

Web UI: format profile table. Refs #2958

Revision d0481192
Added by Giacomo Sanchietti over 6 years ago

Web UI: add support for AD users. Refs #2958

Revision 9dadedb1
Added by Giacomo Sanchietti over 6 years ago

config template: avoid warnings. Refs #2958

Revision 53e5aa9d
Added by Giacomo Sanchietti over 6 years ago

config template: allow Active Directory users. Refs #2958

Revision 8d925b0c
Added by Davide Principi over 6 years ago

Enhance Back button behaviour on Tabs widget. Refs #2958

Revision 7c9017ec
Added by Davide Principi over 6 years ago

Push initial URL fragment into browser history. Refs #2958

Revision 14f5ec19
Added by Davide Principi over 6 years ago

Imported NethServer template from nethserver-base package. Refs #2958

The new template is compatible with the Back button fix from Nethgui.

Revision abe4c0f7
Added by Davide Principi over 6 years ago

Merge branch 'master' into v6.6

Import Back button fixes #2958

Conflicts:
SHA1SUM
nethserver-httpd-admin.spec

Revision 338d5945
Added by Giacomo Sanchietti over 6 years ago

squidGuard.conf: whitelist always wins. Refs #2958

Revision 25bdd6cd
Added by Giacomo Sanchietti over 6 years ago

squidGuard.conf: whitelist always wins in profiles. Refs #2958

Revision 7224113f
Added by Giacomo Sanchietti over 6 years ago

squidGuard.conf: whitelist always wins in profiles. Refs #2958

History

#1 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from NEW to TRIAGED
  • Target version set to v6.5
  • % Done changed from 0 to 20

#2 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from TRIAGED to ON_DEV
  • Assignee set to Giacomo Sanchietti
  • % Done changed from 20 to 30

#3 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from ON_DEV to MODIFIED
  • Assignee deleted (Giacomo Sanchietti)
  • % Done changed from 30 to 60

#4 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from MODIFIED to ON_DEV
  • % Done changed from 60 to 30
SquidGuard can be redirect the client to the block page using two different ways:
  • returning back a 302 HTTP code, the client than takes care to do the new request
  • faking the client and loading the block page then returning it back to the client

See this document for more info: http://wiki.squid-cache.org/Features/Redirectors

First scenario has one major drawback: the client must always be allowed to access the block page. This isn't true if the block page has the IP address of the green interface and the client does requests from the blue network. Beside this, if the client can't access sites with IP addresses (squidGuard ip-addr acl), the browser is forced to an infinite loop.
The second scenario addresses above problems, but it doesn't work correctly with TPROXY because part of the TCP connection is made by the proxy itself, so the client can't display the block page.

The best solution is to switch to a transparent proxy implementation based on DNAT and use the second scenario for squidGuard configuration.

#5 Updated by Giacomo Sanchietti over 6 years ago

  • Assignee set to Giacomo Sanchietti

#6 Updated by Giacomo Sanchietti over 6 years ago

  • Related to Enhancement #2967: Transparent proxy: switch iplementation from TPROXY to REDIRECT added

#7 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from ON_DEV to MODIFIED
  • % Done changed from 30 to 60

#8 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from MODIFIED to ON_QA
  • Assignee deleted (Giacomo Sanchietti)
  • % Done changed from 60 to 70
Packages in nethserver-testing:
  • nethserver-squidguard-1.0.5-36.0gitef146c2d.ns6.noarch.rpm
  • nethserver-squid-1.2.0-18.0gitcfbd3944.ns6.noarch.rpm
Test case 1: default profile
  • On a clean enable the proxy in transparent mode (or manual, but remember to configure the client)
  • Configure a domain inside the black list and one inside the white list
  • Check the client can access the one in white list and not the one in black list
  • Check the client can't access sites using the IP address (blocked for default)
Test case 2
  • Create a host with the IP address of the client you will use for tests
  • Create a custom category with some domains
  • Create a new filter and select the new category
  • Select "Block all, allow selected content* option
  • Create a new profile with the new host and the new filter
  • Check the client can browse only sites inside the selected category
Test case 3
  • Create a host with the IP address of the client you will use for tests
  • Create a custom category with some domains
  • Create a new filter and select the new category
  • Select "Allow all, block selected content* option
  • Create a new profile with the new host and the new filter
  • Check the client can't browse sites inside the selected category
Test case 4
  • Repeat test 3 enabling and disabling global white/black lists
Test case 5
  • Configure the proxy as authenticated
  • Repeat tests 2 and 3
  • Remember to edit the profile and select a user in the "Who" field
Test case 6
  • Free your imagination and try obscure and uncommon option combinations

#9 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from ON_QA to TRIAGED
  • % Done changed from 70 to 20

Fragment for default profile in sguidguard configuration is missing.

#10 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from TRIAGED to ON_DEV
  • Assignee set to Giacomo Sanchietti
  • % Done changed from 20 to 30

#11 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from ON_DEV to MODIFIED
  • Assignee deleted (Giacomo Sanchietti)
  • % Done changed from 30 to 60

#12 Updated by Giacomo Sanchietti over 6 years ago

Package in nethserver-testing:
  • nethserver-squidguard-1.0.5-52.0git8fceeac4.ns6.noarch.rpm

#13 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from MODIFIED to ON_QA
  • % Done changed from 60 to 70

#14 Updated by Filippo Carletti over 6 years ago

In squid.conf https port is 3130, but shorewall redirects to 3129.

--- 90squid    2014-12-05 15:49:52.000000000 +0100
+++ /etc/e-smith/templates/etc/shorewall/rules/90squid    2014-12-04 20:02:19.230206865 +0100
@@ -64,7 +64,7 @@
             $OUT.="REDIRECT\tloc$bypass_src_str\t3129\ttcp\t80\t-\t$bypass_dst_str\n";
             if ($green_mode =~ /ssl/) {
                 $OUT .="?COMMENT transparent proxy on green for port 443\n";
-                $OUT.="REDIRECT\tloc$bypass_src_str\t3129\ttcp\t443\t-\t$bypass_dst_str\n";
+                $OUT.="REDIRECT\tloc$bypass_src_str\t3130\ttcp\t443\t-\t$bypass_dst_str\n";
             }

         }
@@ -84,7 +84,7 @@
             $OUT.="REDIRECT\tblue$bypass_src_str\t3129\ttcp\t80\t-\t$bypass_dst_str\n";
             if ($blue_mode =~ /ssl/) {
                 $OUT .="?COMMENT transparent proxy on blue for port 443\n";
-                $OUT.="REDIRECT\tloc$bypass_src_str\t3129\ttcp\t443\t-\t$bypass_dst_str\n";
+                $OUT.="REDIRECT\tloc$bypass_src_str\t3130\ttcp\t443\t-\t$bypass_dst_str\n";
             }
         }
     }

#15 Updated by Filippo Carletti over 6 years ago

The block page always says: unknown category

#16 Updated by Giacomo Sanchietti over 6 years ago

In squid.conf https port is 3130, but shorewall redirects to 3129.

This is not related to this package.
I just added the same note to #2967

#17 Updated by Giacomo Sanchietti over 6 years ago

Filippo Carletti wrote:

The block page always says: unknown category

You should see something like:
Category: unknown blacklist

The first unknown is the category for the source, and blacklist is the name of destination category.

Please can you post the matching lines from /var/log/squidGuard/urlfilter.log?

Maybe we can hide the unknown case.

#18 Updated by Filippo Carletti over 6 years ago

Please can you post the matching lines from /var/log/squidGuard/urlfilter.log?

Nothing is looged.

#19 Updated by Giacomo Sanchietti over 6 years ago

New package in nethserver-testing:
  • nethserver-squidguard-1.0.5-65.0git53e5aa9d.ns6.noarch.rpm
Test case 7
  • Configure the proxy in authenticated mode
  • Install nethserver-samba and join the server to an existing Active Directory
  • Open the Profiles page and check AD users are listed
  • Select a user from AD and and an associated filter
  • Configure a client with the proxy in authenticated mode and try to login with AD user credentials
  • Check the filter is applied

#20 Updated by Davide Principi over 6 years ago

Fixed Back button issues in Tab widget.

Added packages to nethserver-testing (6.5):
nethserver-base-2.5.4-2.0git488f170c.ns6.noarch.rpm
nethserver-httpd-admin-1.3.5-1.2git14f5ec1.ns6.noarch.rpm

#21 Updated by Davide Marini over 6 years ago

  • Status changed from ON_QA to TRIAGED
  • % Done changed from 70 to 20

test case 1 : ok
test case 2 : ok but blocked sites are not logged on urlfilter.log (may be this is the normal behavior in this filtering mode)
test case 3 : ok
test case 4 : global whitelist override doesn't work (can't enable website access using the global whitelist)
test case 5: auth with local users: the authentications work, but I can't choose users when configuring profiles, just hosts and hosts groups

#22 Updated by Giacomo Sanchietti over 6 years ago

  • NEEDINFO changed from No to Yes

test case 2 : ok but blocked sites are not logged on urlfilter.log (may be this is the normal behavior in this filtering mode)

Yes, it is.

test case 4 : global whitelist override doesn't work (can't enable website access using the global whitelist)

What is the configuration? Can you paste the extract from squidGuard.conf and contentfilter db?

With this configuration, if a site is inside a blocked category and inside the whitelist, the site is still blocked. Should the whitelist always win? Even over custom categories?


[root@localhost ~]# db contentfilter show
blocked=category
    Description=
    Domains=www.nethesis.it
default=filter
    BlackList=enabled
    BlockAll=disabled
    BlockFileTypes=disabled
    BlockIpAccess=enabled
    Categories=blocked
    Description=Default filter
    Removable=no
    WhiteList=enabled
default_profile=profile
    Description=Default profile
    Filter=filter;default
    Removable=no

Extract from squidGuard.conf:

    default {
        pass !blocked  !in-addr  whitelist  !blacklist  all
        redirect     http://192.168.5.246/cgi-bin/nethserver-block.cgi?clientaddr=%a&clientname=%n&clientident=%i&srcclass=%s&targetgroup=%t&url=%u
    }

test case 5: auth with local users: the authentications work, but I can't choose users when configuring profiles, just hosts and hosts groups

I can't reproduce the problem. Have you used the latest package? Current web interface always displays hosts and users, no matter how the proxy is configured.

#23 Updated by Davide Principi over 6 years ago

  • Status changed from TRIAGED to MODIFIED
  • % Done changed from 20 to 60

#24 Updated by Davide Marini over 6 years ago

What is the configuration? Can you paste the extract from squidGuard.conf and contentfilter db?

[root@server ~]# db contentfilter show 
default=filter
    BlackList=enabled
    BlockAll=disabled
    BlockFileTypes=disabled
    BlockIpAccess=disabled
    Categories=test_category
    Description=Default filter
    Removable=no
    WhiteList=enabled
default_profile=profile
    Description=Default profile
    Filter=filter;default
    Removable=no
test_category=category
    Description=fsddf
    Domains=libero.it,repubblica.it

squidguard.conf:

.
.
.
    default {
        pass !test_category  whitelist  !blacklist  all
        redirect     http://192.168.56.115/cgi-bin/nethserver-block.cgi?clientaddr=%a&clientname=%n&clientident=%i&srcclass=%s&targetgroup=%t&url=%u
    }

With this configuration, if a site is inside a blocked category and inside the whitelist, the site is still blocked. Should the whitelist always win? Even over custom categories?

I think the most intuitive and easy to use behavior is : the global whitelist always win, also over custom categories and global blacklist

I can't reproduce the problem. Have you used the latest package? Current web interface always displays hosts and users, no matter how the proxy is configured.

I don't know how but I didn't have the latest package, after upgraded now I can see the users and hosts on dropdown menu.

#25 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from MODIFIED to ON_QA
  • % Done changed from 60 to 70

Implementation changed: now whitelist has priority over blacklist and custom categories.

Package in nethserver-testing:
  • nethserver-squidguard-1.0.5-66.0git338d5945.ns6.noarch.rpm

Please, repeat test case 4.

#26 Updated by Giovanni Bezicheri over 6 years ago

  • Assignee set to Giovanni Bezicheri

#27 Updated by Giovanni Bezicheri over 6 years ago

  • Status changed from ON_QA to TRIAGED
  • Assignee deleted (Giovanni Bezicheri)
  • % Done changed from 70 to 20

#28 Updated by Giovanni Bezicheri over 6 years ago

The web content filter does not block domains specified in custom categories for custom filters (not default).

#29 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from TRIAGED to ON_DEV
  • Assignee set to Giacomo Sanchietti
  • % Done changed from 20 to 30

#30 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from ON_DEV to MODIFIED
  • % Done changed from 30 to 60

The problem should now be fixed.

#31 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from MODIFIED to ON_QA
  • Assignee deleted (Giacomo Sanchietti)
  • % Done changed from 60 to 70
Package in nethserver-testing:
  • nethserver-squidguard-1.0.5-67.0git7224113f.ns6.noarch.rpm

Again, repeat test case 4.

#32 Updated by Giovanni Bezicheri over 6 years ago

  • Assignee set to Giovanni Bezicheri

#33 Updated by Giovanni Bezicheri over 6 years ago

  • Status changed from ON_QA to VERIFIED
  • Assignee deleted (Giovanni Bezicheri)
  • % Done changed from 70 to 90

#34 Updated by Giacomo Sanchietti over 6 years ago

  • Status changed from VERIFIED to CLOSED
  • % Done changed from 90 to 100
Released in nethserver-updates:
  • nethserver-squidguard-1.1.0-1.ns6.noarch.rpm

Also available in: Atom PDF