Anti Spam Email server

Anti Spam Email server

In this post Ill show you how to install an anti-spam smart host relay server, based on Ubuntu 12.04 LTS, that will include:

Postfix w/Bayesian Filtering and Anti-Backscatter (Relay Recipients via look-ahead), Apache2, Mysql, Dnsmasq, MailScanner (Spamassassin, ClamAV, Pyzor, Razor, DCC-Client), Baruwa, SPF Checks, FuzzyOcr, Sanesecurity Signatures, PostGrey, KAM, Scamnailer, FireHOL (Iptables Firewall) and Relay Recipients Script.

Continue reading for the instructions.

Install main packages:

First we need to add Baruwas repo to /etc/apt/sources.list

deb http://apt.baruwa.org/ubuntu precise main

To use this repo add the gpg key:

wget -O - http://apt.baruwa.org/baruwa-apt-keys.gpg | apt-key add -

Now well install some dependencies:

apt-get install binutils cpp fetchmail flex gcc libarchive-zip-perl libc6-dev libcompress-raw-zlib-perl libcompress-bzip2-perl libpcre3 libpopt-dev lynx m4 make ncftp nmap openssl perl perl-modules unzip zip zlib1g-dev autoconf automake1.9 libtool bison autotools-dev g++ build-essential telnet wget gawk

Use bash as default shell:

dpkg-reconfigure dash

Caching Dnsmasq:

apt-get install dnsmasqvi /etc/dnsmasq.conf

and make Dnsmasq listen on localhost:

listen-address=127.0.0.1

Install Mysql:

apt-get install mysql-client mysql-server libdbd-mysql-perl

Install and configure Postfix:

apt-get install postfix postfix-mysql postfix-doc procmail

You will be asked two questions. Answer as follows:

General type of mail configuration: --> Internet Site
System mail name: --> server1.example.com

Stop Postfix

postfix stop

We�ll want to edit Postfix with the below:

vi master.cf

We need to add two items below the pickup service type. The pickup service "picks up" local mail (local meaning "on this machine") and delivers it. This is a way to bypass content filtering for mail generated by this machine.

It should look like this when you are done:

pickup fifo n - - 60 1 pickup -o content_filter= -o receive_override_options=no_header_body_checks

Edit main.cf with a script:

vi /usr/src/postfix.sh

#!/bin/sh
postconf -e "alias_maps = hash:/etc/aliases"
newaliasespostconf -e "myorigin = domain.tld"postconf -e "myhostname = server1.domain.tld"postconf -e "mynetworks = 127.0.0.0/8, 192.168.0.0/24"postconf -e "message_size_limit = 10485760"
postconf -e "local_transport = error:No local mail delivery"
postconf -e "mydestination = "
postconf -e "local_recipient_maps = "
postconf -e "relay_domains = mysql:/etc/postfix/mysql-relay_domains.cf"
postconf -e "relay_recipient_maps = mysql:/etc/postfix/mysql-relay_recipients.cf"
postconf -e "transport_maps = mysql:/etc/postfix/mysql-transports.cf"
postconf -e "virtual_alias_maps = hash:/etc/postfix/virtual"
postconf -e "disable_vrfy_command = yes"
postconf -e "strict_rfc821_envelopes = no"
postconf -e "smtpd_banner = $myhostname ESMTP SpamSnake"
postconf -e "smtpd_delay_reject = yes"
postconf -e "smtpd_recipient_limit = 100"
postconf -e "smtpd_helo_required = yes"
postconf -e "smtpd_client_restrictions = permit_sasl_authenticated, permit_mynetworks, permit"
postconf -e "smtpd_helo_restrictions = permit_sasl_authenticated, permit_mynetworks, permit"
postconf -e "smtpd_sender_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_non_fqdn_sender, reject_unknown_sender_domain, permit"
postconf -e "smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unknown_recipient_domain, reject_unauth_destination, whitelist_policy, grey_policy, rbl_policy, spf_policy, permit"
postconf -e "smtpd_data_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_pipelining"
postconf -e "smtpd_restriction_classes = spf_policy, rbl_policy, grey_policy, whitelist_policy"
postconf -e "spf_policy = check_policy_service unix:private/policy"
postconf -e "rbl_policy = reject_rbl_client zen.spamhaus.org, reject_rbl_client bl.spamcop.net"
postconf -e "grey_policy = check_policy_service unix:private/greyfix"
postconf -e "whitelist_policy = check_client_access mysql:/etc/postfix/mysql-global_whitelist.cf, check_sender_access mysql:/etc/postfix/mysql-global_whitelist.cf"
postconf -e "header_checks = regexp:/etc/postfix/header_checks"
touch /etc/postfix/virtual
echo "root administrator@example.com" >> /etc/postfix/virtual && echo "abuse administrator@example.com" >> /etc/postfix/virtual && echo "postmaster administrator@example.com" >> /etc/postfix/virtual
postmap /etc/postfix/virtual
touch /etc/postfix/header_checks
echo "/^Received:/ HOLD" >> /etc/postfix/header_checks
postmap /etc/postfix/header_checks
cat > /etc/postfix/mysql-global_whitelist.cf <<EOF
#mysql-global_whitelist
user = baruwa
password = password
dbname = baruwa
query = select concat(PERMIT) action from lists where from_address=%s AND list_type=1;
hosts = 127.0.0.1
EOF
cat > /etc/postfix/mysql-relay_domains.cf <<EOF
#mysql-relay_domains
user = baruwa
password = password
dbname = baruwa
query = select concat(address, , OK) domain from user_addresses where user_addresses.address=%s and user_addresses.enabled=1;
hosts = 127.0.0.1
EOF
cat > /etc/postfix/mysql-relay_recipients.cf <<EOF
#mysql-relay_recipients
user = baruwa
password = password
dbname = baruwa
query = select concat(@, address, OK) email from user_addresses where user_addresses.address=%d;
hosts = 127.0.0.1
EOF
cat > /etc/postfix/mysql-transports.cf <<EOF
#mysql-transports
user = baruwa
password = password
dbname = baruwa
query = select concat(smtp:[, mail_hosts.address, ], :, port) transport from mail_hosts, user_addresses where user_addresses.address = %s AND user_addresses.id = mail_hosts.useraddress_id;
hosts = 127.0.0.1
EOF

Now make it executable:

chmod +x /usr/src/postfix.sh

and run it using

/usr/src/postfix.sh

Postfix Recipient Callout:

This feature queries the recipient server to see if the recipient exists. If not, it replies with a 550 error to the sending server and drops the connection. If the user does exist, the SpamSnake will continue processing the email. This is just another method to prevent backscatter, but comes at a price. Read up on it at http://www.postfix.org/ADDRESS_VERIFICATION_README.html. You can skip this method and use the script method (later on in this guide) if you decide it will bog down your server.

vi /etc/postfix/main.cf

and add the following:

verify_recipient = reject_unknown_recipient_domain, reject_unverified_recipientlook_ahead = check_recipient_access hash:/etc/postfix/accessunverified_recipient_reject_code = 550address_verify_map = btree:/var/lib/postfix/verify

Add this to your smtpd_restriction_classes:

verify_recipient, look_ahead

Add this to smptd_recipient_restrictions:

smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, look_ahead, whitelist_policy, grey_policy, rbl_policy, spf_policy, permit

Create the access file:

touch /etc/postfix/access

and add your domains:

domainA.com verify_recipientdomainB.com verify_recipient

Note: Make sure to add valid domains youre filtering for.

Postmap it:

postmap /etc/postfix/access

Final look at the Postfix install:

less /etc/postfix/main.cf

Check the contents of the file for errors and repair if needed. Fire up Postfix:

Start Postfix

postfix start

Check that Postfix responds:

telnet 127.0.0.1 25

You should see:

220 [yourFQDNhere] ESMTP Postfix (Ubuntu)

Install MailScanner (Apparmor, Clamav, DCC, Pyzor, Razor and Spamassassin):

aptitude install razor pyzor clamav-daemon libclamav6 apparmor mailscanner libltdl7

Apparmor configuration for Clamav

Add clamav to the www-data group so that it can access the directory:

usermod -a -G www-data clamav

Now edit the profile for clamd:

vi /etc/apparmor.d/usr.sbin.clamd

and add the Incoming folder to the list of folders:

/usr/sbin/clamd {
#clamav
/var/spool/MailScanner/** rw,
/var/spool/MailScanner/incoming/** rw,
}

Reload apparmor:

/etc/init.d/apparmor reload

DCC 32bit/64bit Configuration:

Install DCC from .deb source, you can just download the necessary packages:

cd /tmp
wget http://ppa.launchpad.net/jonasped/ppa/ubuntu/pool/main/d/dcc/dcc-common_1.3.130-0ubuntu1~ppa2~precise1_$(uname -m | sed -e s/x86_64/amd64/ -e s/i686/i386/).deb && dpkg -i dcc-common_1.3.130-0ubuntu1~ppa2~precise1_$(uname -m | sed -e s/x86_64/amd64/ -e s/i686/i386/).deb
wget http://ppa.launchpad.net/jonasped/ppa/ubuntu/pool/main/d/dcc/dcc-client_1.3.130-0ubuntu1~ppa2~precise1_$(uname -m | sed -e s/x86_64/amd64/ -e s/i686/i386/).deb && dpkg -i dcc-client_1.3.130-0ubuntu1~ppa2~precise1_$(uname -m | sed -e s/x86_64/amd64/ -e s/i686/i386/).deb

Or you can add the PPA and install:

add-apt-repository ppa:jonasped/ppaaptitude updateaptitude install  dcc-client dcc-common

Test our installation with:

cdcc info

You should get requests ok from the servers.

Pyzor Configuration
Because pyzor doesn�t work with python2.6 very well, the workaround is to append the following to the first line of /usr/bin/pyzor to make it look like:

#!/usr/bin/python -Wignore::DeprecationWarning

Here we supply the IP address of the Pyzor server to Pyzor. This will create the servers IP address in a servers file therein. Then it will test the connection. If you are behind a firewall, open port 24441/udp in and out to your server. While youre at it also open up 6277/udp for DCC, 2703/tcp for Razor and 783/tcp for SpamAssassin:

mkdir /var/lib/MailScannerpyzor --homedir=/var/lib/MailScanner discover
pyzor ping

Razor Configuration

Create the .razor configuration:

cd && rm /etc/razor/razor-agent.confmkdir /var/lib/MailScanner/.razorrazor-admin -home=/var/lib/MailScanner/.razor -createrazor-admin -home=/var/lib/MailScanner/.razor -discoverrazor-admin -home=/var/lib/MailScanner/.razor -register

Edit the razor agent conf.

vi /var/lib/MailScanner/.razor/razor-agent.conf

And set:

debuglevel = 0razorhome = /var/lib/MailScanner/.razor/

Install dependencies:

apt-get install libconvert-tnef-perl libdbd-sqlite3-perl libfilesys-df-perl libmailtools-perl libmime-tools-perl libmime-perl libnet-cidr-perl libsys-syslog-perl libio-stringy-perl libfile-temp-perl libole-storage-lite-perl libarchive-zip-perl libsys-hostname-long-perl libnet-cidr-lite-perl libhtml-parser-perl libdb-file-lock-perl libnet-dns-perl libncurses5-dev libdigest-hmac-perl libdigest-sha-perl libnet-ip-perl liburi-perl libfile-spec-perl spamassassin libnet-ident-perl libmail-spf-perl libmail-dkim-perl dnsutils libio-socket-ssl-perl gdebi-core

Edit Crontab:

vi /etc/crontab

And add:

#Mailscanner
37 5 * * * /usr/sbin/update_phishing_sites &> /dev/null
07 * * * * /usr/sbin/update_bad_phishing_sites &> /dev/null
#58 23 * * * /usr/sbin/clean.quarantine &> /dev/null
42 * * * * /usr/sbin/update_virus_scanners &> /dev/null
3,23,43 * * * * /usr/sbin/check_mailscanner &> /dev/null

Install SpamAssassin

First we need to disable the default SpamAssassin configuration file:

mv /etc/spamassassin/local.cf/etc/spamassassin/local.cf.disabled

Now lets backup the SpamAssassin configuration file in MailScanner then edit:

cp /etc/MailScanner/spam.assassin.prefs.conf /etc/MailScanner/spam.assassin.prefs.conf.back

SpamAssassin SQL Bayes
Pre-requisities: Youll need the perl-DBI and perl-DBD-MySQL modules installed.

Assumptions and Variables:

SpamAssassin Bayes Database Name: sa_bayesSpamAssassin Bayes Database UserName: sa_userSpamAssassin Bayes Database Password: sa_password

Create the MySQL database on the server where you intend on storing the bayesian information.

mysql -u root -pmysql> create database sa_bayes;mysql> GRANT ALL ON sa_bayes.* TO sa_user@localhost IDENTIFIED BY sa_password;mysql> flush privileges;

Import database structure:

mysql -u sa_user -p sa_bayes < /usr/share/doc/spamassassin/sql/bayes_mysql.sql

To enable DCC:

vi /etc/spamassassin/v310.pre

And add:

loadplugin Mail::SpamAssassin::Plugin::DCC

Edit Spam Assassins conf file:

vi /opt/MailScanner/etc/spam.assassin.prefs.conf

and add the following to the top:

#pyzoruse_pyzor 1pyzor_options --homedir /var/lib/MailScanner/
#razoruse_razor2 1razor_config /var/lib/MailScanner/.razor/razor-agent.confFix DCC path:
dcc_path /usr/bin/dccproc

Update header string:

bayes_ignore_header X-YOURDOMAIN-COM-MailScannerbayes_ignore_header X-YOURDOMAIN-COM-MailScanner-SpamCheckbayes_ignore_header X-YOURDOMAIN-COM-MailScanner-SpamScorebayes_ignore_header X-YOURDOMAIN-COM-MailScanner-Information#use_auto_whitelist 0

"YOURDOMAIN-COM" should be replaced with whatever you used for "%org-name%" in the MailScanner.conf file. Leave the "X-" in place. This is the same orgname used in the MailScanner.conf above.

Add sql connection string to the bottom:

bayes_store_module Mail::SpamAssassin::BayesStore::SQLbayes_sql_dsn DBI:mysql:sa_bayes:localhostbayes_sql_username sa_userbayes_sql_password sa_passwordbayes_sql_override_username rootvi v310.pre

and comment out domainkeys since DKIM has superseeded it:

#loadplugin Mail::SpamAssassin::Plugin::DomainKeys

Add it to cron:

30 01 * * * /usr/bin/sa-learn --force-expire --sync -p /etc/MailScanner/spam.assassin.prefs.conf

Install Apache and mod-passenger:

aptitude install apache2 libapache2-mod-passenger

Create the following to prevent an error in a lint test:

mkdir /var/www/.spamassassin

Install missing perl packages:

aptitude install libencode-detect-perl libcrypt-openssl-rsa-perlperl -MCPAN -e shellinstall IP::Country::Fastexit

Set permissions to bring it all together:

chown -R postfix:www-data /var/spool/postfix/holdchmod -R ug+rwx /var/spool/postfix/hold

Test out the setup:

spamassassin -x -D -p /opt/MailScanner/etc/spam.assassin.prefs.conf --lint

Check for lines like:

debug: bayes: Database connection establisheddebug: bayes: found bayes db version 3debug: bayes: Using userid: 2

You should see lines come up with DCC, Pyzor and Razor that say loading plugin and hopefully no errors.

MailScanner Configuration:

We need to make a directory for SpamAssassin in the spool and give postfix permissions to it, if you run sa-learn --force as root, bayes databese that is stored in these directories will change to root:root and spamassassin will error looking at the db. Just keep an eye on the mail.log and youll remember to change the permissions back. Also disable the MailScanner default configs:

mkdir /var/spool/MailScanner/spamassassin

Backup your MailScanner.conf file:

cp /etc/MailScanner/MailScanner.conf /etc/MailScanner/MailScanner.conf.dist

Change the following parameters in MailScanner.conf with the following script:

vi /usr/src/mailscanner.sh

Add:

sed -i "/^%org-name% =/ c%org-name% =orgname" /etc/MailScanner/MailScanner.conf
sed -i "/^%org-long-name% =/ c%org-long-name% = longorgname" /etc/MailScanner/MailScanner.conf
sed -i "/^%web-site% =/ c%web-site% = www.domain.tld" /etc/MailScanner/MailScanner.conf
sed -i "/^Run As User =/ cRun As User = postfix" /etc/MailScanner/MailScanner.conf
sed -i "/^Run As Group =/ cRun As Group = www-data" /etc/MailScanner/MailScanner.conf
sed -i "/^Incoming Work Group =/ cIncoming Work Group = clamav" /etc/MailScanner/MailScanner.conf
sed -i "/^Incoming Work Permissions =/ cIncoming Work Permissions = 0640" /etc/MailScanner/MailScanner.conf
sed -i "/^Incoming Queue Dir =/ cIncoming Queue Dir = /var/spool/postfix/hold" /etc/MailScanner/MailScanner.conf
sed -i "/^Outgoing Queue Dir =/ cOutgoing Queue Dir = /var/spool/postfix/incoming" /etc/MailScanner/MailScanner.conf
sed -i "/^MTA =/ cMTA = postfix" /etc/MailScanner/MailScanner.conf
sed -i "/^Quarantine User =/ cQuarantine User = root" /etc/MailScanner/MailScanner.conf
sed -i "/^Quarantine Group =/ cQuarantine Group = www-data" /etc/MailScanner/MailScanner.conf
sed -i "/^Quarantine Permissions =/ cQuarantine Permissions = 0660" /etc/MailScanner/MailScanner.conf
sed -i "/^Quarantine Whole Message =/ cQuarantine Whole Message = yes" /etc/MailScanner/MailScanner.conf
sed -i "/^Virus Scanners =/ cVirus Scanners = clamd" /etc/MailScanner/MailScanner.conf
sed -i "/^Monitors for ClamAV Updates =/ cMonitors for ClamAV Updates = /var/lib/clamav/*.cld /var/lib/clamav/*.cvd" /etc/MailScanner/MailScanner.conf
sed -i "/^Clamd Socket =/ cClamd Socket = /var/run/clamav/clamd.ctl" /etc/MailScanner/MailScanner.conf
sed -i "/^Clamd Lock File =/ cClamd Lock File = /var/run/clamav/clamd.pid" /etc/MailScanner/MailScanner.conf
sed -i "/^Spam Subject Text =/ cSpam Subject Text = ***SPAM***" /etc/MailScanner/MailScanner.conf
sed -i "/^Spam Actions =/ cSpam Actions = deliver store" /etc/MailScanner/MailScanner.conf
sed -i "/^High Scoring Spam Actions =/ cHigh Scoring Spam Actions = store delete" /etc/MailScanner/MailScanner.conf
sed -i "/^Non Spam Actions =/ cNon Spam Actions = deliver store" /etc/MailScanner/MailScanner.conf
sed -i "/^SpamAssassin User State Dir =/ cSpamAssassin User State Dir = /var/spool/MailScanner/spamassassin" /etc/MailScanner/MailScanner.conf

Make the script executable:

chmod +x mailscanner.sh

and run using

/usr/src/mailscanner.sh

Edit /etc/default/mailscanner. There you will have to set the variable run_mailscanner to 1.

Install and configure Saslauthd:

aptitude install sasl2-bin libsasl2-modules-ldap

edit the configuration:

vi /etc/default/saslauthd

change:

MECHANISMS="pam"

to:

MECHANISMS="rimap"

Set MECH_OPTIONS to the ip address of your imap server

And at the bottom change:

OPTIONS="-c -m /var/run/saslauthd"

to:

OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd -r"

Set the correct permissions:

chown postfix /var/spool/postfix/var/run/saslauthdchgrp sasl /var/spool/postfix/var/run/saslauthdchmod 710 /var/spool/postfix/var/run/saslauthd

Start the system:

/etc/init.d/saslauthd start
/etc/init.d/mailscanner start
/etc/init.d/postfix start

Check your logs for errors:

tail -f /var/log/mail.log

Check your mail.log (tail �f /var/log/mail.log) and you should see the following:

Jun 13 12:18:23 hoshi MailScanner[26388]: MailScanner E-Mail Virus Scanner version 4.81.4 starting...

Install Baruwa:

First you need to install rabbitmq-server

aptitude install rabbitmq-server

And configure it:

rabbitmqctl add_user baruwa passwordrabbitmqctl add_vhost baruwarabbitmqctl set_permissions -p baruwa baruwa ".*" ".*" ".*"/etc/init.d/rabbitmq-server restart

Now install Baruwa:

aptitude install baruwa

Youll be prompted to setup the baruwa db for logging, including an access username and password, which will be used by the Baruwa scripts. Youll also be prompted to setup the admin user information for the Baruwa frontend.

Edit Baruwas setings:

vi /usr/share/pyshared/baruwa/settings.py

Change the Quarantine hosturl to your liking:

QUARANTINE_REPORT_HOSTURL = http://baruwa-alpha.local

Setup the Baruwa DB:

baruwa-admin syncdb --noinputbaruwa-admin migratebaruwa-admin syncdb --noinputbaruwa-admin migrate

Set MailScanner to use Baruwas DB:

vi /etc/MailScanner/MailScanner.conf

You need to get