Notes on Setting up Synonym for Email Archiving


    This page contains some notes on how to setup a computer to archive all incoming and outgoing email for a domain. These notes were made using Slackware Linux 12.0 as the O.S. on x86 architeture. Sendmail 8.14.2 was used as the SMTP mail transfer agent. There is nothing special about that, the part that allowed the email archiving was Synonym. We used a separate box for the archiving, we didn't do it on our mail server itself. You should be able to do it all on one server, the process would be just slightly different (and actually it should be even easier). This page assumes the mail archiving is done on a different server than the mail server, though, and that the mail server already exists and is there running, etc.

    We first downloaded Synonym from here:
http://dv8.ro/Synonym/Synonym/synonym_downloads.html
We used synonym-0.3-2.tar.gz , which it had listed as the latest stable release. We un-tar-ed the tarball, cd-ed into the directory it made, and did: 
make
make install


    After that, We did this: 
cd /etc/rc.d/rc.init.d
mv synonym ../rc.synonym

Then we vi-ed /etc/rc.d/rc.local and put this at the bottom of the file: 
# start synonym, milter allowing mail archiving:
/etc/rc.d/rc.synonym start

NOTE: You may not need to do that step, depending on which version of Linux you are using. The SysV-ish *nixes should be fine without doing this step. It's the BSD-ish *nixes that may need this done, I believe it has to do with the differences in the start up scripts between System V and BSD. (the synonym tarball assumes SysV Init, I believe, and some BSD init boxes may need the minor modifications above so synonym automagically starts up).

    Next, move the default synonym config file out of the way and create your own. We did it this way: 
cd /etc
mv synonym.conf synonym.conf.orig
vi synonym.conf

When we vi-ed (edited) the file, we put this into it: 

<Rules>
        <Rule>
                <Condition>
                        <EnvFrom>.*<EnvFrom>
                <Condition>
                <Action>
                        <ActionType>Copy<ActionType>
                        <Address>mailarchive<Address>
                <Action>
        <Rule>
<Rules>


That config file should copy all emails (the Condition section being a .* matches everyting as a wildcard and the ActionType being Copy tells it to copy them all). The Address section tells synonym to copy them to the local email address "mailarchive". So whatever you use for your email address there needs to be created and working as an email address on this box. All mail will be "archived" to the Inbox (usually /var/spool/mail/mailarchive, for example) for that user. It is likely we created it by just using "adduser". I don't recall for sure now how we did that step. This page is about synonym, though, not how to add email addresses in Linux with Sendmail... Let me attempt to return to topic.

    Once that is done, you will need to change the sendmail configuration to use Synonym as a milter (mail filter). We used m4, if I recall. Unfortunately, we do not have this step documented well. m4 dislikes me, and tends to frighten and confuse me. I generally long for the days when one could vi one's /etc/sendmail.cf and not be berated and considered unenlightened for not using m4. I think there was a lot of muddling through m4 in this step for us.
    Update: I searched around a bit after I originally wrote this page, and it seems like someone else already documented how to do this! Cool. They have it documented here:
http://www.geocities.com/sbmilbur/sendmail/email_archiving.html
    Actually. You may just want to head on over to that site and use their instructions. They look more detailed than mine. Up to you. They do have the m4 part documented pretty well there, though.

Shane Milburn says: 
"
Listing 1.
1 VERSIONID(`msgidruleset.m4')
2 VERSIONID(`Copyright 1998, Robert Harker, Harker Systems')
3 VERSIONID(`www.harker.com, info@harker.com, 408-295-6239')
4 VERSIONID(`Permission granted to user this as long as this')
5 VERSIONID(`VERSIONID information is preserved in the M4 macro file')
6 VERSIONID(`and any sendmail.cf files created using thie M4 macro file')
7 VERSIONID(`Modified to handle logging local mail as well as external mail')
8 VERSIONID(`Shane B. Milburn, milburn@netcom.com, 04/19/1999'')
9 ifdef(`_MAILER_smtp_',,
10 `errprint(`*** MAILER(smtp) must appear before copymail mailer')')dnl
11
12 LOCAL_CONFIG
13 CPNOCOPY
14
15 LOCAL_NET_CONFIG
16
17 LOCAL_RULE_0
18 R$+.NOCOPY. $#local @ $: $1
19 R$+<@$j.NOCOPY.> $#local @ $: $1
20 R$+<@$+.NOCOPY.> $#esmtp $@$2 $:$1<@$2.>
21 R$+<@$+.> $#copymail $@nohostneeded $:$1<@$2.NOCOPY>
22 R$+<@$+> $#copymail $@nohostneeded $:$1<@$2.NOCOPY>
23 R$+ $#copymail $@nohostneeded $:$1<@$j.NOCOPY>
24
25 MAILER_DEFINITIONS
26 # Copy a message by sending it back to sendmail with an
27 # additional adress: copyuser
28 Mcopymail, P=/usr/lib/sendmail, F=mDFMuX, S=11/31, R=21, E=\r\n, L=990,
29 T=DNS/RFC822/X-Unix,
30 A=/usr/lib/sendmail -oi copyuser@$j.NOCOPY $u

Here's a description of what mgsidruleset.m4 does. The first eight lines will
insert the text between the quotes into your sendmail.cf file as comments.
Lines 25 through 30 define a new mailer named "copymail". This definition
re-invokes sendmail with the original reciepients ($u) and an additional
recipient named 'copyuser@$j.NOCOPY'. While this does cause additional load on
the server, in my case it was not enough of a load to cause concern. Now let's
look at the rest of the m4 file. Lines 9-10 will print an error message if in
your site-config.mc file you try to declare "copymail" before "smtp". The next
two lines tell sendmail to declare a local class. The LOCAL_NET_CONFIG line
forwards non-local network stuff to SMART_HOST. The LOCAL_RULE_0 statement is
used to introduce new parsing rules. This is where you place any custom
delivery agents and parsing rules you have defined. In Listing 1, the custom
parsing rules are lines 18 through 23.

Installation:

In order to install this, you need to place listing 1 into a file called
msgidruleset.m4 in sendmail-8.9.3/cf/feature/ directory. Don't forget that
there are tabs between the first and second entry on lines 18-23 (if you do,
sendmail will remind you.) Now add the following line to your site-config.mc
file.

FEATURE(msgidruleset) 

Here is what my site-config.mc file looks like.
 VERSIONID(`@(#)mcst-config.mc  Shane B. Milburn 04/21/1999')
 VERSIONID(`@(#)This configuration logs ALL email to copyuser.')
 OSTYPE(solaris2)
 FEATURE(use_cw_file)dnl
 FEATURE(relay_entire_domain)dnl
 FEATURE(always_add_domain)
 FEATURE(rbl)
 MAILER(smtp)
 MAILER(local)
 FEATURE(msgidruleset)


After you create your site-config.mc file, use the m4 program to generate your
sendmail.cf file. In /usr/local/src/sendmail-8.9.3/cf/cf you would use
"m4 ../m4/cf.m4 site-config.mc > sendmail.cf". This would create a sendmail.cf
in the cf directory. You can either move this file into /etc/mail/ "

Thank you Mr. Milburn!


I believe what m4 did differently for us in our sendmail.cf file, was:

Add this line: 
O InputMailFilters=Synonym

I think there was an empty InputMailFilters= line there already, m4 changed it.

It also added this right below that: 
O Milter.macros.connect=j, _, {daemon_name}, {if_name}, {if_addr}
O Milter.macros.helo={tls_version}, {cipher}, {cipher_bits}, {cert_subject}, {cert_issuer}
O Milter.macros.envfrom=i, {auth_type}, {auth_authen}, {auth_ssf}, {auth_author}, {mail_mailer}, {mail_host}, {mail_addr}
O Milter.macros.envrcpt={rcpt_mailer}, {rcpt_host}, {rcpt_addr}
O Milter.macros.eom={msg_id}


Then, way down after the re-writing rules, in "MAIL FILTER DEFINITION", it added: 
XSynonym, S=local:/var/run/synonym/synonym.sock, T=C:10m;S:1s;R:1s;E:5m


In the AddDomain section, it seems to have changed: 
               A=procmail -t -Y -a $h -d $u

to: 
               A=procmail -Y -a $h -d $u


On a related note, I believe we also had to masquerade as the domain we wanted to archive the email for, so we had a line like this: 
DMyour.domain.here

Replace your.domain.here with the domian you are archiving for.

    There are also other Sendmail changes that need to be made. We edited /etc/mail/relay-domains and put in the domains involved. I believe this includes the fully qualified host names of your other mail servers that are involved. Such as something like
mail.your.domain.com
    Also edit /etc/mail/mailertable. Since you will be having your DNS MX records (hope to document that, or at least mention it, below) point to the archive box, you'll need to override those MX records to get the archive box to send the email to the proper place, your other/existing mail server. (I believe you could use the mail archiving box itself as the mail server too, and in that case this step should not be needed). Anyway, edit mailertable and put in something like: 
your.domain.com		smpt:[mail.your.domain.com]
mail.your.domain.com		smpt:[mail.your.domain.com]

    Whenever you edit your mailertable, you then need to (I think) do something like this: 
makemap hash mailertable.db < mailertable

and perhaps HUP or restart sendmail before the changes take effect. I believe you need to HUP sendmail after you edit relay-domains as well (or want sendmail to re-read any changes you've made).

    In order to get the mail to go through the archiver box so that it can get archived, you need to adjust your MX records in DNS for the domain you are archiving for. Basically, instead of the MX records pointing at your mail server, they will need to point at your mail archiver, and your mail archiver box will know to send them on to the mail server from it's mailertable entries. That gets "incoming" email archived.
    As for outgoing email, you need to make sure the only outgoing SMTP server any mail clients or servers are using is the archiver as well. The exact deails on that depend on the mail programs and servers in place.     For example, on a squirrelmail box, you might 
cd /var/www/docs/mail/config
./conf.pl
 2.  server settings
  3. sendmail or SMTP     change to SMTP
     B.  update SMTP settings
            set to IP address of archiver box, colon 25.   ie. 192.168.1.81:25

    Of course, this document doesn't include details such as firewalling rules that will also need to be in place to allow this to work.

    All this email gets archived to one inbox. Depending on the situation, that could add up quickly. You should be able to rotate the inbox every so often. Stop sendmail, rename the inbox, create/chown/chmod a new inbox, start sendmail up again. Maybe something like that. I believe we created a script to rotate the inbox once a day right before midnight and rename it to the date, and put it into a subfolder named the number of the month inside a folder named the year, creating the folders as needed, inside the mailarchive user's home directory. We run it from cron. I think logrotate could also be used.
(more details on this)
    You'll also need some sort of mail client (or some other way) if you ever want to retrieve certain emails from the archive. In our case, we installed squirrelmail onto the archiver box, and we log into squirrelmail with a standard web browser and add in the mail folders needed and use the search to find specific messages. That uses imapd, so really any email client could be setup to speak imap to the archiver and do the same thing. There are also other ways of getting your messages... Whatever you are comfortable with. (more details on this)

We hope this is useful to someone sometime.



spp 2008-04-23