Page 1 of 1

Understanding how bounce management works

Posted: Thu Aug 24, 2006 2:15 pm
by torcolato
Hi all!

We installed OpenEMM on a FreeBSD box and everything seems to work well.
I will post setup instructions when we are sure its perfect.

One question remains right now:

How should bounce management work?

We set up a test list with 2 real recipients and 2 non-existing addresses.

The statistic page of the mailing reports no bounces after having sent the mailing.

What are we doing wrong?

Thanks,
Christian

Posted: Fri Aug 25, 2006 1:39 am
by minichip
Hi torcolato,

I am running OpnenEMM on a debian system and as far as I understand the system there are at least two ways how OpenEMM identifies bounce emails.

The first one is through maillog. If the foreign email server rejects the email during the smtp session an error is log in the maillog file and OpenEMM checks for it and identifies it.

The second way (which is currently not working for me) is through the bav.sock entry in sendmail. Every email is passed through this mail filter and I guess OpenEMM should identify bounce emails. The emails are matched, or at least should be matched, against against the files in openemm-bse/conf/bav/ directory.

That is how far I understand the system, more to follow if I find time at the weekend to investigate further.


One question to the admins here, is the bav.sock feature working correctly in OpenEMM 5.0? I have the bav-X-Headers in my bounce emails, but they are not identified by the OpenEMM system.

best regards

Posted: Fri Aug 25, 2006 8:01 am
by ud
Minichip, you're right, there are these two ways to find bounces. While reading the sendmail log is more or less straight forward, parsing bounces coming back as e-mails isn't that easy, as every systems seems to generate them their own way.

Short description about bounce handling using bav:

1.) The separate domain is required to route all possible bounce mails to a single instance without creating a real user for each new bounce address.

2.) Every mail is then sent to procmail using ~openemm/conf/bav/bav.rc as its configuration file. At this point we use procmails capabilities to recognize automatically generate mails and our own ruleset (the [systemmail] block in ~openemm/conf/bav/bav.rule).

3.) If it does not look like a possible bounce message, its checked against some pattern (like out of office messages) to be ignored, every other mail is handled as configured in the OpenEMM (for performance reasons, this content is read from database and is written to ~openemm/var/spool/bav/bav.conf).

4.) If it is a possible bounce, the section [hard] and [soft] are used to checkout, if this is a hard- or softbounce. If the real recipient of the original mail can then be determinated, a bounce record for this recipient is generated.

5.) A copy of all mails passing bav is written to ~openemm/var/spool/filter.

In both cases (sendmail log and bav) bounce information is written to ~openemm/var/spool/log/extbounce.log and is processed by the update.py process

If you have questions why a bounce is not recognized, please add the complete mail to your post with full, unmodified header information.

-- ud

Posted: Sun Aug 27, 2006 3:32 am
by flyboy
And if I'm not mistaken, there's another piece to the Bounce handling that I'm having trouble getting to work.

If you send a message out and someone replies to it, the configuration in the Bounce-Filters section of "Settings" should handle this. I'm having issues with this though.

I'm on Fedora Core 5, with OpenEMM 5.0.1. My questions are:

1. What's the deal with the emails addresses "ext_1@", "ext_2@", etc.? Shouldn't you be able to set these up as email addresses that you were using as the "Sender E-Mail" in the mailing? I don't see any user interface to edit these.

2. I've set up bounce handling to go to bounce.mydomain.com. I test the bav stuff by:

a) sending an email to an address that is not in the Bounce-Filter section (i.e. one that's not in /home/openemm/var/spool/bav/bav.conf and I get a SMTP 550 error. That seems to be correct :-)
b) sending an email to an address that IS in the Bounce-Filter section (i.e. one that IS in /home/openemm/var/spool/bav/bav.conf) and it accepts it, but then I get a bounce message:

ext_1@bounce.mydomain.com on 8/26/2006 10:20 PM
The destination system is currently not accepting any messages. Please retry at a later time. If that fails, contact your system administrator.
<linux01>

3. As a side note, the PDF documentation talks about a "Reply-To Address" in the Mailing section. My version doesn't have that, it only has a "Reply-To Full name". Is that documentation that has not been updated, or a bug in the code leaving out that field?

Thanks,
Sam

Posted: Tue Sep 05, 2006 10:59 am
by olivier.r
The bounce filtering system works for me in the first case (during SMTP transaction) but not when a Mailer daemon is send .... and my extbounce.log is empty ...
Which log files/configuration files I can send for my problem ?

Thanks.

Posted: Tue Sep 05, 2006 11:46 am
by olivier.r
Why should I set up an MX Record in the DNS for bounce management work ?

Posted: Tue Sep 05, 2006 1:03 pm
by flyboy
You set up an MX record to create a return path for NDRs (Non Delivery Reports).

For example, if you send email to a user called xyz@123domain.com and that user doesn't exist, the server might tell you as you try to deliver it that the user doesn't exist, but it might also accept it and then send back a NDR later as a seperate message. If you don't have an MX record for the return domain, the NDR won't be delivered.

The way mine is set up is:

123domain.com MX --> to my MS Exchange Server
bounce.123domain.com MX --> to my MS Exchange Server

Then my MS Exchange server has a SMTP connector set up that forwards everything to the "bounce.123domain.com" to the OpenEMM SMTP server.

I do this because I have only one external IP address, thus one SMTP "Port 25" available to the public Internet.

The problem I"m having is with the scripts that handle the messages back to the OpenEMM systems Sendmail.

Note: The domain names have been changed to protect the innocent.

Posted: Tue Sep 05, 2006 3:38 pm
by olivier.r
I have solve my probleme ....
it's procmail that doesn't use the bav.rc ... so I copy bav.rc in /etc/procmailrc .... and now it's ok ...

But I have another probleme with bounce filter configuration ... I've configure a forwarding and I see :

ext_1@localhost accept:rid=1,cid=1,fwd=mymail@mydomain.fr

but what's is that "ext1" ?
Note :
I send Mailing using "openemm@openemm.lan" so the answer go back to that email ...

Can you help me please ??

Thanks

Posted: Thu Sep 21, 2006 6:40 pm
by sfantul
There are a lot of unanswered questions in this topic .
I kindly ask one of the developers to iluminate us.

Does anyone make it work as it should?
What are ext_1,ext_2 etc how do we use them.
Should we use them in the sender address for them to work?
In the company_tbl table there is a field called mailloop_domain (empty) (what can u tell us about it?).
In Mailgun.ini file i found a line :
DOMAIN = openemm.invalid
(what is this used for ? doesnt look right)

I would prefer a general explanation rather than a punctual one, extended to the the usage of the filters because i think the user guide is not clear enought.

Posted: Sat Jan 20, 2007 12:27 am
by jim_frey
3.) If it does not look like a possible bounce message, its checked against some pattern (like out of office messages) to be ignored, every other mail is handled as configured in the OpenEMM (for performance reasons, this content is read from database and is written to ~openemm/var/spool/bav/bav.conf).
Where in the database is the information that writes to ~openemm/var/spool/bav/bav.conf

I have changed the name of my subdomain and it's not correct in the bav.conf file anymore. I can change bav.conf, but as you said, it just keep getting written over.


Both of the bounce management scripts seem to be working for me now even with the bav.conf being a little off.

Posted: Sat Jan 20, 2007 4:21 pm
by jim_frey
Does anyone make it work as it should?
Yes, I do have it working on my machine.
What are ext_1,ext_2 etc how do we use them.
When you start the application up, then click on Settings > Bounce-filters and create a new bounce filter, the filter will be given the email address ext_1@bounce.company.com ext_2@bounce.company.com and so on.

These different filters let you set up different auto-replies and forwarders on those email addresses (ext_1@bounce.company.com, ext_2@bounce.company.com, etc. ) This way if two or more people are sending out email from OpenEMM and they each use their own filtered email address, when there is a reply to the email that was sent, the filter scans to see if it was a bounce reply, and if it wasn't it will forward the email along to the correct person.

You can change the name of ext_1@bounce.company.com to something like noreply@bounce.company.com if you alias the name noreply to ext_1@bounce.company.com
Should we use them in the sender address for them to work?
Yes, your filter can forward the email to your personal address. So if someone does reply to the email that you sent, the email is first run through the OpenEMM filter to check that it wasn't a bounce, and if it wasn't it'll forward the email on to whatever address you specify.
In the company_tbl table there is a field called mailloop_domain (empty) (what can u tell us about it?).
I may be wrong, but I think you're suppose to have in there:
bounce.company.com

Posted: Sat Jan 20, 2007 4:23 pm
by jim_frey
You will want to read this post:
http://www.openemm.org/forums/ftopic156.html

I had this same problem, it's a simple fix.

Posted: Mon Feb 05, 2007 5:14 pm
by ud
jim_frey wrote:Where in the database is the information that writes to ~openemm/var/spool/bav/bav.conf
The data from the database is located in the table mailloop_tbl. Domains are read from /etc/mail/mailertable. From these sources the main part of bav.conf is build (the files /etc/mail/local-host-names and /etc/mail/virtusertable are also taken into account, fyi).

-- ud