Understanding how bounce management works

Use this forum for all questions related to bounce management of OpenEMM

Moderator: moderator

torcolato
Posts: 8
Joined: Thu Aug 24, 2006 2:12 pm

Understanding how bounce management works

Post by torcolato »

Hi all!

We installed OpenEMM on a FreeBSD box and everything seems to work well.
I will post setup instructions when we are sure its perfect.

One question remains right now:

How should bounce management work?

We set up a test list with 2 real recipients and 2 non-existing addresses.

The statistic page of the mailing reports no bounces after having sent the mailing.

What are we doing wrong?

Thanks,
Christian
minichip
Posts: 9
Joined: Wed Aug 09, 2006 2:57 pm
Location: Ffm, Germany

Post by minichip »

Hi torcolato,

I am running OpnenEMM on a debian system and as far as I understand the system there are at least two ways how OpenEMM identifies bounce emails.

The first one is through maillog. If the foreign email server rejects the email during the smtp session an error is log in the maillog file and OpenEMM checks for it and identifies it.

The second way (which is currently not working for me) is through the bav.sock entry in sendmail. Every email is passed through this mail filter and I guess OpenEMM should identify bounce emails. The emails are matched, or at least should be matched, against against the files in openemm-bse/conf/bav/ directory.

That is how far I understand the system, more to follow if I find time at the weekend to investigate further.


One question to the admins here, is the bav.sock feature working correctly in OpenEMM 5.0? I have the bav-X-Headers in my bounce emails, but they are not identified by the OpenEMM system.

best regards
Last edited by minichip on Fri Aug 25, 2006 5:58 pm, edited 1 time in total.
ud
Posts: 154
Joined: Thu Aug 17, 2006 9:56 am
Location: Munich, Germany
Contact:

Post by ud »

Minichip, you're right, there are these two ways to find bounces. While reading the sendmail log is more or less straight forward, parsing bounces coming back as e-mails isn't that easy, as every systems seems to generate them their own way.

Short description about bounce handling using bav:

1.) The separate domain is required to route all possible bounce mails to a single instance without creating a real user for each new bounce address.

2.) Every mail is then sent to procmail using ~openemm/conf/bav/bav.rc as its configuration file. At this point we use procmails capabilities to recognize automatically generate mails and our own ruleset (the [systemmail] block in ~openemm/conf/bav/bav.rule).

3.) If it does not look like a possible bounce message, its checked against some pattern (like out of office messages) to be ignored, every other mail is handled as configured in the OpenEMM (for performance reasons, this content is read from database and is written to ~openemm/var/spool/bav/bav.conf).

4.) If it is a possible bounce, the section [hard] and [soft] are used to checkout, if this is a hard- or softbounce. If the real recipient of the original mail can then be determinated, a bounce record for this recipient is generated.

5.) A copy of all mails passing bav is written to ~openemm/var/spool/filter.

In both cases (sendmail log and bav) bounce information is written to ~openemm/var/spool/log/extbounce.log and is processed by the update.py process

If you have questions why a bounce is not recognized, please add the complete mail to your post with full, unmodified header information.

-- ud
flyboy
Posts: 24
Joined: Tue Aug 22, 2006 6:50 pm
Location: Atlanta, Georgia U.S.A.

Post by flyboy »

And if I'm not mistaken, there's another piece to the Bounce handling that I'm having trouble getting to work.

If you send a message out and someone replies to it, the configuration in the Bounce-Filters section of "Settings" should handle this. I'm having issues with this though.

I'm on Fedora Core 5, with OpenEMM 5.0.1. My questions are:

1. What's the deal with the emails addresses "ext_1@", "ext_2@", etc.? Shouldn't you be able to set these up as email addresses that you were using as the "Sender E-Mail" in the mailing? I don't see any user interface to edit these.

2. I've set up bounce handling to go to bounce.mydomain.com. I test the bav stuff by:

a) sending an email to an address that is not in the Bounce-Filter section (i.e. one that's not in /home/openemm/var/spool/bav/bav.conf and I get a SMTP 550 error. That seems to be correct :-)
b) sending an email to an address that IS in the Bounce-Filter section (i.e. one that IS in /home/openemm/var/spool/bav/bav.conf) and it accepts it, but then I get a bounce message:

ext_1@bounce.mydomain.com on 8/26/2006 10:20 PM
The destination system is currently not accepting any messages. Please retry at a later time. If that fails, contact your system administrator.
<linux01>

3. As a side note, the PDF documentation talks about a "Reply-To Address" in the Mailing section. My version doesn't have that, it only has a "Reply-To Full name". Is that documentation that has not been updated, or a bug in the code leaving out that field?

Thanks,
Sam
olivier.r
Posts: 23
Joined: Tue Aug 29, 2006 8:33 am

Post by olivier.r »

The bounce filtering system works for me in the first case (during SMTP transaction) but not when a Mailer daemon is send .... and my extbounce.log is empty ...
Which log files/configuration files I can send for my problem ?

Thanks.
olivier.r
Posts: 23
Joined: Tue Aug 29, 2006 8:33 am

Post by olivier.r »

Why should I set up an MX Record in the DNS for bounce management work ?
flyboy
Posts: 24
Joined: Tue Aug 22, 2006 6:50 pm
Location: Atlanta, Georgia U.S.A.

Post by flyboy »

You set up an MX record to create a return path for NDRs (Non Delivery Reports).

For example, if you send email to a user called xyz@123domain.com and that user doesn't exist, the server might tell you as you try to deliver it that the user doesn't exist, but it might also accept it and then send back a NDR later as a seperate message. If you don't have an MX record for the return domain, the NDR won't be delivered.

The way mine is set up is:

123domain.com MX --> to my MS Exchange Server
bounce.123domain.com MX --> to my MS Exchange Server

Then my MS Exchange server has a SMTP connector set up that forwards everything to the "bounce.123domain.com" to the OpenEMM SMTP server.

I do this because I have only one external IP address, thus one SMTP "Port 25" available to the public Internet.

The problem I"m having is with the scripts that handle the messages back to the OpenEMM systems Sendmail.

Note: The domain names have been changed to protect the innocent.
Fedora Core 5
OpenEMM 5.0.1
Java 1.5.0_08-b03
MySQL v5.0.22

Thanks,
Sam Eads
Sales & Investments
Crown Realty & Management
olivier.r
Posts: 23
Joined: Tue Aug 29, 2006 8:33 am

Post by olivier.r »

I have solve my probleme ....
it's procmail that doesn't use the bav.rc ... so I copy bav.rc in /etc/procmailrc .... and now it's ok ...

But I have another probleme with bounce filter configuration ... I've configure a forwarding and I see :

ext_1@localhost accept:rid=1,cid=1,fwd=mymail@mydomain.fr

but what's is that "ext1" ?
Note :
I send Mailing using "openemm@openemm.lan" so the answer go back to that email ...

Can you help me please ??

Thanks
sfantul
Posts: 40
Joined: Tue Aug 15, 2006 9:34 am

Post by sfantul »

There are a lot of unanswered questions in this topic .
I kindly ask one of the developers to iluminate us.

Does anyone make it work as it should?
What are ext_1,ext_2 etc how do we use them.
Should we use them in the sender address for them to work?
In the company_tbl table there is a field called mailloop_domain (empty) (what can u tell us about it?).
In Mailgun.ini file i found a line :
DOMAIN = openemm.invalid
(what is this used for ? doesnt look right)

I would prefer a general explanation rather than a punctual one, extended to the the usage of the filters because i think the user guide is not clear enought.
jim_frey
Posts: 132
Joined: Wed Nov 29, 2006 11:32 pm
Location: Middle Caicos, Turks and Caicos Islands
Contact:

Post by jim_frey »

3.) If it does not look like a possible bounce message, its checked against some pattern (like out of office messages) to be ignored, every other mail is handled as configured in the OpenEMM (for performance reasons, this content is read from database and is written to ~openemm/var/spool/bav/bav.conf).
Where in the database is the information that writes to ~openemm/var/spool/bav/bav.conf

I have changed the name of my subdomain and it's not correct in the bav.conf file anymore. I can change bav.conf, but as you said, it just keep getting written over.


Both of the bounce management scripts seem to be working for me now even with the bav.conf being a little off.
OpenEMM Moderator
jim_frey
Posts: 132
Joined: Wed Nov 29, 2006 11:32 pm
Location: Middle Caicos, Turks and Caicos Islands
Contact:

Post by jim_frey »

Does anyone make it work as it should?
Yes, I do have it working on my machine.
What are ext_1,ext_2 etc how do we use them.
When you start the application up, then click on Settings > Bounce-filters and create a new bounce filter, the filter will be given the email address ext_1@bounce.company.com ext_2@bounce.company.com and so on.

These different filters let you set up different auto-replies and forwarders on those email addresses (ext_1@bounce.company.com, ext_2@bounce.company.com, etc. ) This way if two or more people are sending out email from OpenEMM and they each use their own filtered email address, when there is a reply to the email that was sent, the filter scans to see if it was a bounce reply, and if it wasn't it will forward the email along to the correct person.

You can change the name of ext_1@bounce.company.com to something like noreply@bounce.company.com if you alias the name noreply to ext_1@bounce.company.com
Should we use them in the sender address for them to work?
Yes, your filter can forward the email to your personal address. So if someone does reply to the email that you sent, the email is first run through the OpenEMM filter to check that it wasn't a bounce, and if it wasn't it'll forward the email on to whatever address you specify.
In the company_tbl table there is a field called mailloop_domain (empty) (what can u tell us about it?).
I may be wrong, but I think you're suppose to have in there:
bounce.company.com
Last edited by jim_frey on Sat Jan 20, 2007 4:27 pm, edited 1 time in total.
OpenEMM Moderator
jim_frey
Posts: 132
Joined: Wed Nov 29, 2006 11:32 pm
Location: Middle Caicos, Turks and Caicos Islands
Contact:

Post by jim_frey »

You will want to read this post:
http://www.openemm.org/forums/ftopic156.html

I had this same problem, it's a simple fix.
OpenEMM Moderator
ud
Posts: 154
Joined: Thu Aug 17, 2006 9:56 am
Location: Munich, Germany
Contact:

Post by ud »

jim_frey wrote:Where in the database is the information that writes to ~openemm/var/spool/bav/bav.conf
The data from the database is located in the table mailloop_tbl. Domains are read from /etc/mail/mailertable. From these sources the main part of bav.conf is build (the files /etc/mail/local-host-names and /etc/mail/virtusertable are also taken into account, fyi).

-- ud
Post Reply