No bounce stats [Plesk + Sendmail + AuthSMTP]

Use this forum for all questions related to bounce management of OpenEMM

Moderator: moderator

enricAD
Posts: 5
Joined: Thu Apr 07, 2011 4:16 pm

No bounce stats [Plesk + Sendmail + AuthSMTP]

Post by enricAD »

Hello,

My server has CentOS with Plesk and it had Qmail as default MTA. So, I'd had to replace Qmail and install Sendmail as default MTA. I've also configured Sendmail to relay all the emails to AuthSMTP servers (in order to avoid being marked as spam).

Everything works fine except to the bounce management.

I use sender addresses with domains different from the domain where OpenEMM is installed. For example: noreply@myclientA.com, noreply@myclientB.com, etc.
That configuration is explained in the section 8.2.A of the last version of "Install Guide". So, I hadn't had to create the file bav.conf-local.

The bounced emails arrive to noreply@myclientX.com account, I forward them automatically to ext_X@bounced.OPENEMMdomain.com and I have configuraed a bounce filter for this email and to forward the bounce emails to root@localhost.

The flow works fine and I receive the bounced emails in the local mailbox of the root user of the server where OpenEMM is installed and, moreover, I can see in my maillog the following lines:

Code: Select all

Apr  7 17:14:45 s15405272 sendmail[6378]: p37FEjZD006378: Milter add: header: X-AGNMailloop: from=<>,rid=3,cid=1,fwd=root@localhost,to=<ext_3@bounced.mydomain.com>
Apr  7 17:14:45 s15405272 sendmail[6378]: p37FEjZD006378: Milter add: header: X-AGNLoop: set
However, my emails are NOT marked as bounced and the bounce tables on the database are empty.

I think that the problem is that my maillog file is in the path /usr/local/psa/var/log/maillog (Plesk path for maillog) and, maybe, it prokoves that the bav daemon doesn't process the bounced mails.
Does anyone know the steps to let OpenEMM know my different maillog path?

Thank you in advance.

Regards
Enric
Last edited by enricAD on Tue Apr 12, 2011 9:13 am, edited 1 time in total.
maschoff
Site Admin
Posts: 2597
Joined: Thu Aug 03, 2006 10:20 am
Location: Munich, Germany
Contact:

Re: No bounce stats [Plesk + Sendmail + AuthSMTP]

Post by maschoff »

Our hunch: Your mail forwarding changes the sender address of the bounce which means that the bounce management can not detect that the incoming mail is a automatically generated bounce reply. If this is the case, we see two possibilities:

1. Extend the filter rules: Add in file /home/openemm/conf/bav/bav.rule, section [systemmail] a line like

Code: Select all

^Return-Path:.*<noreply@(myclientA.com|myclientB.com)>
2. if you do not want to list every single sender address try this instead:

Code: Select all

^Return-Path:.*<noreply@.*>
Does it work now?
OpenEMM Maintainer
enricAD
Posts: 5
Joined: Thu Apr 07, 2011 4:16 pm

Re: No bounce stats [Plesk + Sendmail + AuthSMTP]

Post by enricAD »

Hello maschoff,

Thnak you very much for your answer.

If it helps, this is the full content of a bounced email that arrives to the mailbox of the root user of my server:

Code: Select all

From MAILER-DAEMON@myServer  Mon Apr 11 11:44:29 2011
Delivered-To: 5-noreply@myClientA.com
Date: Mon, 11 Apr 2011 10:44:39 +0100 (BST)
From: Mail Delivery Subsystem <MAILER-DAEMON@punt12.authsmtp.com>
To: <noreply@myClientA.com>
MIME-Version: 1.0
Content-Type: multipart/report; report-type=delivery-status;
        boundary="p3B9idZ8070304.1302515079/punt12.authsmtp.com"
Subject: Returned mail: see transcript for details
Auto-Submitted: auto-generated (failure)
X-AGNMailloop: from=<>,rid=3,cid=1,fwd=bounced@localhost,to=<ext_3@bounced.livecrm.es>
X-AGNLoop: set

This is a MIME-encapsulated message

--p3B9idZ8070304.1302515079/punt12.authsmtp.com

The original message was received at Mon, 11 Apr 2011 10:44:37 +0100 (BST)
from mail-c193.authsmtp.com [62.13.128.118]

   ----- The following addresses had permanent fatal errors -----
<prueba44445@hotmail.com>
    (reason: 550 Requested action not taken: mailbox unavailable)

   ----- Transcript of session follows -----
... while talking to mx1.hotmail.com.:
>>> DATA
<<< 550 Requested action not taken: mailbox unavailable
550 5.1.1 <prueba44445@hotmail.com>... User unknown
<<< 503 Need Rcpt command.

--p3B9idZ8070304.1302515079/punt12.authsmtp.com
Content-Type: message/delivery-status

Reporting-MTA: dns; punt12.authsmtp.com
Received-From-MTA: DNS; mail-c193.authsmtp.com
Arrival-Date: Mon, 11 Apr 2011 10:44:37 +0100 (BST)

Final-Recipient: RFC822; prueba44445@hotmail.com
Action: failed
Status: 5.1.1
Remote-MTA: DNS; mx1.hotmail.com
Diagnostic-Code: SMTP; 550 Requested action not taken: mailbox unavailable
Last-Attempt-Date: Mon, 11 Apr 2011 10:44:38 +0100 (BST)

--p3B9idZ8070304.1302515079/punt12.authsmtp.com
Content-Type: text/rfc822-headers

Return-Path: <noreply@myClientA.com>
Received: from mail-c193.authsmtp.com (mail-c193.authsmtp.com [62.13.128.118])
        by punt12.authsmtp.com (8.14.2/8.14.2/Kp) with ESMTP id p3B9ibZ8070302
        for <prueba44445@hotmail.com>; Mon, 11 Apr 2011 10:44:37 +0100 (BST)
Received: from myServer (myServer [myServerIP])
        (authenticated bits=0)
        by mail.authsmtp.com (8.14.2/8.14.2) with ESMTP id p3B9iXVb035543
        for <prueba44445@hotmail.com>; Mon, 11 Apr 2011 10:44:34 +0100 (BST)
Received: by myServer for <prueba44445@hotmail.com>; Mon, 11 Apr 2011 09:44:06 GMT
Message-ID: <20110411114401-1.1.t.2vyk.0.zec7tpkd7n@openemm.invalid>
Date: Mon, 11 Apr 2011 09:44:06 GMT
From: XXXXX <noreply@myClientA.com>
To: <prueba44445@hotmail.com>
Subject: frede
X-Mailer: OpenEMM V6.2
MIME-Version: 1.0
Content-Type: multipart/alternative;
        boundary="-==AGNITASOUTER164240059B2900001D=="
X-Server-Quench: 4ead1601-6420-11e0-97bb-002264978518
X-AuthReport-Spam: If SPAM / abuse - report it at: http://www.authsmtp.com/abuse
X-AuthRoute: OCd2YQgdAVZTSx4X EicVJDRJQA8hJwNB DA8bIBFbLVwCWEJc KVBXKFNXPFAYGQ0F B2N7AwsQDQwoBX0m NFcfbARCEwoMQAdq UUlZQ1JYEAFiAhhP DhwAUx93aQRFf2B5 ZVlgW3JeQ0Q0NUUP Sxc8cz8GZmJlbGNR WUBcOQcBIVdLeB9E d1Z7VnM
QYGRSZmdp QAU5emhpZ28BdHUK ElxUfUQcW0sHE1YA
X-Authentic-SMTP: 61633531303832.1014:706
X-AuthFastPath: 0 (Was 255)
X-AuthVirus-Status: No virus detected - but ensure you scan with your own anti-virus system.

--p3B9idZ8070304.1302515079/punt12.authsmtp.com--
So, you're right, my forwarding configuration changes the sender address but, fortunately, the "Return-Path" field still has the noreply@xxxxx.com address. So, I've changed my bav.rule file (I've commented the subject rule and the old return-path rule):

Code: Select all

[systemmail]
#^Subject: .*(DELIVER|RETURNED MAIL).*
^Return-Path:.*noreply@.*
#^Return-Path:.*<noreply@.*>
(...)
This is what I see on my maillog file now (the from field is empty --> from=<>, in my first post I've copied it badly, sorry, it was already empty):

Code: Select all

Apr 11 12:07:50 s15405272 sendmail[9693]: p3BA7omx009693: Milter add: header: X-AGNMailloop: from=<>,rid=3,cid=1,fwd=root@localhost,to=<ext_3@bounced.mydomain.es>
Apr 11 12:07:50 s15405272 sendmail[9693]: p3BA7omx009693: Milter add: header: X-AGNLoop: set
And this is what I see on my update log file (my mailing sends 6 emails):

Code: Select all

==> /home/openemm/var/log/20110411-s15405272-update.log <==
[11.04.2011  12:08:01] 9489 INFO/udpBounce: Found 0 hardbounces, 0 softbounces, 6 successes, 0 ignored in 6 lines
My bounced emails are still NOT marked as bounced. How can I know if my bounced mails are beeing processed by the bav daemon?

Thank you very much again.

Kind regards
Enric
Last edited by enricAD on Tue Apr 12, 2011 9:14 am, edited 1 time in total.
maschoff
Site Admin
Posts: 2597
Joined: Thu Aug 03, 2006 10:20 am
Location: Munich, Germany
Contact:

Re: No bounce stats [Plesk + Sendmail + AuthSMTP]

Post by maschoff »

The code sample of the bounce you posted does not contain a return-path line. This is the reason why the mail can not be caught as bounce. A filter rule working for this example could be

Code: Select all

^Delivered-To:.*noreply@.*
OpenEMM Maintainer
enricAD
Posts: 5
Joined: Thu Apr 07, 2011 4:16 pm

Re: No bounce stats [Plesk + Sendmail + AuthSMTP]

Post by enricAD »

Could you please explain me what is the normal flow to detect a bounce mail (slrtscn.py --> bavd.py --> update.py)? I would like to add some log info in order to understand why my bounce emails are nor being caught.

Thank you in advance.

Regards
Enric
enricAD
Posts: 5
Joined: Thu Apr 07, 2011 4:16 pm

Re: No bounce stats [Plesk + Sendmail + AuthSMTP]

Post by enricAD »

Hello maschoff,

That's what I see in my /usr/local/psa/var/log/maillog file when I send a mailing (id = 39) to only one user (teeeeest1234@hotmail.com, ID = 134738):

1) Relay to AUTHSMTP (mail relayed successfully):

Code: Select all

Apr 12 19:13:59 s15405272 sendmail[27107]: ruleset=try_tls, arg1=mail.authsmtp.com, relay=mail.authsmtp.com, reject=550 5.7.1 <teeeeest1234@hotmail.com>... do not try TLS with mail.authsmtp.com [62.13.128.193]
Apr 12 19:13:59 s15405272 sendmail[27107]: 00002700000020E52: to=<teeeeest1234@hotmail.com>, delay=00:00:19, xdelay=00:00:00, mailer=relay, pri=0, relay=mail.authsmtp.com. [62.13.128.193], dsn=2.0.0, stat=Sent (p3CHEFa9090571 Message accepted for delivery)
2) Hotmail bounces the email back to noreply@myClientA.com and, then, I forward it to ext_3@bounced.mydomain.com. So, the bounce filter adds a header and forward the email to root@localhost:

Code: Select all

Apr 12 19:14:07 s15405272 sendmail[27111]: p3CHE7Fr027111: from=<>, size=3662, class=0, nrcpts=1, msgid=<201104121714.p3CHEM5l090051@punt12.authsmtp.com>, proto=ESMTP, daemon=MTA, relay=myClient-MailServerName [myClient-MailServerIP]
Apr 12 19:14:07 s15405272 sendmail[27111]: p3CHE7Fr027111: Milter add: header: X-AGNMailloop: from=<>,rid=3,cid=1,fwd=root@localhost,to=<ext_3@bounced.mydomain.com>
Apr 12 19:14:07 s15405272 sendmail[27111]: p3CHE7Fr027111: Milter add: header: X-AGNLoop: set
Apr 12 19:14:07 s15405272 sendmail[27113]: p3CHE7Fr027111: to=root, delay=00:00:00, xdelay=00:00:00, mailer=local, pri=34008, dsn=2.0.0, stat=Sent
And, then, slrtscn.py writes into /home/openemm/var/spool/log/extbounce.log that line:

Code: Select all

2.0.0;0;39;0;134738;stat=Sent (p3CHEFa9090571 Message accepted for delivery)    relay=mail.authsmtp.com. [62.13.128.193]
When bavd.py or update.py read that line, I guessed that they should mark the user 134738 as bounced, am I right? But they don't...

Any help would be appreciated.

Thank you in advance.

Kind regards
Enric
maschoff
Site Admin
Posts: 2597
Joined: Thu Aug 03, 2006 10:20 am
Location: Munich, Germany
Contact:

Re: No bounce stats [Plesk + Sendmail + AuthSMTP]

Post by maschoff »

At your request the processing of bounces in OpenEMM's backend:

- Syslog realtime scanner slrtscn.py parses sendmail's log files in /var/log/maillog for instant bounces and writes identified bounces in file ~/var/spool/log/extbounce.log and state of processing (cursor) to ~/var/run/slrtscn.save.

- If you have configured OpenEMM for processing of delayed bounces, email replies are forwarded to Procmail (via configuration in mailertable).

- Procmail analyses via control file bav.rc (bounce address validation filter) email replies with ruleset in file bav.rule and forwards result to bavd (bounce address validation daemon).

- bavd writes copies of all incoming replies to ~/var/spool/filter and filtered bounces in file ~/var/spool/log/extbounce.log.

- update.py reads and processes extbounce.log and writes result to bounce_tbl.

- Softbounces in bounce_tbl are processed by softbounce.sh and written to softbounce_email-tbl. Softbounce rules are in script softbounce.py, beginning at about line 200.
OpenEMM Maintainer
enricAD
Posts: 5
Joined: Thu Apr 07, 2011 4:16 pm

Re: No bounce stats [Plesk + Sendmail + AuthSMTP]

Post by enricAD »

Hello again,

I've removed the relaying to AUTHSmtp (of my local Sendmail) and, then, the softbounced detection started to work.

But the detection of the delayed bounceds emails doesn't work yet. I've configured the automatic forward to the ext_N@bounced.mydomain.com (the one specified by my bounce-filter) on the account noreply@myClientA.com (section 8.2.A of the Install Guide) but I see the following error in the Qmail logs of the server who had this account:

Code: Select all

May 10 20:23:26 s15405200 qmail: 1305051806.218529 delivery 16167: failure: [MY-SERVER-IP]_does_not_like_recipient./Remote_host_said:_550_5.1.1_<ext_5@bounced.mydomain.es>..._[b]User_unknown[/b]/Giving_up_on_[MY-SERVER-IP]./

So, I have 2 questions:

1) What else do I have to configure on my local Sendmail in order to avoid the "User unknown error"? Remember that I followed the instructions of the section 8.2.A, I mean, I don't have a bav.conf-local file.

2) Do you know any success story with OpenEMM relaying to an external SMTP like AuthSMTP?

Thank you very much!

Regards
Enric
Daniell
Posts: 6
Joined: Thu Sep 08, 2011 1:45 pm

Re: No bounce stats [Plesk + Sendmail + AuthSMTP]

Post by Daniell »

enricAD wrote: So, I have 2 questions:

1) What else do I have to configure on my local Sendmail in order to avoid the "User unknown error"? Remember that I followed the instructions of the section 8.2.A, I mean, I don't have a bav.conf-local file.

2) Do you know any success story with OpenEMM relaying to an external SMTP like AuthSMTP?

Thank you very much!

Regards
Enric
1) Look through: helpdesk.rootsweb.com/help/sendboun.html
2) Concerning SMTP relay by an external mail server:
forums.techarena.in/small-business-server/983240.htm
waleeedijaz
Posts: 1
Joined: Wed Mar 02, 2016 8:22 am

Re: No bounce stats [Plesk + Sendmail + AuthSMTP]

Post by waleeedijaz »

My server has CentOS with Plesk and it had Qmail as default MTA. So, I'd had to replace Qmail and install Sendmail as default MTA. I've also configured Sendmail to relay all the emails to AuthSMTP servers (in order to avoid being marked as spam).
== Solitaire ==
Post Reply