Bounce management with postfix relay host

Use this forum for all questions related to bounce management of OpenEMM

Moderator: moderator

perun
Posts: 9
Joined: Wed Dec 04, 2019 12:39 pm

Bounce management with postfix relay host

Post by perun » Wed Dec 04, 2019 12:47 pm

Hi,

We have OpenEMM installed on AWS on Amazon Linux 2 OS with DB on remote MySQL Aurora and local postfix relay linked to our main postfix-based mailserver (on another EC2).

It works in principle but I still struggle with bounce management. Is there any chance I could get some help here? OpenEMM server has MX configured on a subdomain, my initial idea was to configure mail forwarding from the main mailserver to OpenEMM for any bounces. It doesn't seem to be working though. What is the best approach here to make bounce management working here?

Thanks,

Matt

perun
Posts: 9
Joined: Wed Dec 04, 2019 12:39 pm

Re: Bounce management with postfix relay host

Post by perun » Fri Dec 20, 2019 1:58 pm

Hi,

No answer yet. In case it helps anyone, I reverted to using our internal bounce management script (based on Sisimai) to update relevant tables on OpenEMM. It feels like the most reliable and simplest solution for our architecture, we get users blacklisted the way we expect.

Thanks,

Matt

cdambrosio
Posts: 21
Joined: Thu Oct 08, 2020 5:40 pm

Re: Bounce management with postfix relay host

Post by cdambrosio » Fri Nov 13, 2020 10:32 am

Hi, I have the same problem.

I configured the transport.maps in this way:

* smtp:[my.relay.host]:25
my.mailloop.domain mailloop:


The OpenEMM system (version 20.10) can send emails without any problem and I can receive the "human" replies (the mailloop forwards them to a real email address), but bounces (either instant or asynchronous) are not processed.

Unfortunately I need the relayhost to obey to the Organizational policies, but the strange thing is that I have the same configuration on an old instance of OpenEMM (2015) with sendmail instead of postfix and it correcly processes the bounces (either instant or asynchronous).

How can I diagnose this problem?

Thank you.

Regards,

Carlo

cdambrosio
Posts: 21
Joined: Thu Oct 08, 2020 5:40 pm

Re: Bounce management with postfix relay host

Post by cdambrosio » Sun Nov 15, 2020 10:49 am

Good morning,
I add also another problem: after a clean installation, my bounce_translate_tbl is empty... what's wrong with it? Maybe an installation problem?

I've installed from scratch (two times) using the latest runtime and downloading packages by Agnitas Website (choice 5 of the installer) without errors.

Thank you.

Regards,

Carlo

cdambrosio
Posts: 21
Joined: Thu Oct 08, 2020 5:40 pm

Re: Bounce management with postfix relay host

Post by cdambrosio » Sun Nov 15, 2020 5:21 pm

Hi,
another clue (I hope).

I think that bounces aren't working for this reason, I have the error below in my slrtscn3 logfile:

[15.11.2020 17:14:08] 72091 INFO/parse/4CYy276ZpDz3jTsN: Failed to parse message_id <DB6PR06MB4037E004774DEA4BCD116BAAACE40@DB6PR06MB4037.bouncedomain.com>: Invalid formated UID

It seems to be a UID problem, what do you think about it?

Thank you.

Regards,

Carlo

cdambrosio
Posts: 21
Joined: Thu Oct 08, 2020 5:40 pm

Re: Bounce management with postfix relay host

Post by cdambrosio » Sun Nov 15, 2020 9:49 pm

Hi again,
sorry for this further post, but I think that I found the cause of the problem. There's already another post about the argument (viewtopic.php?t=2519), but unfortunately it has no answer.

Doing a tcpdump, I found that OpenEMM is setting the option NOTIFY=NEVER when it sends the RCPT TO command, here is what I have on my system (obviously I've anonymized the email address);

RCPT TO:<externalrecipient@domain.com> ORCPT=rfc822;externalrecipient@domain.com NOTIFY=NEVER

If I send an email by the sendmail command I correctly receive the NDR, cause that option isn't set. I couldn't find a way to remove it, also setting the option smtpd_command_filter doesn't work, cause Postfix send the command to the relayhost using the smtp part and the command is passed as is.

If anyone has an idea about how to solve the problem, that would be a great help.

Thank you.

Regards,

Carlo

ud
Posts: 147
Joined: Thu Aug 17, 2006 9:56 am
Location: Munich, Germany
Contact:

Re: Bounce management with postfix relay host

Post by ud » Mon Nov 16, 2020 10:48 am

You can enable the status notification of sendmail by editing the file $HOME/scripts/config.sh and extend the environment variable SYSTEM_CONFIG near the top of the file. The content of the variable has to be valid JSON, so be careful when modifying it. The default version should look like this:

Code: Select all

export  SYSTEM_CONFIG='{
  "trigger-port": 8450,
  "direct-path-incoming": "/home/openemm/var/spool/DIRECT",
  "direct-path-archive": "/home/openemm/var/spool/ARCHIVE",
  "direct-path-recover": "/home/openemm/var/spool/RECOVER",
  "direct-path-queues": "/home/openemm/var/spool/QUEUE",
  "licence": 0,
  "dbid": "openemm",
  "merger-address": "127.0.0.1",
  "filter-name": "localhost",
  "mailout-server": "localhost"
}'
The extended version looks then like this:

Code: Select all

export  SYSTEM_CONFIG='{
  "trigger-port": 8450,
  "direct-path-incoming": "/home/openemm/var/spool/DIRECT",
  "direct-path-archive": "/home/openemm/var/spool/ARCHIVE",
  "direct-path-recover": "/home/openemm/var/spool/RECOVER",
  "direct-path-queues": "/home/openemm/var/spool/QUEUE",
  "licence": 0,
  "dbid": "openemm",
  "merger-address": "127.0.0.1",
  "filter-name": "localhost",
  "mailout-server": "localhost",
  "enable-sendmail-dsn": true
}'
I just added one parameter (enable-sendmail-dsn) at the end of block to enable status notification.

-- ud

cdambrosio
Posts: 21
Joined: Thu Oct 08, 2020 5:40 pm

Re: Bounce management with postfix relay host

Post by cdambrosio » Mon Nov 16, 2020 4:16 pm

Great! It works like a charm!

Thank you very much UD!

Regards,

Carlo

maschoff
Site Admin
Posts: 2401
Joined: Thu Aug 03, 2006 10:20 am
Location: Munich, Germany
Contact:

Re: Bounce management with postfix relay host

Post by maschoff » Mon Nov 16, 2020 5:38 pm

Regarding empty bounce_translate_tbl: Can you confirm that you have the problem with OpenEMM 20.10 BETA? Because 20.04 does not support this feature.
OpenEMM Maintainer

cdambrosio
Posts: 21
Joined: Thu Oct 08, 2020 5:40 pm

Re: Bounce management with postfix relay host

Post by cdambrosio » Mon Nov 16, 2020 5:46 pm

Hi Martin,
I have that table also in version 20.04.000.188 (frontend) and 20.04.000.190 (backend), but with four columns instead of many more.

If I don't insert data there, the hardbounces are not reported.

I take advantage for another question: I see all sent emails as "Without confirmation of receipt": could I have another misconfiguration somowhere?

Thank you.

Regards,

Carlo

maschoff
Site Admin
Posts: 2401
Joined: Thu Aug 03, 2006 10:20 am
Location: Munich, Germany
Contact:

Re: Bounce management with postfix relay host

Post by maschoff » Mon Nov 16, 2020 6:06 pm

Confirmed, table exist, but it is only used beginning with 20.10.

"Without confirmation of receipt" means, postfix/sendmail sent out mails but did not get a DSN back from the receiving end. Either you use a web proxy which does not report back to OpenEMM properly, or ISPs do not like your mails (mostly because of incomplete DNS configuration).
OpenEMM Maintainer

cdambrosio
Posts: 21
Joined: Thu Oct 08, 2020 5:40 pm

Re: Bounce management with postfix relay host

Post by cdambrosio » Mon Nov 16, 2020 6:43 pm

Hi Martin,
I have success DSN in ./var/spool/mail/store/20201116-mbox, but OpenEMM reports the email as without confirmation of receipt.

Here is the (anonymized) success DSN:

From MAILER-DAEMON Mon Nov 16 19:14:53 2020
Return-Path: <MAILER-DAEMON>
Received: from server.externaldomain.com (server.externaldomain.com [10.10.10.10])
by my.postfix.host.com (Postfix) with ESMTP id 4CZcfy5kPtz3jgCx
for <sender@my.mailloop.domain.com>; Mon, 16 Nov 2020 19:14:50 +0100 (CET)
MIME-Version: 1.0
From: <postmaster@externaldomain.com>
To: <sender@mypublicsenderdomain.com>
Date: Mon, 16 Nov 2020 19:14:50 +0100
Content-Type: multipart/report; report-type=delivery-status;
boundary="76e4b5c3-8719-41a1-bac3-cb03af3b46e1"
Content-Language: it-IT
Message-ID: <e5327634-8372-439e-bc9c-6aa2d9053478@mysenderdomain.com>
In-Reply-To: <20201116181450_0.D.A.g.E.A.A.0BfNHVMUkhy4vdEURTKHXKmLvrJcFkjunaCvYFKo7jl8GhxHXf27dV8a1K0rDVjE1o-xmYF2jAc49fb97opfEQ@mypublicsenderdomain.com>
References: <20201116181450_0.D.A.g.E.A.A.0BfNHVMUkhy4vdEURTKHXKmLvrJcFkjunaCvYFKo7jl8GhxHXf27dV8a1K0rDVjE1o-xmYF2jAc49fb97opfEQ@mypublicsenderdomain.com>
Subject: Recapitato: Spedizione di Test n. 2 del 20201116
X-AGNMailloop: from=<>,rid=1,cid=1,fwd=feedbackuser@mysenderdomain.com,to=<sender@my.mailloop.domain.com>
X-AGNLoop: set

--76e4b5c3-8719-41a1-bac3-cb03af3b46e1
Content-Type: multipart/alternative; differences=Content-Type;
boundary="73522db2-3b26-4048-8731-b400a19990ac"

--73522db2-3b26-4048-8731-b400a19990ac
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Ho recapitato il tuo messaggio ai seguenti destinatari:

myrecipient@externaldomain.com <mailto:myrecipient@externaldomain.com>

Oggetto: Spedizione di Test n. 2 del 20201116

--73522db2-3b26-4048-8731-b400a19990ac
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html>
<Head></head><body>
<p><b><font color=3D"#000066" size=3D"3" face=3D"Arial">Ho recapitato il tu=
o messaggio ai seguenti destinatari:</font></b></p>
<font color=3D"#000000" size=3D"2" face=3D"Tahoma"><p><a href=3D"mailto:myrecipient@externaldomain.com">myrecipient@externaldomain.com</a></p>
<p>Oggetto: Spedizione di Test n. 2 del 20201116</p>
</font>
</body>
</html>=

--73522db2-3b26-4048-8731-b400a19990ac--

--76e4b5c3-8719-41a1-bac3-cb03af3b46e1
Content-Type: message/delivery-status

Reporting-MTA: dns;externaldomain.com
Received-From-MTA: dns;my.postfix.host.com
Arrival-Date: Mon, 16 Nov 2020 18:14:50 +0000

Original-Recipient: rfc822;myrecipient@externaldomain.com
Final-Recipient: rfc822;myrecipient@externaldomain.com
Action: delivered
Status: 2.0.0


What do you think about it?

Thank you.

Regards,

Carlo
Last edited by cdambrosio on Mon Nov 16, 2020 11:58 pm, edited 1 time in total.

cdambrosio
Posts: 21
Joined: Thu Oct 08, 2020 5:40 pm

Re: Bounce management with postfix relay host

Post by cdambrosio » Mon Nov 16, 2020 11:58 pm

Hi Martin,
I think that this is the problem in my slrtscn3 logfile:

[16.11.2020 19:14:51] 551198 INFO/parse/4CZcfy5kPtz3jgCx: Failed to parse message_id <e5327634-8372-439e-bc9c-6aa2d9053478@mysenderdomain.com>: Invalid formated UID

It seems that OpenEMM doesn't process the success DSN because they came with a different message-id, but that's normal, cause they came from the recipient mail server.

Let me know what do you think about it.

Thank you.

Regards,

Carlo

cdambrosio
Posts: 21
Joined: Thu Oct 08, 2020 5:40 pm

Re: Bounce management with postfix relay host

Post by cdambrosio » Tue Nov 17, 2020 9:15 am

Hi Martin,
just an update.

I've replaced postfix with sendmail and now OpenEMM correctly processes the success DSN.

To allow sendmail to request DSN, I had to make this replacements (in addition to the change suggested by UD):

/home/openemm/bin/mta.sh --> row 76 - replace "-NNEVER" con "-RHDRS -NSUCCESS,FAILURE,DELAY"
/home/openemm/bin/mlcontrol.sh --> rows 56 e 57 - sostituire -NNEVER con "-RHDRS -NSUCCESS,FAILURE,DELAY"
/home/openemm/scripts/config.sh --> rows 214 - sostituire "SENDMAIL_DSN_OPT="" con SENDMAIL_DSN_OPT="-RHDRS -NSUCCESS,DELAY,FAILURE"

Thank you.

Regards,

Carlo

Post Reply