Memory Leaks in OpenEMM?
Posted: Fri Nov 05, 2010 9:45 am
We using OpenEMM two newsletters in a week, each has ~150.000 Recipients.
We use OpenEMM 6.1 and we use Web-services for Mail-Creation and Newsletter-Subscribers.
Multiple times we must restart OpenEMM cause Java-Errors in Statistics-Heatmap, which shows a Null-Pointer exception and in logs came this: "Cannot allocate memory".
OpenEMM runs under a Debian-Lenny Server with 3 GB Memory, 1,5 GB for OpenEMM and 1,5 GB for the System (sendmail & co).
The REAL problem:
xmlback breaks on sending emails! I noticed it on a Test-Mailing, but after some times of research, it also happened in productive Mailings! That cannot be tolerated, many mailings to recipients could be lost because that error.
And please, no solutions like "Increase the Memory on the Server", 3 GB is much enough, the problem is that java / openemm did not release the memory somewhere. I would profile it if i know how.
Here are some files:
/home/openemm/bin/core.sh
/home/openemm/var/log/20101105-s4-mailgun.log
/home/openemm/webapps/core/log/openemm_stderr.log
We use OpenEMM 6.1 and we use Web-services for Mail-Creation and Newsletter-Subscribers.
Multiple times we must restart OpenEMM cause Java-Errors in Statistics-Heatmap, which shows a Null-Pointer exception and in logs came this: "Cannot allocate memory".
OpenEMM runs under a Debian-Lenny Server with 3 GB Memory, 1,5 GB for OpenEMM and 1,5 GB for the System (sendmail & co).
The REAL problem:
xmlback breaks on sending emails! I noticed it on a Test-Mailing, but after some times of research, it also happened in productive Mailings! That cannot be tolerated, many mailings to recipients could be lost because that error.
And please, no solutions like "Increase the Memory on the Server", 3 GB is much enough, the problem is that java / openemm did not release the memory somewhere. I would profile it if i know how.
Here are some files:
/home/openemm/bin/core.sh
Code: Select all
export LANG=en_US.ISO8859_1
export RESIN_HOME=$HOME
$HOME/bin/httpd.sh -J-Xms1536m -J-Xmx1536m -conf $HOME/conf/core.conf -server-root $HOME/webapps/core -jvm-log $HOME/webapps/core/log/openemm_jvm.log -stderr $HOME/webapps/core/log/openemm_stderr.log -stdout $HOME/webapps/core/log/openemm_stdout.log -server core -pid $HOME/var/run/core.pid $args $*
Code: Select all
[05.11.2010 08:49:14] ERROR/writer/meta/(1/2/103/125): command /home/openemm/bin/xmlback -vql -E /tmp/error5891881651176551113.tmp -o generate:temporary=true;syslog=false;account-logfile=/home/openemm/var/spool/log/account.log;bounce-logfile=log/extbounce.log;media=email;path=/home/openemm/var/spool/ADMIN /home/openemm/var/spool/META/AgnMail=D20101105084914=1=103=00001=liaMngA.xml failed (Missing binary? Wrong permissions?): java.io.IOException: Cannot run program "/home/openemm/bin/xmlback": java.io.IOException: error=12, Cannot allocate memory
/home/openemm/webapps/core/log/openemm_stderr.log
Code: Select all
[...]
triggerMailing: java.lang.Exception: Execution of /home/openemm/bin/xmlback -vql -E /tmp/error5891881651176551113.tmp -o generate:temporary=true;syslog=false;account-logfile=/home/openemm/var/spool/log/account.log;bounce-logfile=log/extbounce.log;media=email;path=/home/openemm/var/spool/ADMIN /home/openemm/var/spool/META/AgnMail=D20101105084914=1=103=00001=liaMngA.xml failed: java.io.IOException: Cannot run program "/home/openemm/bin/xmlback": java.io.IOException: error=12, Cannot allocate memory
org.agnitas.backend.MailWriterMeta.startXMLBack(MailWriterMeta.java:239)
org.agnitas.backend.MailWriterMeta.done(MailWriterMeta.java:289)
org.agnitas.backend.MailgunImpl.doExecute(MailgunImpl.java:583)
org.agnitas.backend.MailgunImpl.executeMailgun(MailgunImpl.java:116)
org.agnitas.beans.impl.MailingImpl.triggerMailing(MailingImpl.java:656)
org.agnitas.web.MailingSendAction.sendMailing(MailingSendAction.java:486)
org.agnitas.web.MailingSendAction.execute(MailingSendAction.java:218)
org.apache.struts.chain.commands.servlet.ExecuteAction.execute(ExecuteAction.java:58)
org.apache.struts.chain.commands.AbstractExecuteAction.execute(AbstractExecuteAction.java:67)
org.apache.struts.chain.commands.ActionCommandBase.execute(ActionCommandBase.java:51)
org.apache.commons.chain.impl.ChainBase.execute(ChainBase.java:190)
org.apache.commons.chain.generic.LookupCommand.execute(LookupCommand.java:304)
org.apache.commons.chain.impl.ChainBase.execute(ChainBase.java:190)
org.apache.struts.chain.ComposableRequestProcessor.process(ComposableRequestProcessor.java:283)
org.apache.struts.action.ActionServlet.process(ActionServlet.java:1913)
org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:449)
javax.servlet.http.HttpServlet.service(HttpServlet.java:115)
javax.servlet.http.HttpServlet.service(HttpServlet.java:92)
com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:106)
org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:174)
[...]