Mail attachment limitations (magic numbers)

[diax]

Hi all,

I'm experimenting with the attachment-functionality of our OpenEMM-System (5.5.1 / Windows).
I want to attach big (> 1MB) pdf-files.

By testing and browsing the source and forums, i found that a file ending with ".pdf" will be rejected by OpenEMM saying its too big, if the file size is larger than 436906 Bytes. I've tested this with two pdf-files, one of size 436906 the other of size 436907, the first one got accepted, the second rejected.

The reason is that files ending with ".pdf" are tested whether their size times 2.4 exceeds a hardcoded limit of 1048576 Bytes (exactly one Megabyte):

Code: Select all

        try {
        	double size = newAttachment.getFileSize();
        	String fileName = newAttachment.getFileName().toLowerCase();
        	if(fileName.endsWith(".pdf")) {
        		size = size * 2.4;
        	}
            if(size != 0  && size < 1048576) {
               [...]
                  // attachment gets saved
               [...]
            } else if(size >= 1048576) {
            	errors.add(ActionMessages.GLOBAL_MESSAGE, new ActionMessage("error.attachment"));
            }
        } catch(Exception e) {
            AgnUtils.logger().error("saveAttachment: "+e);
        }

My Question is: will changing the hardcoded limit and compiling and exchanging just this class solve my problem?

And will this limit be configurable in future-versions (maybe in OpenEMM 6) ?

Thanks in advance and happy OpenEMM'ing to all!

-diax

[diax]

I also found a possible bug:

Trying to attach a the ~700K plain text file that can be found here:

http://www.gutenberg.org/files/29558/29558.txt

will result in a reproducible error:

Code: Select all

java.lang.IllegalStateException: forward() not allowed after buffer has
committed.
	at com.caucho.server.webapp.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:139)
	at com.caucho.server.webapp.RequestDispatcherImpl.error(RequestDispatcherImpl.java:113)
	at com.caucho.server.webapp.ErrorPageManager.sendServletError(ErrorPageManager.java:363)
	at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:180)
	at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:229)
	at com.caucho.server.http.HttpRequest.handleRequest(HttpRequest.java:268)
	at com.caucho.server.port.TcpConnection.run(TcpConnection.java:389)
	at com.caucho.util.ThreadPool.runTasks(ThreadPool.java:507)
	at com.caucho.util.ThreadPool.run(ThreadPool.java:433)
	at java.lang.Thread.run(Unknown Source)

I tested and found this behaviour on two independent OpenEMM Installations. (Both 5.5.1 / Windows)

[maschoff]

To be honest, this is a bug we tried to hide with the magic number, because older releases of MySQL do not accept big files saved as BLOB. We will look into this after release of 6.0 and see if we can find a workaround (split big files in smaller chunks or whatever).

[omiba]

we can't attach anything more than a 240K pdf file, it's fairly frustrating.

-Scott

[pheelix]

i'm not sure, but what if you change the mysql blob field to a longblob, with a max. filesize of 2/4GB? or mediumblob.

neither i tried it, nor i know what type the field is (whether it is really just blob, or already a longblob)

Code: Select all

BLOB = 65KB
MEDIUMBLOB = 16MB
LONGBLOB = 4GB

http://bytes.com/topic/mysql/answers/569102-blob-size

[maschoff]

If you use a 5.x version of MySQL: Change the value of 2²° in class MailingAttachmentsAction.java to a bigger number, recompile it and replace the original class file.

[omiba]

What if we are using 6.0?

-Scott

[maschoff]

As far as I know the latest stable release of MySQL is 5.1.x, I would not go beyond it for a production system!

[omiba]

Sorry, was talking about OpenEMM version.

Do you know when we will not have to re-compile the source to make this change?

[maschoff]

No, but compilation (please use Java 6) and replacement is done within minutes.

[omiba]

Do you know how small of a multiple we can go before running into an issue?

My assumption is that we should be changing:

Code: Select all

        try {
        	double size = newAttachment.getFileSize();
        	String fileName = newAttachment.getFileName().toLowerCase();
        	if(fileName.endsWith(".pdf")) {
        		size = size * 2.4;
        	}

Can we just change it to:

Code: Select all

size = size * 1;

Thanks in advance!

-Scott

[maschoff]

We don't know.

[omiba]

Okay, i'll try it out, and let everybody know.

-Scott

[maschoff]

Since omiba did not let us know what he learnt we will look into this issue again.

[reenfoo]

Code:

try {
double size = newAttachment.getFileSize();
String fileName = newAttachment.getFileName().toLowerCase();
if(fileName.endsWith(".pdf")) {
size = size * 2.4;

this change really helped!

peace~