IOException: "The multipart stream ended unexpectedly"

General
1

Hi Everybody,

I have a comma separated file (csv-file) containing around 120.000 datasets. My code is mainly based on the
Upload processing example from the Sampler
. The input is processed right away (checks if a dataset already exist, persists new datasets, does some heavy calculation, etc.). When starting the process from outside my Vaadin application the whole thing takes about 45 min.

BUT when I start the upload within my Vaadin application I get the error listed below. I got it with Vaadin 6.6.0 and followed the advice from Henri in this
thread
 and upgraded to Vaadin 6.6.6. But I still get the same error.

Can it have something to do with a time limitation I am missing? It can’t be session timeout as I set it to 60 min.

Any help is really appreciated as I am desperate.

Thanks in advance
Kurt


com.vaadin.terminal.gwt.server.UploadException: Upload failed
	at com.vaadin.terminal.gwt.server.AbstractCommunicationManager.streamToReceiver(AbstractCommunicationManager.java:616)
	at com.vaadin.terminal.gwt.server.AbstractCommunicationManager.doHandleSimpleMultipartFileUpload(AbstractCommunicationManager.java:469)
	at com.vaadin.terminal.gwt.server.CommunicationManager.handleFileUpload(CommunicationManager.java:257)
	at com.vaadin.terminal.gwt.server.AbstractApplicationServlet.service(AbstractApplicationServlet.java:495)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
	at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1534)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
	at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)
	at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)
	at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:98)
	at com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:91)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:162)
	at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:326)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:227)
	at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:170)
	at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:822)
	at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:719)
	at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1013)
	at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)
	at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
	at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
	at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
	at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
	at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
	at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
	at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
	at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
	at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
	at java.lang.Thread.run(Thread.java:717)
Caused by: java.io.IOException: The multipart stream ended unexpectedly
	at com.vaadin.terminal.gwt.server.AbstractCommunicationManager$SimpleMultiPartInputStream.read(AbstractCommunicationManager.java:2231)
	at java.io.InputStream.read(InputStream.java:171)
	at java.io.InputStream.read(InputStream.java:102)
	at com.vaadin.terminal.gwt.server.AbstractCommunicationManager.streamToReceiver(AbstractCommunicationManager.java:571)
	... 29 more
2

I’m not sure what happens there, but apparently either the browser or the application server has a timeout. Session timeout is just one thing, there’s also connection timeout, which is normally rather short. For regular HTTP requests I think it is 60 seconds, but might be different for upload requests. At least Tomcat has
separate settings
 for upload timeouts.

One reason could be that chunked or buffered data is processed so slowly that the connection expires before one chunk/bufferful is processed. For example, if a chunk is 10 MB and it takes over 1 minute to process it, it could be a problem. How large are the uploaded files? How much time is spent processing one chunk of a data received from the browser (perhaps you can observe it in Firebug)? I don’t know how the data in a multipart upload is chunked or buffered but it is somehow.

I assume that you have a good reason to process the data while uploading and not do it after the upload.

3

Thank you very much Marco for your quick reply and help.

Yes and no. My first approach was to upload the data and do processing afterwards. But I ran into problems there as well. So I started with the demo example hoping that this would be the way to go.

Anyways, while waiting for an answer I started a new try. This time using Progress Indicator. My approach here is (again) to upload the file first and then start data processing in a new thread. So far this looks good. But I can not tell for sure right now. It needs a bit more coding. I will let you know first thing in the morning because although this is really urgent I need some hours sleep right now. n looks like an m already even there is no alcohol involved. :smiley:

Kurt

4

OK, I did some more coding and testing and using Progress Indicator is the way to go. Everything works fine now.

By the way, do I have to call getApplication() after each line I read? Or does it hurt in anyway when calling getApplication() every 10th line?

At the moment getApplication() gets called after every line read:


bufReader = new BufferedReader(new FileReader(file));
while ((line = bufReader.readLine()) != null) {
        lineCnt++;
        processLine(lineCnt, line);
        synchronized (getApplication()) {
                processed(true);
        }
}

Might this be ok as well?


bufReader = new BufferedReader(new FileReader(file));
while ((line = bufReader.readLine()) != null) {
        lineCnt++;
        processLine(lineCnt, line);
        if((lineCnt % 10) == 0) {
                synchronized (getApplication()) {
                        processed(true);
                }
        }
}

Of course this does not clarify my original IOException problem. But as said, this stuff is so urgent that I do not have the time right now to investigate it any further. Sorry for that.

5

Mmm, I don’t know why you are synchronizing the processed(true) call (what does it do?), but if you are using the application object as a synchronization lock, you can just use an object reference to it and not call getApplication() every time.

6

processed(true) updates a label. Kind of the same way the
Progress Indicator example method processed()
 updates the progress indicator.

I see. Thanks again for your quick and excellent help Marko.

Kurt

P.S.: I will mark this thread as solved.