UIDL Compressor problem

Hello,

I just droped in the uidl compressor addon to my application.
I also modified the web.xml.

But the browser now always shows this error:

Communication problem

Take note of any unsaved data, and click here to continue.


(SyntaxError): identifier starts immediately after numeric literal stack: d6("AAAAMVU/27aMBB+lcjSpE1KIIFSWtD+QLBq68pWlan9A9DkJBfw5tiR40BTxNvsTfZiOzuJVko3TdqkgXS5X/7Ovs/nRKqXw+Gr4XxHbimNmfBmEBWK6dJ7DyUZkF73JPD7/XPvJOkF3km3E3jhCT31/MSHsBf2o06/S1wSralYQU4G83mtE3dHEqlSqsnAIQWLOaZlLDbW9buJT/bunPgm65HPJWtgq7VdEvh+y3+Bri2L9frAE9FMMymMb3wzdUZUhZKq2Fm0jZ4p+QUi7Xg/A6hEuBGXCJqChUJdr6EyaBNVkLMHGnL0alWAS1LKRKNzWspC3wCnmm3grtpUFdqQwY5oGjIRwz0ZeIFL8khJzq8gwbP4jflJZtbKZM7MAercxiwrM+Iyxx0klOewN10KDrp0+mxLUqpWTCABQc8llLOVSEFotJ3d3iVwn1ER3+DOZeXaL4+63z/ofgdxIX1cqtOrPQpoLAUvm9P/PRvPkpHKmPKmxhEzv2EjL8L/QszZPyEGYTsHsOdPpuKXvOS65LaFG49LrGmmz05Z9wAv+OMx02Vm8UIltzkoU0JFlqMsGyzai/bo+nrRDhZtW86claXQLNJwrxfttU750MEXQeWgXxc68c7Ifln9XIdgOiUOnhsJznDW9IjXlO8xiqTLQkX4rDi2W47d0RSrM7GqnA6JZNra2KerVbDWHTIttybkO+7T4C0ozSLKr+zdMUnBcdK4yLVMxzLNpECiTFbnOOtNGkIcQ2zCXcdsjUtEti/grr7XJIbP47emaykOx5xcUtOjCwhRTtUDylFmmjqlDOVlIaw0z+SoWKGcQYby41czDB/kBuUEHsgS29zAFVRViKrSpt+/NbiMP0auPpxV2HjCCl4DHkNVRWSlYaHGidUqFUvmsa05kwbViInBmhoxMeaFWTCjJrXJFELTlU2vlQkDkdc+pvVWRmu7XAhQtf9CAau0GU2tDwETw+gA5y9OTFPj1nTaKkv7ZkT1NOKFTvF+43+/3C9/ACh5V4/VBgA")@http://localhost:8080/VAADIN/widgetsets/com.aarboard.aarcat2.admin.vaadin.Aarcat2VaadinWidgetset/AC797A1D881F8C6F06483D6036C90FD5.cache.html:2517 q6([object Object]
,[object Object]
)@http://localhost:8080/VAADIN/widgetsets/com.aarboard.aarcat2.admin.vaadin.Aarcat2VaadinWidgetset/AC797A1D881F8C6F06483D6036C90FD5.cache.html:4068 rq([object Object]
,[object Object]
)@http://localhost:8080/VAADIN/widgetsets/com.aarboard.aarcat2.admin.vaadin.Aarcat2VaadinWidgetset/AC797A1D881F8C6F06483D6036C90FD5.cache.html:3247 Rq([object XMLHttpRequest]
)@http://localhost:8080/VAADIN/widgetsets/com.aarboard.aarcat2.admin.vaadin.Aarcat2VaadinWidgetset/AC797A1D881F8C6F06483D6036C90FD5.cache.html:4065 ([object Event]
)@http://localhost:8080/VAADIN/widgetsets/com.aarboard.aarcat2.admin.vaadin.Aarcat2VaadinWidgetset/AC797A1D881F8C6F06483D6036C90FD5.cache.html:2471 ie((function () {a.Vc(b);}),[object XMLHttpRequest]
,[object Arguments]
)@http://localhost:8080/VAADIN/widgetsets/com.aarboard.aarcat2.admin.vaadin.Aarcat2VaadinWidgetset/AC797A1D881F8C6F06483D6036C90FD5.cache.html:1951 le((function () {a.Vc(b);}),[object XMLHttpRequest]
,[object Arguments]
)@http://localhost:8080/VAADIN/widgetsets/com.aarboard.aarcat2.admin.vaadin.Aarcat2VaadinWidgetset/AC797A1D881F8C6F06483D6036C90FD5.cache.html:3514 ([object Event]
)@http://localhost:8080/VAADIN/widgetsets/com.aarboard.aarcat2.admin.vaadin.Aarcat2VaadinWidgetset/AC797A1D881F8C6F06483D6036C90FD5.cache.html:2639 fileName: http://localhost:8080/VAADIN/widgetsets/com.aarboard.aarcat2.admin.vaadin.Aarcat2VaadinWidgetset/AC797A1D881F8C6F06483D6036C90FD5.cache.html lineNumber: 2517 - Original JSON-text:AAAAMVU/27aMBB+lcjSpE1KIIFSWtD+QLBq68pWlan9A9DkJBfw5tiR40BTxNvsTfZiOzuJVko3TdqkgXS5X/7Ovs/nRKqXw+Gr4XxHbimNmfBmEBWK6dJ7DyUZkF73JPD7/XPvJOkF3km3E3jhCT31/MSHsBf2o06/S1wSralYQU4G83mtE3dHEqlSqsnAIQWLOaZlLDbW9buJT/bunPgm65HPJWtgq7VdEvh+y3+Bri2L9frAE9FMMymMb3wzdUZUhZKq2Fm0jZ4p+QUi7Xg/A6hEuBGXCJqChUJdr6EyaBNVkLMHGnL0alWAS1LKRKNzWspC3wCnmm3grtpUFdqQwY5oGjIRwz0ZeIFL8khJzq8gwbP4jflJZtbKZM7MAercxiwrM+Iyxx0klOewN10KDrp0+mxLUqpWTCABQc8llLOVSEFotJ3d3iVwn1ER3+DOZeXaL4+63z/ofgdxIX1cqtOrPQpoLAUvm9P/PRvPkpHKmPKmxhEzv2EjL8L/QszZPyEGYTsHsOdPpuKXvOS65LaFG49LrGmmz05Z9wAv+OMx02Vm8UIltzkoU0JFlqMsGyzai/bo+nrRDhZtW86claXQLNJwrxfttU750MEXQeWgXxc68c7Ifln9XIdgOiUOnhsJznDW9IjXlO8xiqTLQkX4rDi2W47d0RSrM7GqnA6JZNra2KerVbDWHTIttybkO+7T4C0ozSLKr+zdMUnBcdK4yLVMxzLNpECiTFbnOOtNGkIcQ2zCXcdsjUtEti/grr7XJIbP47emaykOx5xcUtOjCwhRTtUDylFmmjqlDOVlIaw0z+SoWKGcQYby41czDB/kBuUEHsgS29zAFVRViKrSpt+/NbiMP0auPpxV2HjCCl4DHkNVRWSlYaHGidUqFUvmsa05kwbViInBmhoxMeaFWTCjJrXJFELTlU2vlQkDkdc+pvVWRmu7XAhQtf9CAau0GU2tDwETw+gA5y9OTFPj1nTaKkv7ZkT1NOKFTvF+43+/3C9/ACh5V4/VBgA

Any ideas what might be missing ?
(I have recompiled the widgetset)

Hi, thanks for trying out the addon.

I can’t say for sure what might cause that error, first time I see that kind of error.

One thing I can think of is that if you are using another addon that modifies the ApplicationConnection (like TouchKit for instance) then it won’t work. For this addon to work you need to be using the standard ApplicationConnection.

Of course this could also be a browser issue, what browser are you testing with? Does it occur in other browsers as well?

Not that I’m aware of…
The app is running on Tomcat 7.0.23

It occurs with IE9 and Firefox 8.0.1 too…

Okay, just tried out your setup with Tomcat 7 + FF8/IE9 and it is working over here. (with a trivial application)

I think the issue might be either in the compilation of the widgets or that you have a old cached widgetset in use.

First could you please check that you do have the following line in your widgetset module definition xml file (.gwt.xml):

<inherits name="fi.jasoft.uidlcompressor.UIDLCompressorWidgetset" />

The vaadin plugin adds it automatically but if you are using a build script or another IDE you will need to do it manually. I probably should add that to the wiki as well…

Hi John,

Do you think your add-on will work with the navigator7 pattern ?

Hi,

I haven’t used the Navigator7 so I cannot say for sure.

Basically the UIDL Compressor has two demands on the application:

  1. On the server side it has to be able to use its own CommunicationManager (or an extension of it) to encode and compress the payload. It does not necessary need to use it’s own ApplicationServlet, if you already have your own ApplicationServlet extension in place you can just override the createCommunicationManager() method and return the UIDLCompressorCommunicationManager.
  2. On the client side the default ApplicationConnection will be replaced by the UIDL Compressors version.

If Navigator7 allows those two things then it will work, if it doesn’t it wont.

I did a clean build of my project and now it’s working.

But now I see that the german umlauts (öäüéàè etc.) are not displayed correctly in labels and button captions any longer…
Currently I test on a localhost (Windows 7 german) with a local tomcat.

Looks like there is a charset/codepage mismatch between compress/uncompress
12131.gif

I am getting the same with some finnish characters as well. Most likely the UTF-8 strings are not getting decoded correctly on the client side.

I’ll look into the issue as soon as I can.

Released version 0.2.1 of UIDL Compressor which should fix those encoding issues.

Yep,

now the encoding is correct.

Thanks

Nice addon!

However, I wonder, if the size reduction weighs up to the extra client side processing.

Does this plugin cause a faster response for the user?

In some cases it will, and probably in some cases it will not (An exception might be really big UIDLs where the network transfer time is really significant). I have been testing on a average computer with a recent browser version and getting client side decoding times between 50-100ms for UIDLs over 300kB. If I would have a slow connection then I would easily save the decoding time in network transfer, if I have a fast connection I might not.

Basically the two edge cases are that if you have a fast connection and not very powerful computer/browser then the decoding overhead might slow things down while if you have an slow or average connection and an average or fast computer/browser then the decoding overhead will not be an issue.

Even though the network response time might get better with this addon the perceived responsiveness of the Vaadin application probably will not. This is because a Vaadin application spends most of its time in the rendering phase doing layout measurements and component specific rendering. The decoding time or JSON parsing time is in many cases insignificant to the time it takes to render a big view in a Vaadin application.

If you’re looking to speed up a Vaadin application then optimizing the layouts used by the application will matter the most. If you’re looking to save some bandwidth (and maybe some bucks in the process) then take a look at this addon. Either way, I do not think the addon will either effect negatively or positively on the perceived responsiveness of the application when using average sized UIDLs.

EDIT: Removed the word “standard” for UIDLs over 300kB since if you are using just standard Vaadin components then you probably will not see those kinds of UIDLs. With graphing components it is a different story.

That sounds like awfully big UIDLs to me, even if it is kilobits and not kilobytes.
I wonder if the units are incorrect or if you are doing lots and lots of full repaints of large tables…

With standard Vaadin components I do not expect to see that kind of UIDLs either. Standard Vaadin components really use very little data to render since they mostly deal with text. In those cases the UIDL is probably somewhere between 50kB-100kB at best.

However, when you start using graphing components (like the MandelBrot addon in the demo or the Vaadin Timeline for instance) then you will see those kinds of UIDLs since they usually are very data intensive and a lot of information needs to be transmitted.

thanks for your elaborate answer. I will experiment with it

Hi,

First of, I’d like to emphasis that this is a cool add-on demonstrating the strength that Javascript has these days. But at the same time I’m bit worried that somebody decides to use this add-on instead of robust industry proven methods to enable compression at http level. That’s why I’d like to discuss if there is a real world usage for this add-on.

In which case can you use this add-on, but cannot set http level compression via server settings or with a servlet filter? E.g. jetty-util package has a very handy filter that can be used in virtually all servlet engines (see e.g. this
blog post that introduces its usage
). Protocol level compression is also more efficient as it don’t need an extra base64 encoding and decoding happens with native code by browser instead of javascript.

Protocol level compression is a better option also because it compresses e.g. largish GWT module (aka widget set), CSS files and the host html page.

cheers,
matti

You are rightly concerned, compression is too little mentioned in Vaadin documentation for the users to know about, it should play a bigger role since it’s benefits are huge. But I think you misunderstand, this addon is not meant to replace the vendor specific industry proven solutions, it can either complement or be used as an alternative where you only want the UIDLs compressed. If you are seeking for more, then I urge everybody to use those servlet filters or deflate modules provided by vendors, they are powerful technologies.

As with the jetty and all other servlet container implementations the GZip enabled servlet filter always comes in util/dependency/external package which you will need to include to use the servlet filter. This introduces a new dependency which I need to include for each container I want to support and if I distribute my software as a downloadable WAR I cannot know on what container it will be run on. Also the notation and features are different for each filter implementation, for instance compare the Jetty filter notation you mention in the blog post with the tomcat one I mention in the addon resource links. Basically the same but not quite, so I would need to have the user of my application to set the correct one. I hate having the users of my application having to do that kind of a thing because it is error prone and using the addon relieves the user of doing it.

But let me emphasize again, so everybody is happy. The addon does one specific thing, it compresses the UIDL stream, it does NOT however compress your Javascript or CSS files. So, if you need that and know where your application is going to be deployed, please use the servlet containers features to the fullest, they are always more robust and proven.

Hi,

With servlet filters you can choose just those uri schemes you want to compress. So for compressing just UIDL streams I don’t see this any better than jetty gzip filter.

I think you have got the servlet filter bit wrong. Servlet filters is a standard that servlet containers support, just like servlets. E.g. the Jetty project has several general purpose servlets and servlet filters that they have built. GzipFilter is one of those, another I have used is their handy ProxyServlet. Both of those are server agnostic (at least in older versions), and they are packaged in a separate jar files. With maven relevant modules are jetty-util (in 6.x) and jetty-servlets with 7.x and 8.x. So you can have one war file with a gzip filter configured and run it on virtually all application servers.

Note, that it may still not be sane to always put compression on with a servlet filter or with an add-on like this. E.g. some hosting setups do compression in a reverse proxy.

cheers,
matti

Didn’t say better, I said alternative, as in you can have it both ways.

Sorry to take that out out of context, but that is exactly what I do not want in my use case. What I want in my use case is to distribute a WAR file without dependencies to any utility jars. I don’t want to have my application users have to build the war with Maven shiver or anything else, I want them to be able to download a WAR from my site, drop it wherever they please and have working compression. No web.xml modifying, no application server config files, just the WAR, nothing else. Then if they want and can they can enable servlet compression by rebuilding the war with those jars in place. I am thinking packaging, and you are thinking development. Please document this process somewhere if you can do it with filters, I have yet to find an article where this is done.

So let me state this again so Matti can rest in peace. Use container filters if you want, the addon is there if you want to use it, I am not forcing it down anybodies throat here. It is not an offical way to do it, it is just an addon you can use if you want.

PS. I really do not see why this addon would offend anyone, my view is that in open source alternatives are always good, it lets the community decide for its self what is best for them, without alternatives nothing new would exist. That is why we are building the Vaadin framework even though there are lots of other frameworks around and that is why the directory and incubator exists. The UIDL Compressor shows one way of doing things and there are other ways as well, you are free to choose. That is the beauty of it all.

Hi,

I probably don’t get it, but what is the difference of having your add-on package or jetty-util in your war file? Both are basically jar packages.

Here is an example war with jetty gzip filter and the very same war deployed to jetty 7.4 and tomcat 6. Both have compression on although it is not set on in the server:
http://dl.dropbox.com/u/4041822/vaadinstuff/app-with-gzipfilter.war
http://matti.virtuallypreinstalled.com/app-with-gzipfilter/ (tomcat 6)
http://vehje.tahvonen.fi:8080/app-with-gzipfilter/ (jetty 7)

And to rephrase myself, I think all this kind of add ons are good thing for Vaadin community. I brought up this discussion just to ensure that nobody thinks this is the de-facto (or the one and only) method of setting compression on for their web apps. At least I still believe that using the http level compression (in any of the several methods) is the superior over UIDL Compressor add-on.

cheers,
matti