redundant JSON traffic

I only update the caption on my buttons, yet the traffic in JSON is so much more, although the control was unchanged:
for(;;);[{“changes”:[[“change”,{“format”: “uidl”,“pid”: “PID89”},[“7”,{“id”: “PID89”,“height”: “50px”,“width”: “50px”,“immediate”:true,“caption”: “20”,“v”:{“state”:false}}]
]], “meta” : {}, “resources” : {}, “locales”:}]

all these are redundant:
“height”: “50px”
“width”: “50px”
“immediate”:true

IDK about the rest as i’m not experienced with JSON.

Hi!

Vaadin terminal marks “dirty” parts of the UI at the component level. If a component is modified, all its properties are sent over to the client side. Components may of course do their own caching internally.

We sure could do the same thing at “attribute level” (and sending only caption in your case), but this would add complexity for both the client and the server side terminal of Vaadin. IMO the current solution is quite good compromise among cpu(client and server), memory usage(client and server), transferred data and complexity (which finally affects the quality of the library in terms of bugs and extensibility).

cheers,
matti

I don’t know how the server side is built, i only know i wouldn’t take the top down approach, but the approach in which everything is built based on need, not on the need to write code alone.

When a button’s caption is changed, the smallest set of data needed for the notification of the client is needed.
That is an id of the button beeing addressed, an id for the attribute of the control, and a value. The client’s terminal can do the rest.

When a button is pressed only its id needs be sent over to the server. Click beeing assumed the default control action, because with the id you can determine it’s a button at server side, if you have a client status.

EDIT: i guess optimizations are done last, but Vaadin is already robust enough, me thinks it’s ready to be optimized. From what you say i guess that’s build upon something else that has this redundancy by default. Managing less information however is easier on the CPU and bandwidth as well though.

EDIT2: Also, i was under the impression for a while that the MVP frameworks would do this:
inject a terminal into the client, which then initializes communication with the server, and only loads the controls needed, inlining css maybe as needed per control, or some other tehnique, in a way that would streamline the loading process to allow smallest bandwidth usage, wich would also speed up the loading.

Just a note - in the case of button the whole UIDL probably fits in one IP packet and thus optimizations would not help in practice.

Dear Raneti,

I understand well what you are suggesting. As I tried to explain there are several aspects in optimization of a software product like Vaadin. If the amount of transferred data on updates to already rendered components was our top priority, we would have already done optimization you suggested. If the transferred amount of data in whole (for small applications) was the top priority we would have taken totally different approaches in our implementation.

Although there are now days also big “web sites” build with Vaadin, Vaadin is designed for large enterprise applications, where hosting expenses are most often negligible compared to their development cost. Our aim is to develop a UI library that is highly productive for Java experts, who don’t speak browsers quirks as their second language or who don’t want to think of the complexities of web development. If I’d do something for the communication messages, design decisions would aim for easier API for component(extension) developers - the size of transferred XHR messages would not be to top priority and it could event get larger.

If the amount of transferred data is a problem in your application, I’d start by ensuring you have gzip compression on in your web server, then progress to optimizing widgetset size and strip away unused CSS (or even build a more simplified theme). The rest depends greatly on the characteristics of your application.

cheers,
matti

You’re right and you’re right.

First you’re right, a probable 1KB package traffic when it comes to updating the UI wouldn’t affect the client much, it would affect the server however me thinks, if it is on a low bandwidth such as mine, ~2MB/s upload, by ~400% perhaps. In this case i’m updating 6 buttons at once, and the target board may get bigger. I’m not thinking of web sites that attract less than 100 visitors a day at least, or intranet.

Then you’re right, you have to have prioritisation. The most important optimisation can be done in the code splitting. As i spoke about my impression how a MVP framework should be able to handle things, doing code splitting all the way down to the individual controls would be a bandwidth improvement, and would raise Vaadin’s scalability all the way to the top. The client should contain either ‘placeholders’ for starting time, and add controls as they appear in the client status maintained on the server side. Of course this wouldn’t matter on intranet, but it would definitely in the open spaces of the internet, and it would help the hosts very much. Again i’m not thinking of web sites that attract less than 100 visitors a day at least.

So, in short, my subject is about firstly about the further granularization of ‘code splitting’ or lazy loading, and secondly the granularization of JSON traffic, towards selective choosing of only the needed arguments.

The CPU overhead would be minimal; detection of data meaning sent to the server is already close to optimal. The issue beeing the choosing of an id that needs be updated on the client, and that’s readily available as before from the server side logic, so again we have an optimal case. The code splitting would already work browser based as it should, so there’s only the issue of picking only the updated value, instead of a predefined dozen (which doesn’t make sense to me). From what i understand from what you say, there’s a layer upon which Vaadin works that has the smallest grain, the control, wich has this predefined set of attributes to sent over, that i see in JSON. This implies modifying that layer so that it doesn’t figure out what really changed from the predefined set, but it only accepts and sends what changed in the logic. Is that layer under some sort of license?

The 6 buttons JSON (i’m only at the beginning, i can’t make assumptions, but it will definitely get worse, i’ll have a stage where a dozen controls, tables, buttons, Items get updated in content and maybe just color, but instead they will send a lot of other meaningless info):

In the above, only the pid and caption is relevant, not to mention going through more data requires a bit more time.
11341.png

Roadmap for 2010H2 - while as still being a bit under construction - includes redesign of the server-client communication protocol for Vaadin 7. I think that the attribute level caching you are proposing could be considered as a part of this redesign.

I’m glad Vaadin keeps going in the right direction. I should not be worried for another 6 months about what i mentioned, but afterwards, i might :grin: Besides i don’t think that thinking ahead is bad, don’t want to realise later that my work is impracticable based on the tools i chose.

Is code splitting per control? Meaning, it loads the controls lazily from the server side, only as needed, not from the initial Vaadin cache?
I already used it and i noticed some stuff still gets through with the Vaadin initial cache, such as some rich edit control and others i don’t need (that i understand i can weed out, but hoped code splitting would do it for me). Will it ever work the way i hope?

Also, about the css file, wich is 134K (only ~45% used in my app) and would take time to fix by hand, wouldn’t it be good if along with needed controls detection for a precompiled cache for control level code splitting, it would get automatically trimmed down for only the controls needed in the application? Perhaps not as easy as said, but i can dream can’t i? :grin:

The way i see it, priority should be:

  1. control level lazy loading, or code splitting (with precompiling for only the controls detected in the application)
    1.a or 3. parsing of the main css file determining only the needed classes
    1.b or 4. the Vaadin initial cache could be determined perhaps automatically from a mock run of the app, picking up the PIDs that get into the client status on the server side, then precompiled, to separate the ones that should make it in the initial cache (and show up at first in the client side), from the rest that should be lazy loaded
  2. a solution for the extra JSON traffic; can’t say more as i don’t understand why it was thought up as a control level update, or control layer, and not sent directly in the form of target element, attribute changed w/ attribute value.
    Based on the assumption that, quote: “HTTP/1.1 can reuse the same connection to download, for instance, images for the just served page. Hence HTTP/1.1 may be faster as it takes time to set up such connections” from (
    WIKI source
    ), Vaadin could let the updates run freely around in an async fashion, while putting the controls in place dynamically in the order they’ve arrived or whatever’s fastest.

Some interesting tools (although coversion to and from XML beats the point of be fast, there should be a point to this):

http://jsonml.org/


http://jsonml.org/bst/

In this example, you can change the control w/o talking to the server:

http://jsonml.org/BST/Example/

And it’s free open source licensed:

http://jsonml.org/License/

The point is I seem to remember something similar in GWT though, by changing the id you get a different control with no hassle with the server.

The ideea is to keep the control data server side depending on a choice, be able to inflate the client with controls dynamically and lazily, then use the cache of the client for UI changes alone, while only keeping data integrity with the server available on choice, the choice beeing if the server side logic is to be compiled into javascript and thus available from the client (no server talk needed), or keep it on the server side for security purposes. This implies the developer beeing aware that he’s in need to do server side validation on incoming data, that he needs to persist.

How about redundant email traffic coming for a single posting …

As an FYI, each time you click SAVE in the forums, you create the posting, and for those who receive emails about new postings, an email is sent out. So if you compose a single message and click SAVE 4 times as you add more text, you end up sending out 4 emails about your 1 topic.

Please use PREVIEW if you feel compelled to send the data to the server without posting it and use SAVE only once when done. I know we all post and then want to add more, but it would reduce the number of emails received per topic if PREVIEW were used until you were really ready to share your thoughts with all forum members.

Thanks!

Actually i was documenting while writing, so i added a few things, then i spotted some errors that i deleted, so i probably made 10 edits :grin: This would be the first forum in my life that sends out mails on edits :blink:
I’ll try to edit less, though i’m not a fan of rules instead of solutions (see Jaque Franco). I’m not perfect so I improve my postings whenever i find something, when I find it. It’s still faster to delete mails than write a reply :stuck_out_tongue: Probably the forum should send an [edit]
tag notification as well (that you can filter in your email box). Personally i don’t subscribe to any threads at all. Some forums send you mail about your own reply, while some mailing lists send you 95% of the time uninteresting stuff. It’s better that you filter what you can, as i don’t feel confortable about not editing what i wrote, as i can’t know now what i did wrong, and can’t leave errors unedited, to lead people astray.

You should be able to configure code splitting already in 6.4. Thus any combination of load in the first js, load when needed and load after application has been initialized should be achievable now for any widgets.

CSS analysis and automatic reduction would be great feature. Unfortunately server side analysis of CSS parsing in all different browsers (internet explorer) would be challenging and prone to introducing hard to debug visual bugs. Better alternative could be to bundle CSS together with widgets and let the GWT compiler handle combining of the CSS. Unfortunately this might introduce problems for theme switching and inheritance.

Actually it would be easy since you already know which classes to add to each control, and if you find out which control makes it in the precompiled bundle, you only need to pick up the relevant classes from styles.css and put them in a new file, at the end deleting styles.css and renaming the new one to styles.css, along with the global classes of course. Sure, it would be a trial and error process a couple of times, but in the end I’m sure it’s deterministic. Firebug has nice detection of css classes and is open source. IE web developer has that detection also. As for lazy loading controls, i don’t know, can css files inherit ‘dynamically’ the existing css? Either way it can be inlined. I don’t think anyone reads the source code on the client side other than for debug. It wouldn’t look nice, but it would be effective.

P.S. I have a problem with the widgetset and Helios. Helios won’t turn off the validators and is going through all the js reporting missing semicolon and such, which takes forever.