I am evaluating ITMill and Wicket for our new development project. How is ITMill compared to Wicket performance wise? Have anyone run performance test against Wicket? Would ITMill scale?
Not that I know anything about the Wicket framework, but it seems that it is not same kind of Ajax framework as the IT Mill Toolkit.
What it comes to the performance of applications I would boldly say that both perform well and are scalable enough. 
Actually, I have made some performance tests years ago with version 3 of the IT Mill Toolkit (known as Millstone back then), but could not find the results anymore. Back then the server-side was capable of handling several thousands of simultaneous users.
Of course this depends on the application itself and back-end logic. For example back then bottleneck was identified to be the Hibernate-layer.
As the IT Mill Toolkit uses GWT, I think that the client-side performance can be directly compared to performance of GWT applications. Any studies on this?
Wicket is new to me too, but I noticed with a quick look that making the UI’s seems very different in the frameworks.
In Wicket you write html directly which is shown to the user. With IT Mill Toolkit you code it in Java, more comparable with Swing. Sizes, swapping components etc. is directly calculated for you. Result wise I’d say you get a more of an application like look’n’feel with Toolkit where as in Wicket it looks more like an traditional web page.
The benefit with toolkit here is that you can do more powerful and dynamic interfaces. The benefit with Wicket is that you probably have some knowledge in HTML and get a nice steep learning curve, being instantly ready to just jump into (that part of) the framework.
I’d say the choice depends more on how the end result shall look, and possibly your preference.
This had nothing to do with performance, I’d say your well set performance wise choosing either one. This was more of an comparison of developing the view layer.
I took a quick look and Wicket looks a bit like IT Mill Toolkit Release 3 was many years ago: it’s a dynamic HTML application framework. It produces HTML pages where you can have form fields and such. Clicking the submit button sends the page to the server with a GET or POST request, and not using Ajax. Wicket does appear to have some support for Ajax, but it is not the core of the framework, as is the case with IT Mill Toolkit. In practice that could mean that if you need Ajax in your application even a bit, development can become many times more difficult.
As the pages in Wicket are normally just plain HTML, they probably load much faster than with IT Mill Toolkit, which first loads the client-side engine and then renders the page with Ajax (doing HTML DOM manipulation with JavaScript compiled with GWT). But after loading the page, the complete Ajax approach of IT Mill Toolkit should usually make the UI faster, especially if the application is complex.
I’m not sure how much Wicket has support for Ajax. Their examples do not include a live demo for the only Ajax example they have. If the components in Wicket are not fully Ajax-enabled, you’ll be implementing a lot of components yourself while in IT Mill Toolkit they all come ready.
Some benefits of IT Mill Toolkit over Wicket:
- A lot less wiring to do (in wicket you must connect the html-templates to java code with unique ids)
- Less code to write - in Toolkit one Java-file is enough
- Widgets in IT Mill Toolkit have a lot more features than the ones in Wicket. Just see the demos in both frameworks
- In IT Mill Toolkit everything is Ajax by default (no extra work), see Wicket Ajax example on how you must enable/build Ajax for each widget in Wicket
Thanks for replies. Actually all of the mentioned benefits of using ITMill over Wicket has been noted in our evaluation process. We just need to know that if our users based expand in the future, will we be able to cope with the load. We don’t mind upgrading the new servers. However, the real concern is “Will we be forced to rewrite our application because of our application might not scale?”. In that case, we might just as well invest labor more now by using Wicket instead. (Just the same reason we still doing Java, a seemingly more laborious choice “now”, instead of other “cheesy” dynamic languages. Anyone have more input on just “Performance and Scalability” issues, not other benefits?
I have run scalability tests for Toolkit 5 and the question is pretty much the same as “Would servlets scale?”. Of course Toolkit contains more Java objects than plain servlets that simply write HTML to sockets (a’ka JSP pages) but the difference is negligible. Whatever the UI framework, in any case you need various tools on the server-side to handle communication between client and server in properly, data layer and persistence etc. Therefore, my answer is that scalability is no concern for Toolkit, it concerns other parts in your architecture such as how do you code your business logic or how do you handle your data layer (persistency / ORM).
If you wish to find out the facts yourself then there’s two excellent tools that I have used often and can recommend.
- Apache jMeter
- record few testcases by interacting with the application using web browser
- playback testcases with 1000 concurrent threads
- results tell you facts for throughput
- e.g. various latency values
- e.g. even graphs
- jProfiler
- start your server through profiler
- results tell you memory consumption
- e.g. how much one application instance eats up memory?
- e.g. how much 500000 queries with 10000 unique sessions eat up memory?
- e.g. does it leak memory?
With these tools you can even do robustness testing easily, bomb servers with extreme load => observe how resources cannot cope with the load => ensure that when peak no longer exists all servers resume normal operation…
Tools are quite simple to use but scalability and robustness testing does require some experience to do it correctly. I’ve done above testing few times during last years, always we have spend time optimizing our business logic (not Toolkit UI related code). I’ll try to see if I can still find any of my results somewhere for some application.
Best of luck with your evaluation of RIA frameworks!
Thanks Jani. Excellent suggestion.
Feel free to throw me private message or skype chat in case you need sparring with your ideas or troubleshooting help… (search for Jani Laakso at Finland).
PS. Do you got business to business or business to consumer app? What kind of concurrent app count are you expecting?
Our application is a complete ERP solution. Currently, it is a Swing application. We would like have it available as web application which we host it at a hosting center.
Your case seems to be exactly what IT Mill Toolkit is made for - if something within the framework prevents your app from scaling, it’s essentially a bug and should be fixed.
As noted earlier, it will scale pretty much in the same way that servlets generally scale (which of course depends on how your application is built). Since there is a per-user session with a server state, it’s for instance quite possible to make an application that stores too much per-user data in-memory, but that’s of course the case with any application. Provided you build the application sanely, minimizing the application-specific resource usage by sharing data between users as appropriate and constructing the views lazily etc, the overhead compared to a ‘plain’ servlet is quite small.
We’re aware that we should gather saleability data into a study or whitepaper on this subject - there have been several studies, but they have all been made in the context of customer applications, so we’re unable to provide that data as-is. If I remember correctly, there is even a trac ticket for this task… But in the mean time, the best you can do is to follow Janis suggestions. And report back!
Best Regards,
Marc
Important notice!
You may have to use the following init-parameter (but it into web.xml inside tags of your application) when testing your toolkit application, otherwise the JMeter testing may fail:
<init-param>
<param-name>disable-xsrf-protection</param-name>
<param-value>true</param-value>
</init-param>
But, don’t forget to remove the parameter when the testing is done!
I wrote short tutorial of how to set up JMeter test of Toolkit application. You could find the tutorial from Technical Articles section of dev.itmill.com wiki:
Can someone explain what’s it for and what it does?
It turns of the automatic Cross Site Request Forgery (aka CSRF or XSRF) protection. You really want the protection enabled, only disable it for specific testing purposes.
Wikipedia writes about CSRF/XSRF here: http://en.wikipedia.org/wiki/Cross-site_request_forgery
Best Regards,
Marc
Sorry this thread is a bit hold but I thought I would still give some feedback, or rather a question about the xsrf parameter.
When in a scenario with “login, some actions, logout” it seems that we use session scope and therefore the Vaadin_Key (in JSON) [+maybe the cookie]
to re-attach a request in jmeter with its corresponding session.
is this true ? is the cookie enough ? do you need the Vaadin_Key in each request ?
My experience so far is that when disabling the xsrf parameter and running many threads in parallel, the whole session mechanisms does not work properly anymore.
I made the jmeter stress test working with the xsrf parameter enabled and some “Regular Expression” extraction to manage the Vaadin_Key passing in each request of the scenario.
Can someone give me some more info on the mechanism behind this Cookie, Vaadin_Key and session management ?
Hi,
The JSESSIONID cookie is used to identify the session, while the Vaadin-Security-Key is used to protect from cross-site scripting. Both are needed to communicate with the server - first the session is identified using the cookie, then the security key is checked for that session. If the security key does not match, the session will also be invalidated. So you do need both the session cookie and the security key to maintain the sesssion.
Note that the Vaadin-Security-Key is used for UIDL (JSON, ajax) requests, not when drawing the HTML init page, or other types of requests (resource etc).
Does this answer your question?
Best Regards,
Marc
Hi All,
We are trying do load test of our vaadin application using jmeter. But while recording using jmeter record controller , it is not able to record tab clicking or button clicking event on vaadin portlets.
Any suggestion regarding this will be highly appreciated.
Thanks
Chinmaya Sahu