Selenium integration?

I am just starting to look at vaadin as a potential framework for a new portal application.

A major requirement is to be able to have browser based automated tests of the portal.

I have not been able to find any information about vaadins ability to be tested by browser based tools such as Selenium or webtest.

I tried a quick selenium test against the AddressBook demo and was a little disappointed. Many things did not seem to work “right out of the box” (try closing the help window - it seems to be dynamically named with a different name every test run).

Can anyone provide a pointer to some information that would help me to learn the best practise or capability of vaadin to be browser tested in an automated fashion?

Thanks

Hi Ken,

Selenium integration exists and it is called the Vaadin Testbench. A short info may be found
here
.

As you noticed, powerful AJAX-frameworks are pretty demanding for general and all-purpose UI testing software like Selenium. This is why we made our own integration and now it is available for Vaadin Pro customers. Yes, the Testbench is a commercial piece of software and it is included in various support options for Vaadin. More info
here
. So if you need Selenium styled UI testing, it is available.

Please don’t hesitate to contact our Vaadin Pro team if you wish to learn more about our Testbench.

Cheers,

  • Ville

P.S. If you are developing a portlet with Vaadin, I would suggest that you do as much developing and testing as possible outside the portlet container. This keeps your development environment a bit swifter. For example after testing your latest modifications it is just a matter of re-packing your Vaadin application to get in running inside a portlet container.

Hi there!

As you’ve noticed, Selenium doesn’t work “as-is” with Vaadin. The biggest problem is the lack of wide enough support for ajax in Selenium. When you click on something, it waits unti the page has finished loading in the traditional web fashion (“loading…” text in the bottom corner of most browsers) before doing the next step of the test. With vaadin, all of the loading happens in javascript after the initial load. Thus, Selenium doesn’t understand when something is still loading, and will think that it is always ready. This means that it will perform all the actions in the test way too fast, and it will fail because the application doesn’t keep up. I guess this is a problem when using ajax frameworks with Selenium.

Luckily, Vaadin has a fix for this. It has a own version, extension, out of Selenium called TestBench, where code has been coded in both Vaadin and Selenium, making them understand each others. Testbench’s UI is identical to Selenium IDE, and it also has an extension to Selenium RC.

Another feature is that Testbench has a better handling of the DOM tree than Selenium, which will make it more robust to changes. Selenium produces DOM paths like this:

//div[@id='sampler-1864843272']
/div/div[2]
/div/div[2]
/div/div/div/div[1]
/div/div/div[1]
/div/div/div/div[2]
/div/div/div/div[1]
/div[2]
/div/div[2]
/div/table/tbody/tr[18]
/td[2]
/div

while Testbench does this:

vaadin=sampler::/VVerticalLayout[0]
/ChildComponentContainer[1]
/VSplitPanelHorizontal[0]
/VPanel[0]
/VGridLayout[0]
/AbsolutePanel[0]
/ChildComponentContainer[1]
/VButton[0]

on the same UI component. Let’s say I change the page by adding a new label at top of the page. This will cause one more div somewhere in the Selenium path and the test will break. In Testbench, it would produce a VLabel between ChildComponentContainer and VButton. As the test does not look for a label, it looks for a button, it will work even if there is something new somewhere in between. You can take this even one step longer and put into java button.setDebugId(“theButtonIWantToPress”); and the Testbench url would look something like:

vaadin=sampler::/theButtonIWantToPress . That’ll work what ever changes you do around the components your testing. No need to re-record.

The third benefit with Testbench is a jar file for your project. You can save your tests in html, save them into your project. and you have the ability to translate them into Java on the fly with Testbench for jUnit integration. Thus, you can give them to some testing server, like cruiseControl for daily testing. You can do the html->java conversion in Selenium IDE as well, but you can’t change them back to html and modify the test after you have converted it.

There was the good news. Here comes the ‘not as good as that’-news. Testbench is not licenced under Apache Licence and open source, like Vaadin. One of the benefits of getting the support package from the company behind Vaadin is getting Testbench, along with many others.You can find more information about Testbench from

here.

Contact us on this forum or contact me directly if you wish to know more.

Best regards,
Jens.

Ah, Ville had the time to answer the question while I was writing. Oh well. :slight_smile:

You can also use free selenium without testbench additions BUT you cannot use test recorder and you will have to write all test manually.

To do it manually, add debugId’s in server-side and then use those in tests like clicking button:


click("//input[@id='myButton1DebugId']
");
waitForElementPresent("//div[@id='myLabelThatUpdates']
");
verifyText("//div[@id='myLabelThatUpdates']
", "Some result");

Jens & Ville: Thanks for the quick turn around response.

I was aware of the 2 links you provided, but the testbench page really did not provide enough useful information. It is a little light on content. The additional information in your posts has been very helpful (maybe you want to spruce the web page up a bit?).

So, in a nutshell: Testbench is Seleinium based, but with some Ajax improvements and some built in Vaadin specific knowledge that allows it to “work better” with vaadin.

I will contact you directly for more info.

I have the following question: I have a portlet developed with Vaadin which has two buttons, the first randomly displays the elements of a table, the second runs a search based on a captured text. Creating a Selenium test case I have the problem that the search is not executed with the command click, apparently has to do with which element has focus when you press the button. There is some way to do this? Only with testbench is the only way to do it?

Hi,

Vaadin Testbench exists because of this. With Vaadin Testbench you are able to test your Vaadin apps just like vanilla Selenium+non-Ajax-frameworks.

Testbench is going to be available in Vaadin Directory shortly and a free beta for 30-days evaluation is available as we speak. If you are interested, just send me an email.

Cheers,

Ville Ingman / ville-at-vaadin-dot-com

Hi all,
I started to use Vaadin recently and I find it very powerfull. I tried to integrate it with Selnium but I found some problems in particular with testing drag and drop. With my company we are thinking to buy a license for the Vaadin TestBench addon but will the addon let us to test vaadin drag and drop functionalities? Please let us know as soon as possible.
Thanks in advance,
Roberto

I’m not an expert on the Testbench, but all of our dual licensed and commercial add-ons include a 30-day trial period. This allows you to try it yourself and see if it fits your needs before making the purchasing decision.

More info about Vaadin add-on licensing:
http://vaadin.com/forum/-/message_boards/message/232868

TestBench 2.1 supports drag-n-drop with
some limitations
(to be fixed in 2.2):

In my professional life we are using a package from New Zealand (OrionHealth) to do the bulk of an Electronic Health Record interface, but in our next release some of the self-service configuration features will be done with Vaadin. I was wondering

  1. whether the enhancements made in TestBench would preclude using it for testing a non-Vaadin application (I am referring to Jens’ post earlier in the thread regarding DOM tree massaging). The non-Vaadin parts are boring HTML with some JavaScript DHTML gunk, and some light AJAX.

  2. has anyone done workflow testing in a push scenario: user 1 performs actions that change what user 2 sees and user 2 then changes something which changes what user 1 and 3 see, and so on. In other words, can testbench test the calendar push demo in true multiuser fashion ?

TestBench is in principle Selenium with the following additions:[list]

[]
Tons of bugfixes
[
]
Support for the stuff in Vaadin that is challenging for stock Selenium
[]
Screenshots and pixel-level regression testing
[
]
Enhancements for CI process

[/list]

Not that I know of. Sound challenging scenario to test with any tooling.

Selenium can use several locators to get a path for a component, and Vaadin adds/extends some to handle Vaadin components more robustly and neatly. I think the explicit buttons to the Selenium DOM and xPath locators have been removed from the TestBench UI or not, but the Vaadin locator should work fine also outside Vaadin components/applications, and any Selenium-supported path format should work at playback time. Sub-elements within HTML embedded in a Vaadin application will have a standard Selenium path rooted in the nearest suitable Vaadin path.

The TestBench recording and playback take place in the context of a single browser, so this is not trivial. You could try several test scripts run on different Selenium “remote controls” (agents) in parallel, waiting for specific events from each other, but I’m afraid this will be a little complicated and could be somewhat fragile depending on your runtime setup.

Most robust test on whether TestBench would work for you: Download and try it out to see how it works with your application.