Has anyone go any ideas how best to test a Vaadin application?
I guess you could use Selenium but you would end up with slow application tests that are very fragile.
I would like to write a JUnit test typically would check that (say) pressing button X results in a particular lump of code being called.
I have been considering using the “humble view” pattern proposed by Martin Fowler. I suspect it would work rather well as Vaadin components implements lots of interfaces. Has anyone explored this option.
Just to follow up - TestBench will be added to
Vaadin Directory shortly with 1199 USD /
developer license price tag. If you do not want to wait, please
contact sales - they will provide you with a beta copy of the upcoming version (that you can try out free for 30 days) and could even give a discount if your development team is larger than a handful of developers.
With TestBench you can package any screen recorded tests as JUnit tests and run them from your continuous integration server. In addition to DOM-based assertions - you can also do pixel level comparisons to detect visual regressions in your theme with a number of web browsers. Vaadin Framework itself uses such testing extensively with hundreds of pixel-level tests run for each Vaadin Framework build.
It is good to know that one can write
application tests using selenium, but I am more concerned about
unit testing of my code - I would leave the testing of Vaadin code to you guys
-_-
Taking the a simple example of a button click changing the text on a label. I would like to write a unit code that does something like the following
button.fireClickEvent();
assertEquals( "new Message", label.getValue());
Not that I am not trying to test that the label actually works by updating the users browser, but only that my code called the setValue() method in the label.
I guess I could mock the label using Mockito an do something like
The correct method to call is button.fireClick(). The problem is that the event is protected. Created a ticket for it:
http://dev.vaadin.com/ticket/4530 .
Before this is resolved, you can call button.handleAction(null,null). It should do the exactly same thing, but sounds unintuitive.
How is this done currently?
I would like to Unit Test one screen that contains a Form with a search button and a result table.
More specifically I want to test that when the search button is clicked, the result table is updated.
I tried invoking .fireEvent, but like you said it’s protected (still is). Moreover, handleAction() does not exist anymore either.
I second this request. TestBench is not an answer to unit testing.
I’d like to understand how others are testing their Vaadin apps and hopefully getting advices. From what I can tell, we would need an extra support from the core to exercice the code touching the Vaadin API. But I’d be glad to be wrong.
Let’s take an example: I have created a wizard using the excellent Vaadin wizard add-on and I’d like to test that wizard. Say the wizard creates a contract, has multiple validations rules (including when/if you can go back) and lookup for existing relationship (a insured person). The transient state of the wizard is kept in a context attached to the wizard.
What I would like to do is to create such a wizard, fill in the form and trigger the next state (typically by “clicking” the next button). We can’t click a button programmatically (the method is protected). I think it should be that way. Putting it public is just fixing the button issue but I am sure there are much more (like triggering events, etc). What I think would be useful is a TestHelper or something around a Vaadin component/add-on that would be made for testing. A simple button could be wrapped in a ButtonTestHelper (not very interesting). But we could imagine a WizardTestHelper with test features.
The other thing that seems to be missing to me is layout validation. I don’t count the number of times where the debug window is giving me warnings about width and height of a component. Why can’t I just have a test validating my layout during the build?
Testing abilities is becoming a concern for us. On our PoC, we realized that the code coverage was lower than our other projects. When we tried to fix it, we hit this problem of not being able to exercice the code in layout and ui components.
How about the other use cases I am bringing? Again, unless you guys want to programmatically click a button in an application, I wouldn’t make that part of the public API. For a button that seems reasonable but for other components, it’s less obvious. Say that I want to validate what happens when I enter some value in a text field and the filed looses focus. Are we going to also offer a public method for that? A public method that anybody can access in production code?
Gave this some thought this morning. Basically when talking about TDD and Vaadin applications you want to separate the business logic from the UI completely so you are able to test the business logic using standard JUnit tests that do not involve the UI. Based on this you know that all your business logic methods works as expected and what remains is to test that the UI works as expected, i.e. after entering text “abc” into field “inputfield” and pressing “save” the method “save” in my business logic should be called with “abc” as a parameter (or alternatively a value “abc” should appear in the database). The issue is then “how can I create a test that enters ‘abc’ and pushes ‘save’” so I after that can in some way verify that the things that should have happened really happened. Or a test that does virtually anything that the user can do in the application? Or, in other words, how do I simulate a user?
Solution number 1 that is being is, use TestBench and record test scripts. This will ensure that you are actually testing the interface that the end user will see and you will be able to detect all issues that an end user could run into.
Solution number 2 which, if I understand correclty, is what Stéphane is looking for is a server side way to simulate a user so the tests can be pure JUnit tests runnable without a UI. Looking at the way Vaadin 6 components work it can be a bit tricky. Some things can be easily tested using the server side API, like the simple test above (inputfield.setValue(“abc”); button.click(); assert… assuming click() would exist as a public method). Some things, as mentioned, are not available through the server side API. On the other hand, all input from the client is passed through the changeVariables method and if you want to go there it should be possible to create a “Tester” class for each component that would provide an API for all the possible actions the client/user can perform. This Tester class would construct a suitable variables map for the changeVariables method and then call the method. You could then create tests like new TextFieldTester(inputfield).enterText(“abc”);new ButtonTester(saveButton).click();
There are potentially a few issues with this
The variables are internal to the component and there is no documentation on what they really mean or what (if any) variables the component assumes are always included in the map. This would have to be figured out separately for each component and would be a whole lot of work.
You need a way to access the components from your application. This might be solveable by e.g. creating fields for all components and marking them as package protected or public, or creating getters for each component.
Thinking about this lead to me considering what we are doing for Vaadin 7 at the moment, i.e. refactoring how the communication between the server and the client works. We are introducing client → server RPC methods that replace the changeVariables/updateVariables in Vaadin6 so what we really are doing is step 1 above although not for that particular reason. By introducing separate RPC methods for separate actions we are splitting the changeVariables in its logical parts and could provide a public API through which you would be able to call the RPC methods also from server side. In this sense Vaadin 7 should make step 1 a whole lot easier (or trivial).
If anybody is doing work on this or something similar it would be interesting to hear how well it is working or what other obstacles there are in getting this to work. I can imagine that in addition to the 2 issues above you will run into additional, smaller or larger, problems when doing this, for instance the communication might use internal identifiers to make the communication efficient and you are unable to get hold of the identifiers.
I believe a great reference for such an implementation is the UISpec4J project. I does exactly this for swing applications. The trick it uses is to wrap all the components with their test implementation (which include methods for dispatching the events and assertion). I think for Vaadin it may be actually simpler since you have a server side representation of the UI. So by injecting actions, making the test emulate a remote terminal, you should get the expect test behavior.
Again wrapping the implementations with test doubles should do the trick. Any change to the internal way would only affect the test doubles, not the actual test cases.
Again UISpec4J has a smart way to do this. The first thing you do is wrap the JFrame. This wrapper has a series of methods to find component, which return the wrapper classes for that component. E.g.:
I sign up to this request. I have done testing with UISpec4J and Mockito, and it’s really nice to have these test. They run really fast.I do believe this is a nice project to add to Vaadin. To me the critical thing is knowing how the internal operations are executed (specially at the terminal controller level).
Please try to use the [Karibu-testing]
(https://github.com/mvysny/karibu-testing) unit-test library. That way, you don’t need to use the Humble Pattern, nor you have to use Selenium. Karibu-Testing is 100x faster than Selenium and 100% reliable; it allows you to call button.click() to run your business logic “server-side” in the same JVM that your tests run in.