Running Tests on Multiple Browsers
A distributed test environment (i.e., a "Test grid") consists of a hub and many test nodes. The hub acts as an orchestrator, tracking which browsers are available in the nodes and making sure that a node is only used by one test at a time. The nodes may have one or several browsers installed. A node is where the actual test is executed.
When running a test on a hub, the TestBench test asks the hub for a certain browser — based on a list of capabilities — instead of launching a local browser. The hub waits until a suitable browser is available on some node, reserves it and redirects the test to that given node. The test is then executed. After it has finished, the node reservation is removed. The node may then be used for another test.
Note
| When running on a hub, you don’t need a local WebDriver installed. The WebDriver needs to be installed instead on the node. |
Preparing Your Tests for Running on a Test Grid
The tests created previously are set up only to run on a single browser since a single ChromeDriver
or other WebDriver
instance is created in a @BeforeEach
(@Before
for JUnit 4) method. When running on multiple browsers in parallel, it’s easier not to handle the driver instances manually but instead let TestBench handle creation and removal when needed.
To accomplish this, you need to do the following:
-
Let TestBench take care of creating and destroying driver instances as needed by annotating test with
@BrowserTest
(JUnit 5) or extendingParallelTest
instead ofTestBenchTestCase
(JUnit 4). -
Define the grid hub URL using either
-
@RunOnHub("hub.testgrid.mydomain.com")
on the test class (or a super class) or the system propertycom.vaadin.testbench.Parameters.hubHostname
or -
Configure Sauce Labs credentials and use Sauce Connect proxy to use Sauce Labs test grid. See Using Sauce Labs Test Grid
-
You’ll also need a test class annotated with @BrowserTest
(JUnit 5) or extending ParallelTest
(Junit 4) automatically to do the following:
-
Executes test methods on the hub defined using
@RunOnHub
or the corresponding system property; -
Creates a suitable WebDriver instance;
-
Terminates the driver after the test ends;
-
Grabs a screenshot if the test fails;
-
Supports running the test locally on only one browser for debugging, using
@RunLocally
or the corresponding system property; and -
Runs tests in parallel (see Advanced Testing Concepts for more information).
Using Sauce Labs Test Grid
To run tests on Sauce Labs, you first need to get an account at https://saucelabs.com/. Take note of your user name and access key.
Before starting a test, you need to set up the Sauce Connect proxy.
The proxy creates a secure connection between the Sauce Labs browser environment and your machine so that the Sauce Labs browsers can access a server you’re running locally. You should run the proxy with a tunnel identifier (-i some-tunnel-identifier
) to avoid conflicts with other tunnels that you or a CI server might create later.
When the proxy is up and running, you can run your tests by passing the Sauce Labs username and access key to it. This can be done using system properties like so:
mvn verify -Dsauce.user=<yourusername> -Dsauce.sauceAccessKey=<youraccesskey> -Dsauce.tunnelId=some-tunnel-identifier
or by first setting the credentials as environment variables and then running the tests. For example
export SAUCE_USERNAME=<yourusername>
export SAUCE_ACCESS_KEY=<youraccesskey>
export SAUCE_TUNNEL_ID=some-tunnel-identifier
mvn verify
When writing your tests, there’s nothing special about Sauce Labs that needs to be taken into account. You write them as described in Preparing Your Tests for Running on a Test Grid
When running tests on a CI/build server, you typically have a build step before the tests — where the Sauce Connect Proxy is started — and a build step after the tests — where the tunnel is torn down.
Defining the Browsers to Run Tests On
You can define the tested browsers and their versions in an environment variable TESTBENCH_GRID_BROWSERS
or system property com.vaadin.testbench.Parameters.gridBrowsers
with a comma-separated list. For example: -Dcom.vaadin.testbench.Parameters.gridBrowsers=chrome,firefox,safari-11,safari-9
.
If you don’t want to use the environment variable, you can define the configuration in your test class, in a method annotated with @BrowserConfiguration
. It returns a list of DesiredCapabilities
, typically describing which platform, operating system, and browser name and version should be used.
Typically, this is defined in a superclass for the test, so that all tests are run on the same browsers.
Here’s an example of this:
@RunOnHub("hub.testgrid.mydomain.com")
public abstract class AbstractIT extends ParallelTest {
@BrowserConfiguration
public List<DesiredCapabilities> getBrowserConfiguration() {
List<DesiredCapabilities> browsers =
new ArrayList<DesiredCapabilities>();
// Add all the browsers you want to test
browsers.add(BrowserUtil.firefox());
browsers.add(BrowserUtil.chrome());
browsers.add(BrowserUtil.ie11());
return browsers;
}
}
Note
|
The BrowserUtil helper methods create a DesiredCapability object which often works. To customize the versions and other values, annotate your test class using @BrowserFactory(MyBrowserFactory.class) and implement MyBrowserFactory by extending DefaultBrowserFactory .
|
Local Multi-Browser Tests
To run a multi-browser test locally, you can use the com.vaadin.testbench.Parameters.runLocally
system property — or a @RunLocally
annotation on the test class — to override what browser to run on. The value of the property or annotation should be the browser on which to run (e.g., chrome
or @RunLocally(Browser.CHROME)
). When RunLocally
is used, any hub configuration is also ignored and a local WebDriver is used.
Please remember that different browsers may require additional preparation before running tests. It’s always good to run initial browser configuration in @BeforeClass
/ @BeforeAll
phase.
Below are examples of drivers preparation:
@RunLocally(Browser.CHROME)
public abstract class AbstractIT {
@BeforeAll
public static void prepareDriver() {
WebDriverManager.chromedriver().setup();
}
}
1562D591-B570-45C4-8813-A278ADA35A7C