You can take and compare screenshots with reference screenshots taken earlier. If there are differences, you can fail the test case.

The screenshot configuration parameters are defined with static methods in the com.vaadin.testbench.Parameters class.

screenshotErrorDirectory (default: null)
Defines the directory where screenshots for failed tests or comparisons are stored.
screenshotReferenceDirectory (default: null)
Defines the directory where the reference images for screenshot comparison are stored.
captureScreenshotOnFailure (default: true)
Defines whether screenshots are taken whenever an assertion fails.
screenshotComparisonTolerance (default: 0.01)
Screen comparison is usually not done with exact pixel values, because rendering in browser often has some tiny inconsistencies. Also image compression may cause small artifacts.
screenshotComparisonCursorDetection (default: false)
Some field component get a blinking cursor when they have the focus. The cursor can cause unnecessary failures depending on whether the blink happens to make the cursor visible or invisible when taking a screenshot. This parameter enables cursor detection that tries to minimize these failures.
maxScreenshotRetries (default: 2)
Sometimes a screenshot comparison may fail because the screen rendering has not yet finished, or there is a blinking cursor that is different from the reference screenshot. For these reasons, Vaadin TestBench retries the screenshot comparison for a number of times defined with this parameter.
screenshotRetryDelay (default: 500)
Delay in milliseconds for making a screenshot retry when a comparison fails.

For example:

@Before
public void setUp() throws Exception {
    Parameters.setScreenshotErrorDirectory(
        "screenshots/errors");
    Parameters.setScreenshotReferenceDirectory(
        "screenshots/reference");
    Parameters.setMaxScreenshotRetries(2);
    Parameters.setScreenshotComparisonTolerance(1.0);
    Parameters.setScreenshotRetryDelay(10);
    Parameters.setScreenshotComparisonCursorDetection(true);
    Parameters.setCaptureScreenshotOnFailure(true);
}

Vaadin TestBench allows taking screenshots of the web browser window with the compareScreen() command in the TestBenchCommands interface. The method has a number of variants.

The compareScreen(File) takes a File object pointing to the reference image. In this case, a possible error image is written to the error directory with the same file name. You can get a file object to a reference image with the static ImageFileUtil.getReferenceScreenshotFile() helper method.

assertTrue("Screenshots differ",
           testBench(driver).compareScreen(
               ImageFileUtil.getReferenceScreenshotFile(
                   "myshot.png")));

The compareScreen(String) takes a base name of the screenshot. It is appended with browser identifier and the file extension.

assertTrue(testBench(driver).compareScreen("tooltip"));

The compareScreen(BufferedImage, String) allows keeping the reference image in memory. An error image is written to a file with a name determined from the base name given as the second parameter.

Screenshots taken with the compareScreen() method are compared to a reference image stored in the reference image folder. If differences are found (or the reference image is missing), the comparison method returns false and stores the screenshot in the error folder. It also generates an HTML file that highlights the differing regions.

Reference images are expected to be found in the reference image folder, as defined with the screenshotReferenceDirectory parameter described in Section 20.6.1, “Screenshot Parameters”. To create a reference image, just copy a screenshot from the errors/ directory to the reference/ directory.

For example:

$ cp screenshot/errors/tooltip_firefox_12.0.png screenshot/reference/

Now, when the proper reference image exists, rerunning the test outputs success:

$ java ...
JUnit version 4.5
.
Time: 18.222

OK (1 test)

You can also supply multiple versions of the reference images by appending an underscore and an index to the filenames. For example:

tooltip_firefox_12.0.png
tooltip_firefox_12.0_1.png
tooltip_firefox_12.0_2.png

This can be useful in certain situations when there actually are more than one "correct" reference.