You can take and compare screenshots with reference screenshots taken earlier. If there are differences, you can fail the test case.
The screenshot configuration parameters are defined with static methods in
the com.vaadin.testbench.Parameters
class.
screenshotErrorDirectory
(default: null
)screenshotReferenceDirectory
(default: null
)captureScreenshotOnFailure
(default: true
)screenshotComparisonTolerance
(default: 0.01
)screenshotComparisonCursorDetection
(default: false
)maxScreenshotRetries
(default: 2)screenshotRetryDelay
(default: 500
)For example:
@Before public void setUp() throws Exception { Parameters.setScreenshotErrorDirectory( "screenshots/errors"); Parameters.setScreenshotReferenceDirectory( "screenshots/reference"); Parameters.setMaxScreenshotRetries(2); Parameters.setScreenshotComparisonTolerance(1.0); Parameters.setScreenshotRetryDelay(10); Parameters.setScreenshotComparisonCursorDetection(true); Parameters.setCaptureScreenshotOnFailure(true); }
Vaadin TestBench takes screenshots automatically when a test fails, if the
captureScreenShotOnFailure
is enabled in TestBench
parameters. The screenshots are written to the error directory defined
with the screenshotErrorDirectory
parameter.
You need to have the following in the setup method:
@Before public void setUp() throws Exception { Parameters.setScreenshotErrorDirectory("screenshots/errors"); Parameters.setCaptureScreenshotOnFailure(true); ... }
Vaadin TestBench allows taking screenshots of the web browser window with
the compareScreen()
command in the
TestBenchCommands
interface. The method has a
number of variants.
The compareScreen(
takes a File
)File
object pointing to the reference
image. In this case, a possible error image is written to the error
directory with the same file name. You can get a file object to a
reference image with the static
ImageFileUtil.getReferenceScreenshotFile()
helper
method.
assertTrue("Screenshots differ", testBench(driver).compareScreen( ImageFileUtil.getReferenceScreenshotFile( "myshot.png")));
The compareScreen(
takes a base name of the screenshot. It is appended with browser
identifier and the file extension.
String
)
assertTrue(testBench(driver).compareScreen("tooltip"));
The compareScreen(
allows keeping the reference
image in memory. An error image is written to a file with a name
determined from the base name given as the second parameter.
BufferedImage
,
String
)
Screenshots taken with the compareScreen()
method
are compared to a reference image stored in the reference image folder. If
differences are found (or the reference image is missing), the comparison
method returns false
and stores the screenshot in the
error folder. It also generates an HTML file that highlights the differing
regions.
Screenshots with errors are written to the error folder, which is
defined with the screenshotErrorDirectory
parameter described in Section 20.6.1, “Screenshot Parameters”.
For example, the error caused by a missing reference image could be
written to
screenshot/errors/tooltip_firefox_12.0.png
. The
image is shown in Figure 20.13, “A screenshot taken by a test run”.
Screenshots cover the visible page area in the browser. The size of
the browser is therefore relevant for screenshot comparison. The
browser is normally sized with a predefined default size. You can set
the size of the browser window with, for example,
driver.manage().window().setSize(new Dimension(1024,
768));
in the @Before
method. The size
includes any browser chrome, so the actual screenshot size will be
smaller.
Reference images are expected to be found in the reference image
folder, as defined with the
screenshotReferenceDirectory
parameter described in
Section 20.6.1, “Screenshot Parameters”. To create a
reference image, just copy a screenshot from the
errors/
directory to the
reference/
directory.
For example:
$
cpscreenshot/errors/tooltip_firefox_12.0.png
screenshot/reference/
Now, when the proper reference image exists, rerunning the test outputs success:
$
java ...
JUnit version 4.5
.
Time: 18.222
OK (1 test)
You can also supply multiple versions of the reference images by appending an underscore and an index to the filenames. For example:
tooltip_firefox_12.0.png tooltip_firefox_12.0_1.png tooltip_firefox_12.0_2.png
This can be useful in certain situations when there actually are more than one "correct" reference.
You can make masked screenshot comparison with reference images that have non-opaque regions. Non-opaque pixels in the reference image, that is, ones with less than 1.0 value, are ignored in the screenshot comparison.
Vaadin TestBench supports advanced difference visualization between a
captured screenshot and the reference image. A difference report is
written to a HTML file that has the same name as the failed
screenshot, but with .html
suffix. The reports are
written to the same errors/
folder as the
screenshots from the failed tests.
The differences in the images are highlighted with blue rectangles. Moving the mouse pointer over a square shows the difference area as it appears in the reference image. Clicking the image switches the entire view to the reference image and back. Text "Image for this run" is displayed in the top-left corner to identify the currently displayed screenshot.
Figure 20.14, “The reference image and a highlighed error image” shows a difference report with three differences. Date fields are a typical cause of differences in screenshots.
Access to the screenshot reference image directory should be arranged
so that a developer who can view the results can copy the valid images
to the reference directory. One possibility is to store the reference
images in a version control system and check-out them to the
reference/
directory.
A build system or a continuous integration system can be configured to automatically collect and store the screenshots as build artifacts.
Screenshots when running Internet Explorer 9 in Compatibility Mode
Internet Explorer prior to version 9 adds a two-pixel border around the content area. Version 9 no longer does this and as a result screenshots taken using Internet Explorer 9 running in compatibility mode (IE7/IE8) will include the two pixel border, contrary to what the older versions of Internet Explorer do.