What is Testerra?
Getting Started
1. Create a new project
1.1. System requirements
-
Testerra is based on Java. You need a JDK 8 or later.
-
Execute your tests with Maven or Gradle
1.2. Testerra Skeleton project
We provide a skeleton project to demonstrate the basic features of Testerra.
1.3. Testerra manual setup
1.3.1. Setup
Testerra and all its components are deployed to MavenCentral: https://mvnrepository.com/artifact/io.testerra
For Testerra you need at least the following dependencies.
// build.gradle
apply plugin: 'java'
// Its highly recommended to normalize your project to Unicode
compileJava.options.encoding = 'UTF-8'
compileTestJava.options.encoding = "UTF-8"
dependencies {
compile 'io.testerra:driver-ui-desktop:2.0-RC-5'
compile 'io.testerra:report-ng:2.0-RC-5'
}
<!-- pom.xml -->
<project>
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<!-- Its highly recommended to normalize your project to Unicode -->
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>io.testerra</groupId>
<artifactId>driver-ui-desktop</artifactId>
<version>2.0-RC-5</version>
</dependency>
<dependency>
<groupId>io.testerra</groupId>
<artifactId>report-ng</artifactId>
<version>2.0-RC-5</version>
</dependency>
<!-- These dependency are required to get logging to work in Maven -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j18-impl</artifactId>
<version>2.13.2</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.13.2</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.13.2</version>
</dependency>
</dependencies>
</project>
1.3.2. Create project structure
Your project structure should comply with these simple constraints.
-
src/main
Contains all the code for your project like PageObjects, Models and language specific resources. -
src/test
Contains all test related code like your Tests, Test-Suites, Test-Data and Testerra related setup.
1.3.3. Create test.properties
Create a new file at src/test/resources
with the name test.properties
.
# Setting the browser
tt.browser=chrome
# Setting the start page
tt.baseurl=http://example.org
All defined properties can be overwritten later by adding system parameters to your command (e.a -Dtt.browser=firefox )
|
All supported browsers are listed in WebdriverManager properties
1.3.4. Create Page Class
Now it’s time to create a first simple page class. It should be saved at path src\main\java\<package>
. The following example represents the website example.org. It contains one possible link to click and one method to test.
import eu.tsystems.mms.tic.testframework.pageobjects.Page;
import eu.tsystems.mms.tic.testframework.pageobjects.UiElement;
public class ExamplePage extends Page {
@Check
private UiElement moreInformationLink =
find(By.partialLinkText("More information"));
public ExamplePage(WebDriver driver) {
super(driver);
}
public void clickOnMoreInformation() {
moreInformationLink.click();
}
}
The basic Page class added all the page object functionality of Testerra to your project. See PageObjects chapter for more details. The UiElement describes the elements like links, buttons, etc. on your page. Learn more about UiElements in UiElements. |
1.3.5. Create Test Class and Test Method
The easiest way to create a new test, is by creating a new class in the path of src\test\java\<package>
and let it extend from TesterraTest
.
If you already have test classes that extend, you can add the TesterraListener
manually. Both ways do basically the same.
To stick to the example above, here is a very simple test class which navigates to example.org and clicks on the link defined on the example page. Again, probably imports must be made in IDE.
import eu.tsystems.mms.tic.testframework.testing.TesterraTest;
import eu.tsystems.mms.tic.testframework.testing.PageFactoryProvider;
public class ExampleTest extends TesterraTest implements PageFactoryProvider {
@Test
public void testT01_My_first_test() {
ExamplePage examplePage = PAGE_FACTORY.createPage(ExamplePage.class);
examplePage.clickOnMoreInformation();
}
}
Be aware of using If you import |
import eu.tsystems.mms.tic.testframework.report.TesterraListener;
import org.testng.annotations.Listeners;
@Listeners(TesterraListener.class)
public class ExampleTest {
}
1.3.6. Setup Selenium
If you don’t have a remote selenium yet, you can easily install it by the package manager of your choice.
choco install selenium selenium-chrome-driver
apt-get install chromium-chromedriver
brew install selenium-server-standalone chromedriver
Read here, if you want to setup another Selenium configuration.
1.3.7. Setup a test suite
To customize the executing of your tests, you have to create a TestNG suite file suite.xml
and locator it at src/test/resources
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="Suite1" verbose="1" thread-count="10" configfailurepolicy="continue" parallel="false">
<test name="Test1" parallel="methods">
<classes>
<class name="ExampleTest"/>
</classes>
</test>
</suite>
1.3.8. Setup test build target
In order to get tests to work, you need to setup a build target test
in your project.
// build.gradle
test {
useTestNG() {
suites file('src/test/resources/suite.xml')
}
testLogging {
outputs.upToDateWhen { false }
showStandardStreams = true
}
// Important: Forward all JVM properties like proxy settings to TestNG
options {
systemProperties(System.getProperties())
}
// basically execution returns "GREEN" (framework exits with exit code > 0 if there were failures)
ignoreFailures = true
}
<!-- pom.xml -->
<project>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>true</skip>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>mySuite</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>false</skip>
<suiteXmlFiles>
<suiteXmlFile>src/test/resources/suite.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
1.4. Using a proxy
There are three ways for setting up a proxy for your test run environment.
-
System proxy settings for the build environment (Maven, Gradle), TestNG, JVM and Selenium
-
Browser proxy settings for the SUT, which is done by capabilities as described here Proxy setup
1.4.1. Command line (recommended)
To setup a proxy for the whole system, including the build environment (Maven, Gradle), the JVM and Testerra, the recommended way is to pass it by command line arguments like
gradle test -Dhttps.proxyHost=your-proxy-host.com -Dhttps.proxyPort=8080
1.4.2. Property file
You can also put your proxy settings to the system Property files with the following content
https.proxyHost=your-proxy-host.com
https.proxyPort=8080
https.proxyUser=
https.proxyPassword=
https.nonProxyHosts=localhost|192.168.0.1
http.proxyHost=your-proxy-host.com
http.proxyPort=8080
http.proxyUser=
http.proxyPassword=
1.4.3. Access the system proxy URL
The system proxy can be accessed by Proxy Utilities
1.4.4. Implicit setup (not recommended)
Since Java 11, it is possible to pass the system’s preconfigured proxy into the JVM.
gradle test -Djava.net.useSystemProxies=true
This affects all Java internal network connections which uses ProxySelector
, but it will not set the environment variables and are transparent to Proxy Utilities and any Browser capabilities.
1.5. Logging
The log configuration prints out to System.out
by default.
If you want to have more control over several log levels of classes, add a log4j2.xml
to your resources/
.
<?xml version="1.0" encoding="UTF-8" ?>
<Configuration packages="eu.tsystems.mms.tic.testframework.logging">
<Appenders>
<Console name="CONSOLE">
<!--
The marker %contextIds gets replaced by internal plugins registered from
plugins packages in the <configuration> node
-->
<PatternLayout pattern="%d{dd.MM.yyyy HH:mm:ss.SSS} [%t][%p]%contextIds: %c{2} - %m%n" />
</Console>
</Appenders>
<Loggers>
<Root level="info">
<AppenderRef ref="CONSOLE"/>
</Root>
</Loggers>
</Configuration>
You can also change the root log level from the command line via.
-Dlog4j.level=DEBUG
Testerra Framework
2. WebDriverManager
2.1. Overview
The WEB_DRIVER_MANAGER
is the central component to create and close your WebDriver
sessions and becomes available by implementing the WebDriverManagerProvider
interface. It uses the standard Selenium Webdriver, but it is easier to configure.
2.2. WebDriver sessions
Before you can use WebDriver
sessions, you have to setup a Selenium-compatible server.
2.2.1. Setup remote sessions
For using a remote Selenium server (e.g. a Selenium Grid) you only have to tell Testerra where it can be find.
tt.webdriver.mode=remote
# Selenium host and port
tt.selenium.server.host=localhost
tt.selenium.server.port=4444
# Or combined as URL
tt.selenium.server.url=http://localhost:4444/wd/hub
The browser support depends on the remote selenium setup. |
2.2.2. Create a local session (not recommended)
Regularly, there is no reason to use local Selenium sessions based on binaries. We won’t recommend using that, because we’ve experienced slightly differences between the local and remote behaviour.
However, Testerra supports using local web drivers.
-
Search for your browser’s selenium driver online, like selenium gecko driver for Firefox or selenium crome driver for Chrome based browsers
-
Download your WebDriver binary from browser vendor’s website to a local location
-
Make sure the driver version supports your installed browser
-
Since the properties are system properties, you need to put the location of the binaries to the
system.properties
file as mentioned in the Property files section.
tt.webdriver.mode=local
webdriver.gecko.driver="C:\\absolute\\path\to\\your\\geckodriver.exe"
# or for other browsers
webdriver.chrome.driver=...
webdriver.edge.driver=...
webdriver.ie.driver=...
You can also pass the wedriver by the command line using
-Dwebdriver.gecko.driver=C:\absolute\path\to\your\geckodriver.exe
2.2.3. Usage of WebDriver sessions
Before starting a WebDriver
session, you should configure your desired browser like.
# Browser name and version
tt.browser=firefox
tt.browser.version=65
# Or both combined
tt.browser.setting=firefox:65
On the first call of
WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
Selenium is triggered to open a new Browser windows with the defined URL.
Use remote Selenium server as often as possible, also for development.
So your project is independent of any WebDriver configuration and needed Webdriver binary files.
|
For every other call of getWebDriver()
in the same test context WebDriverManager always returns the existing session.
This makes it possible to retrieve the current session in any context and avoids to force the user to pass the instance around.
2.2.4. WebDriver lifecycle
The default behaviour of Testerra’s WebDriverManager
is, to create unique WebDrivers for each thread and/or test method.
That prevents issues in mutual interference between multiple threads.
-
Every WebDriver created in a thread, will be closed in this thread after the test method. See Use multiple sessions
-
You can keep a WebDriver in a thread over multiple methods. See Shared sessions in one thread
-
You can reuse WebDrivers by sharing sessions over different threads
2.2.5. Use multiple sessions
The WebDriverManager can handle more than one session in one test context.
Every session has a defined session key.
If no key was set, the default session key is called default
.
The following example creates two independent browser sessions:
WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
WebDriver driverWindow2 = WEB_DRIVER_MANAGER.getWebDriver("window2");
// Get the session key
String key1 = WEB_DRIVER_MANAGER.getSessionKey(driver); // key1 contains 'default'
String key2 = WEB_DRIVER_MANAGER.getSessionKey(driverWindow2); // key2 contains 'window2'
2.2.6. Close a session
In most cases it is not needed to close your session manually. Testerra always closes all open session created in the thread at the end of a test method.
Anyway, to close active sessions manually, do the following:
// Shutdown a session by key
WEB_DRIVER_MANAGER.shutdownSession(String);
// Shutdown an explicit driver
WEB_DRIVER_MANAGER.shutdownSession(WebDriver);
// Close all active session in the current test context.
WEB_DRIVER_MANAGER.shutdownAllThreadSessions();
// Close all active session in all current parallel test threads.
WEB_DRIVER_MANAGER.shutdownAllSessions()
Please do not use Selenium provided methods to close |
2.2.7. Shared sessions in one thread
You can reuse the WebDriver session over multiple methods in the same thread by setting the following property:
tt.wdm.closewindows.aftertestmethods=false
@Test
public void test1() {
WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
}
@Test
public void test2() {
// You get the already opened session from test1
WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
}
You can only reuse the session in the current thread. If you run parallel tests, you have no access to the session between parallel test threads. |

Special behaviour for config methods (deprecated)
When you create a WebDriver in a setup method by is annotated with @Before…
or @After…
, the session will not be closed after that method, even when tt.wdm.closewindows.aftertestmethods
is true.
@BeforeMethod
public void setupWebDriver() {
// This WebDriver will not be closed, because its a setup method
WebDriver driver = WebDriverManager.getWebDriver();
}
@Test
public void test1() {
// You get the already opened session
WebDriver driver = WebDriverManager.getWebDriver();
}
2.2.8. Shared sessions over different threads
To use a WebDriver session over different test threads, you need an exclusive session.
Exclusive sessions are identified by a special uuid, not by the standard session key.
private static String uuid = null;
@Test
public void test1() {
WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
uuid = WEB_DRIVER_MANAGER.makeExclusive(driver);
}
@Test
public void test2() {
// Get the exclusive session
WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver(uuid);
}

An exclusive session has an unlimited lifetime. You need do close this session manually.
|
2.3. WebDriver configuration
An WebDriver can be configured in the following ways.
A global configuration applies to all new sessions created by WebDriverManager. You can set a global configuration by.
-
Setting defaults as Properties
-
Configure the WebDriver on request
-
at runtime by using the WebDriverManagerConfig (
@deprecated
)
2.3.1. Property configuration
Like shown above all session properties can be set in test.properties
.
Property | default | Description |
---|---|---|
tt.browser |
na. |
Browser label, the following browsers are possible
|
tt.browser.version |
na. |
Version label |
tt.browser.setting |
You can combine browser type and version as single string like |
|
tt.baseurl |
na. |
URL of the first site called in a new browser session |
tt.webdriver.mode |
remote |
Sets the webdriver mode. remote uses an external Selenium server |
tt.selenium.server.url |
na. |
The complete URL to a remote Selenium server. This setting overrides the following two properties. |
tt.selenium.server.host |
localhost |
The host name of the remote Selenium server. |
tt.selenium.server.port |
4444 |
The port of the remote Selenium server. |
tt.browser.maximize |
false |
Try to maximize the browser window. |
tt.browser.maximize.position |
self |
Screen position for the window to maximize. If you have several screens and want to maximize the window on another screen than your default screen, you can choose between (left, right, top or bottom) |
tt.display.resolution |
1920x1200 |
Fall-back resolution, if maximize does not work. |
tt.wdm.closewindows.aftertestmethods |
true |
If true, after every test method all open browser windows are closed. |
tt.wdm.closewindows.onfailure |
true |
If true, after failed test methods all open browser windows are closed |
tt.wdm.timeouts.seconds.selenium.command.stuck |
300 |
Kills a stuck selenium command after this timeout |
tt.wdm.timeouts.seconds.window.switch.duration |
5 |
Maximum duration to wait for on a |
webdriver.timeouts.seconds.pageload |
120 |
Defines the Selenium timeout for page load seconds. |
webdriver.timeouts.seconds.script |
120 |
Defines the Selenium timeout for execution of async scripts in seconds. |
2.3.2. Request configuration
If you only want to change the settings for one session, you can use WebDriverRequest
.
All defined attributes overrides the standard configuration.
If an attribute is not set, the global definition is used. |
DesktopWebDriverRequest myRequest = new DesktopWebDriverRequest();
myRequest.setBaseUrl("http://example.org");
myRequest.setBrowser(Browsers.firefox);
myRequest.setBrowserVersion("66");
myRequest.setSessionKey("mysession"); // if no custom session defined, the default value 'default' is set
WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver(myRequest);
2.3.3. Configure with WebDriverManagerConfig (@deprecated
)
Since this WebDriverManagerConfig
is actually a DesktopWebDriverRequest
, you should prefer using that when creating a new WebDriver.
Some of the WebDriverManager settings you can change as follows
// tt.wdm.closewindows.aftertestmethods
WEB_DRIVER_MANAGER.getConfig().setShutdownSessionAfterTestMethod(true);
// tt.wdm.closewindows.onfailure
WEB_DRIVER_MANAGER.getConfig().setShutdownSessionOnFailure(false);
// tt.browser.maximize
WEB_DRIVER_MANAGER.getConfig().setMaximizeViewport(true);
Keep in mind, that after the complete test run a session shutdown is being forced. |
You can reset the settings to the default values or defined in test.properties
as follows
WEB_DRIVER_MANAGER.getConfig().reset();
2.4. Working with sessions
2.4.1. Get current session
WebDriver driver = WebDriverManager.getWebDriver();
Optional<SessionContext> sessionContext = WebDriverSessionsManager.getSessionContext(driver);
2.4.2. Switching windows
Sometimes you will come across testing websites with multiple windows, such as popups or something similar.
// Switch to a window by matching title
WebDriver webDriver = WEB_DRIVER_MANAGER.switchToWindowTitle(WebDriver webDriver, String windowTitle);
// Switch to a given window handle
WebDriver webDriver = WEB_DRIVER_MANAGER.switchToWindowHandle(WebDriver webDriver, String windowHandle);
// Switch to a window by a custom condition
Optional<WebDriver> optionalWebDriver = WEB_DRIVER_MANAGER.switchToWindow(WebDriver webDriver, webDriver -> webDriver.getCurrentUrl().contains("login"));
Assert.assertTrue(optionalWebDriver.isPresent(), "Login window not found");
3. Browser capabilities
You can customize your WebDriver session by setting capabilities in the following ways:
-
Global capabilities that affect every WebDriver session created
-
Request capabilities that affect only WebDrivers for a given request
-
User agent configuration that affect only WebDrivers for a specified browser
When creating a new WebDriver, these capabilities get merged together in this exact order.
3.1. Global capabilities
You can customize your browser session by setting capabilities for every browser type before the WebDriver has been initialized.
WEB_DRIVER_MANAGER.setGlobalCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true);
WEB_DRIVER_MANAGER.removeGlobalCapability(CapabilityType.ACCEPT_INSECURE_CERTS);
Do NOT set browser capabilities with WebDriverManager like:
|
3.2. Request capabilities
Some WebDriverRequests support setting capabilities, like the DesktopWebDriverRequest
.
WebDriverRequest myRequest = new DesktopWebDriverRequest();
myRequest.getDesiredCapabilities().put(CapabilityType.ACCEPT_INSECURE_CERTS, true);
WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver(myRequest);
3.3. User agent configuration
The user agent configuration is most precise, because it provides explicit browser options based on the Selenium driver.
import eu.tsystems.mms.tic.testframework.useragents.FirefoxConfig;
WEB_DRIVER_MANAGER.setUserAgentConfig(Browsers.firefox, (FirefoxConfig) options -> {
options.addPreference("intl.accept_languages", "de-DE");
});
Have a look into Browser specific knowledge for specific browser options. |
3.4. Setting local capabilities
A local defined capability means its only available in the current test execution (current TestNG testmethod).
WebDriverManager.addThreadCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true);
Have a look into [Useful browser capabilities] for specific browser options. |
3.5. Proxy setup
If you want that the browser uses a proxy for the SUT, you can just configure that by default Selenium capabilities.
Make sure that your WebDriver supports the Proxy-Capability. For example the MicrosoftWebDriver for Legacy Edge does not support proxy setup (see Edge WebDriver Capabilities). |
If you want to setup a proxy for the runtime environment but the browser, you have to follow the instructions at Using a proxy |
The following code setups a proxy based on the System’s proxy configuration and a custom proxy.
import org.testng.annotations.BeforeSuite;
import org.openqa.selenium.remote.DesiredCapabilities;
import eu.tsystems.mms.tic.testframework.webdrivermanager.WebDriverManagerUtils;
import eu.tsystems.mms.tic.testframework.webdrivermanager.WebDriverProxyUtils;
import org.openqa.selenium.Proxy;
public abstract class AbstractTest extends TesterraTest {
@BeforeSuite
public void proxySetup() {
WebDriverProxyUtils utils = new WebDriverProxyUtils();
/**
* Global browser proxy configuration
*/
Proxy defaultProxy = utils.getDefaultHttpProxy();
WEB_DRIVER_MANAGER.setGlobalCapability(CapabilityType.PROXY, defaultProxy);
/**
* Browser specific proxy configuration
*/
Proxy otherProxy = utils.createHttpProxyFromUrl(
new URL("http://proxyUser:secretPassword@my-proxy:3128")
);
WEB_DRIVER_MANAGER.setUserAgentConfig(Browsers.chrome, (ChromeConfig) options -> {
options.setProxy(otherProxy);
});
}
}
4. UiElements
4.1. Overview
UiElements are representations of elements of the tested website, like buttons, search fields, checkboxes or even just DOM elements.
UiElements are not, but based on the Selenium WebElement and add more functionality to them. Since a UiElement is just a pointer to a locator, it’s using the same definition as WebElements By
(Selenium docs).
UiElements are self refreshing: Every action on it will trigger a find call, so the current state is always up to date when the requested action takes place. There is de facto no StaleElementReferenceException on UiElements like it could be when using vanilla WebElements.
4.2. Creation
4.2.1. Create default UiElement
For every UiElement you need an implementation of the UiElementFinder
interface (like PageObjects) and a locator (Selenium docs).
UiElement myElement = find(By.id("elementId"));
UiElement button1 = find(By.name("button"));
UiElement textOutputField = find(By.xpath("//p[@id='99']"));
A GuiElement always points to the first element found by the given locator. Even when your locator would return multiple elements, it just represents one. You can make your locators to force uniqueness or use element lists. |
4.2.2. Create SubElements
The UiElement
also implements UiElementFinder
. Elements will only be searched in the DOM tree below the given parent element on which the method was called.
UiElement upper = find(By.name("upperElement"));
// Create the sub elements
UiElement lower = upper.find(By.name("lowerElement"));
UiElement lower = upper.find(By.xpath(".//p[@id='element']")); (1)
UiElement lower = upper.find(By.xpath("//p[@id='element']")); (2)
UiElement lower = upper.find(By.xpath("./p[@id='element']")); (3)
1 | Find any matching descendant |
2 | Corrects the selector prefix to './/' |
3 | Find any matching child |
4.2.3. Implement UiElementFinder
When you don’t want to use the PageObject pattern, you can instantiate an UiElementFinder
on your own.
import eu.tsystems.mms.tic.testframework.pageobjects.UiElementFinder;
import eu.tsystems.mms.tic.testframework.testing.TesterraTest;
import eu.tsystems.mms.tic.testframework.testing.UiElementFinderFactoryProvider;
import eu.tsystems.mms.tic.testframework.testing.WebDriverManagerProvider;
class MyTest extends TesterraTest implements
UiElementFinderFactoryProvider,
WebDriverManagerProvider
{
protected UiElementFinder createFinder() {
return UI_ELEMENT_FINDER_FACTORY.create(WEB_DRIVER_MANAGER.getWebDriver());
}
@Test
public void test() {
UiElementFinder finder = createFinder();
UiElement myElement = finder.find(By.id("elementId"));
UiElement button1 = finder.find(By.name("button"));
UiElement textOutputField = finder.find(By.xpath("//p[@id='99']"));
}
}
4.2.4. Advanced UiElement locating
The UiElementFinder
extends the LocatorFactoryProvider
interface, which provides a LOCATE
factory that supports more features than standard Selenium By
.
Locate unique elements
If you want to make sure, that your element is unique.
UiElement myElement = find(LOCATE.by(By.id("elementId")).unique());
This will throw an exception, if not exact one WebElement has been found.
Locate displayed items only
UiElement myElement = find(LOCATE.by(By.xpath(".//button")).displayed());
Prepared xpath expressions
Using prepared expressions makes complex selectors more readable.
PreparedLocator byText = LOCATE.prepare("//button[text()='%s'])");
PreparedLocator byClass = LOCATE.prepare("//%s[contains(@class, '%s')][%d]");
UiElement loginButton = find(byText.with("Login"));
UiElement logoutButton = find(byClass.with("button", "btn-logout", 1));
Filtering elements
You can also filter elements during find.
Locator byText = LOCATE.by(By.xpath("//button"))
.filter(webElement -> webElement.getText().equals("Open again"));
UiElement buttonContainsText = find(byText);
4.2.5. UiElements inside frames
Accessing WebElements inside frames requires changing the active frame before any action. UiElements do that automatically as long you identify every <frame>
or <iframe>
as own element.
UiElement frame1 = find(By.id("frame1"));
// frame2 is child of frame1
UiElement frame2 = frame1.find(By.id("frame2"));
// target is child of frame2 which is child of frame1,
UiElement target = frame2.find(By.id("target"));
And easier way is to use the findDeep
method, which search for the element recursive in the frame hierarchy every time.
// in this case the frames are searched recurse automatically
UiElement target = findDeep(By.id("target"));
4.2.6. Element lists
The UiElement is like a pointer to the first element of the given locator. But if you want to retrieve a list of all found elements by the given locator, you can use the UiElementList
like the following.
<div>First</div>
<div>Second</div>
<div>Third</div>
UiElement div = find(By.tagName("div"));
div.list().first().expect().text("First");
div.list().last().expect().text("Third");
They can also be iterated and streamed.
div.list().forEach(uiElement -> {});
div.list().stream().forEach(uiElement -> {});
4.2.7. Empty elements
To prevent null pointers or any other exception that will break you program flow, you can use and empty UiElement using createEmpty()
of UiElementFinder
.
UiElement empty = createEmpty(Locator);
All interactive operations on this element will do nothing, all wait methods will be false and all assertions will fail.
4.2.8. Sensible Data
Sensible data, such as passwords, can be displayed obfuscated in the logs during the actions type and sendKeys.
UiElement sensibleElement = findById("secret").sensibleData();
Only the placeholder * is logged in the report instead of the real value.
4.2.9. Trace elements hierarchy
In most cases, elements are part of a view hierarchy. The Nameable
interface provides some methods to retrieve this information.
Nameable parent = element.getParent();
A parent could be any PageObject
like UiElement
, Component
or Page
.
Be aware that getParent() could return NULL , when the element has been created without a hierarchy or the element is a Page . So you should always perform a null or instanceof check.
|
If you want to trace the hierarchy beginning from top-down, you can use the traceAncestors()
method.
element.traceAncestors(ancestor -> true);
When the given Predicate
return FALSE
, the tracing will stop.
This will not supply the calling element. |
4.3. XPath builder
The static XPath
class helps you to build failsafe xPathes optimized for HTML. But it’s restricted to search elements top-down.
This is what the basic syntax looks like
UiElement div = find(XPath.from("div"));
But it supports many other features you need when you select elements from the DOM, like first and last element.
XPath.from("tr", 1);
XPath.from("tr", -1);
The xPath is generated greedy by default. When you create an xPath like
XPath.from("body").select("div");
this equivalent result would be //body//div
If you want to restrict the selection to a child element, use the /
prefix:
XPath.from("/body").select("/div");
which equivalent would be /body/div
It is also possible to pass groups in the from
method only.
XPath.from("(//iframe|//frame)");
4.3.1. Elements that have classes
XPath.from("div").classes("navigation", "header");
This will find elements like
<div class="header large navigation">
but not
<div class="navigation-header">
4.3.2. Select an element that encloses another element
XPath.from("nav")
.classes("mobile")
.enclose("/div")
.classes("navigation", "header");
This will find the <nav>
element
<nav class="mobile">
<div class="navigation header"></div>
</nav>
4.3.3. Select an element by its text
XPath.from("*").text().hasWords("Login", "here");
XPath.from("*").text().contains("first");
XPath.from("*").text().endsWith("here ");
This will find elements like
<a> Login first
here </a>
4.4. Checks
Checks if the element is present in the DOM
element.expect().present(boolean);
Checks if the element is present in the Viewport,
if it’s visible by it’s display
and visibility
style properties
and it’s width
and height
are both greater than 0.
element.expect().displayed(boolean);
Checks if the element is displayed and if it’s partially or complete
visible
in the scroll area of the Viewport.
element.expect().visible(boolean complete).is(boolean);
It doesn’t relate to opacity or z-index style properties. If you need to test the perceptually visibility to the human eye, you should consider to implement an image based Layout Check.
|
4.5. Assertions
UiElements provide many kinds of assertion methods to verify your elements.
If an assertion fails, it will make the whole test fail and abort. You can control that by using a TestController
UiElement element = find(By.id("button"));
// Expect the element text
element.expect().text("Hello World");
element.expect().text().contains("World").is(boolean);
element.expect().text().map(String::trim).startsWith("Hello").is(boolean);
// Expect the existence of an attribute
element.expect().attribute(Attribute.DISABLED).isNot(null);
// Expect the value of an input element matches an regular expression
element.expect().value().matches("^hello\\s.orld").is(boolean);
// CSS property checks
element.expect().css("display").is("none");
// CSS class checks
element.expect().classes("active", "button").is(boolean);
4.5.1. Layout assertions
UiElement can be checked for their relative layouts.
UiElement left = find(By.id("left"));
UiElement right = find(By.id("right"));
left.expect().bounds().leftOf(right).is(true);

UiElement image1 = find(By.xpath("//..."));
UiElement image2 = find(By.xpath("//..."));
UiElement image3 = find(By.xpath("//..."));
// Assertions are true
image1.expect().bounds().leftOf(image2).is(true);
image1.expect().bounds().fromTop().toTopOf(image2).is(0);
image1.expect().bounds().fromBottom().toBottomOf(image3).is(0);
// Assertions are false
image1.expect().bounds().fromBottom().toBottomOf(image2).is(0);
4.6. Actions
An UiElement provides a variety of action methods. Beside the known WebElement methods there are some more useful methods to interact with the web site.
4.6.1. Click on elements
UiElement element = find(By.id("button"));
element.click();
element.doubleClick();
element.contextClick();
If you have troubles using these methods, take a look to the fallback solution Desktop WebDriver utilities. |
4.6.2. Enter text
UiElement element = find(By.id("input"));
// Enters the given text in a input or textfield.
// Any old values are automatically deleted before input.
// The type method has a value check. If the given string is NULL or empty, the method does nothing.
element.type("my text");
// The standard Selenium method is used.
// You can also use the Selenim Keys class to enter special keys.
element.sendKeys("my text");
element.sendKeys("my text" + Keys.ENTER);
// Delete the content of an input field.
element.clear();
4.6.3. Use select boxes
UiElement element = find(By.id("select"));
// Get the Select WebElement of a UiElement
element.findWebElement(webElement -> {
Select selectElement = new Select(webElement);
// You can use all Selenium Select methods to interact.
select.selectByIndex(2);
select.selectByVisibleText("option");
List<WebElements> list = select.getAllSelectedOptions();
});
4.6.4. Use check boxes
UiElement element = find(By.id("checkbox"));
// Check and uncheck the check box
element.select();
element.deselect();
// true = check, false = uncheck
element.select(boolean);
4.6.5. Scrolling
You can scroll the browser viewport until the element is in the middle viewport if possible.
element.scrollIntoView();
// Lets offset pixel distance from the top of the viewport
element.scrollIntoView(new Point(0, -20))
4.6.6. Mouse over
You can simulate the mouse pointer is moved over an element.
element.hover();
If you have troubles using this method, take a look to the fallback solution Desktop WebDriver utilities. |
4.6.7. Drag and drop actions
With the utils class MouseActions
you can execute a drag-and-drop actions.
Source and target UiElements can be located in different frames.
UiElement source = find(By.id("elem1"));
UiElement target = find(By.id("elem2"));
MouseActions.dragAndDropJS(source, target);
// You can add one or more DragAndDropActions
MouseActions.dragAndDropJS(source, target, DragAndDropOption.CLICK_AFTER_RELEASE);
// This method provides a swipe of an element to a relative position from the element.
int offsetX = 50; // Pixel
int offsetY = 125; // Pixel
MouseActions.swipeElement(source, offsetX, offsetY);
4.7. Waiters
In testing practice the test automation code must tolerate delays caused e.g. by page loading or javascript activities when checking conditions on UiElements.
If the condition (which is checked continuously) is met within the timeout then the wait methods return true.
Otherwise, after the timeout has passed they return false without any further action or assertion.
boolean result;
result = element.waitFor().displayed(boolean);
// Overrides preconfigured internal timeout
result = element.waitFor(int seconds).displayed(boolean);
4.8. Access the WebElement
UiElement provides all Selenium methods when retrieving the WebElement
.
<a href="newpage.html" style="font-size: 20px;">My link</a>
UiElement element = find(By.xpath("//a"));
element.findWebElement(webElement -> {
String text = webElement.getText(); // returns "My link"
String attr = webElement.getAttribute("href"); // returns "newpage.html"
String name = webElement.getTagName(); // returns "a"
Point point = webElement.getLocation(); // returns the top left corner of the element
Dimension dim = webElement.getSize(); // returns width and heigth of the element
Rectangle rect = webElement.getRect(); // returns rectangle with location and size
String value = webElement.getCssValue("font-size"); // returns "20px"
});
4.9. Internals
The find
mechanism for a UiElement
in Testerra works different as in plain Selenium.
When using a constructor for a UiElement
instantiation, Testerra internally will add some facades / decorators to make things easier.
The most important decorator that is added by default is the GuiElementCoreSequenceDecorator
-which adds a sequence to all method calls against a GuiElement
.
Example: When calling the isPresent()
method on a UiElement
the added GuiElementSequenceDecorator
will fire up an internal find()
call to the GuiElement
and therefore a find()
call to the underlying Selenium WebElement
.
But instead of calling the find()
method once, it will execute this call in a default sequence every 500ms.
Therefor the property tt.element.timeout.seconds defined in test.properties
will be used as a hard timeout for this sequence.
If the find()
does not run successfully after the defined timeout it will fail.
5. PageObjects
5.1. Overview
5.1.1. What is a page object?
A page objects represents a HTML pages and or a subpage. It contains UiElements with describe the actual page and methods to provide actions on them.
In your test you only uses the provided actions of your page like an API. The page object himself uses the UiElements as an API to interact with the website.
5.1.2. Navigation Principle
In a regular Web Application there is a defined navigation flow. This means there are pages with actions on it that let you navigate to other pages.
In the example below we have a search dialog with a search action on it that lets you navigate to a ResultPage
with the search result.
When a search is performed the browser will navigate to the ResultPage
. In your page you create a new object of your next page.
This new page object is used for the next steps in your test.

5.1.3. Example
The following page contains two UiElements and one method for a user action 'search a string'.
Within the method search
the defined UiElements are used to execute a search.
The annotation Check
marks the UiElements as mandatory for the page. Testerra automatically verifies these elements when this page is instantiated (Check Annotations).
public class SearchPage extends Page {
@Check
private final UiElement searchButton = find(By.name("searchButton"));
@Check
private final UiElement inputField = find(By.name("inputField"));
// constructor
public SearchPage(WebDriver driver) {
super(driver);
}
// search action on page
public ResultPage search(String text) {
inputField.type(text);
searchButton.click();
return createPage(ResultPage.class);
}
}
The following lines demonstrate how to use page objects in your test method.
public class TestClass extends TesterraTest implements PageFactoryProvider {
@Test
public void myTest() {
HomePage homePage = PAGE_FACTORY.createPage(HomePage.class);
SearchPage searchPage = homePage.openSearch();
ResultPage resultPage = searchPage.search("search text");
resultPage.assertResultSetIsNotEmpty();
homePage = resultPage.close();
}
}
5.2. Instantiation
5.2.1. PageFactory
Pages need to be created initially via. PageFactory
interface, which will be provided by the PageFactoryProvider
as PAGE_FACTORY
instance.
When the page is instantiated, Testerra automatically checks its annotated elements.
HomePage homePage = PAGE_FACTORY.createPage(HomePage.class);
// Or use an explicit WebDriver
HomePage homePage = PAGE_FACTORY.createPage(HomePage.class, WebDriver otherWebDriver);
5.2.2. Inline creating
Once you created a page, you can create other pages directly from this page, to keep track of the same WebDriver.
OtherPage otherPage = homePage.createPage(OtherPage.class);
5.2.3. Optional pages
You can also try to create a page, when you want to handle unexpected redirects at some point.
homePage.waitForPage(OtherPage.class).ifPresent(otherPage -> {});
homePage.waitForPage(OtherPage.class, int seconds).ifPresent(otherPage -> {});
5.2.4. Lifecycle
When a page has been created and all Check Annotations have been performed, the pageLoaded()
method will be called, which can be used to perform some additional initializing without any test related interaction.
class MyPage extends Page {
@Override
protected void pageLoaded() {
super.pageLoaded();
// additional actions
}
}
5.2.5. Page Prefixes (deprecated)
Using page prefixes is an uncommon feature and therefore marked as @deprecated
|
Page Prefixes can influence which concrete classes get instantiated by the PageFactory
. They work together with a inheritance scheme of page classes. This can be useful if there is a base page which can come in different concrete variations. Example:
There is a BaseClass
which inherits from the Page
class and contains the basic functionality of a page.
Then the Page
can come in 2 different variations.
We can represent this as Variation1BaseClass
and Variation2BaseClass
.
They both inherit from BaseClass
.
Before instantiation, we can set the prefix using the PageFactory
.
Then we instantiate it and we can get our variation of the base class.
PAGE_FACTORY.setGlobalPagesPrefix("Variation1");
//this actualy creates a Variation1BaseClass
BaseClass baseClass = PAGE_FACTORY.createPage(BaseClass.class);
Default is no prefix.
Usage:
// Set a global Prefix
PAGE_FACTORY.setGlobalPagesPrefix("prefix");
// Set a thread local prefix. See next row about cleaning this prefix.
PAGE_FACTORY.setThreadLocalPagesPrefix("prefix");
// The thread local pages prefix is not cleared automatically,
// be sure to always set the correct one or clear it after using.
PAGE_FACTORY.removeThreadLocalPagePrefix();
5.3. Components
You can improve your PageObjects by using components. Components are like they are in actual web development environments: Containers with functionality. With components, you don’t need to try to create reusable PageObjects in a complex inheritance hierarchy, you can follow the pattern that composition before polymorphism.
In Testerra, components are hybrids of both UiElements and PageObjects. They can contain more UiElements and even Components, but they don’t provide features restricted to Pages or UiElements and their finder API is restricted to its root container element by default.
You can create a component like
import eu.tsystems.mms.tic.testframework.pageobjects.AbstractComponent;
public class MyComponent extends AbstractComponent<MyComponent> {
@Check
UiElement button = find(By.tagName("button"));
public MyComponent(UiElement rootElement) {
super(rootElement);
}
}
To instantiate components, use the createComponent()
method the same way as you create pages.
public class MyPage extends Page {
@Check
MyComponent component = createComponent(MyComponent.class, find(By.className("container")));
}
5.3.1. Component lists
Since components are hybrid UiElements, they can also act as lists.
UiElement table = find(By.tagName("table"));
TableRow rows = createComponent(TableRow.class, table.find(By.tagName("tr")));
rows.list().forEach(row -> {
row.getNameColumn().text().contains("Hello").is(true);
});
5.4. Check Annotations
The @Check
annotation is used to verify the actual presence of an element on the site.
All UiElements that are marked with the @Check
annotation are automatically checked when instantiated by the PageFactory
.
In the example, the first UiElement has the @Check
annotation, the second doesn’t.
The result is, that the presence of the first element will be checked by the constructor, the second won’t.
If a checked element is not found, the constructor will throw a PageFactoryException
.
@Check
private UiElement checked = find(By.name("checked"));
//no @Check here
private UiElement unchecked = find(By.name("unchecked"));
The @Check
annotation will use the default CheckRule
defined in test.properties
.
It is also possible to overwrite the default CheckRule
for a single @Check
annotation.
@Check(checkRule = CheckRule.IS_PRESENT)
private UiElement uiElement;
Change the check rules for the whole project (global) with the following:
tt.guielement.checkrule=IS_PRESENT
Available CheckRules are
-
IS_DISPLAYED
-
IS_NOT_DISPLAYED
-
IS_PRESENT
-
IS_NOT_PRESENT
The default is IS_DISPLAYED.
With the optional
attribute, the check only adds an optional assertion to the report.
The test will not be interrupted at this position.
@Check(optional = true)
private UiElement uiElement;
You can also define a special error message with the prioritizedErrorMessage
attribute.
@Check(prioritizedErrorMessage = "My error message.")
private UiElement uiElement;
Use the timeout
attribute to define a specific timeout only for that element to optimize check timeouts on Page instantiation. This overrides the Page timeout setting.
@Check(timeout = 60)
private UiElement uiElement;
5.5. Timeout Setting
The checks are performed with a timeout. The default timeout is set by the tt.element.timeout.seconds
property.
With the following annotation, the check can be optimized for all UiElements during page instantiation:
@PageOptions(elementTimeoutInSeconds = 60)
public class ExamplePage extends Page {
// insert your code
}
5.6. Restrict element access
The PageObject pattern encapsulates UiElements for all activities on the page. Thats why every element should be private accessible by the page only. But for testing purposes, it could be useful to allow access to some elements. In this case, you can create public methods and return the element casting to TestableUiElement
.
public class MyPage extends Page {
private UiElement saveButton = find(By.tagName("button"));
public void performSave() {
saveButton.click();
}
public TestableUiElement getSaveButton() {
return saveButton;
}
}
In the test, you can perform assertions without breaking the PageObject pattern.
@Test
public void testSaveFunctionality() {
MyPage page = getPage();
page.performSave();
page.getSaveButton().expect().text("Saved");
}
Excecution and controlling
6. Test controlling
The TestControllerProvider
interface provides the TestController
instance CONTROL
for controlling the test and assertion flow.
6.1. Collected assertions
Collecting assertion means, that a failing assertion will not abort the test method, but it will throw an exception at the end of the test method. So you have a chance to validate many more aspects in one test run.
CONTROL.collectAssertions(() -> {
element.expect().text("Hello World");
page.performLogin();
Assert.assertEquals("hello", "world");
});
6.2. Optional assertions
Optional asserts do not let the test fail, but the assertion message will be added to the log with loglevel WARN
and will result in an minor failure aspect.
CONTROL.optionalAssertions(() -> {
element.expect().text("Hello World");
page.performLogin();
Assert.assertEquals("hello", "world");
});
6.3. Change internal timeout
To change the timeout for internal assertions, you can override it for a specified block.
CONTROL.withTimeout(int seconds, () -> {
element.expect().text("Hello World");
}
Please mind that you also can pass already implemented methods.
@Test
public void test_something_fast() {
CONTROL.withTimeout(0, this::test_something);
}
withTimeout() overrides all internal timeouts except the explicit set timeout in waitFor(int seconds) methods.
|
6.4. Retries
In some situations you cannot rely on single assertions or waits anymore and need to continue trying something out before performing an alternative solution. Use control methods for repeating a couple of actions within a loop until a timeout has reached.
For example, this retry block tries to click a button until it’s disabled.
CONTROL.retryFor(int seconds, () -> {
button.click();
button.expect().enabled(false);
});
You can also perform something when the retry block fails.
CONTROL.retryFor(int seconds,() -> {
element.expect().text(String);
}, () -> {
element.getWebDriver().reload();
});
You can also combine these control features.
CONTROL.retryFor(int seconds, () -> {
CONTROL.withTimeout(int seconds, () -> {
button.click();
uiElement.scrollIntoView();
uiElement.expect().visible(boolean complete).is(boolean);
}
);
7. Regular Assertions
When implementing the AssertProvider
interface, you get an instance named ASSERT
, which provides more features than standard TestNG Assert
You can use this interface for assertions they cannot be covered by the UiElement.
import eu.tsystems.mms.tic.testframework.testing.AssertProvider;
public class MyTest extends TesterraTest implements AssertProvider {
@Test
public void test() {
ASSERT.assertContains("Hello world", "planet");
}
}
This will throw an AssertionError
with the message Expected [Hello world] contains [planet]
.
You can also add some more detailed information to the subject.
ASSERT.assertContains("Hello world", "planet", "the greeting");
Which will result in Expected that the greeting [Hello world] contains [planet]
Please take a look into the Assertion
interface for a full feature overview.
8. Test execution
Testerra has several features to handle and adjust a test execution, which are described in the following paragraphs.
8.1. Conditional behaviour
For managing the execution behaviour of tests in suites there are means to skip tests and avoid closing browser windows after failures.
# all browser windows remain open after first failure, default = false
tt.on.state.testfailed.skip.shutdown=true
# skip all tests after first failure, default = false
tt.on.state.testfailed.skip.following.tests=true
8.2. Failure Corridor
This mechanism is used to define the test goal of test runs so that it only fails with an invalid failure corridor.
This feature is enabled by default with the following property.
tt.failure.corridor.active=true
With an enabled failure corridor, you need to define the maximum amount of failures per weight:
tt.failure.corridor.allowed.failed.tests.high=0
tt.failure.corridor.allowed.failed.tests.mid=1
tt.failure.corridor.allowed.failed.tests.low=2
If you do not define any failure corridor, the default value |
To change the weight for each test, just annotate it with @FailureCorridor
, where High
is default.
// This testcase is marked with a high weight.
@FailureCorridor.High
@Test
public void test1() throws Exception {
Assert.fail();
}
// This testcase is not marked, but the default weight is high.
@Test
public void test2() throws Exception {
Assert.fail();
}
// This testcase is marked with a middle weight.
@FailureCorridor.Mid
@Test
public void test3() throws Exception {
Assert.fail();
}
// This testcase is additional marked with @Fails.
// So the test result is ignored by the Failure corridor.
@Fails
@FailureCorridor.Mid
@Test
public void test4() throws Exception {
Assert.fail();
}
// This testcase is marked with a low weight.
@FailureCorridor.Low
@Test
public void test5() throws Exception {
Assert.fail();
}
8.3. Element Highlighting
8.3.1. Demo mode
In the demo mode actions on pages are marked with distinctive coloured frames around the element of the action. This mechanism is set by a property
# activate demo mode, default = false
tt.demomode=true
The following colours are used for highlighting
-
red: failed visibility checks and asserts
-
green: successful visibility checks and asserts
-
yellow: mouseOver
-
blue: click
8.4. Expected Fails
For known issues on the SUT the annotation @Fails
can used to mark a test method as failing. These test cases are marked as Expected failed
separately in the report.
If tests are passed again, you get a note in the report to remove the Fails
annotation.
@Test
@Fails()
public void testItWillFail() {
Assert.assertTrue(false);
}
The result is technically still a failure and only visually elevated to facilitate the evaluation of the report. Please keep in mind that
|
8.4.1. Add additional information
You can add additional information to describe the cause in more detail. All information are added to the report.
@Test
@Fails(description="This test fails for reasons")
public void testItWillFail() {
Assert.assertTrue(false);
}
Attribute | Description |
---|---|
description |
Give more details about the failure. |
ticketString |
Define a bug ticket ID or URL as a String value. |
intoReport |
If |
validFor |
Define the conditions of the expected failed. |
8.4.2. Define conditions
You can specify some conditions for expected fails. Only if all conditions are true, the test is marked as Expected failed
, otherwise as Failed
.
test.properties
# test.properties
environment=test
country=de
@Fails
annotation@Test
@Fails(description = "Failing for environment 'test' and country 'de'", validFor = {"environment=test", "country=de"})
public void testExpectedFailed() {
Assert.fail();
}
If the test is executed with other values of the properties (like country=uk) the test will marked as Failed
.
8.5. Retry analyzer
8.5.1. Common
Testerra provides an adjustable mechanism to automatically retry failed tests.
The default retry count is 1
. Each failed method is executed exactly one more time, when matching the retry criteria.
Retried methods are shown in the section Retried
of the report.
You can change the default with the following property.
tt.failed.tests.max.retries=1
The retry mechanism always ignores testcases with a valid Fails annotation. |
8.5.2. Specific retry count for test methods
You can change the retry count for specific test methods.
@Test()
@Retry(maxRetries = 2)
public void testMethod() {
...
}
8.5.3. Specific retries
Testerra can also retry failed methods when matching certain criteria. The filtering process contains of checks of classes and messages matching the thrown Exception, which are set within the test.properties file.
# set additional classes that engage a retry,
# instances of TimeoutException are always retried
tt.failed.tests.if.throwable.classes=java.sql.SQLRecoverableException
# set additional messages of Throwable that engage a retry,
# by default the following messages are evaluated
# "Error communicating with the remote browser. It may have died",
# "was terminated due to TIMEOUT",
# "was terminated due to SO_TIMEOUT",
# "The requested URL could not be retrieved",
# "Squid is unable to create a TCP socket"
tt.failed.tests.if.throwable.messages=failed to connect, error communicating with database
8.5.4. Customize retry behaviour
For further adjustment additional analyzers can be registered expanding the default behaviour.
InstantiationException
// custom Retryanalyzers need to implement the functional interface AdditionalRetryAnalyzer
public class InstantiationExceptionRetryAnalyzer implements AdditionalRetryAnalyzer {
final String message = "failed instantiation";
@Override
public Throwable analyzeThrowable(Throwable throwable, String tMessage) {
if (throwable instanceof InstantiationException) {
if (tMessage != null) {
final String tMessageLC = tMessage.toLowerCase();
boolean match = tMessageLC.contains(message);
if (match) {
return throwable;
}
}
}
return null;
}
}
public class AbstractTest extends TesterraTest {
static {
// register the additional Analyzer,
// which checks for "InstantiationException" and the message "failed instantiation"
RetryAnalyzer.registerAdditionalRetryAnalyzer(new InstantiationExceptionRetryAnalyzer());
}
}
8.5.5. @NoRetry
With this annotation the Retry Analyzer won’t retry these methods if previously failed. This is characteristically shown in the report by the badge NoRetry
.
You can customize the NoRetry
annotation with the attributes name
and color
.
Attribute | Description |
---|---|
name |
Changes the shown text in the report. Default is |
color |
Change the background color of the shown text. Default is
|
NoRetry
@Test()
@NoRetry(name = "No retry because it's not allowed.", color="rgb(255, 236, 139)")
public void testMethod() {
...
}
8.6. WebDriverWatchDog
The WebDriverWatchDog
is your vigilant pet watching the test execution and reacting on blocked tasks. With two properties it is set up.
# activate watchdog, default = true
tt.watchdog.enable = true
# timeout in seconds after the test execution is terminated, default = 300
tt.watchdog.timeout.seconds = 500
8.6.1. How does it work?
With the first Usage of WedDriverManager
the WebDriverWatchDog
is initiated.
It internally starts a Thread running in parallel to the current test execution checking the stacktrace every ten seconds for
stacktrace entries with thread name "Forwarding" containing an Element "java.net.SocketInputStream".
These potentially blocking stacktrace entries are updated every time found.
Upon reaching the maximum timeout of 500 seconds the whole test execution is terminated with exit code 99 and a readable error output in your log.
A valid report is always generated. |
8.7. Annotations for Report adjustment
To improve the readability and clarity of the report there are several annotations for marking the test class and test methods, which are described in the following paragraphs as well as in @Fails, @Retry, @InDevelopment and @NoRetry.
8.7.1. @New
This marks a test method as New
, which is shown with this text in the report. Customization is possible with two attributes name and color in the annotation.
-
name: change the shown text in the report. Default is
New
-
color: change the background color of the shown text. Default is
cadetblue
. Values need to be valid for html colors.-
name of the color, e.g. red
-
RGB values, e.g. rgb(255, 236, 139)
-
RGBA values, e.g. rgba(252, 156, 249, 0.75)
-
HSL values, e.g. hsl(217, 97%, 57%)
-
Hex values, e.g. #57c0ff
-
8.7.2. @ReadyForApproval
This marks a test method as Ready For Approval
, which is characteristically shown with this text the report. Customization is possible with the two attributes name and color in the annotation.
-
name: change the shown text in the report. Default is
Ready For Approval
-
color: change the background color of the shown text. Default is
indianred
. Values need to be valid for html colors.
See @New for explanation.
8.7.3. @SupportMethod
This marks a test method as Support Method
, which is characteristically shown with this text the report. Retests won’t skip these methods if previously passed. Customization is possible with the two attributes name and color in the annotation.
-
name: change the shown text in the report. Default is
Support Method
-
color: change the background color of the shown text. Default is
#848282
. Values need to be valid for html colors.
See @New for explanation.
8.7.5. @TestClassContext
With this annotation you can set the test context for the given test class. There two attributes for adjustments.
-
name: name of the context, default =
""
-
mode:
TestClassContext.Mode.ONE_FOR_ALL
orTestClassContext.Mode.ONE_FOR_EACH
, default =TestClassContext.Mode.ONE_FOR_ALL
The Executed tests are then shown in the classes overview of the report as a entry labeled with name
from @TestClassContext
.
8.7.6. @InDevelopment
This aforementioned annotations is further adjustable with the two attributes name and color.
-
name: change the shown text in the report. Default is
In Development
-
color: change the background color of the shown text. Default is
#a7a5a5
. Values need to be valid for html colors:-
name of the color, e.g. red
-
RGB values, e.g. rgb(255, 236, 139)
-
RGBA values, e.g. rgba(252, 156, 249, 0.75)
-
HSL values, e.g. hsl(217, 97%, 57%)
-
Hex values, e.g. #57c0ff
-
8.8. Dry Run
8.8.1. Overview
With this execution mode all methods marked with TestNG Annotations are only called but their code isn’t executed, hence the name dry run. It’s designed to simply check the callstack of TestNG related methods without executing their logic, e.g. to find missing method calls in your test setup. For using this you just need to set the following property.
# activate the dry run, default = false
tt.dryrun=true
The report indicates a dry run with the suffix Dry Run
in the headlines of each section.
The rest is visually identical to a normal run. |
All called methods are shown, but probably as passed. With a closer look into the report details you will just notice a really low test duration, something below one second.
8.9. JVMMonitor
The JVMMonitor
is the Observer of the hardware utilization for memory and cpu.
With the start of a test while using the TesterraListener
the latter implicitly starts the JVMMonitor
.
Thus a concurrent thread for monitoring purposes only is initiated next to the actual test execution. Every ten seconds the following parameters are logged
-
JVM Memory usage in MB
-
JVM Memory reserved in MB
-
JVM CPU usage in per cent
The JVMMonitor
is automatically terminated after the test execution and a graph showing the memory consumption is put into the report.
Testerra Features
9. Modules
9.1. BrowserUp Proxy
BrowserUp Proxy (BUP) is a simple HTTP proxy utility developed by browserup.com. It offers functionality to track, manipulate and modify HTTP requests and responses, as well as capture HTTP traffic for analysis.
Testerra offers a simple integration to spin multiple local proxy servers or manage remote proxy servers via HTTP API.
9.1.1. Project setup
Latest release
compile 'io.testerra:bup:2.0-RC-5'
<dependencies>
<dependency>
<groupId>io.testerra</groupId>
<artifactId>bup</artifactId>
<version>2.0-RC-5</version>
</dependency>
</dependencies>
9.1.2. External Proxy Server
Best practice for using a Testerra with an external proxy, is to use a dedicated BrowserUp Instance. To start these instance, please have a further read on the BrowserUp documentation.
import eu.tsystems.mms.tic.testerra.bup.BrowserUpRemoteProxyManager;
import eu.tsystems.mms.tic.testerra.bup.BrowserUpRemoteProxyServer;
import eu.tsystems.mms.tic.testframework.testing.TesterraTest;
import org.openqa.selenium.Proxy;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.BeforeSuite;
import java.net.MalformedURLException;
import java.net.URL;
public class AbstractTest extends TesterraTest {
private static BrowserUpRemoteProxyServer bupProxy;
@BeforeSuite
public void setupProxy() throws MalformedURLException {
final URL apiUrl = new URL("http://localhost:8080");
final BrowserUpRemoteProxyManager bupRemoteProxyManager = new BrowserUpRemoteProxyManager(apiUrl);
bupProxy = bupRemoteProxyManager.startServer();
/* Additional Proxy setup here */
final String bmpProxyAddress = String.format("%s:%d", apiUrl.getHost(), bupProxy.getPort());
// For selenium usage.
final Proxy proxy = new Proxy();
proxy.setHttpProxy(bmpProxyAddress)
.setFtpProxy(bmpProxyAddress)
.setSslProxy(bmpProxyAddress);
WebDriverManager.setGlobalExtraCapability(CapabilityType.PROXY, proxy);
}
@AfterSuite
public void tearDownProxy() throws MalformedURLException {
final URL apiBaseUrl = new URL("http://localhost:8080");
final BrowserUpRemoteProxyManager browserUpRemoteProxyManager = new BrowserUpRemoteProxyManager(apiBaseUrl);
for (final Integer proxyPort : browserUpRemoteProxyManager.getProxies()) {
final BrowserUpRemoteProxyServer bupToStop = new BrowserUpRemoteProxyServer();
bupToStop.setPort(proxyPort);
browserUpRemoteProxyManager.stopServer(bupToStop);
}
}
}
BrowserUp creates a new proxy server at the next free port beginning with port 8081 (BrowserUp default).
If you need a dedicated port, use `startServer(BrowserUpRemoteProxyServer proxyServer)`method the following way.
BrowserUpRemoteProxyServer browserUpRemoteProxyServer = new BrowserUpRemoteProxyServer();
browserUpRemoteProxyServer.setPort(8088);
browserUpRemoteProxyServer = browserUpRemoteProxyManager.startServer(browserUpRemoteProxyServer);
If the port already used, the BrowserUpRemoteProxyManager
will do nothing, and just return the given config-object of type BrowserUpRemoteProxyServer
.
Basic Auth
If your SUT is protected by HTTP basic auth, you can set up these credentials as following.
URL baseUrl = WEB_DRIVER_MANAGER.getConfig().getBaseUrl();
final String basicAuthUser;
final String basicAuthPassword;
final URL apiBaseUrl = new URL(LOCAL_PROXY_FOR_TEST);
final BrowserUpRemoteProxyManager browserUpRemoteProxyManager = new BrowserUpRemoteProxyManager(apiBaseUrl);
final BrowserUpRemoteProxyServer bup1 = browserUpRemoteProxyManager.startServer();
browserUpRemoteProxyManager.setBasicAuth(bup1, baseUrl.getHost(), basicAuthUser, basicAuthPassword);
Upstream proxy
If your SUT should use a proxy, you can set up BrowserUp proxy instance to use an upstream proxy like in SUT Browser Proxy
BrowserUpRemoteProxyServer bup1 = new BrowserUpRemoteProxyServer();
bup1.setUpstreamProxy(ProxyUtils.getSystemHttpProxyUrl());
final BrowserUpRemoteProxyManager browserUpRemoteProxyManager = new BrowserUpRemoteProxyManager(apiBaseUrl);
bup1 = browserUpRemoteProxyManager.startServer();
Other features
/*
Check if proxy alread runs on port...
*/
final BrowserUpRemoteProxyManager browserUpRemoteProxyManager = new BrowserUpRemoteProxyManager(apiBaseUrl);
BrowserUpRemoteProxyServer bup1 = new BrowserUpRemoteProxyServer();
bup1.setPort(8088);
bup1 = browserUpRemoteProxyManager.startServer(bup1);
boolean isRunning = browserUpRemoteProxyManager.isRunning(nup1);
/*
Maps specific host names to another host names or IP adresses
*/
browserUpRemoteProxyManager.setHostMapping(BrowserUpRemoteProxyServer proxyServer, Map<String, String> hostNameMapping);
/*
Capture the traffic and return it as a JsonElement
You can choose, if you want to capture only the headers, the content or both via the boolean flags.
*/
browserUpRemoteProxyManager.startCapture(BrowserUpRemoteProxyServer proxyServer, String initialPageRef, boolean isCaptureHeaders, boolean isCaptureContent);
JsonElement stopCapture(BrowserUpRemoteProxyServer proxyServer);
/*
Adds additional key-value pairs to the headers.
*/
browserUpRemoteProxyManager.addHeader(final BrowserUpRemoteProxyServer proxyServer, final String key, final String value);
9.1.3. Local browser instances
If you want to quickly spin up a proxy isntance on your local system while testing, you can use the BrowserUpLocalProxyManager
.
List<Integer> portPool = new ArrayList<>();
ports.add(8090);
ports.add(8091);
ports.add(8092);
ports.add(8093);
ports.add(8094);
ports.add(8095);
BrowserUpLocalProxyManager bupLocalManager = new BrowserUpLocalProxyManager(ports);
// Start instance
BrowserUpProxyServer browserUpProxyServer = new BrowserUpProxyServer();
browserUpProxyServer = bupLocalManager.startServer(browserUpProxyServer);
// assert that a port of given port pool was used.
Assert.assertTrue(portPool.contains(port), "Port of range was used.");
// assert proxy is started.
Assert.assertTrue(bup1.isStarted(), "Proxy started");
The local proxy manager works with a defined port pool, which has to be declared on instantiation of the manager class. This port pool will be used to spin up multiple proxy servers for a multi threading test execution.
The port pool has to be declared by yourself, respectively your code, because, only you can know which ports are currently free to use on your local test execution machine.
To use upstream proxies, add headers or do other things on the local proxy server, please take a closer look on BrowserUp documentation.
9.2. Browser Mob Proxy (deprecated)
Browser Mob Proxy (BMP) is a utility for manipulating network traffic. You can use it to enable Basic Auth for your SUT and other features.
9.2.1. Project setup
Latest release
compile 'io.testerra:bmp:2.0-RC-5'
<dependencies>
<dependency>
<groupId>io.testerra</groupId>
<artifactId>bmp</artifactId>
<version>2.0-RC-5</version>
</dependency>
</dependencies>
9.2.2. External BMP server (recommended)
Best practice for using a Testerra with a proxy, is to use a dedicated BMP instance.
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.AfterSuite;
import eu.tsystems.mms.tic.testframework.bmp.BmpRestClient;
import eu.tsystems.mms.tic.testframework.webdrivermanager.WebDriverManager;
import org.openqa.selenium.Proxy;
public abstract class AbstractTest extends TesterraTest {
private static BmpRestClient bmpClient;
@BeforeSuite
public void proxySetup() {
final URL apiUrl = new URL("http://localhost:8080");
bmpClient = new BmpRestClient(apiUrl);
/* Additional Proxy setup here */
final int proxyPort = bmpClient.startServer();
final String bmpProxyAddress = String.format("%s:%d", apiUrl.getHost(), proxyPort);
// For selenium usage.
final Proxy proxy = new Proxy();
proxy.setHttpProxy(bmpProxyAddress)
.setFtpProxy(bmpProxyAddress)
.setSslProxy(bmpProxyAddress);
WEB_DRIVER_MANAGER.setGlobalCapability(CapabilityType.PROXY, proxy);
}
@AfterSuite
public void proxyTeardown() {
bmpClient.stopServer();
}
}
BMP creates a new proxy server at the next free port beginning with port 8081 (BMP default).
If you need a dedicated port, use startServer
with a port number.
If the port already used, BMP client only returns the port number.
...
int customPort = 8085;
final int proxyPort = bmpClient.startServer(customPort);
...
9.2.3. Basic Auth
If your SUT is protected by HTTP basic auth, you can setup these credentials as following.
import java.net.URL;
/* Additional Proxy setup here */
URL baseUrl = WEB_DRIVER_MANAGER.getConfig().getBaseUrl();
final String basicAuthUser;
final String basicAuthPassword;
bmpClient.setBasicAuth(baseUrl.getHost(), basicAuthUser, basicAuthPassword);
9.2.4. Upstream proxy
If your SUT should use proxy, you can setup BMP to use a upstream proxy like in SUT Browser Proxy
import eu.tsystems.mms.tic.testframework.utils.ProxyUtils;
bmpClient.setUpstreamProxy(ProxyUtils.getSystemHttpProxyUrl());
9.2.5. Other features
/*
Checks if a proxy server is available by a custom port.
*/
int port = 8085;
boolean isUsed = bmpClient.isProxyRunningAtPort(port);
/*
Maps specifc host names to another host names or IP adresses
*/
bmpClient.setHostMapping(Map<String, String> hostNameMapping);
/*
Capture the traffic and return it as a JsonElement
You can choose, if you want to capture only the headers, the content or both via the boolean flags.
*/
bmpClient.startCapture(boolean headers, boolean content);
JsonElement element = client.stopCapture();
/*
Adds additional key-value pairs to the headers.
*/
bmpClient.setHeader(String key, String value);
9.3. CSV Reader (deprecated)
This class is not supported anymore and therefore marked as @deprecated . If you want to read CSV files into Java beans, please consider the documentation of the OpenCSV library
|
The CSV reader class provides a basic reader for CSV files. This is useful to handle testdata. It provides two ways to read a csv file.
For all code examples we assume the following CSV files located in your project resource directory, e.g. src/test/resources
.
id;name;firstName
AH_1;Muster;Max
AH_2;Paula;Paul
id;name;firstName;subModel
AH_1;Muster;Max;SUB_1
AH_2;Paula;Paul;SUB_2
id;street;city
SUB_1;Street A 12;Dresden
SUB_2;Street B 13;Leipzig
9.3.1. Read file into map
With following code snippet you can read a CSV file into a simple key-value based map.
The values of the header row will be used as key. All other rows are then values of the given key.
final CSVTestDataReader csvTestDataReader = new CSVTestDataReader();
final List<Map<String, String>> testDataMap = csvTestDataReader.readCsvTestDataFromResource("testfiles/TestCsvReader.csv");
Assert.assertEquals(testDataMap.size(), 2);
Assert.assertEquals(testDataMap.get(0).get("id"), "AH_1");
Assert.assertEquals(testDataMap.get(1).get("id"), "AH_2");
9.3.2. Read files into beans
The other way to read a CSV file is to convert it into Java models. Therefore you have to specify the class matches the Java domain model pattern.
final CSVTestDataReader csvTestDataReader = new CSVTestDataReader();
final List<TestCsvReaderBean> testDataList = csvTestDataReader.readCsvIntoBeans("testfiles/TestCsvReader.csv", TestCsvReaderBean.class);
Assert.assertNotNull(testDataList);####
Assert.assertNotEquals(testDataList.size(), 0);
final TestCsvReaderBean testCSVReader = testDataList.get(0);
final String id = testCSVReader.getId();
Assert.assertEquals(id, "AH_1");
Further to this approach it is possible to concat objects like you do in your Java model as well.
Given the following classes and the given CSV, you will notice, that TestCsvReaderWithSubModel
has a property of type TestCsvReaderSubModel
.
In order to solve this, the CSV reader will try to find a CSV file named like your class TestCsvReaderSubModel
in your resources.
If it exists, the CSV reader will read this "sub-file" as well and will inject the object into the main type.
Just ensure that you provide a unique line identifier. The CSV reader will then take care.
public class TestCsvReaderBeanWithSubModel {
private String id;
private String name;
private String firstName;
private TestCsvReaderSubModel subModel;
// getter and setter here...
}
public class TestCsvReaderSubModel {
private String id;
private String street;
private String city;
// getter and setter here...
}
Simple data types must be represented by their object data types. E.g.
* |
9.3.3. Customize the CSV reader
The following table shows how you can customize the CSV reader.
Method | Default value | Description |
---|---|---|
|
0 |
Set the row of the hedaer in CSV file, beginning with 0. |
|
; |
Set the column separater. |
setQuoteChar(char quoteChar) |
\" |
Set the characters, how strings can be quoted. |
9.4. Layout Check
Layout tests always mean checking whether a GUI is designed according to the guidelines.
9.4.1. Introduction
Are the position, size, color and shape of the elements correct? Are distances maintained? Perhaps even certain elements are missing? In test automation, functionality is primarily tested. Here it is actually irrelevant how a button is designed and where it is positioned. The main thing is that the element can be uniquely recognized via XPath or CSS selectors.
However, in some frontends such as content management systems with a high level of relevance to end users (certain portal solutions, shops) the management is also extremely important. However, testing this with Selenium’s previous means is not in any relation between effort and benefit. Manual visual inspection is usually still the fastest way to do this.
Although, manual inspection can never be pixel-accurate. An inspection of perhaps more than 100 elements is too costly. Smaller shifts or colour deviations are easily overlooked.
At this point an automatic comparison of reference screenshots can help. This test is to be seen in addition to the functional tests, but cannot replace them. In a test case, it would also be conceivable to combine the check of function and layout.
9.4.2. Description
A layout test with the Testerra utilities is actually a comparison between a reference and the actual state. This is done via the screenshot functionality of Selenium, in which both screenshots (reference and actual) are compared pixel by pixel from top to bottom. In this comparison, a third image is created in which differences (ReferencePixel != ActualPixel) are marked in red. The LayoutCheck can perform a check in two ways:
-
PIXEL: Pixel-based image comparison of the entire reference image without exception. You get a quantified value for pixel displacement in percent (0-100%)
-
ANNOTATED: The image comparison is based on annotations in the reference image. The annotations are rectangles in a certain color, which include the areas to be checked. You get a quantified value for displaced annotations in percent.
Prerequisites
The following prerequisites must be met
-
Concrete execution environment: Layout tests should run in a concrete browser environment to ensure the comparability of the screenshots.
-
Size of browser window: Define fixed size to exclude different sizes of the images at different VM resolutions
-
Screen resolution and scaling: Make sure you have the same screen resolution and scaling on each presentation device (ea. Testerra uses a resolution of 1920x1200 (16:10) with a scaling of 1 per default in headless environments)
-
-
Build Server: The library underlying ANNOTATED mode supports the operating systems
-
Windows 32 - and 64 - Bit
-
Linux Debian 32 - and 64 - Bit (glibc >= 2.15; Jenkins)
-
MacOSX (untested)
-
9.4.3. Configuration
In order to get the layout check to work, you need at least a reference image and a place where it’s located.
tt.layoutcheck.reference.path=src/test/resources/layout-references/{tt.browser}
tt.layoutcheck.reference.nametemplate=%s.png
# Highly recommended to disable full screen for browser
tt.browser.maximize=false
# Highly recommended to switch of the demo mode for layout tests
tt.demomode=false
The directory for reference image may result in src/test/resources/layout-references/chrome/WelcomePage.png
for example.
9.4.4. PIXEL Mode
The comparison is generally carried out over the entire reference image. In this case, it is a prerequisite that the reference screenshot and the screenshot created during the test runtime have the same resolution (height and width are identical).
Mostly the website consists of many more elements than should be checked. Ingredients such as the header,
menus or footer may falsify the test result if there are deviations from the reference,
but these are not currently the subject of the test. For this reason,
the tt.layoutcheck.use.ignore.color
property can be used to determine that a particular color in the image can be excluded from comparison.
To validate the layout in Pixel Mode, you can check the whole page or a single UiElement.
Check a single UiElement
To check the layout of a single UiElement only, you can use the standard asserts implementation.
uiElement.expect().screenshot().pixelDistance("HeaderComponent").isLowerThan(10);
Take reference screenshots on first run
When you have configured the reference screenshots location and implemented the tests, you can now set the option
tt.layoutcheck.takereference=true
to enable taking automatically screenshots based on the current browser and size configuration and storing them to the reference image’s location.
All concrete distance values in this tt.layoutcheck.takereference-mode will return 0 (zero) and always pass the tests.
|
9.4.5. ANNOTATED Mode
For this mode, an arbitrary number of rectangles with a line width of one pixel are defined in the reference images. The color of the rectangles is to be communicated to Testerra via the upper left pixel (x=1, y=1). In the current screenshot, Testerra searches for the area within the annotated selection. The resolution of the current screenshot is irrelevant.
9.5. Localization
Websites come in many languages, where the functionality may not change, but labels of buttons, input or other interactive elements.
Testerra provides an easy and simple way to support the localization of UiElements based on Locale
.
9.5.1. Add localization resource bundle
The localization is based on Unicode .properties
files loaded during runtime.
To add such files, create a new resource bundle lang
in src/main/resources
and add all required locales.
BTN_LOGIN=Login
BTN_LOGIN=Anmelden
You can change the default encoding for .properties files in IntellJ at File → Settings → File Encodings
|

9.5.2. Access localization text
Now you can instance UiElements by localized strings.
import eu.tsystems.mms.tic.testframework.l10n.SimpleLocalization;
UiElement loginBtn = find(By.linkText(SimpleLocalization.getText("BTN_LOGIN")));
But you can use your own localized bundles.
import eu.tsystems.mms.tic.testframework.l10n.LocalizedBundle;
LocalizedBundle bundle = new LocalizedBundle("testdata");
bundle.getString("TEST_KEY");
9.5.3. Change locale from command line (recommended)
The best way to change the locale for your tests is, to pass the language property as command line argument.
gradle test -Duser.language=de
For Maven, you need some extra configuration
<project>
<properties>
<user.language>de</user.language>
<user.country>DE</user.country>
<argLine>-Duser.language=${user.language} -Duser.country=${user.country}</argLine>
</properties>
</project>
before running the command
mvn test -Duser.language=de -Duser.country=DE
9.5.4. Change locale from property
When you want to change the locale from a property. Do the following
String testLanguage = PROPERTY_MANAGER.getProperty("test.language", "de");
Locale.setDefault(Locale.forLanguageTag(testLanguage));
9.5.5. Change locale for the user agent
The WebDriver API official doesn’t support changing the language of a browser session. But there exists non-standard or experimental ways on Stackoverflow to change your browser locale.
Anyways, it’s currently the better way to visit your SUT with a predefined language and change it there (with a language switcher).
9.6. Mail connector
The module mail-connector allows you to send and receive/read emails as well as solve mail surrounding tasks like encoding or encrypting.
9.6.1. Project setup
Latest release
// build.gradle
compile 'io.testerra:mail-connector:2.0-RC-5'
<!-- pom.xml -->
<dependencies>
<dependency>
<groupId>io.testerra</groupId>
<artifactId>mail-connector</artifactId>
<version>2.0-RC-5</version>
</dependency>
</dependencies>
Configuration Parameters
All MailConnector classes need parameters to connect to the corresponding server. These are set within the mailconnection.properties
file or by system properties. The following parameters must be set.
#SMTP
SMTP_SERVER=smtp.mailserver.com
SMTP_SERVER_PORT=25
SMTP_USERNAME=user
SMTP_PASSWORD=password
#POP3
POP3_SERVER=pop.mailserver.com
POP3_SERVER_PORT=110
POP3_FOLDER_INBOX=inbox
POP3_USERNAME=user
POP3_PASSWORD=password
# IMAP
IMAP_SERVER=imap.mailserver.com
IMAP_SERVER_PORT=143
IMAP_FOLDER_INBOX=INBOX
IMAP_USERNAME=user
IMAP_PASSWORD=password
# TIMING for POP3 and IMAP
# sets the timeout between polls, default = 50s
POLLING_TIMER_SECONDS=10
# sets the maximum number of polls, default = 20
MAX_READ_TRIES=20
If you want to use an SSL-encrypted connection, you need to set SMTP_SSL_ENABLED
/ POP3_SSL_ENABLED
to true. The ports must then be adjusted according to the server.
The actual connection to the mail server is implicitly opened within each MailConnector
class.
9.6.2. POP3MailConnector
The POP3MailConnector
provides access to a POP3 server. Emails are read and processed by using methods of the superclass AbstractInboxConnector
. Specific mails can be extracted with serverside filtering by matching given criteria expressed with SearchTerm (see JavaMail API)
POP3MailConnector pop3MailConnector = new POP3MailConnector();
// wait until the email with subject 'test' arrived in the InboxFolder
EmailQuery query = new EmailQuery().setSearchTerm(new SubjectTerm(subject));
Email email = pop3MailConnector.query(query).findFirst().get();
// delete all emails with this subject from the server while setting timeout and max number of polls explicitly
query.setRetryCount(5);
query.setPauseMs(2000);
Email email = pop3MailConnector.query(query).findFirst().get();
// delete mails matching the given criteria
String recipient = email.getRecipients().get(0));
String subject = email.getSubject();
String messageId = null;
pop3MailConnector.deleteMessage(recipient, RecipientType.TO, subject, messageId);
// wait until the email with subject 'test' arrived in the InboxFolder
EmailQuery query = new EmailQuery().setSearchTerm(new SubjectTerm(subject));
Email email = pop3MailConnector.query(query).findFirst().get();
try {
Multipart content = (Multipart) email.getMessage().getContent();
int contentCnt = content.getCount();
String attachmentFileName = null;
for (int i = 0; i < contentCnt; i++) {
Part part = content.getBodyPart(i);
// Retrieve attachment
if (part.getDisposition().equals(Part.ATTACHMENT)) {
attachmentFileName = part.getFileName();
}
}
} catch (IOException e) {
e.printStackTrace();
} catch (MessagingException e) {
e.printStackTrace();
}
9.6.3. SMTPMailConnector
This entity allows sending emails via the SMTP protocol.
SMTPMailConnector smtpMailConnector = new SMTPMailConnector();
// send a created message
MimeMessage createdMessage = new MimeMessage(session);
try {
msg.addRecipients(RecipientType.TO, RECIPIENT);
msg.addFrom(new Address[]{new InternetAddress(SENDER)});
msg.setSubject("testerra");
msg.setText("mail text");
} catch (MessagingException e) {
LOGGER.error(e.toString());
}
smtpMailConnector.sendMessage(createdMessage);
// send an existing message
MimeMessage existingMessage = MailUtils.loadEmailFile("test-mail.eml");
smtpMailConnector.sendMessage(existingMessage);
9.6.4. ImapMailConnector
The ImapMailConnector
operates like the POP3MailConnector with an additional method to mark all mails as seen.
ImapMailConnector imapMailConnector = new ImapMailConnector();
EmailQuery query = new EmailQuery().setSearchTerm(new SubjectTerm(subject));
imapMailConnector.query(query).findFirst().ifPresent(email -> {
// EMail found
});
// mark all mails in inbox as seen
imapMailConnector.markAllMailsAsSeen();
// delete all mails in inbox
imapMailConnector.deleteAllMessages();
9.6.5. Get simply the message count
You can get the message count for the inbox, of a specified folder name.
connector.getMessageCount();
connector.getMessageCount("FolderName");
9.6.6. SSL settings and trusting hosts for self-signed certificates
SSL is enabled per default for POP3 and IMAP and can be configured via. properties.
IMAP_SSL_ENABLED=false
POP3_SSL_ENABLED=false
SMTP_SSL_ENABLED=true
The MailConnector uses Certificate Utilities for trusting hosts.
9.6.7. Custom configuration
You can set properties to the JavaMail
framework like:
connector.configureSessionProperties(properties -> {
properties.put("mail.imaps.auth.ntlm.disable", true);
});
See the original documentation for more information: * IMAP: https://eclipse-ee4j.github.io/mail/docs/api/com/sun/mail/imap/package-summary.html * SMTP: https://eclipse-ee4j.github.io/mail/docs/api/com/sun/mail/smtp/package-summary.html * POP3: https://eclipse-ee4j.github.io/mail/docs/api/com/sun/mail/pop3/package-summary.html
9.6.8. Debugging the MailConnector
Enable the debug mode programatically
connector.getSession().setDebug(true);
or via Properties
DEBUG_SETTING=true
9.6.9. Best Practices
Combine search terms
You can combine search terms the following way
EmailQuery query = new EmailQuery();
query.withAllOfSearchTerms(
new SubjectTerm("My Subject"),
new ReceivedDateTerm(DateTerm.EQ, new Date())
);
// or
SearchTerm oneOf = new OrTerm(
new SubjectTerm("My Subject"),
new SubjectTerm("PetsOverNight.com"),
);
query.setSearchTerm(oneOf);
// or
List<SearchTerm> searchTerms = new ArrayList<>();
searchTerms.add(oneOf);
searchTerms.add(new ReceivedDateTerm(DateTerm.EQ, new Date()));
query.withAllOfSearchTerms(searchTerms);
Find emails by specified date
To find emails for a specified date, you should combine the SentDateTerm
and an explicit filter, because the internal library is not able to filter by exact
date with the IMAP protocol.
EmailQuery query = new EmailQuery();
Date now = new Date();
// Query emails that arrived today
query.setSearchTerm(new SentDateTerm(ComparisonTerm.GE, now));
connector.query(query)
.filter(email -> email.getSentDate().after(now))
.forEach(email -> {
// EMail found
});
9.6.10. MailUtils
This helper class contains methods which facilitate reoccurring tasks when working with mails, e.g. encrypting, decrypting, and comparing mails.
String pahtKeyStore = "your/path/to/cacert.p12";
String password = "123456";
String subject = "test";
String sentContent = "Testerra Testmail"
SMTPMailConnector smtpMailConnector = new SMTPMailConnector();
Session session = smtpMailConnector.getSession();
MimeMessage sentMessage = new MimeMessage(session);
sentMessage.setText(sentContent);
sentMessage.setSubject(subject);
// encrypt message
MimeMessage encryptedMsg = MailUtils.encryptMessageWithKeystore(sentMessage, session, pahtKeyStore, password);
smtpMailConnector.sendMessage(encryptedMsg);
Email receivedMsg = waitForMessage(subject);
// compare Mails and verify difference due to encryption
boolean areContentsEqual = MailUtils.compareSentAndReceivedEmailContents(sentMessage, receivedMsg);
Assert.assertFalse(areContentsEqual);
// decrypt message
MimeMessage decryptedMsg = MailUtils.decryptMessageWithKeystore(encryptedMsg, session, pahtKeyStore, password);
// verify receivedContent is equal to sentContent
String receivedContent = ((Multipart) decryptedMsg.getContent()).getBodyPart(0).getContent().toString();
Assert.assertEquals(receivedContent, sentContent);
9.7. PropertyManager
The PropertyManagerProvider
interface provides a PROPERTY_MANAGER
instance of IPropertyManager
String property = PROPERTY_MANAGER.getProperty("my.property", "default");
Long longProperty = PROPERTY_MANAGER.getLongProperty("my.long", 200L);
Double doubleProperty = PROPERTY_MANAGER.getDoubleProperty("my.double", 200.0);
Boolean booleanProperty = PROPERTY_MANAGER.getBooleanProperty("my.boolean", false);
It will look for the property definition appearance in the following order:
-
System.getProperties()
(system.properties
) -
Property files (
test.properties
)
When it’s still not defined, in will fall back to the given default value.
9.7.1. Property files
Testerra supports and loads automatically .properties
files located under test/resources
.
The test.properties
is called the Test Configuration and contains everything required by the test, like Browser setup, SUT credentials or Layout test thresholds.
You can load additional properties files at any time.
PROPERTY_MANAGER.loadProperties("my.property.file");
This will override already defined properties. |
9.7.2. System properties
When a system.properties
exists, Testerra loads and sets the given properties via System.setProperty()
if they are not present already.
This will not override already defined system properties given by the command line. |
The path of this file can be changed by tt.system.settings.file
and will be automatically loaded with the following message
common.PropertyManager - Load system properties: /path/to/your/system.properties
9.8. Connectors
While using the framework some external connectors could be useful. The maintainers provide some of them in separate repositories. Please take a look at the following list if there is a connector mapping your need.
Name | Description |
---|---|
Uses the open source standard Appium to run web tests based on Testerra on mobile devices. |
|
Automatic test result synchronization for Microsoft AzureDevOps platform. |
|
Provides the opportunity to use Cucumber and Gherkin to specify |
|
Automatic test result synchronization to HP Application Lifecycle Management, former called HP QualityCenter. |
|
Using a Selenium Grid based on Selenoid this module provides access to videos and VNC streams. |
|
A simple notification service for Jetbrains TeamCity. |
|
Automatic test result synchronization Jira XRay. |
10. Utilities
10.1. Assert Utilities (deprecated)
This class has been replaced by the Assertion interface, therefore marked as @deprecated and should not be used anymore.
|
This class provides some extra assertion methods for TestNG:
AssertUtils.assertContains("Hello World", "Martin", "Greeting");
// Greeting [Hello World] contains [Martin] expected [true] but found [false]
AssertUtils.assertContainsNot("Hello World", "World", "Greeting");
// Greeting [Hello World] contains [World] expected [false] but found [true]
AssertUtils.assertGreaterEqualThan(new BigDecimal(2), new BigDecimal(4), "High number");
// High number [2] is greater or equal than [4] expected [true] but found [false]
AssertUtils.assertLowerThan(new BigDecimal(2), new BigDecimal(-1), "Low number");
// Low number [2] is lower than [-1] expected [true] but found [false]
10.2. Certificate Utilities
This is an utility class that offers the possibility to trust all SSL certificates.
Accepting all SSL certs is a risk and should only be used in trusted environments. |
// Trust all certificates in the current Java VM instance.
CertUtils.trustAllCerts();
// Trust only certificates for a special connection and a SSL socket factory.
HttpsURLConnection CertUtils.trustAllCerts(
HttpsURLConnection connection,
SSLSocketFactory sslSocketFactory
);
10.3. Desktop WebDriver utilities
This utility class provides some additional methods to interact with web elements.
All methods using JavaScript snippets to execute an action instead the Selenium way.
Please consider this utility class as a fallback solution. It could be useful if elements are hidden or not reachable by Selenium. |
10.3.1. Supported actions
UiElement element = find(By.id("label"));
DesktopWebDriverUtils utils = new DesktopWebDriverUtils();
utils.clickJS(element);
The following methods are supported:
-
clickJS()
-
rightClickJS()
-
doubleClickJS()
-
mouseOverJS()
-
clickAbsolute()
-
mouseOverAbsolute2Axis()
See Mouse over for more details about the mouseOverAbsolute2Axis
method.
10.3.2. mouseOver
vs. mouseOverAbsolute2Axis
The mouseOverAbsolute2Axis
method does not move the mouse pointer relativly from the last position.
The following graphic shows 2 different mouse pointer paths beginning at the upper right image to the button 1.

The standard path (green) goes over a menu with hover effects. This could hide your target element. The blue path variant always goes first to the point (0,0) of the viewport, then in x and y direction the the target element.
10.4. JavaScript Utilities
JSUtils
provide some helper methods to inject and execute JavaScript code via WebDriver
.
10.4.1. Execute JavaScript
Sometimes in automated testing for web applications you want to access your system under test by JavaScript or you just want to implement a code snippet to run custom validations. For example, Testerras Demo mode will use this behaviour to highlight elements while asserting or typing to visualize current actions.
UiElement hiddenUploadField = findById("hidden-upload-field");
hiddenUploadField.expect().present(true);
hiddenUploadField.findWebElement(webElement -> {
// will change style to display a hidden element
JSUtils.executeScript(hiddenUploadField.getWebDriver(), "arguments[0].setAttribute('style', 'display:block; opacity: 1')", webElement);
});
10.4.2. Inject JavaScript (deprecated)
Content Security Policies disallow injection JavaScript into the DOM. Therefore, the following feature is @deprecated .
|
For executing more than a single line of JavaScript code it is recommended to write a JavaScript file and store it in src/main/resources
or src/test/resources
directory.
Then you can inject the full JavaScript file with following method.
// WebDriver, Path to resource file, id of the script-tag
JSUtils.implementJavascriptOnPage(driver, "js/inject/custom.js", "customJs");
Testerra will then inject your resource file into the current DOM of your WebDriver
instance according to the template.
<script id="customJs" type="text/javascript">
// your javascript code here.
</script>
10.5. Download Utilities
10.5.1. FileDownloader
The FileDownloader
utility provides methods for downloading resources from web sites. It uses the default proxy configuration.
// define the url where to download the resource from
String downloadUrl = "https://upload.wikimedia.org/wikipedia/commons/thumb/0/0a/"+
"S%C3%A5g_med_arm_av_valn%C3%B6t_-_Skoklosters_slott_-_92588.tif/"+
"lossy-page1-1024px-S%C3%A5g_med_arm_av_valn%C3%B6t_-_Skoklosters_slott_-_92588.tif.jpg";
// request access to the current web driver instance
WebDriver driver = WebDriverManager.getWebDriver();
// construct the downloader
FileDownloader downloader = new FileDownloader();
// perform the download
File downloadFile = downloader.download(driver, downloadUrl);
System.out.println("downloaded file exists:" + downloadFile.exists());
FileDownloader downloader = new FileDownloader();
downloader.setDownloadLocation("/some/directory");
downloader.setImitateCookies(true);
downloader.setTrustAllCertificates(true);
downloader.setProxy(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("123.0.0.1", 8080)));
downloader.cleanup();
10.6. PDF Utilities
This is an utility class for extracting text content of PDF documents for verifications.
// reads text from a file given by a filename (absolute path of file).
String textFromPdf = PdfUtils.getStringFromPdf(String pdfFileLocation);
//reads text from a pdf given as input steam.
String textFromPdf = PdfUtils.getStringFromPdf(InputStream stream);
10.7. Proxy Utilities
This is a static helper class based for reading the proxy configuration from system properties.
import java.net.URL;
import eu.tsystems.mms.tic.testframework.utils.ProxyUtils;
// Format e.g.: http://{http.proxyHost}:{http.proxyPort}
URL httpProxyUrl = ProxyUtils.getSystemHttpProxyUrl();
URL httpsProxyUrl = ProxyUtils.getSystemHttpsProxyUrl();
10.8. UITest Utilities (deprecated)
This class is marked as @deprecated and should not be used anymore.
|
The UITestUtils
supports you to generate additional screenshots.
You can add the screenshots into the report to the method steps.
@Test
public void testDemo() {
UITestUtils.takeScreenshot(WebDriverManager.getWebDriver(), true);
}
class ExamplePage extends Page {
public void doAnything() {
UITestUtils.takeScreenshot(this.getWebDriver(), true);
}
}
You can also store a simple screenshot to your project directory.
@Test
public void testDemo() {
...
// FileSystems.getDefault() returns the current working directory
File path = FileSystems.getDefault().getPath("screen.png").toFile();
UITestUtils.takeWebDriverScreenshotToFile(WebDriverManager.getWebDriver(), path);
...
}
Screenshots are always saved in the PNG image format. |
10.9. Timer Utilities
The timer utilities provide some useful time related classes.
10.9.1. Sleepy interface
The Sleepy
interface provides for proper sleep logging.
import eu.tsystems.mms.tic.testframework.utils.Sleepy;
class MyWorkflow implements Sleepy {
public void doSomething() {
sleep(); // Sleeps for an internal default timeout
sleep(long); // Sleeps for milliseconds
}
}
This will log something like
MyWorkflow - sleep(200ms) on MyWorkflow@62b635fe
10.9.2. TimerUtils
If you want to pause your current test execution, because you may have to wait for something or need a hard timeout you can use the TimerUtils
.
// Will pause current thread for 1 second.
TimerUtils.sleep(1000);
// You can pass a comment as well
TimerUtils.sleep(5000, "Wait for something to happen.");
Both of these methods calls of sleep()
will produce a log message.
If you want to sleep without a message, TimerUtils
of Testerra provides a method sleepSilent()
.
10.9.3. Timer (deprecated)
Timings can be controlled using TestController, therefore this class is marked as @deprecated and should not be used anymore.
|
The Timer
provides a basic timer feature, if you have to execute the same snippet of code in a defined interval. It stops after timeout or your sequence code was executed without any exceptions. If your sequence code was not successful the Timer
will occur a TimeoutException
.
final Timer timer = new Timer(500, 15_000);
timer.executeSequence(new Timer.Sequence<Boolean>() {
@Override
public void run() throws Throwable {
// sequence code here
}
});
With this approach you will block your current thread, mostly your main thread. If you want to execute your |
You can also return an object. In that case no TimeoutException
will occur. Therefor you have to verify your returning object.
public MyObject runTimer() {
final Timer timer = new Timer(500, 15_000);
ThrowablePackedResponse<MyObject> myObjectResponse
= timer.executeSequenceThread(new Timer.Sequence<MyObject>() {
@Override
public void run() throws Throwable {
// sequence code here
setPassState(boolean); // exit the sequence if your condition is true before timeout
setReturningObject(new MyObject());
}
});
return myObjectResponse.getResponse();
}
10.10. WebDriver Utilities
10.10.1. Keep a session alive
Sometimes when interacting with a remote selenium grid provider you will get in touch with the given session timeout of the provider. If your session is idle, the grid provider will close your session due to resource reasons after a timeout set by the provider itself. This is a helpful feature for selenium grid providers, but maybe you just want to set your WebDriver session on hold, because you are interacting with some other WebDriver session in the same thread.
To keep a session alive while processing some other tasks in the main thread you have to send some interactions. For this you can use the managed methods in Testerra framework. This will help you to get your things done and will ensure that you can’t keep sessions alive forever to avoid grid abuse.
WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
// Do some test stuff with session your session
// Starting a second session for example to test concurrent interactions.
WebDriver driverWithOtherUser = WEB_DRIVER_MANAGER.getWebDriver("Session2");
// Keep alive driver while doing actions on driverWithOtherUser for 90 seconds, while refreshing all 10 seconds
WEB_DRIVER_MANAGER.keepAlive(driver, 10, 90);
// Do your things with your second driver
// NOTE: Please release your WebDriverKeepAliveSequence as you dont need the lock
WEB_DRIVER_MANAGER.stopKeepingAlive(driver);
10.11. JVM Utilities
This is an utility class that gives you access to the jvm system performance indicators.
// the jvm's performance indicators
int cpuPercent = JVMUtils.getCPUUsagePercent();
int memoryPercent = JVMUtils.getMemoryUsagePercent();
long usedMemory = JVMUtils.getUsedMemory();
long maximumMemory = JVMUtils.getUsedMemory();
System.out.println("cpu usage: " + cpuPercent + " %");
System.out.println("memory usage: " + memoryPercent + " %");
System.out.println("used memory: " + usedMemory);
System.out.println("maximum memory: " + maximumMemory);
// a typical output of these utility methods would be:
//
// cpu usage: 33 %
// memory usage: 20 %
// used memory: 370
// maximum memory: 370
Extending Testerra
11. Extending Testerra
Testerra provides several ways for extensions.
11.1. Modules
To register your module, you need to create a module injection configuration based on the Google Guice framework.
package eu.tsystems.mms.tic.mymodule;
import com.google.inject.AbstractModule;
public class ConfigureMyModule extends AbstractModule {
@Override
protected void configure() {
// inject dependencies here
}
}
Please be aware, that the package namespace prefix eu.tsystems.mms.tic is required in order to find your module configuration.
|
All module configuration instances are loaded in alphabetical order. |
11.2. Hooks
When you need more module features, like registering to Events or perform setup and teardown functionality, you can implement the ModuleHook
interface.
package eu.tsystems.mms.tic.mymodule;
import eu.tsystems.mms.tic.testframework.hooks.ModuleHook;
public class ConfigureMyModule extends AbstractModule implements ModuleHook {
@Override
public void init() {
//
}
@Override
public void terminate() {
//
}
}
While the init()
method is called on startup of Testerra as one of the earliest steps, you are able to make your customizations as soon as possible.
The terminate()
method is called at the most latest point for Testerra right before terminating the execution.
You should use this method to cleanup your module, if necessary, for example closing database connections in a database module.
11.3. Events and Listeners
Testerra provides a Google Guava event bus with custom events for test execution. The following events are currently implemented:
Event | Description |
---|---|
|
Called on start of every test method annotated by TestNG |
|
Called at the end of every test method annotated by TestNG |
|
Called at the end of test run to trigger report generation and other output worker. |
|
Called on test run abortion due to unclear circumstances like hanging sessions, JVM exit or other. |
|
Called before suite execution. The events methods list provides a list of tests to execute. Read more about this in Intercept test method execution |
|
Called every time the internal context data has been changed significantly. This can be used to synchronize the internal test context model. |
|
Called on the very end of the test execution when the execution model has been finalized. Use this event to generate a report. |
11.3.1. Create custom event listeners
The simplest way to get in touch with the event bus is to write and register your own implementation of the event’s Listener
interface and add the @Subscribe
annotation.
import com.google.common.eventbus.Subscribe;
import eu.tsystems.mms.tic.testframework.events.MethodStartEvent;
import eu.tsystems.mms.tic.testframework.logging.Loggable;
public class LogEventListener implements MethodStartEvent.Listener, Loggable {
@Override
@Subscribe
public void onMethodStart(MethodStartEvent event) {
log().info("Event logged: " + event.getMethod());
}
}
If you want to react to some more events, you can just implement multiple interfaces.
11.3.2. Register custom event listener
After you defined your first custom listener you now have to register it to the TesterraListener
.
Testerra.getEventBus().register(new LogEventListener());
11.3.3. Fire events by yourself
While Implementing your own module you may reach a point, where you want to inform other components or modules about an important event. You can achieve this by just posting this event to the bus.
For example, if your module changes some data in the underlying data model, you have to inform all other "participants" about your change by firing an ContextUpdateEvent
event.
// Update some data in data model...
methodContext.name = "new_Test_Method_Name";
Testerra.getEventBus().post(new ContextUpdateEvent().setContext(methodContext));
11.3.4. Create custom event types
If you want to provide some custom events to other modules. You can implement these by implementing any kind of class for the event bus.
public class CustomEvent {
public interface Listener {
void onCustomEvent(CustomEvent event);
}
}
With your CustomEvent
created, you now can fire these events or react to them in the way described in the sections Fire events by yourself and Create custom event listeners.
@Override
@Subscribe
public void onCustomEvent(CustomEvent event) {
log().info("Custom Event started!");
}
11.3.5. Intercept test method execution
With the test InterceptMethodsEvent
, you are able to modify the list of tests being executed before execution.
import eu.tsystems.mms.tic.testframework.events.InterceptMethodsEvent;
public class MyTest extends TesterraTest {
public class MyListener implements InterceptMethodsEvent.Listener {
@Override
@Subscribe
public void onInterceptMethods(InterceptMethodsEvent event) {
event.getMethodInstances().removeIf(iMethodInstance -> true);
}
}
@BeforeTest
public void setupListener() {
Testerra.getEventBus().register(new MyListener());
}
}
11.3.6. Listen to TestNG events
Since the TesterraListener
listens to TestNG
events, it also forwards some of these events the same way like any other events.
import eu.tsystems.mms.tic.testframework.logging.Loggable;
import com.google.common.eventbus.Subscribe;
import org.testng.ISuite;
import org.testng.ISuiteListener;
class MySuiteListener implements ISuiteListener, Loggable {
@Subscribe
@Override
public void onStart(ISuite suite) {
log().info("Suite started");
}
}
11.4. Provide Properties
When you want to provide some properties, you can use the IProperties
interface within your module.
package eu.tsystems.mms.tic.mymodule;
import eu.tsystems.mms.tic.testframework.common.IProperties;
public class MyModule {
public enum Properties implements IProperties {
GREETING("greeting", "hello world"),
ENABLED("enabled", false),
ANSWER("answer", 42),
;
private final String property;
private final Object defaultValue;
Properties(String property, Object defaultValue) {
this.property = property;
this.defaultValue = defaultValue;
}
@Override
public String toString() {
return String.format("tt.mymodule.%s",property);
}
@Override
public Object getDefault() {
return defaultValue;
}
}
}
Override the default values in a .properties
file.
tt.mymodule.greeting=hello planet
tt.mymodule.enabled=true
And access them in your code like
MyModule.Properties.GREETING.toString(); // tt.mymodule.greeting
MyModule.Properties.GREETING.asString(); // hello planet
MyModule.Properties.ENABLED.asBool(); // true
MyModule.Properties.ANSWER.asLong(); // 42
11.5. Create a report
When you want to create a custom report, you have to add the report-model
module as a dependency to your module and listen to the FinalizeExecutionEvent
.
import com.google.common.eventbus.Subscribe;
import eu.tsystems.mms.tic.testframework.events.FinalizeExecutionEvent;
import eu.tsystems.mms.tic.testframework.logging.Loggable;
public class GenerateReportListener implements FinalizeExecutionEvent.Listener, Loggable {
@Override
@Subscribe
public void onFinalizeExecution(FinalizeExecutionEvent event) {
log().info("Generate report");
}
}
11.5.1. Generate the protobuf report model
Testerra ships Google Protobuf model adapters for the internal context model.
You can automatically generate all the models during the execution, when you register the AbstractReportModelListener
in your module hook.
import com.google.common.eventbus.EventBus;
import eu.tsystems.mms.tic.testframework.listeners.AbstractReportModelListener;
import eu.tsystems.mms.tic.testframework.hooks.ModuleHook;
import eu.tsystems.mms.tic.testframework.report.TesterraListener;
public class CustomReportModuleHook implements ModuleHook {
@Override
public void init() {
Report report = Testerra.getInjector().getInstance(Report.class);
EventBus eventBus = Testerra.getEventBus();
eventBus.register(new GenerateReportModelListener(report.getReportDirectory()));
}
}
This will generate Protobuf models in test-report/models
.
Appendix
12. Best Practices
This section contains several articles for best practices for the usage and implementation of tests with Testerra.
12.1. Browser specific knowledge
12.1.1. Firefox
Prevent Firefox download confirmation dialog
WEB_DRIVER_MANAGER.setUserAgentConfig(Browsers.firefox, (FirefoxConfig) options -> {
// You have to add every mimetype you want no confirmation for
options.addPreference("browser.helperApps.neverAsk.saveToDisk", "application/zip");
options.addPreference("browser.download.manager.showAlertOnComplete", false);
options.addPreference("browser.download.manager.showWhenStarting", false);
});
12.1.2. Chrome
Chrome in a container
If using Selenium with Chrome in a Docker container it may comes to some random WebDriverException
, because of some internal container limits.
If you are getting random selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash
, may this code snippet will solve your problem, by disabling the usage of dev/shm
memory and enabling the usage of /tmp
instead.
This will may slow down your execution time, because you are using disk instead of memory.
The reason is, that chrome / chrome driver leads to errors when /dev/shm
is too small.
WEB_DRIVER_MANAGER.setUserAgentConfig(Browsers.chromeHeadless, (ChromeConfig) options -> {
options.addArguments("--disable-dev-shm-usage");
});
12.1.3. Internet Explorer
Skip certificate warning page
When testing websites with own certificates you may encounter issues and warning pages in each browser you use. For Chrome or Firefox these warnings can be skipped by setting properties, but for the Internet Explorer you have to define a small helper method, that you can call right after opening the base url.
public void skipInternetExplorerSecurityWarning(WebDriver driver, boolean handleAlert) {
driver.navigate().to("javascript:document.getElementById('overridelink').click()");
if (handleAlert) {
driver.switchTo().alert().accept();
}
}
12.2. Working with HTML elements
12.2.1. Radio buttons
Since radio buttons share the same name, as the following example shows
<input type="radio" name="beverage" value="tee">
<input type="radio" name="beverage" value="coffee">
It’s not a good practice to select it By.name
.
Its better to select both options separately.
// Good practice
PreparedLocator locator = LOCATE.prepare("//input[@name='%s' and @value='%s']").unique();
UiElement teeOption = find(locator.with("beverage", "tee"));
UiElement coffeeOption = find(locator.with("beverage", "coffee"));
// Bad practice
UiElement options = find(By.name("beverage"));
12.2.2. Shadow Roots
Working with Shadow Roots and Shadow DOM elements is not supported by Selenium. It’s a highly-experimental feature and not all browsers can work well with these elements in browser automation. The best browser by our experience is a chromium-based browser. |
Modern web applications are allowed to use some third-party components, which can be integrated with Shadow DOM. This is the modern art of an iframe, because the components will be loaded via asynchronous JavaScript.
Each embedded Shadow DOM component will have its own shadow root.
To work with shadow root elements Testerra provide the method shadowRoot()
on the UiElement
class.
Given the following HTML code snippet it will be easier how to get the corresponding UiElement
of the Shadow DOM component.
<body>
<div id="wrapper">
<!-- HTML code-->
<my-custom-shadow-root-element>
<!-- #shadowRoot -->
<div class="custom-component">
<input id="custom-component-login-name" name="name">
</div>
</my-custom-shadow-root-element>
</div>
<!-- HTML code-->
</body>
UiElement shadowRootElement = find(By.cssSelector("my-custom-shadow-root-element")).shadowRoot();
UiElement inputName = shadowRootElement.find(By.xpath("./custom-component-login-name"));
Calling displayed(boolean) on shadowRoot() -notated UiElement will always return false, but present(boolean) will work as designed.
|
Because of technical limitations and internal handling of browser-different access to Shadow DOM elements, it is not possible to use any other By Selector-type than By.xpath() working with find() on a shadow root.
|
12.3. Support multiple profiles
Supporting multiple profiles is useful, for different environments or test suites.
test {
def profiles = [
"mySuite": "suite.xml"
]
def suiteFiles = []
profiles.each { k, v ->
if (project.hasProperty("" + k)) {
def f = 'src/test/resources/' + v
suiteFiles << f
}
}
useTestNG() {
suites(suiteFiles as String[])
}
}
<project>
<profiles>
<profile>
<id>mySuite</id>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>false</skip>
<suiteXmlFiles>
<suiteXmlFile>src/test/resources/suite.xml</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
gradle test -PmySuite
mvn test -PmySuite
12.4. Test on multiple environments
If you run your tests on multiple test environments, you need for every environment a specific set of settings, for example another tt.baseurl
or tt.browser
.
12.4.1. Define your properties
Your test.properties
describes your default setting. If you want to change some properties, you can define another property file and locate it into the resource
folder.
tt.browser=chrome
tt.baseurl=http://example.org
tt.browser=firefox
tt.baseurl=http://qa.example.org
12.4.2. Load specific properties at startup
Load your custom property file at test startup if its necessary.
@BeforeSuite
public void setup() {
String env = PROPERTY_MANAGER.getProperty("env", "");
if (!"".equals(env)) {
PROPERTY_MANAGER.loadProperties("test_" + env + ".properties");
}
}
12.4.3. What happens?
Testerra is loading the test.properties
automatically at its initialization. Loading another property file will overwrite already existing values.
If you add your env
property at the Gradle or Maven call, you can control the execution depending on the test environment.
gradle test -Denv=qa
12.5. Project setup
Project setup recommendations.
12.6. Debugging tests
When you want to debug tests in your IDE, you can use the following setup to debug tests after they failed.
// Check if the JVM is in Debug mode
boolean isDebug = java.lang.management.ManagementFactory.getRuntimeMXBean().getInputArguments().toString().indexOf("-agentlib:jdwp") > 0;
if (isDebug) {
// Disable close windows after test
WEB_DRIVER_MANAGER.getConfig().setShutdownSessionAfterTestMethod(false);
// Register to the end event
EventBus eventBus = Testerra.getEventBus();
eventBus.register(new MethodEndEvent.Listener() {
@Override
@Subscribe
public void onMethodEnd(MethodEndEvent event) {
if (event.isFailed()) {
// Set your breakpoint here
log().error("Failed");
}
}
});
}
13. Migration from Testerra 1
The main goal for re-inventing the GuiElement API was at first, to provide a better user experience for implementing tests scripts in the role of a Test Engineer.
Automated testing on web pages is hard, especially on large enterprise environments we address with Testerra. And yes, we know the trouble what happens when tests (suddenly) fails.
As an Test Engineer, you may want expressive info messages for bugs in the System under Test or to keep your tests up to date to the specification. You may need easy to understand test code and a robust extensible Page Object model for large test environments by keeping the full potential of a Software Developer?
The new Testerra API provides it all.
If you’re a developer and want to know how the new API works, you can jump straight to The internals or just proceed with the next chapter to get an overview.
13.1. Page object changes
When implementing the PageFactoryProvider
interface, you get an PAGE_FACTORY
instance within your class.
import eu.tsystems.mms.tic.testframework.testing.PageFactoryProvider;
class MyTest extends TesterraTest implements PageFactoryProvider {
@Test
public void test_MyPageTest() {
MyPage page = PAGE_FACTORY.createPage(MyPage.class);
// Or using a different WebDriver
MyPage page = PAGE_FACTORY.createPage(MyPage.class, WEB_DRIVER_MANAGER.getWebDriver());
}
}
but you can instantiate pages within pages more easily:
class MyPage extends Page {
public AnotherPage navigateToAnotherPage() {
return createPage(AnotherPage.class);
}
}
Using the static
PageFactory is now @deprecated
|
Passing variables to the Page constructor is now
@deprecated
|
Constructor instantiation of Pages is prohibited!
|
13.1.1. Components pattern
The new standard way to implement Sub Pages aka Components is now
public class MyForm extends AbstractComponent<MyForm> {
public MyForm(UiElement rootElement) {
super(rootElement);
}
}
Instantiate components
class MyPage extends Page {
private MyForm form = createComponent(MyForm.class, find(By.tagName("form")));
}
13.1.2. Implicit Element checks
The standard way of implicit GuiElement checks is now
class MyPage extends Page {
@Check
private UiElement uiElement = findById(42);
}
Performing explicit page checks is prohibited!
|
13.2. Responsive PageFactory
The responsive page factory features have been removed from the default implementation. To use it anyway, you need to inject in in your Modules configuration.
import eu.tsystems.mms.tic.testframework.pageobjects.internal.ResponsivePageFactory;
public class MyModule extends AbstractModule {
protected void configure() {
bind(PageFactory.class).to(ResponsivePageFactory.class).in(Scopes.SINGLETON);
}
}
13.3. Element creation changes
The new standard way to instantiate GuiElements is now
class MyPage extends Page {
private UiElement uiElement = findById(42);
private UiElement uiElement = find(By.xpath("//div[1]"));
}
Constructor instantiation of GuiElements is now
@deprecated
|
For descendant elements
class MyPage extends Page {
private UiElement parent = findById(42);
private UiElement sub = parent.find(By.xpath("//div[1]"));
}
getSubElement is now @deprecated
|
List elements
UiElement anchors = find(By.tagName("a"));
anchors.expect().foundElements().is(3); (1)
UiElementList<UiElement> list = anchors.list();
list.first().expect().value(Attribute.TITLE).is("StartPage"); (2)
list.get(1).expect().value(Attribute.TITLE).is("About Us"); (3)
list.last().expect().value(Attribute.TITLE).is("Contact"); (4)
list.forEach(anchor -> anchor.expect().value(Attribute.HREF).startsWith("https")); (5)
GuiElement lists are now
@deprecated
|
For elements in frames
class MyPage extends Page {
private UiElement frame = find(By.tagName("frame")); (1)
private UiElement uiElement = frame.findById(14); (2)
}
Passing frames to the constructor is not supported anymore.
|
13.4. Assertion changes
13.4.1. Element assertions
The new standard way to perform assertions on elements like Pages and GuiElements is now
uiElement.expect().displayed(true); (1)
uiElement.expect().value().contains("Hallo Welt").is(true); (2)
Using the GuiElement assertions is now
@deprecated
|
Perform decisions on occurrence with the waitFor
prefix.
if (uiElement.waitFor().displayed(true)) {
// Optional element became visible
}
Using the GuiElement waits is now
@deprecated
|
Support of more features through consistent assertion API
uiElement.expect().css("display").is("none"); (1)
uiElement.expect().text()
.map(value -> value.toLowerCase()) (2)
.matches("^hello\\s.orld") (3)
.is(true);
1 | Perform assertions on the element’s CSS properties |
2 | Map values |
3 | Regular expression assertions |
Custom failure messages
uiElement.expect().displayed(true, "Element is displayed");
13.4.2. Page assertions
Assert that a text is visible on a page
page.expect().url().endsWith("index.html").is(true); (1)
class MyPage extends Page {
public void checkIfPageContainsText(String text) {
this.getFinder()
.findDeep(XPath.from("*").text().contains(text))
.expect().displayed(true); (2)
}
}
Using the text assertions is now
@deprecated
|
13.4.3. Screenshot based Assertions
The new standard way to perform screenshot based assertions is now
uiElement.expect().screenshot().pixelDistance("ElementReference").isLowerThan(1);
page.expect().screenshot().pixelDistance("PageReference").isBetween(0, 10);
Add screenshot to the report
page.screenshotToReport();
Using the static
UITestUtils is now @deprecated
|
13.4.4. Layout based Assertions
To check if a element is beside another element
UiElement left = find(By.id("left"));
UiElement right = find(By.id("right"));
left.expect().bounds().leftOf(right).is(true); (1)
left.expect().bounds().intersects(right).is(false);
Elements aligned to the same right
UiElement top = find(By.id("top"));
UiElement bottom = find(By.id("bottom"));
top.expect().bounds().fromRight().toRightOf(bottom).is(0); (2)
Element contains another element
UiElement body = find(By.tagName("body"));
UiElement nav = parent.find(By.tagName("nav"));
body.expect().bounds().contains(nav).is(true);
Using the
assertLayout() method is now @deprecated
|
13.5. New Control API
With the TestController
API, you are able to control your test flow during runtime. Like timeouts, assertion handling and retry intervals.
The Control
instance is availabe as soon you implement the TestControllerProvider
interface.
import eu.tsystems.mms.tic.testframework.testing.TestControllerProvider;
class MyTest implements TestControllerProvider {
}
13.5.1. Collected Assertions
The new standard way to collect assertions of elements in tests or pages is now
CONTROL.collectAssertions(() -> uiElement.expect().displayed(true));
For many elements or pages
CONTROL.collectAssertions(() -> {
MyPage page = PAGE_FACTORY.create(MyPage.class);
page.expect().title().is("TestPage");
uiElement.expect().value().contains("Hello");
});
For custom assertions
CONTROL.collectAssertions(() -> {
String data = loadSomeData();
Assert.assertEquals(data, "Hello World", "some data");
});
For other test methods
@Test
public void test_CollectEverything() {
CONTROL.collectAssertions(() -> test_TestSomething());
}
Using the static
AssertCollector is now @deprecated
|
Using the GuiElement’s assert collector is now
@deprecated
|
Forcing standard assertions is now
@deprecated
|
Setting collected assertions by default is now
@deprecated
|
13.5.2. Optional Assertions
The new standard way for optional assertions works like Collected Assertions
CONTROL.optionalAssertions(() -> uiElement.expect().displayed(true));
Using the static
NonFunctionalAssert is now @deprecated
|
Using the GuiElement’s non functional asserts are now
@deprecated
|
13.6. Timeouts and Retry API
13.6.1. @Check timeouts
The new standard way for setting GuiElement timeouts on @Check
is now
class MyPage extends Page {
@Check(timeout = 1)
private UiElement uiElement;
}
Setting and restoring explicit timeouts on the GuiElement is now
@deprecated
|
For the whole Page
@PageOptions(elementTimeoutInSeconds = 1)
class MyPage extends Page {...}
Setting explicit timeouts on the Page is now
@deprecated
|
Override during runtime
CONTROL.withTimeout(1, () -> uiElement.expect().displayed(true));
For many elements
CONTROL.withTimeout(1, () -> {
MyPage page = PAGE_FACTORY.create(MyPage.class);
page.expect().title().is("TestPage");
uiElement.expect().value().contains("Hello");
});
For other test methods
@Test
public void test_TestSomething_fast() {
CONTROL.withTimeout(1, () -> test_TestSomething());
}
Setting timeouts using static
POConfig is now @deprecated
|
13.7. Modul migration
We want to make Testerra more SOLID. Thats why we finally introduced Dependency Injection via. Google Guice.
To enable you ModuleHook
for v2, you need to extend this class from AbstractModule
.
import com.google.inject.AbstractModule;
import eu.tsystems.mms.tic.testframework.hooks.ModuleHook;
public class MyModuleHook extends AbstractModule implements ModuleHook {
}
13.8. The internals
This chapter explains how the new API works internally.
13.8.1. Everything is timed, but once
Every assertions is performed multiple times with a maximum timeout of tt.element.timeout.seconds. If this timeout has reached, the assertion will finally fail.
But there is only one timeout for each assertion now. No more implicit timeouts on sub method calls like getWebElement()
, isPresent()
etc.
This is what an assertion internally does, when you perform uiElement.expect().text().contains("Something")
.
-
Find web element using WebDriver
-
Check if element is present
-
Retrieve the text of the element
-
If the text does not contain "Something", start over with 1.
-
Otherwise when the timeout has reached, an assertion error message will be displayed that the text of the element you’re looking for doesn’t contain the string "Something".
13.8.2. More consistence, less complexity
There will be only one interface for everything you need in a manner of an easy to read fluent API. It is not too abstract like TestNG Assert, and not to technically like AssertJ.
The new interface will always act exactly like you expect to, no matter in which context you are. You don’t have to decide which method you should use. The standard way will be the best fit for most cases. Let the framework handle the workarrounds for you.
13.8.3. Strict Page Object pattern
Testerra was built with the Page Object pattern in mind. The new API makes it easier for your team, to keep you on track makes it harder to break out, even if your project contains hundreds of Pages and thousands of Tests.
The new components extension allows you to implement page objects like a web developer would do, by separating functionality into reusable components.
13.8.4. Smaller codebase and less boilerplate
The API provides abstract assertion implementations for several properties.
-
StringAssertion
allows you to perform assertions on strings likecontains("Something")
-
QuantityAssertion
allows you to perform assertions on quantified values likeisBetween(-2,3)
-
BinaryAssertion
allows to assert if an value is boolean or a string that represents a boolean value withis(true)
These generic assertions are used in many other assertions and supports a hierarchical order.
This is what the hierarchy looks like when you perform uiElement.screenshot().file().extension().is("png")
-
Take a screenshot and return a
ScreenshotAssertion
-
Return a generic
FileAssertion
with the taken screenshot file -
Return a generic
StringAssertion
with the given file name extension
This implementation helps to keep the internal assertion code small, easy extensible and maintainable.
14. Known issues
Because we depend on other frameworks and tools like TestNG and Selenium we may encounter issues that we want to fix, but are bound to releases and fixes in our dependencies.
Every known issue in our dependencies that will lead to an error, an unexpected or unsightly behaviour in Testerra framework will be documented here, as well as a solution or a workaround.
15. Overview of all Testerra properties
Properties are managed by the PropertyManager
15.1. Testerra core properties
Property | default | Description |
---|---|---|
tt.system.settings.file |
system.properties |
File name of property file for proxy settings. |
tt.cert.trusted.hosts |
(empty) |
Whitespace separated list of trusted hosts (for SSL sockets) |
15.2. WebdriverManager properties
Property | default | Description |
---|---|---|
tt.browser |
na. |
Browser label, the following browsers are possible
|
tt.browser.version |
na. |
Version label |
tt.browser.setting |
You can combine browser type and version as single string like |
|
tt.baseurl |
na. |
URL of the first site called in a new browser session |
tt.webdriver.mode |
remote |
Sets the webdriver mode. remote uses an external Selenium server |
tt.selenium.server.url |
na. |
The complete URL to a remote Selenium server. This setting overrides the following two properties. |
tt.selenium.server.host |
localhost |
The host name of the remote Selenium server. |
tt.selenium.server.port |
4444 |
The port of the remote Selenium server. |
tt.browser.maximize |
false |
Try to maximize the browser window. |
tt.browser.maximize.position |
self |
Screen position for the window to maximize. If you have several screens and want to maximize the window on another screen than your default screen, you can choose between (left, right, top or bottom) |
tt.display.resolution |
1920x1200 |
Fall-back resolution, if maximize does not work. |
tt.wdm.closewindows.aftertestmethods |
true |
If true, after every test method all open browser windows are closed. |
tt.wdm.closewindows.onfailure |
true |
If true, after failed test methods all open browser windows are closed |
tt.wdm.timeouts.seconds.selenium.command.stuck |
300 |
Kills a stuck selenium command after this timeout |
tt.wdm.timeouts.seconds.window.switch.duration |
5 |
Maximum duration to wait for on a |
webdriver.timeouts.seconds.pageload |
120 |
Defines the Selenium timeout for page load seconds. |
webdriver.timeouts.seconds.script |
120 |
Defines the Selenium timeout for execution of async scripts in seconds. |
15.3. PageFactory properties
Property | default | Description |
---|---|---|
tt.project.package |
eu.tsystems.mms.tic |
The package where the PageFactory searches for screen resolution specific subclasses. |
tt.page.factory.loops |
20 |
The loop detections prevents endless recursive creation of new page instances. This property defines the max count of loops. |
15.4. UiElement properties
Property | default | Description |
---|---|---|
tt.element.timeout.seconds |
8 |
Default timeout for UiElement actions and assertions |
tt.guielement.default.assertcollector |
false |
Sets the behavior of |
tt.guielement.checkrule |
|
Rule for Page objects validation of UiElements |
tt.delay.after.guielement.action.millis |
0 |
Waits in milliseconds after an action on a UiElement. |
tt.delay.before.guielement.action.millis |
0 |
Waits in milliseconds before an action on a UiElement. |
15.5. Execution properties
Property | default | Description |
---|---|---|
tt.demomode |
false |
Visually marks every UiElement that is being processed by click, type or assert. This may break layout checks. |
tt.dryrun |
false |
All testmethods are executed with ignoring all steps. Also all setup methods (before, after) are ignored. |
tt.report.list.tests |
false |
Lists all test methods in the current context without executing them. |
tt.on.state.testfailed.skip.shutdown |
false |
If true all browser sessions are left open. |
tt.on.state.testfailed.skip.following.tests |
false |
If true, all follwoing tests are skipped in case a test failed. |
tt.failed.tests.if.throwable.classes |
na. |
Failed tests condition: Throwable Class(~es, devided by ','). |
tt.failed.tests.if.throwable.messages |
na. |
Failed tests condition. Throwable Message(~s, devided by ','). |
tt.failed.tests.max.retries |
1 |
How often tests should be retried by the Testerra RetryAnalyzer. |
tt.reuse.dataprovider.driver.by.thread |
false |
Reuse existing Webdriver session for a thread of dataprovider. |
tt.execution.omit.indevelopment |
false |
If |
tt.watchdog.enable |
true |
Enables/Disables the |
tt.watchdog.timeout.seconds |
300 |
Sets the timeout in seconds after the |
tt.failure.corridor.active |
true |
Activate the failure corridor. |
tt.failure.corridor.allowed.failed.tests.high |
0 |
Number of test methods with weighting high allowed to fail to still mark the suite as passed. |
tt.failure.corridor.allowed.failed.tests.mid |
0 |
Number of test methods with weighting mid allowed to fail to still mark the suite as passed. |
tt.failure.corridor.allowed.failed.tests.low |
0 |
Number of test methods with weighting low allowed to fail to still mark the suite as passed. |
tt.perf.test |
false |
If true, activates performance test related behaviour sets default values for the performance test. |
tt.perf.page.thinktime.ms |
0 |
Sets a thinktime in ms for each page load. |
tt.perf.generate.statistics |
false |
If true, activates generation of graphs for the performance measurements. |
15.6. Report properties
Property | default | Description |
---|---|---|
tt.report.dir |
/target/surefire-reports/report/ |
Creates the report in the specified directory below the working directory. |
tt.report.name |
na. |
Names the report (e.g. the project where Testerra is used) |
tt.runcfg |
na. |
Set a run configuration to use different variations (test sets) of a test scope within a build task. |
tt.screenshotter.active |
true |
If true, screenshots are fetched and added to the report. |
tt.screenshots.preview |
true |
If true a screenshots preview is added to the test methods in the report |
tt.screenshot.on.pageload |
false |
If true, screenshot after page is loaded will be taken |
tt.screencaster.active |
true |
If true, all screencasts are fetchted and added to the report depending on the enabled test method states by tt.screencaster.active.on.failed and tt.screencaster.active.on.success. |
tt.screencaster.active.on.failed |
true |
If true, all screencasts for failed tests are fetched and added to the report. |
tt.screencaster.active.on.success |
false |
If true, all screencasts for successful tests are fetched and added to the report. |
tt.report.activate.sources |
true |
If true, adds source information to report |
tt.module.source.root |
src |
Root directory for searching source info. |
tt.source.lines.prefetch |
5 |
Amount of lines taken into account before the actual error occurred (print lines between error line and error line minus tt.source.lines.prefetch) |
15.7. Layout Check properties
Property | default | Description |
---|---|---|
tt.layoutcheck.reference.path |
|
Path where the reference screenshots where saved |
tt.layoutcheck.reference.nametemplate |
|
Prefix for ReferenceScreenshots |
tt.layoutcheck.takereference |
false |
Determines whether reference images where taken in the current run |
tt.layoutcheck.use.ignore.color |
false |
Specifies whether the upper left pixel in the reference image defines an "ignore color". If true, then every pixel with this color will be ignored during the later comparison. |
tt.layoutcheck.use.area.color |
false |
Specifies whether the upper left pixel in the reference image defines an "area color". If true, then every area surrounded by pixels with this color will be used for later comparison, other areas are dismissed. Opposite of tt.layoutcheck.use.ignore.color. |
tt.layoutcheck.actual.nametemplate |
|
Filename scheme for saving current screenshots. The value must contain a '%s' which is replaced by the specified target file name during test execution. |
tt.layoutcheck.distance.nametemplate |
|
Filename scheme for saving distance images. The value must contain a '%s' which is replaced by the specified target file name during test execution. |
tt.layoutcheck.distance.path |
|
Directory path under which the calculated distance images are stored. |
tt.layoutcheck.actual.path |
|
Directory path under which the current screenshots for the comparison are saved |
tt.layoutcheck.match.threshold |
0 |
Max allowed difference in rgb values between actual and reference image in percentage. If R, G, and B percentages are higher than tt.layoutcheck.match.threshold the corresponding pixel is marked as false (=red color in distance image) |
tt.layoutcheck.match.threshold |
0.95d |
Defines at which score a region is considered a match. Should be as high as possible and as low as needed. |
tt.layoutcheck.displacement.threshold |
5 |
Displacement distance of matches that is considered as error (when distance > tt.layoutcheck.displacement.threshold) |
tt.layoutcheck.intra.grouping.threshold |
5 |
Threshold for grouping movement errors (errors ⇐ tt.layoutcheck.intra.grouping.threshold are grouped) |
tt.layoutcheck.min.match.distance |
5 |
Max distance for grouping multiple matches as single match (distance of matches < tt.layoutcheck.min.match.distance are marked as single match) |
tt.layoutcheck.min.size.difference.sub.images |
10 |
Minimal difference in size of the reference and actual image, to consider the reference image as sub image. |
tt.layoutcheck.distance.multiple.matches |
14 |
Max distance between matches until a warning message about the parameter setting is logged. (matches have distance < tt.layoutcheck.distance.multiple.matches a warning is logged) |
tt.layoutcheck.ignore.ambiguous.movement |
- |
when true ignore ambiguous movement, which means for a template, several matches were found at different positions and it is unclear to which match the template belongs to. |
tt.layoutcheck.ignore.movement |
- |
when true ignore movement, which mean exactly one match was found for a template, but it is in the wrong position. |
tt.layoutcheck.ignore.group.movement |
false |
when true ignore group movement, which means for a set of templates, displacement errors have been found which have the same displacement vector. |
tt.layoutcheck.ignore.missing.elements |
- |
when true ignore missing elements, which mean no match was found for a template. |
tt.layoutcheck.ignore.ambiguous.match |
- |
when true ignore ambiguous match, which means several templates have been matched to this position, but only one template can be correct. |
Architecture

Glossary
- SUT
-
System under test
- Tapas
-
Test Automation Platform as a Service (https://tapa-as.io)