What is Testerra?

Testerra logo

It is an integrated Java framework for automating tests for (web) applications. Testerra can also be understood as a building block for test automation projects with various basic components.

Testerra is based on Selenium but makes it much easier to create your automation solution to test your application.

The framework is developed by our Test Automation Experts at Telekom MMS in Dresden (Website). In numerous projects Testerra is used as the standard test automation framework and includes the experience of more then 10 years of test automation.

This manual belongs to Testerra 2.11. For older versions check out this table.

Getting Started

1. Create a new project

1.1. System requirements

  • Testerra is based on Java. You need a JDK 11 or later.

  • Execute your tests with Maven or Gradle

1.2. Testerra Skeleton project

We provide a skeleton project to demonstrate the basic features of Testerra.

1.3. Testerra manual setup

1.3.1. Setup

Testerra and all its components are deployed to MavenCentral: https://mvnrepository.com/artifact/io.testerra

For Testerra you need at least the following dependencies.

Gradle
// build.gradle

apply plugin: 'java-library'

// Its highly recommended to normalize your project to Unicode
compileJava.options.encoding = 'UTF-8'
compileTestJava.options.encoding = "UTF-8"

repositories {
    mavenCentral()
}

dependencies {
    implementation 'io.testerra:driver-ui-desktop:2.11'
    implementation 'io.testerra:report-ng:2.11'
}
Maven
<!-- pom.xml -->
<project>
    <properties>
        <maven.compiler.source>11</maven.compiler.source>
        <maven.compiler.target>11</maven.compiler.target>
        <!-- It's highly recommended to normalize your project to Unicode -->
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>
    <dependencies>
        <dependency>
            <groupId>io.testerra</groupId>
            <artifactId>driver-ui-desktop</artifactId>
            <version>2.11</version>
        </dependency>

        <dependency>
            <groupId>io.testerra</groupId>
            <artifactId>report-ng</artifactId>
            <version>2.11</version>
        </dependency>

        <!-- These dependency are required to get logging to work in Maven -->
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-slf4j18-impl</artifactId>
            <version>2.16.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>2.16.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.16.0</version>
        </dependency>

    </dependencies>
</project>

1.3.2. Create project structure

Your project structure should comply with these simple constraints.

  • src/main Contains all the code for your project like PageObjects, Models and language specific resources.

  • src/test Contains all test related code like your Tests, Test-Suites, Test-Data and Testerra related setup.

1.3.3. Create test.properties

Create a new file at src/test/resources with the name test.properties.

test.properties
# Setting the browser
tt.browser.setting=chrome

# ... or with version
tt.browser.setting=chrome:120

# Setting the start page
tt.baseurl=http://example.org
All defined properties can be overwritten later by adding system parameters to your command.
(e.a -Dtt.browser.setting=firefox)

All supported browsers are listed in WebdriverManager properties

The tests will be started using the browser that is installed on the local machine if you do not provide a version in the properties. If you want to use a specific browser version that is not installed, Selenium Manager will automatically download the required version of Firefox or Chrome in the background.

For further information about the Selenium Manager, you can read the corresponding section of the Selenium documentation.

1.3.4. Create Page Class

Now it’s time to create a first simple page class. It should be saved at path src\main\java\<package>. The following example represents the website example.org. It contains one possible link to click and one method to test.

New page class
import eu.tsystems.mms.tic.testframework.pageobjects.Page;
import eu.tsystems.mms.tic.testframework.pageobjects.UiElement;

public class ExamplePage extends Page {

    @Check
    private UiElement moreInformationLink =
        find(By.partialLinkText("More information"));

    public ExamplePage(WebDriver driver) {
        super(driver);
    }

    public void clickOnMoreInformation() {
        moreInformationLink.click();
    }
}

The basic Page class added all the page object functionality of Testerra to your project. See PageObjects chapter for more details.

The UiElement describes the elements like links, buttons, etc. on your page. Learn more about UiElements in UiElements.

1.3.5. Create Test Class and Test Method

The easiest way to create a new test, is by creating a new class in the path of src\test\java\<package> and let it extend from TesterraTest.

If you already have test classes that extend, you can add the TesterraListener manually. Both ways do basically the same. To stick to the example above, here is a very simple test class which navigates to example.org and clicks on the link defined on the example page. Again, probably imports must be made in IDE.

TesterraTest
import eu.tsystems.mms.tic.testframework.testing.TesterraTest;
import eu.tsystems.mms.tic.testframework.testing.PageFactoryProvider;

public class ExampleTest extends TesterraTest implements PageFactoryProvider {

    @Test
    public void testT01_My_first_test() {
        ExamplePage examplePage = PAGE_FACTORY.createPage(ExamplePage.class);
        examplePage.clickOnMoreInformation();
    }
}

Be aware of using @Test annotation at your test method. You have to use the TestNG annotation, not from JUnit.

If you import JUnit lib, no test is executed via Maven or Gradle.

TesterraListener
import eu.tsystems.mms.tic.testframework.report.TesterraListener;
import org.testng.annotations.Listeners;

@Listeners(TesterraListener.class)
public class ExampleTest {
}

1.3.6. Setup Selenium (optional)

However, if you want to set up a selenium driver by yourself and if you don’t have a remote selenium yet, you can easily install it by the package manager of your choice.

chocolately for Windows
choco install selenium selenium-chrome-driver
Ubuntu/Debian
apt-get install chromium-chromedriver
homebrew for Mac
brew install selenium-server-standalone chromedriver

Read here, if you want to set up another Selenium configuration.

1.3.7. Setup a test suite

To customize the executing of your tests, you have to create a TestNG suite file suite.xml and locator it at src/test/resources

suite.xml
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="Suite1" verbose="1" thread-count="10" configfailurepolicy="continue" parallel="false">
    <test name="Test1" parallel="methods">
        <classes>
            <class name="ExampleTest"/>
        </classes>
    </test>
</suite>
If configfailurepolicy is not set to "continue" and a configuration method fails, the test methods will be skipped.

1.3.8. Setup test build target

In order to get tests to work, you need to set up a build target test in your project.

Gradle
// build.gradle
test {
    useTestNG() {
        suites file('src/test/resources/suite.xml')
    }

    testLogging {
        outputs.upToDateWhen { false }
        showStandardStreams = true
    }

    // Important: Forward all JVM properties like proxy settings to TestNG
    options {
        systemProperties(System.getProperties())
    }

    // basically execution returns "GREEN" (framework exits with exit code > 0 if there were failures)
    ignoreFailures = true
}
Maven
<!-- pom.xml -->
<project>
    <build>
        <plugins>
            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <configuration>
                    <skip>true</skip>
                    <testFailureIgnore>true</testFailureIgnore>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <profiles>
        <profile>
            <id>mySuite</id>
            <activation>
                <activeByDefault>true</activeByDefault>
            </activation>
            <build>
                <plugins>
                    <plugin>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <configuration>
                            <skip>false</skip>
                            <suiteXmlFiles>
                                <suiteXmlFile>src/test/resources/suite.xml</suiteXmlFile>
                            </suiteXmlFiles>
                        </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>
</project>

1.3.9. Run the tests

Finally, you are good to run your very first test by entering the following command:

Gradle
gradle test
Maven
mvn test

1.4. Using a proxy

There are three ways for setting up a proxy for your test run environment.

  • System proxy settings for the build environment (Maven, Gradle), TestNG, JVM and Selenium

  • Browser proxy settings for the SUT, which is done by capabilities as described here Proxy setup

To setup a proxy for the whole system, including the build environment (Maven, Gradle), the JVM and Testerra, the recommended way is to pass it by command line arguments like

gradle test -Dhttps.proxyHost=your-proxy-host.com -Dhttps.proxyPort=8080

1.4.2. Property file

You can also put your proxy settings to the system Property files with the following content

Example of system.properties
https.proxyHost=your-proxy-host.com
https.proxyPort=8080
https.proxyUser=
https.proxyPassword=
https.nonProxyHosts=localhost|192.168.0.1

http.proxyHost=your-proxy-host.com
http.proxyPort=8080
http.proxyUser=
http.proxyPassword=

1.4.3. Access the system proxy URL

The system proxy can be accessed by Proxy Utilities

Since Java 11, it is possible to pass the system’s preconfigured proxy into the JVM.

gradle test -Djava.net.useSystemProxies=true

This affects all Java internal network connections which uses ProxySelector, but it will not set the environment variables and are transparent to Proxy Utilities and any Browser capabilities.

1.5. Logging

The log configuration prints out to System.out by default. If you want to have more control over several log levels of classes, add a log4j2.xml to your resources/.

debug level in log4j2.xml
<?xml version="1.0" encoding="UTF-8" ?>
<Configuration packages="eu.tsystems.mms.tic.testframework.logging">
    <Appenders>
        <Console name="CONSOLE">
            <!--
                The marker %contextIds gets replaced by internal plugins registered from
                plugins packages in the <configuration> node
            -->
            <PatternLayout pattern="%d{dd.MM.yyyy HH:mm:ss.SSS} [%t][%p]%contextIds: %c{2} - %m{nolookups}%n" />
        </Console>
    </Appenders>
    <Loggers>
        <Logger name="org.asynchttpclient" level="info" additivity="false">
            <AppenderRef ref="CONSOLE"/>
        </Logger>
        <Root level="debug">
            <AppenderRef ref="CONSOLE"/>
        </Root>
    </Loggers>
</Configuration>

You can also change the root log level from the command line via.

-Dlog4j.level=DEBUG
The shown log4j2.xml sets the INFO level for the package org.asynchttpclient. Otherwise Selenium 4 spams the logs with request/response information.

1.5.1. Log own messages

The Loggable interface provide some logging features and always uses the current instance class as logger name.

class MyClass implements Loggable {

    public void doSomething() {
        log().info("Do something");
    }
}

Which results in a log message similar to

[main][INFO][MCID:xyz][SCID:abc]: MyClass - Do something

The markers MCID and SCID are referencing to the current MethodContext respectively SessionContext ID.

Important messages can be prompted to the Report. See here for more details.

Testerra Framework

2. WebDriverManager

2.1. Overview

The WEB_DRIVER_MANAGER is the central component to create and close your WebDriver sessions and becomes available by implementing the WebDriverManagerProvider interface. It uses the standard Selenium Webdriver, but it is easier to configure.

2.2. WebDriver sessions

Before you can use WebDriver sessions, you have to set up a Selenium-compatible server.

2.2.1. Setup remote sessions

For using a remote Selenium server (e.g. a Selenium Grid) you only have to tell Testerra where it can be found.

Additional settings in test.properties
tt.selenium.server.url=http://localhost:4444/wd/hub
The browser support depends on the remote Selenium setup.
Manually configure WebDriver binaries
  • Download your WebDriver binary from browser vendor’s website to a local location

  • Make sure the driver version supports your installed browser

  • Since the properties are system properties, you need to put the location of the binaries to the system.properties file as mentioned in the Property files section.

system.properties
webdriver.gecko.driver="C:\\absolute\\path\to\\your\\geckodriver.exe"

# or for other browsers
webdriver.chrome.driver=...
webdriver.edge.driver=...
webdriver.ie.driver=...

You can also pass the WebDriver by the command line using

-Dwebdriver.gecko.driver=C:\absolute\path\to\your\geckodriver.exe

2.2.2. Define browser type and version

Before starting a WebDriver session, you should configure your desired browser like.

# Only browser type
tt.browser.setting=firefox

# ... or with version
tt.browser.setting=firefox:65

You can also define browser config via the settings tt.browser and tt.browser.version, but the version is independent of browser type.

If you have different browser configurations in your Selenium grid you have to take care about the correct combination!

2.2.3. Usage of WebDriver sessions

On the first call of

WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();

Selenium is triggered to open a new Browser windows with the defined URL.

Use remote Selenium server as often as possible, also for development. So your project is independent of any WebDriver configuration and needed Webdriver binary files.

For every other call of getWebDriver() in the same test context WebDriverManager always returns the existing session.

This makes it possible to retrieve the current session in any context and avoids to force the user to pass the instance around.

2.2.4. WebDriver lifecycle

The default behaviour of Testerra’s WebDriverManager is, to create unique WebDrivers for each thread and/or test method. That prevents issues in mutual interference between multiple threads.

2.2.5. Use multiple sessions

The WebDriverManager can handle more than one session in one test context. Every session has a defined session key. If no key was set, the default session key is called default.

The following example creates two independent browser sessions:

WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
WebDriver driverWindow2 = WEB_DRIVER_MANAGER.getWebDriver("window2");

// Get the session key
String key1 = WEB_DRIVER_MANAGER.getSessionKey(driver);        // key1 contains 'default'

String key2 = WEB_DRIVER_MANAGER.getSessionKey(driverWindow2); // key2 contains 'window2'

2.2.6. Close a session

In most cases it is not needed to close your session manually. Testerra always closes all open session created in the thread at the end of a test method.

Anyway, to close active sessions manually, do the following:

// Shutdown a session by key
WEB_DRIVER_MANAGER.shutdownSession(String);

// Shutdown an explicit driver
WEB_DRIVER_MANAGER.shutdownSession(WebDriver);

// Close all active session in the current test context.
WEB_DRIVER_MANAGER.shutdownAllThreadSessions();

// Close all active session in all current parallel test threads.
WEB_DRIVER_MANAGER.shutdownAllSessions()

Please do not use Selenium provided methods to close WebDriver sessions. Have a further read in our known issue section: Close WebDriver sessions without WebDriverManager.

2.2.7. Shared sessions in one thread

You can reuse the WebDriver session over multiple methods in the same thread by setting the following property:

test.properties
tt.wdm.closewindows.aftertestmethods=false

In this case, the sessions will not be closed until the very end of the execution.

Working with shared sessions
@Test
public void test1() {
    WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
}

@Test
public void test2() {
    // You get the already opened session from test1
    WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
}

You can implement that behaviour in a slightly different way, by setting up your sessions in @Before methods and close them in @After methods, which gives you more control.

Setting up and closing sessions
@BeforeMethod
public void setupBrowser() {
    DesktopWebDriverRequest request = new DesktopWebDriverRequest();
    request.setShutdownAfterTest(false);
    WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver(request);
}

@Test
public void test() {
    WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
}

@AfterMethod(alwaysRun = true)
public void shutdownBrowser() {
    WEB_DRIVER_MANAGER.shutdownAllThreadSessions();
}
You can only reuse the session in the current thread. If you run parallel tests, you have no access to the session between parallel test threads.
resuse webdriver sessions 1
Figure 1. Limitations of reusing shared sessions
Special behaviour for config methods (deprecated)

When you create a WebDriver in a setup method annotated with @Before…​ or @After…​, the session will not be closed after that method, even when tt.wdm.closewindows.aftertestmethods is true.

Reuse sessions in setup methods
@BeforeMethod
public void setupWebDriver() {
    // This WebDriver will not be closed, because it's a setup method
    WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
}

@Test
public void test1() {
    // You get the already opened session
    WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
}
This implicit behaviour is @deprecated and will be removed in future versions.

2.2.8. Shared sessions over different threads

To use a WebDriver session over different test threads, you need an exclusive session.

Exclusive sessions are identified by a special uuid, not by the standard session key.

Create exclusive browser sessions
private static String uuid = null;

@Test
public void test1() {
    WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
    uuid = WEB_DRIVER_MANAGER.makeExclusive(driver);
}

@Test
public void test2() {
    // Get the exclusive session
    WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver(uuid);
}
resuse webdriver sessions 2
Figure 2. Reuse a session in different test threads

An exclusive session has an unlimited lifetime. You need do close this session manually.

@AfterTest
public void cleanup() {
    WEB_DRIVER_MANAGER.shutdownSession(uuid);
}

2.3. WebDriver configuration

A global configuration applies to all new sessions created by WebDriverManager. You can set a global configuration by.

2.3.1. Property configuration

The main configuration your WebDriver session is done in test.properties.

The most important properties are:

  • tt.browser.setting (or tt.browser, tt.browser.version and tt.browser.platform)

  • tt.baseurl

  • tt.selenium.server.url (or tt.selenium.server.host and tt.selenium.server.port)

The complete list of WebDriver properties can be found at WebdriverManager properties.

2.3.2. Request configuration

If you only want to change the settings for one session, you can use WebDriverRequest. All defined attributes overrides the standard configuration.

If an attribute is not set, the global definition is used.
DesktopWebDriverRequest myRequest = new DesktopWebDriverRequest();
myRequest.setBaseUrl("http://example.org");
myRequest.setBrowser(Browsers.firefox);
myRequest.setBrowserVersion("66");
myRequest.setSessionKey("mysession");
myRequest.setWindowSize(new Dimension(2560,1440));  // tt.window.size
myRequest.setShutdownAfterTest(false);              // applies to all test method results
                                                    // tt.wdm.closewindows.aftertestmethods
myRequest.setShutdownAfterTestFailed(false);        // applies only to failed test methods
                                                    // tt.wdm.closewindows.onfailure
myRequest.setMaximizeBrowser(true);                 // tt.browser.maximize

WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver(myRequest);

Via the SessionContext you can get the current configuration of a WebDriver session:

Optional<SessionContext> sessionContext = WEB_DRIVER_MANAGER.getSessionContext(webDriver);
// In case of a desktop browser you can cast to 'DesktopWebDriverRequest'
// Be careful if you are using other session types like Appium sessions!
DesktopWebDriverRequest webDriverRequest
    = (DesktopWebDriverRequest) sessionContext.get().getWebDriverRequest();
With DesktopWebDriverRequest you can also define Browser capabilities.

2.3.3. Configure with WebDriverManagerConfig (@deprecated)

Do not use anymore set methods of WebDriverManagerConfig or WEB_DRIVER_MANAGER.getConfig().

2.4. Working with sessions

2.4.1. Get current session

Get the current session context with information about your WebDriver session.
WebDriver driver = WebDriverManager.getWebDriver();
Optional<SessionContext> sessionContext = WEB_DRIVER_MANAGER.getSessionContext(driver);

2.4.2. Switching windows or tabs

In some cases you need to switch to a new browser window or tab which was opened by the application.

// Switch to a window by matching title with 'equals()'
WEB_DRIVER_MANAGER.switchToWindowTitle(WebDriver webDriver, String windowTitle);

// Switch to a given window handle
WEB_DRIVER_MANAGER.switchToWindowHandle(WebDriver webDriver, String windowHandle);

// Switch to a window by a custom condition
boolean switched = WEB_DRIVER_MANAGER
        .switchToWindow(WebDriver webDriver, webDriver -> webDriver.getCurrentUrl().contains("login"));

Assert.assertTrue(switched, "Login window not found");

You can also handle a delayed switch of a window by using CONTROL.

int secondsForRetry = 6;
CONTROL.retryFor(secondsForRetry, () -> {
    WEB_DRIVER_MANAGER.switchToWindowTitle(webDriver, "MyWindowTitle");
});

// Using a predicate you have to check the result of switching
CONTROL.retryFor(secondsForRetry, () -> {
    boolean result = WEB_DRIVER_MANAGER
            .switchToWindow(webDriver, driver -> driver.getTitle().contains("MyWindowTitle"));
    Assert.assertTrue(result);
});

3. Browser capabilities

You can customize your WebDriver session by setting capabilities in the following ways:

When creating a new WebDriver, these capabilities get merged together in this exact order.

3.1. User agent configuration

The user agent configuration is most precise, because it provides explicit browser options based on the Selenium driver.

The setting applies to all created sessions for a browser type (global).

import eu.tsystems.mms.tic.testframework.useragents.FirefoxConfig;

WEB_DRIVER_MANAGER.setUserAgentConfig(Browsers.firefox, (FirefoxConfig) options -> {
    options.addPreference("intl.accept_languages", "de-DE");
});

Non-standard capabilities need a vendor prefix and can be set as follows:

Map<String, Object> customCaps = new HashMap<>();
customCaps.put("foo", "bar");

WEB_DRIVER_MANAGER.setUserAgentConfig(Browsers.firefox, (FirefoxConfig) options -> {
    options.setCapability("custom:caps", customCaps);
});

3.2. Request capabilities

Some WebDriverRequests support setting capabilities, like the DesktopWebDriverRequest. It’s used to specify a single WebDriver session.

Set capabilities to a DesktopWebDriverRequest object
DesktopWebDriverRequest request = new DesktopWebDriverRequest();
MutableCapabilities caps = request.getMutableCapabilities();
caps.setCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true);

// Start your session with the DesktopWebDriverRequest object
WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver(request);

3.3. Global capabilities with WEB_DRIVER_MANAGER (@deprecated)

This feature is deprecated, please use User agent configuration

You can customize your browser session by setting capabilities for every browser type.

Be in mind that not every browser could handle all types of capabilities.

WEB_DRIVER_MANAGER.setGlobalCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true);
WEB_DRIVER_MANAGER.removeGlobalCapability(CapabilityType.ACCEPT_INSECURE_CERTS);

Do NOT set browser capabilities with WebDriverManager. This will added to the capabilities to all browser types!

FirefoxOptions options = new FirefoxOptions();
options.addPreference("intl.accept_languages", "de-DE");
// This cannot be merged correctly!
WEB_DRIVER_MANAGER.setGlobalCapability(FirefoxOptions.FIREFOX_OPTIONS, options);

3.4. Proxy setup

If you want that the browser uses a proxy for the SUT, you can just configure that by default Selenium capabilities.

Make sure that your WebDriver supports the Proxy-Capability. For example the MicrosoftWebDriver for Legacy Edge does not support proxy setup (see Edge WebDriver Capabilities).
If you want to setup a proxy for the runtime environment but the browser, you have to follow the instructions at Using a proxy

The following code setups a proxy based on the System’s proxy configuration and a custom proxy.

import org.testng.annotations.BeforeSuite;
import eu.tsystems.mms.tic.testframework.webdrivermanager.WebDriverManagerUtils;
import eu.tsystems.mms.tic.testframework.webdrivermanager.WebDriverProxyUtils;
import org.openqa.selenium.Proxy;

public abstract class AbstractTest extends TesterraTest {

    @BeforeSuite
    public void proxySetup() {
        WebDriverProxyUtils utils = new WebDriverProxyUtils();

        Proxy otherProxy = utils.createHttpProxyFromUrl(
            new URL("http://proxyUser:secretPassword@my-proxy:3128")
        );
        WEB_DRIVER_MANAGER.setUserAgentConfig(Browsers.chrome, (ChromeConfig) options -> {
            options.setProxy(otherProxy);
        });
    }
}
WebDriverProxyUtils.getDefaultHttpProxy() only returns the proxy configuration for HTTP, HTTPS and non-proxy connections.

4. UiElements

4.1. Overview

UiElements are representations of elements of the tested website, like buttons, search fields, checkboxes or even just DOM elements.

UiElements are not, but based on the Selenium WebElement and add more functionality to them. Since a UiElement is just a pointer to a locator, it’s using the same definition as WebElements By (Selenium docs).

UiElements are self refreshing: Every action on it will trigger a find call, so the current state is always up to date when the requested action takes place. There is de facto no StaleElementReferenceException on UiElements like it could be when using vanilla WebElements.

4.2. Creation

4.2.1. Create default UiElement

For every UiElement you need an implementation of the UiElementFinder interface (like PageObjects) and a locator (Selenium docs).

UiElement myElement = find(By.id("elementId"));
UiElement button1 = find(By.name("button"));
UiElement textOutputField = find(By.xpath("//p[@id='99']"));
A GuiElement always points to the first element found by the given locator. Even when your locator would return multiple elements, it just represents one. You can make your locators to force uniqueness or use element lists.

4.2.2. Create SubElements

The UiElement also implements UiElementFinder. Elements will only be searched in the DOM tree below the given parent element on which the method was called.

UiElement upper = find(By.name("upperElement"));

// Create the sub elements
UiElement lower = upper.find(By.name("lowerElement"));
UiElement lower = upper.find(By.xpath(".//p[@id='element']")); (1)
UiElement lower = upper.find(By.xpath("//p[@id='element']")); (2)
UiElement lower = upper.find(By.xpath("./p[@id='element']")); (3)
1 Find any matching descendant
2 Corrects the selector prefix to './/'
3 Find any matching child

4.2.3. Implement UiElementFinder

When you don’t want to use the PageObject pattern, you can instantiate an UiElementFinder on your own.

import eu.tsystems.mms.tic.testframework.pageobjects.UiElementFinder;
import eu.tsystems.mms.tic.testframework.testing.TesterraTest;
import eu.tsystems.mms.tic.testframework.testing.UiElementFinderFactoryProvider;
import eu.tsystems.mms.tic.testframework.testing.WebDriverManagerProvider;

class MyTest extends TesterraTest implements
    UiElementFinderFactoryProvider,
    WebDriverManagerProvider
{
    protected UiElementFinder createFinder() {
        return UI_ELEMENT_FINDER_FACTORY.create(WEB_DRIVER_MANAGER.getWebDriver());
    }

    @Test
    public void test() {
        UiElementFinder finder = createFinder();
        UiElement myElement = finder.find(By.id("elementId"));
        UiElement button1 = finder.find(By.name("button"));
        UiElement textOutputField = finder.find(By.xpath("//p[@id='99']"));
    }
}

4.2.4. Advanced UiElement locating

The UiElementFinder extends the LocatorFactoryProvider interface, which provides a LOCATE factory that supports more features than standard Selenium By.

Locate unique elements

If you want to make sure, that your element is unique.

UiElement myElement = find(LOCATE.by(By.id("elementId")).unique());

This will throw an exception, if not exact one WebElement has been found.

Locate displayed items only
UiElement myElement = find(LOCATE.by(By.xpath(".//button")).displayed());
Prepared xpath expressions

Using prepared expressions makes complex selectors more readable.

PreparedLocator byText = LOCATE.prepare("//button[text()='%s'])");
PreparedLocator byClass = LOCATE.prepare("//%s[contains(@class, '%s')][%d]");

UiElement loginButton = find(byText.with("Login"));
UiElement logoutButton = find(byClass.with("button", "btn-logout", 1));
Filtering elements

You can also filter elements during find.

Locator byText = LOCATE.by(By.xpath("//button"))
                    .filter(webElement -> webElement.getText().equals("Open again"));

UiElement buttonContainsText = find(byText);
Default locator configurator

When you want to preconfigure all locators in the current thread, you can use

LOCATE.setThreadLocalConfigurator(locator -> {
   // Configure your locator here
});
GuiElements inside frames

4.2.5. UiElements inside frames

Accessing WebElements inside frames requires changing the active frame before any action. UiElements do that automatically as long you identify every <frame> or <iframe> as own element.

UiElement frame1 = find(By.id("frame1"));

// frame2 is child of frame1
UiElement frame2 = frame1.find(By.id("frame2"));

// target is child of frame2 which is child of frame1,
UiElement target = frame2.find(By.id("target"));

And easier way is to use the findDeep method, which search for the element recursive in the frame hierarchy every time.

// in this case the frames are searched recurse automatically
UiElement target = findDeep(By.id("target"));

4.2.6. Element lists

The UiElement is like a pointer to the first element of the given locator. But if you want to retrieve a list of all found elements by the given locator, you can use the UiElementList like the following.

<div>First</div>
<div>Second</div>
<div>Third</div>
UiElement div = find(By.tagName("div"));
div.list().first().expect().text("First");
div.list().last().expect().text("Third");

They can also be iterated and streamed.

div.list().forEach(uiElement -> {});
div.list().stream().forEach(uiElement -> {});

4.2.7. Empty elements

To prevent null pointers or any other exception that will break you program flow, you can use and empty UiElement using createEmpty() of UiElementFinder.

UiElement empty = createEmpty(Locator);

All interactive operations on this element will do nothing, all wait methods will be false and all assertions will fail.

4.2.8. Sensitive Data

Sensitive data, such as passwords, can be displayed obfuscated in the logs during the actions type and sendKeys.

UiElement sensitiveElement = findById("secret").sensitiveData();

Only the placeholder * is logged in the report instead of the real value.

4.2.9. Trace elements hierarchy

In most cases, elements are part of a view hierarchy. The Nameable interface provides some methods to retrieve this information.

Nameable parent = element.getParent();

A parent could be any PageObject like UiElement, Component or Page.

Be aware that getParent() could return NULL, when the element has been created without a hierarchy or the element is a Page. So you should always perform a null or instanceof check.

If you want to trace the hierarchy beginning from top-down, you can use the traceAncestors() method.

element.traceAncestors(ancestor -> true);

When the given Predicate return FALSE, the tracing will stop.

This will not supply the calling element.

4.3. XPath builder

The static XPath class helps you to build failsafe xPathes optimized for HTML. But it’s restricted to search elements top-down.

This is what the basic syntax looks like

UiElement div = find(XPath.from("div"));

But it supports many other features you need when you select elements from the DOM, like first and last element.

XPath.from("tr", 1);
XPath.from("tr", -1);

The xPath is generated greedy by default. When you create an xPath like

XPath.from("body").select("div");

this equivalent result would be //body//div

If you want to restrict the selection to a child element, use the / prefix:

XPath.from("/body").select("/div");

which equivalent would be /body/div

It is also possible to pass groups in the from method only:

XPath.from("(//iframe|//frame)");

Or use functions:

XPath.from("*").attribute("local-name()", "svg");

4.3.1. Elements that have classes

XPath.from("div").classes("navigation", "header");

This will find elements like

<div class="header large navigation">

but not

<div class="navigation-header">

4.3.2. Select an element that encloses another element

XPath.from("nav")
    .classes("mobile")
    .enclose("/div")
        .classes("navigation", "header");

This will find the <nav> element

<nav class="mobile">
    <div class="navigation header"></div>
</nav>

4.3.3. Select an element by its text

XPath.from("*").text().hasWords("Login", "here");
XPath.from("*").text().contains("first");
XPath.from("*").text().endsWith("here ");

This will find elements like

<a> Login first
    here </a>

4.3.4. Select by attributes

XPath.from("*").attribute("src").endsWith(".png");

This will find elements like

<img src="http://example.com/image.png"/>

4.3.5. Select a sub element

XPath.from("form")
    .attribute("name", "login")
    .select("button")
        .text().hasWords("Login here");

This will find the <button> element

<form name="login">
    <button> Login here * </button>
</form>

4.4. Assertions

UiElements provide many kinds of assertion methods to verify your elements.

If an assertion fails, it will make the whole test fail and abort. You can control that by using a TestController

4.4.1. UiElement checks

Checks are assertions which verify the condition of an UiElement.

Checks if the element is present in the DOM

element.expect().present(boolean);

Checks if the element is present in the Viewport, if it is visible by it’s display and visibility style properties and if it’s width and height are both greater than 0.

element.expect().displayed(boolean);

Checks if the element is displayed and if it’s partially or fully visible in the scroll area of the viewport.

element.expect().visiblePartial(boolean);
element.expect().visibleFull(boolean);
It doesn’t relate to opacity or z-index style properties. If you need to test the perceptually visibility to the human eye, you should consider to implement an image based Layout Check.

Some more WebElement checks

// The following methods are calling the standard webelement method
element.expect().enabled(boolean);
element.expect().selected(boolean);

// Tries to find out if an element could be selected.
element.expect().selectable(boolean);

4.4.2. Assertions of UiElement attributes

UiElement element = find(By.id("button"));

// Expect the element text
element.expect().text("Hello World");
element.expect().text().contains("World").is(boolean);
element.expect().text().isContaining("World");  // Short form
element.expect().text().map(String::trim).startsWith("Hello").is(boolean);

// Expect the existence of an attribute
element.expect().attribute(Attribute.DISABLED).isNot(null);

// Expect the value of an input element matches an regular expression
element.expect().value().matches("^hello\\s.orld").is(boolean);

// CSS property checks
element.expect().css("display").is("none");

// CSS class checks
element.expect().hasClasses("active", "button").is(boolean);

// Visibility checks
element.expect().visiblePartial(boolean);
element.expect().visibleFull(boolean);

4.4.3. Layout assertions

UiElement can be checked for their relative layouts.

UiElement left = find(By.id("left"));
UiElement right = find(By.id("right"));

left.expect().bounds().leftOf(right).is(true);
GuiElement layout comperator
Figure 3. A simple example for a layout check
UiElement image1 = find(By.xpath("//..."));
UiElement image2 = find(By.xpath("//..."));
UiElement image3 = find(By.xpath("//..."));

// Assertions are true
image1.expect().bounds().leftOf(image2).is(true);
image1.expect().bounds().fromTop().toTopOf(image2).is(0);
image1.expect().bounds().fromBottom().toBottomOf(image3).is(0);

// Assertions are false
image1.expect().bounds().fromBottom().toBottomOf(image2).is(0);

4.4.4. Custom assertion message

Testerra generates a readable assertion message with name of UiElement, page and checked attribute.

Example of a page and a test method
public class DragAndDropPage extends Page {
    ...
    private UiElement columnA = find(By.id("column-a"));

    public TestableUiElement getColumnA() {
        return this.columnA;
    }
}
...
public MyTest extends TesterraTest implements PageFactoryProvider {
    @Test
    public void myTest() {
        DragAndDropPage page = PAGE_FACTORY.createPage(DragAndDropPage.class);
        page.getColumnA().expect().displayed().is(false); (1)
        page.getColumnB().expect().displayed().is(false, "Foo"); (2)
    }
}
1 Error message: UiElementAssertionError: Expected that DragAndDropPage → columnB displayed is false
2 Error message: UiElementAssertionError: Expected that Foo is false

4.5. Actions

An UiElement provides a variety of action methods. Beside the known WebElement methods there are some more useful methods to interact with the web site.

4.5.1. Click on elements

UiElement element = find(By.id("button"));

element.click();
element.doubleClick();
element.contextClick();
If you have troubles using these methods, take a look to the fallback solution Desktop WebDriver utilities.

4.5.2. Enter text

UiElement element = find(By.id("input"));

// Enters the given text in a input or textfield.
// Any old values are automatically deleted before input.
// The type method has a value check. If the given string is NULL or empty, the method does nothing.
element.type("my text");

// The standard Selenium method is used.
// You can also use the Selenim Keys class to enter special keys.
element.sendKeys("my text");
element.sendKeys("my text" + Keys.ENTER);

// Delete the content of an input field.
element.clear();

4.5.3. Use select boxes

UiElement element = find(By.id("select"));

// Get the Select WebElement of a UiElement
element.findWebElement(webElement -> {
    Select select = new Select(webElement);

    // You can use all Selenium Select methods to interact.
    select.selectByIndex(2);
    select.selectByVisibleText("option");
    List<WebElements> list = select.getAllSelectedOptions();
});

4.5.4. Use check boxes

UiElement element = find(By.id("checkbox"));

// Check and uncheck the check box
element.select();
element.deselect();

// true = check, false = uncheck
element.select(boolean);

4.5.5. Scrolling

You can scroll the browser viewport until the element is in the middle viewport if possible.

element.scrollIntoView();

// Lets offset pixel distance from the top of the viewport
element.scrollIntoView(new Point(0, -20))

4.5.6. Mouse over

You can simulate the mouse pointer is moved over an element.

element.hover();
If you have troubles using this method, take a look to the fallback solution Desktop WebDriver utilities.

4.5.7. Drag and drop actions

With the utils class MouseActions you can execute a drag-and-drop actions. Source and target UiElements can be located in different frames.

UiElement source = find(By.id("elem1"));
UiElement target = find(By.id("elem2"));

MouseActions.dragAndDropJS(source, target);

// You can add one or more DragAndDropActions
MouseActions.dragAndDropJS(source, target, DragAndDropOption.CLICK_AFTER_RELEASE);

// This method provides a swipe of an element to a relative position from the element.
int offsetX = 50;   // Pixel
int offsetY = 125;  // Pixel
MouseActions.swipeElement(source, offsetX, offsetY);

4.5.8. Highlight elements

This method draws a coloured frame around the UiElement.

element.highlight();

// Or by a given color
element.highlight(new Color(int, int, int));

4.6. Waiters

In testing practice the test automation code must tolerate delays caused e.g. by page loading or javascript activities when checking conditions on UiElements.

If the condition (which is checked continuously) is met within the timeout then the wait methods return true.

Otherwise, after the timeout has passed they return false without any further action or assertion.

boolean result;
result = element.waitFor().displayed(boolean);

// Overrides preconfigured internal timeout
result = element.waitFor(int seconds).displayed(boolean);

4.7. Properties

4.7.1. Use the getActual()

The values of most properties of a UiElement can be returned by the getActual() method.

The given HTML snippet
<a href="newpage.html" class="foolink" style="font-size: 20px;">My link</a>
UiElement element = find(By.xpath("//a"));

String text = element.waitFor().text().getActual();             // returns "My link"
String href = element.waitFor().attribute("href").getActual();  // returns "newpage.html"
String tag = element.waitFor().tagName().getActual();           // returns "a"
String classes = element.waitFor().classes().getActual();       // returns "foolink"
String css = element.waitFor().css("font-size").getActual();    // returns "20px"

4.7.2. Access the WebElement

UiElement provides all Selenium methods when retrieving the WebElement. You have access to some more properties.

The given HTML snippet
<a href="newpage.html" style="font-size: 20px;">My link</a>
UiElement element = find(By.xpath("//a"));

element.findWebElement(webElement -> {
    Point point = webElement.getLocation();     // returns the top left corner of the element
    Dimension dim = webElement.getSize();       // returns width and heigth of the element
    Rectangle rect = webElement.getRect();      // returns rectangle with location and size
});

Testerra’s UiElement does not support a public method like element.getWebElement() as GuiElement did this. Testerra wants to save you from a Selenium StaleElementReferenceException.

We absolutely do not recommend to create you own Selenium WebElement by
driver.findElement(By…​). You fall back to plain Selenium and lose the Testerra error handling.

4.8. Internals

The find mechanism for a UiElement in Testerra works different as in plain Selenium. When using a constructor for a UiElement instantiation, Testerra internally will add some facades / decorators to make things easier. The most important decorator that is added by default is the GuiElementCoreSequenceDecorator -which adds a sequence to all method calls against a GuiElement.

Example: When calling the isPresent() method on a UiElement the added GuiElementSequenceDecorator will fire up an internal find() call to the GuiElement and therefore a find() call to the underlying Selenium WebElement. But instead of calling the find() method once, it will execute this call in a default sequence every 500ms.

Therefor the property tt.element.timeout.seconds defined in test.properties will be used as a hard timeout for this sequence. If the find() does not run successfully after the defined timeout it will fail.

5. PageObjects

5.1. Overview

5.1.1. What is a page object?

A page objects represents a HTML pages and or a subpage. It contains UiElements with describe the actual page and methods to provide actions on them.

In your test you only uses the provided actions of your page like an API. The page object himself uses the UiElements as an API to interact with the website.

We recommend the usage of page objects:

  • They are easy to maintain.

  • They improve the readability of test scripts.

  • They reduce or eliminate duplicity of code.

  • The pages are reusable.

5.1.2. Navigation Principle

In a regular Web Application there is a defined navigation flow. This means there are pages with actions on it that let you navigate to other pages.

In the example below we have a search dialog with a search action on it that lets you navigate to a ResultPage with the search result. When a search is performed the browser will navigate to the ResultPage. In your page you create a new object of your next page.

This new page object is used for the next steps in your test.

PageFlowExample
Figure 4. Example of a page flow

5.1.3. Example

The following page contains two UiElements and one method for a user action 'search a string'.

Within the method search the defined UiElements are used to execute a search.

The annotation Check marks the UiElements as mandatory for the page. Testerra automatically verifies these elements when this page is instantiated (Check Annotations).

public class SearchPage extends Page {

    @Check
    private final UiElement searchButton = find(By.name("searchButton"));

    @Check
    private final UiElement inputField = find(By.name("inputField"));

    // constructor
    public SearchPage(WebDriver driver) {
        super(driver);
    }

    // search action on page
    public ResultPage search(String text) {
        inputField.type(text);
        searchButton.click();
        return createPage(ResultPage.class);
    }
}

The following lines demonstrate how to use page objects in your test method.

public class TestClass extends TesterraTest implements PageFactoryProvider {

    @Test
    public void myTest() {
        HomePage homePage = PAGE_FACTORY.createPage(HomePage.class);
        SearchPage searchPage = homePage.openSearch();
        ResultPage resultPage = searchPage.search("search text");
        resultPage.assertResultSetIsNotEmpty();
        homePage = resultPage.close();
    }
}

The method PAGE_FACTORY.createPage() uses the default WebDriver session via WEB_DRIVER_MANAGER.getWebDriver() if no driver object is given.

This could mean that the usage of the PAGE_FACTORY creates a new browser session automatically (like in the given demo test case).

5.2. Instantiation

5.2.1. PageFactory

Pages need to be created initially via. PageFactory interface, which will be provided by the PageFactoryProvider as PAGE_FACTORY instance.

When the page is instantiated, Testerra automatically checks its annotated elements.

HomePage homePage = PAGE_FACTORY.createPage(HomePage.class);

// Or use an explicit WebDriver
HomePage homePage = PAGE_FACTORY.createPage(HomePage.class, WebDriver otherWebDriver);

5.2.2. Inline creating

Once you created a page, you can create other pages directly from this page, to keep track of the same WebDriver.

OtherPage otherPage = homePage.createPage(OtherPage.class);

5.2.3. Optional pages

You can also try to create a page, when you want to handle unexpected redirects at some point.

homePage.waitForPage(OtherPage.class).ifPresent(otherPage -> {});
homePage.waitForPage(OtherPage.class, int seconds).ifPresent(otherPage -> {});

5.2.4. Lifecycle

When a page has been created and all Check Annotations have been performed, the pageLoaded() method will be called, which can be used to perform some additional initializing without any test related interaction.

class MyPage extends Page {

    @Override
    protected void pageLoaded() {
        super.pageLoaded();

        // additional actions
    }
}

5.2.5. Page Prefixes (deprecated)

Using page prefixes is an uncommon feature and therefore marked as @deprecated

Page Prefixes can influence which concrete classes get instantiated by the PageFactory. They work together with a inheritance scheme of page classes. This can be useful if there is a base page which can come in different concrete variations. Example:

There is a BaseClass which inherits from the Page class and contains the basic functionality of a page. Then the Page can come in 2 different variations. We can represent this as Variation1BaseClass and Variation2BaseClass. They both inherit from BaseClass. Before instantiation, we can set the prefix using the PageFactory. Then we instantiate it and we can get our variation of the base class.

PAGE_FACTORY.setGlobalPagesPrefix("Variation1");
//this actualy creates a Variation1BaseClass
BaseClass baseClass = PAGE_FACTORY.createPage(BaseClass.class);

Default is no prefix.

Usage:

// Set a global Prefix
PAGE_FACTORY.setGlobalPagesPrefix("prefix");

// Set a thread local prefix. See next row about cleaning this prefix.
PAGE_FACTORY.setThreadLocalPagesPrefix("prefix");

// The thread local pages prefix is not cleared automatically,
// be sure to always set the correct one or clear it after using.
PAGE_FACTORY.removeThreadLocalPagePrefix();

5.3. Components

You can improve your PageObjects by using components. Components are like they are in actual web development environments: Containers with functionality. With components, you don’t need to try to create reusable PageObjects in a complex inheritance hierarchy, you can follow the pattern that composition before polymorphism.

In Testerra, components are hybrids of both UiElements and PageObjects. They can contain more UiElements and even Components, but they don’t provide features restricted to Pages or UiElements and their finder API is restricted to its root container element by default.

5.3.1. Create a component

The following HTML snippet is given:

..
<div id="container">
    <input type="text" id="mytext">
    <button id="button">Send</button>
</div>
..

The component should contain all elements of the div element.

import eu.tsystems.mms.tic.testframework.pageobjects.AbstractComponent;

public class MyComponent extends AbstractComponent<MyComponent> {

    @Check
    UiElement input = find(By.id("mytext"));
    @Check
    UiElement button = find(By.id("button"));

    public MyComponent(UiElement rootElement) {
        super(rootElement);
    }
}

To instantiate components, use the createComponent() method the same way as you create pages.

The second parameter is the root element of your component (here the div element).

public class MyPage extends Page {

    MyComponent component = createComponent(MyComponent.class, find(By.id("container")));

}

All UiElements of your component are always sub elements of the root element!

The button element of the example is found like find(By.id("container").find(By.id("button").

5.3.2. Component lists

Since components are hybrid UiElements, they can also act as lists.

UiElement table = find(By.tagName("table"));
TableRow rows = createComponent(TableRow.class, table.find(By.tagName("tr")));

rows.list().forEach(row -> {
    row.getNameColumn().text().contains("Hello").is(true);
});

5.3.3. Break out from components

Some website components have no unique root element because some elements were added dynamically to the end of the DOM at runtime.

In that case your UiElements are no subelements of your component’s root element. You can access outside elements as follows:

public class MyComponent extends AbstractComponent<MyComponent>
        implements UiElementFinderFactoryProvider {

    // This element is a subelement from component's root element
    private UiElement input = find(By.id("mytext"));

    // This element can be outside
    private UiElement button = this.getExternalElement(By.id("button"));

    public MyComponent(UiElement rootElement) {
        super(rootElement);
    }

    private UiElement getExternalElement(By by) {
        UiElementFinder uiElementFinder = UI_ELEMENT_FINDER_FACTORY.create(this.getWebDriver());
        UiElement uiElement = uiElementFinder.find(by);
        if (uiElement instanceof NameableChild) {
            ((NameableChild<?>) uiElement).setParent(this);
        }
        return uiElement;
    }

}

Setting the parent of the outside UiElement is optional, but in case of Assertion errors the error message is well formatted like Expected that <Page> → <Component> → <Element> …​

5.4. Check Annotations

The @Check annotation is used to verify the actual presence of an element on the site. All UiElements that are marked with the @Check annotation are automatically checked when instantiated by the PageFactory.

In the example, the first UiElement has the @Check annotation, the second doesn’t. The result is, that the presence of the first element will be checked by the constructor, the second won’t. If a checked element is not found, the constructor will throw a PageFactoryException.

@Check
private UiElement checked = find(By.name("checked"));
//no @Check here
private UiElement unchecked = find(By.name("unchecked"));

The @Check annotation will use the default CheckRule defined in test.properties. It is also possible to overwrite the default CheckRule for a single @Check annotation.

@Check(checkRule = CheckRule.IS_PRESENT)
private UiElement uiElement;

Change the check rules for the whole project (global) with the following:

test.properties
tt.guielement.checkrule=IS_PRESENT

Available CheckRules are

  • IS_DISPLAYED

  • IS_NOT_DISPLAYED

  • IS_PRESENT

  • IS_NOT_PRESENT

The default is IS_DISPLAYED.

With the optional attribute, the check only adds an optional assertion to the report. The test will not be interrupted at this position. See Optional assertions for more details.

@Check(optional = true)
private UiElement uiElement;

With the collect attribute you can collect all checks. The test will not be interrupted, but if failes at the end. See Collected assertions for more details.

@Check(collect = true)
private UiElement uiElement;
It does not make sense to set optional and collect at the same UiElement!
Because optional is also a kind of collect, the collect attribute will be ignored.

You can also define a special error message with the prioritizedErrorMessage attribute.

@Check(prioritizedErrorMessage = "My error message.")
private UiElement uiElement;

Use the timeout attribute to define a specific timeout only for that element to optimize check timeouts on Page instantiation. This overrides the Page timeout setting.

@Check(timeout = 60)
private UiElement uiElement;

5.5. Page loaded callback

You can add a custom action if a page was loaded successfully.

public class MyPage extends Page {

    ...

    @Override
    protected void pageLoaded() {
        super.pageLoaded();
        // Add here your custom action
    }
}

5.6. Timeout Setting

The checks are performed with a timeout. The default timeout is set by the tt.element.timeout.seconds property.

With the following annotation, the check can be optimized for all UiElements during page instantiation:

@PageOptions(elementTimeoutInSeconds = 60)
public class ExamplePage extends Page {
	// insert your code
}

5.7. Information hiding

The PageObject pattern encapsulates UiElements for all activities on the page. That’s why every element should be private accessible by the page only. But for testing purposes, it could be useful to allow access to some elements. In this case, you can create public methods and return the element casting to TestableUiElement.

public class MyPage extends Page {
    private UiElement saveButton = find(By.tagName("button"));

    public void performSave() {
        saveButton.click();
    }

    public TestableUiElement getSaveButton() {
        return saveButton;
    }
}

In the test, you can perform assertions without breaking the PageObject pattern.

@Test
public void testSaveFunctionality() {
    MyPage page = getPage();
    page.performSave();
    page.getSaveButton().expect().text("Saved");
}

Excecution and controlling

6. Regular Assertions

When implementing the AssertProvider interface, you get an instance named ASSERT, which provides more features than standard TestNG Assert

You can use this interface for assertions they cannot be covered by the UiElement.

import eu.tsystems.mms.tic.testframework.testing.AssertProvider;

public class MyTest extends TesterraTest implements AssertProvider {

    @Test
    public void test() {
        ASSERT.assertContains("Hello world", "planet");
    }
}

This will throw an AssertionError with the message Expected [Hello world] contains [planet].

You can also add some more detailed information to the subject.

ASSERT.assertContains("Hello world", "planet", "the greeting");

Which will result in Expected that the greeting [Hello world] contains [planet]

Please take a look into the Assertion interface for a full feature overview.

7. Test controlling

The TestControllerProvider interface provides the TestController instance CONTROL for controlling the test and assertion flow.

7.1. Collected assertions

Collecting assertion means, that a failing assertion will not abort the test method, but it will throw an assertion error at the end of the test method. So you have a chance to validate many more aspects in one test run.

CONTROL.collectAssertions(() -> {
    element.expect().text("Hello World");
    page.performLogin();
    ASSERT.assertEquals("hello", "world");
});
Assertions can only be collected by using Regular Assertions or expect()/assertThat() of UiElements.
CONTROL.collectAssertions(…​) does not work with org.testng.Assert.

See here how collected assertions are presented in Report-NG.

7.2. Optional assertions

Optional asserts do not let the test fail, but the assertion message will be added to the log with loglevel WARN and will result in an minor failure aspect.

CONTROL.optionalAssertions(() -> {
    element.expect().text("Hello World");
    page.performLogin();
    ASSERT.assertEquals("hello", "world");
});
Assertions can only be optional by using Regular Assertions or expect()/assertThat() of UiElements.
CONTROL.optionalAssertions(…​) does not work with org.testng.Assert.

See here how optional assertions are presented in Report-NG.

7.3. Change internal timeout

To change the timeout for internal assertions, you can override it for a specified block.

CONTROL.withTimeout(int seconds, () -> {
    element.expect().text("Hello World");
}

Please mind that you also can pass already implemented methods.

@Test
public void test_something_fast() {
    CONTROL.withTimeout(0, this::test_something);
}
withTimeout() overrides all internal timeouts except the explicit set timeout in waitFor(int seconds) methods.

7.4. Retries

In some situations you cannot rely on single assertions or waits anymore and need to continue trying something out before performing an alternative solution. Use control methods for repeating a couple of actions within a loop until a timeout has reached.

For example, this retry block tries to click a button until it’s disabled.

CONTROL.retryFor(int seconds, () -> {
    button.click();
    button.expect().enabled(false);
});

Or if you want to retry something multiple times.

CONTROL.retryTimes(int times, () -> {
    button.click();
    button.expect().enabled(false);
});

You can also perform something when the retry block fails.

CONTROL.retryFor(int seconds,() -> {
    element.expect().text(String);
}, () -> {
    element.getWebDriver().reload();
});

You can also combine these control features.

CONTROL.retryFor(int seconds, () -> {
    CONTROL.withTimeout(int seconds, () -> {
        button.click();
        uiElement.scrollIntoView();
        uiElement.expect().visiblePartial(boolean);
    }
);

7.5. Waits

If you need to wait for something to happen. You can use the control method waitFor which does the same as retryFor but without throwing any exception.

boolean loginOpened = CONTROL.waitFor(int seconds, () -> {
    WEB_DRIVER_MANAGER.switchToWindowTitle("Login");
});

Or use waitTimes similar to retryTimes.

boolean loginOpened = CONTROL.waitTimes(int times, () -> {
    WEB_DRIVER_MANAGER.switchToWindowTitle("Login");
});

8. Test execution

Testerra has several features to handle and adjust a test execution, which are described in the following paragraphs.

8.1. Conditional behaviour

For managing the execution behaviour of tests in suites there are means to skip tests and avoid closing browser windows after failures.

test.properties
# all browser windows remain open after first failure, default = false
tt.on.state.testfailed.skip.shutdown=true

# skip all tests after first failure, default = false
tt.on.state.testfailed.skip.following.tests=true

8.2. Failure Corridor

This mechanism is used to define the test goal of test runs so that it only fails with an invalid failure corridor.

This feature is enabled by default with the following property.

tt.failure.corridor.active=true

With an enabled failure corridor, you need to define the maximum amount of failures per weight:

test.properties
tt.failure.corridor.allowed.failed.tests.high=0
tt.failure.corridor.allowed.failed.tests.mid=1
tt.failure.corridor.allowed.failed.tests.low=2

If you do not define any failure corridor, the default value 0 is used for all three levels.

To change the weight for each test, just annotate it with @FailureCorridor, where High is default.

Examples of method weighting
// This testcase is marked with a high weight.
@FailureCorridor.High
@Test
public void test1() throws Exception {
    Assert.fail();
}

// This testcase is not marked, but the default weight is high.
@Test
public void test2() throws Exception {
    Assert.fail();
}

// This testcase is marked with a middle weight.
@FailureCorridor.Mid
@Test
public void test3() throws Exception {
    Assert.fail();
}

// This testcase is additional marked with @Fails.
// So the test result is ignored by the Failure corridor.
@Fails
@FailureCorridor.Mid
@Test
public void test4() throws Exception {
    Assert.fail();
}

// This testcase is marked with a low weight.
@FailureCorridor.Low
@Test
public void test5() throws Exception {
    Assert.fail();
}

8.3. Element Highlighting

8.3.1. Demo mode

In the demo mode actions on pages are marked with distinctive coloured frames around the element of the action. This mechanism is set by a property:

test.properties
# activate demo mode, default = false
tt.demomode=true

The following colours are used for highlighting

  • red: failed visibility checks and asserts

  • green: successful visibility checks and asserts

  • yellow: mouseOver

  • blue: click

The highlighting is removed after a timeout of 2000 ms. You can set a custom timeout:

test.properties
# Set a custom timeout in ms
# 0 -> infinite timeout
# default is 2000
tt.demomode.timeout=5000

8.4. Expected Fails

For known issues on the SUT the annotation @Fails can used to mark a test method as failing. These test cases are marked as Expected failed separately in the report.

If tests are passed again, you get a note in the report to remove the Fails annotation.

@Test
@Fails()
public void testItWillFail() {
    Assert.assertTrue(false);
}

The result is technically still a failure and only visually elevated to facilitate the evaluation of the report.

Please keep in mind that @Fails has an impact to Failure Corridor.

@Fails should not be used in conjunction with TestNG @DataProvider because the detected failure is ambiguous and might not be valid for all provided data.

8.4.1. Add additional information

You can add additional information to describe the cause in more detail. All information are added to the report.

@Test
@Fails(description="This test fails for reasons")
public void testItWillFail() {
    Assert.assertTrue(false);
}
Table 1. Possible attributes for the Fails annotation
Attribute Description

description

Give more details about the failure.

ticketString

Define a bug ticket ID or URL as a String value.

intoReport

If true the failing test is shown as Failed instead of Expected Failed (default: false).

validator

Define a method that checks if the expected failure is valid.

validatorClass

Define a class for the validator method (optional)

8.4.2. Defining a validator

With expected fails validators, you can define if the Expected Failed state is valid or not. When the validator returns true, the expected failed status is valid, otherwise, the test will result in a regular Failed. You can use that feature to mark a test as expecting to fail for known circumstances, like browser or environment configurations.

You define a validator the following way:

public boolean browserCouldFail(MethodContext methodContext) {
    return methodContext.readSessionContexts()
                .map(SessionContext::getActualBrowserName)
                .anyMatch(s -> s.contains("internet explorer"));
}

@Test
@Fails(validator = "browserCouldFail")
public void testSomething() {
    // Perform your tests here
}

Or as a class:

public class FailsValidator {
    public boolean expectedFailIsValid(MethodContext methodContext) {
        return true;
    }
}
@Test
@Fails(validatorClass = FailsValidator.class, validator = "expectedFailIsValid")
public void testSomething() {
    // Perform your tests here
}

Example for expected fails on a specific Exceptions:

public class FailsValidator {
    public boolean expectedFailOnCertainException(MethodContext methodContext) {
        // of course it's possible to check for more things inside the throwable
        return methodContext.getTestNgResult().isPresent()
                && methodContext.getTestNgResult().get().getThrowable()
                   instanceof CertainException;
    }
}
@Test
@Fails(validatorClass = FailsValidator.class, validator = "expectedFailOnCertainException")
public void test_customFails() throws CertainException {
    throw new CertainException();
}

8.5. Retry analyzer

Testerra provides an adjustable mechanism to automatically retry failed tests.

The default retry count is 1. Each failed method is executed exactly one more time, when matching the retry criteria. Retried methods are shown in the section Retried of the report.

You can change the default with the following property.

test.properties
tt.failed.tests.max.retries=1
The retry mechanism always ignores testcases with a valid Fails annotation.

8.5.1. Specific retry count for test methods

You can change the retry count for specific test methods.

@Test()
@Retry(maxRetries = 2)
public void testMethod() {
    // ...
}

8.5.2. Default retries

The following default retry analyzers are registered from modules:

  • driver-ui registeres WebDriverRetryAnalyzer which retries tests on specific internal WebDriver exceptions they look like temporary communication problems.

  • driver-ui-desktop registeres SeleniumRetryAnalyzer which retries tests on general Selenium communication problems with requested Desktop user agents:

    • org.openqa.selenium.json.JsonException

    • org.openqa.selenium.remote.UnreachableBrowserException

8.5.3. Specific retries

Testerra can also retry failed methods when matching certain criteria. The filtering process contains of checks of classes and messages matching the thrown Exception, which are set within the test.properties file.

test.properties
# Set additional classes that engage a retry,
tt.failed.tests.if.throwable.classes=java.sql.SQLRecoverableException

# Set additional messages of Throwable that engage a retry,
tt.failed.tests.if.throwable.messages=failed to connect, error communicating with database

8.5.4. Customize retry behaviour

For further adjustment additional analyzers can be registered expanding the default behaviour.

Defining AdditionalRetryAnalyzer for InstantiationException
// custom Retryanalyzers need to implement the functional interface AdditionalRetryAnalyzer
public class InstantiationExceptionRetryAnalyzer implements AdditionalRetryAnalyzer {

    final String message = "failed instantiation";

    @Override
    public Optional<Throwable> analyzeThrowable(Throwable throwable) {
        String tMessage = throwable.getMessage();
        if (throwable instanceof InstantiationException) {
            if (tMessage != null) {
                final String tMessageLC = tMessage.toLowerCase();
                boolean match = tMessageLC.contains(message);
                if (match) {
                    return Optional.of(throwable);
                }
            }
        }

        return Optional.empty();
    }
}
Register your retry analyzer
public class AbstractTest extends TesterraTest {

    static {
        // register the additional Analyzer,
        // which checks for "InstantiationException" and the message "failed instantiation"
        RetryAnalyzer.registerAdditionalRetryAnalyzer(new InstantiationExceptionRetryAnalyzer());
    }

}

8.5.5. @NoRetry

With this annotation the Retry Analyzer won’t retry these methods if previously failed. This is characteristically shown in the report by the badge NoRetry.

You can customize the NoRetry annotation with the attributes name and color.

Table 2. Possible attributes for the NoRetry annotation
Attribute Description

name

Changes the shown text in the report. Default is No Retry

color

Change the background color of the shown text. Default is grey.
Values need to be valid for HTML colors like:

  • name of the color, e.g. red

  • RGB values, e.g. rgb(255, 236, 139)

  • RGBA values, e.g. rgba(252, 156, 249, 0.75)

  • HSL values, e.g. hsl(217, 97%, 57%)

  • Hex values, e.g. #57c0ff

An example with customization of NoRetry
@Test()
@NoRetry(name = "No retry because it's not allowed.", color="rgb(255, 236, 139)")
public void testMethod() {
    ...
}

8.6. WebDriverWatchDog

The WebDriverWatchDog is your vigilant pet watching the test execution and reacting on blocked tasks. With two properties it is set up.

test.properties
# activate watchdog, default = false
tt.watchdog.enable = true

# timeout in seconds after the test execution is terminated, default = 300
tt.watchdog.timeout.seconds = 500

8.6.1. How does it work?

With the first Usage of WedDriverManager the WebDriverWatchDog is initiated. It internally starts a Thread running in parallel to the current test execution checking the stacktrace every ten seconds for stacktrace entries with thread name "Forwarding" containing an Element "java.net.SocketInputStream". These potentially blocking stacktrace entries are updated every time found. Upon reaching the maximum timeout of 500 seconds the whole test execution is terminated with exit code 99 and a readable error output in your log.

A valid report is always generated.

8.7. Annotations for Report adjustment

To improve the readability and clarity of the report there are several annotations for marking the test class and test methods, which are described in the following paragraph as well as in @Fails, @Retry and @NoRetry.

8.7.1. @TestClassContext

With this annotation you can set the test context for the given test class. There two attributes for adjustments.

  • name: name of the context, default = ""

  • mode: TestClassContext.Mode.ONE_FOR_ALL or TestClassContext.Mode.ONE_FOR_EACH, default = TestClassContext.Mode.ONE_FOR_ALL

The Executed tests are then shown in the classes overview of the report as a entry labeled with name from @TestClassContext.

8.8. Dry Run

8.8.1. Overview

With this execution mode all methods marked with TestNG Annotations are only called but their code isn’t executed, hence the name dry run. It’s designed to simply check the callstack of TestNG related methods without executing their logic, e.g. to find missing method calls in your test setup. For using this you just need to set the following property.

test.properties
# activate the dry run, default = false
tt.dryrun=true

The report indicates a dry run with the suffix Dry Run in the headlines of each section.

The rest is visually identical to a normal run.

All called methods are shown, but probably as passed. With a closer look into the report details you will just notice a really low test duration, something below one second.

8.8.2. @DismissDryRun

When this is annotated at a method it will be executed completely, regardless of the value of tt.dryrun. There is no dedicated visual elevation for these methods in the report.


Testerra Features

9. Reporting with Report-NG

9.1. Overview

Testerra provides an advanced reporting of the test results with the Report-NG module. A report generated with Report-NG consists of these different views:

  • Dashboard - a general overview of the test run

  • Tests - a detailed list of the individual tests

  • Failure Aspects - a summary of the different types of failures

  • Logs - a panel that contains detailed logging information

  • Threads - a graphical view of the thread activities during the parallel test execution

  • Timings - a detailed overview of the execution time of test cases and loading time of sessions

  • History - an overview of past test runs, highlighting issues, their durations, and the ability to compare runs

Report-NG Dashboard

The report is generated after the test execution and can be viewed with this html file:

test-report/report-ng/index.html

Unfortunately, because the report is a highly dynamic application based on java script, the security settings of most browsers do not allow to directly view the report in your local folder. In this case you can use one of the simple http servers from this list to serve the files and to access the report.

9.2. Setup

The simplest way to add Report-NG test reports to your project is to use the Testerra project skeleton as a foundation. See Testerra Skeleton project how to start with it. You can use Report-NG instantly because there it is already configured for use with Maven as well as Gradle.

If you are not using the Testerra skeleton, checkout the Maven and Gradle build files in chapter Testerra manual setup.

9.3. History

The Testerra report can display historical data from previous test runs. By default, results from up to 50 executions are stored and displayed. This limit can be adjusted by setting the tt.report.history.maxtestruns property to a different value. If the limit is exceeded, the oldest results are automatically deleted.

9.3.1. How to use the history?

When executing a test set for the first time, placeholders are shown because there is no previous run available yet. You need to execute the test set at least twice to view historical data in the report. Testerra uses the locally saved test results, stored in the report directory (which can be configured via the tt.report.dir property), to build the history. Therefore, ensure that previous test results are locally available to enable history features.

9.4. Dashboard View

9.4.1. Overview

The dashboard view provides an overview of the latest test execution. It displays the total number of executed tests, along with counts of passed, skipped, and failed tests.

Dashboard

The Breakdown panel includes a pie chart showing the proportions of test results, color-coded by test status.

Directly below the pie chart, a panel displays detailed information about test execution duration, as well as start and end times. Beneath that, another panel lists the executed test classes, showing how many tests passed, failed, or were skipped within each class.

9.4.2. Status definitions

Testerra defines the following final 4 statuses which describe the final test result:

Test Status Description
status passed

The test was successful.

status failed

The test was failed.

status skipped

The test was not executed because a precondition failed like a configuration method, a data provider or an another test.

status expected failed

The test failed as expected and the test method is annotated with @Fails because of a known bug or problem.

Additional to the statuses above Testerra defines some more sub statuses:

Test Status Description
status retried

The test failed at it’s first try. Testerra executed that test again if the RetryAnalyzer was active.

status recovered

The test was passed at it’s second (third…​) execution if the RetryAnalyzer was active.

status repaired

The test was passed but it is still annotated with @Fails. The annotation can be removed now.

  • The dashboard only shows the final statuses in the 'Breakdown' and 'Test Classes' charts.

  • The 'Tests' table summarize all executed tests inclusive retried tests. Recovered and repaired tests belong to passed tests.

  • The tests view shows all tests with their detailed status.

9.5. Test Details View

9.5.1. Overview

The report provides a list of all executed tests that can be filtered according to the status, the class or the name of the test. If you want to include the setup and configuration methods in this list you must activate the switch on the right side:

Test case list filter

When you click on the name of a test case in this list, you can see all details of its last execution.

On the top you will find general status information, test timing information and if a screenshot was taken during the test execution it can be viewed here as well.

Test details

Below this section different tab panes provide more information:

  • Error Details - a tab pane that is present when the test failed which shows the type (aspect) of the failure, the lines of test code causing it and a stack trace.

  • Steps - a tab pane which lists all test steps of the test execution in detail, which is further described here.

  • Sessions - a tab pane which shows the web driver sessions with its capabilities when a web driver session was active.

  • Dependencies - a tab pane that shows the dependencies of this test case in case there are any.

  • Method History – a tab showing previous runs the method, along with additional statistics.

9.5.2. Browser Info tab pane

Webdriver Sessions

9.5.3. Video tab pane

If you are using the (Testerra Selenoid connector) and a (Selenoid grid), you can add recorded video streams to your report.
Video tab pane

9.5.4. Dependencies tab pane

Dependencies tab pane

9.5.5. Method History tab pane

Method History tab pane

9.6. Failure Aspects View

In case of many failures it is important to be able to find out the most severe failure reasons (aspects) that caused the most failed tests cases.

For this the Failure Aspect View was designed.

Failure Aspect View

It comes with a filter as well and shows the most occurred problems on the top with the number of tests that they caused to fail.

9.7. Log View

The Log View shows all log information that were recorded during test execution. It allows to filter by message log level and to search log messages according to a given search term or keyword.

Log View

9.8. Threads View

The Threads View displays all activities on the individual worker threads in a gantt chart. The y-axis delineates the thread names, while the x-axis portrays the timeline. Test cases are visualized within the respective threads where they were executed. For details hover over a test case to view a tooltip showing the name, its Run-Index, Start- and End-time and the duration.

Navigation:

  • Use the mouse wheel to zoom in or out on the time axis.

  • Click the left button while moving the mouse to scroll horizontally.

  • Use the scroll bar on the right to scroll vertically.

Filter:

  • Use the status filter to focus on all test cases with the selected status.

  • Use the method input field to zoom in on a specific test case.

Threads View

9.9. Timings View

9.9.1. Tests tab pane

All executed test cases are included in a bar chart, grouped in ranges by their execution time. By hovering over a bar the methods within this range are listed in a tooltip.

It is possible to adjust the number of method ranges by selecting a value of 5, 10, 15 or 20 in the corresponding drop-down menu. This way you can adjust the resolution of the chart.

You can also search for a specific method by entering the name of the test case in the searchbar. The corresponding bar is then highlighted.

By default, only the test methods themselves are included. However, it is possible to incorporate the configuration methods by toggling the switch located in the top right corner of the view.

Tests

9.9.2. Sessions tab pane

In the scatter chart you can see every session that was started in the test execution. On the x-axis, the starting time of session is indicated, while the loading duration is represented on the y-axis.

When hovering over any of the dots, a tooltip appears providing details regarding the browser and its version, the session name and ID, along with the test cases executed within that session.

If a base URL is configured, the load duration and timestamp of the base URL are also displayed. In this scenario, the two dots are linked, as the base URL is consistently loaded at session startup. Note that setting up a base URL is optional. If both dots exist, the hover effect will clarify which base URL dot corresponds to each session dot.

Navigation:

  • Use the mouse wheel to zoom in or out on the y-axis.

  • Move the mouse while holding the left button to scroll vertically.

  • It is also possible to use the buttons in the upper right corner to zoom into a specific area, reset the zoom and restore the chart.

Sessions

9.10. History View

9.10.1. Test Run tab pane

The Test Run view provides an overview of the most important aspects of the history and highlights test cases with problematic behavior:

Test Run history

9.10.2. Run Comparison tab pane

You can compare any previous run with the current one in the Run Comparison tab:

Run Comparison

9.10.3. Run Duration tab pane

The Run Duration tab offers more insights into timing-related aspects of the history:

Run Duration history

9.10.4. Test Classes tab pane

The Classes view displays all executed test classes along with their previous executions:

Classes history

You can also select a class to view the history of all methods within that class:

Single class history

9.11. Test Steps

It is a good and recommended practice to use test steps to increase the transparency of the test. In case of failures it is easier to understand their context.

Test steps are defined within the test code by marking their begin as demonstrated in this example test of the Testerra skeleton project:

    @Test
    public void testT04_TableEntryNotPresent() {
        final UserModel userNonExisting = userModelFactory.createNonExisting();

        TestStep.begin("Init driver and first page");
        StartPage startPage = PAGE_FACTORY.createPage(StartPage.class);

        TestStep.begin("Navigate to tables");
        TablePage tablePage = startPage.goToTablePage();

        TestStep.begin("Assert user shown.");
        Assert.assertTrue(tablePage.isUserShown(userNonExisting));
    }
Test Steps
All steps are numbered automatically. The step Setup is added by Testerra.

9.12. Priority Messages

9.12.1. For test methods

In case you would like to have some log messages displayed on a prominent place on the test details page, you can log priority messages like this:

public class MyTestClass extends TesterraTest implements Loggable {

    @Test
    public void testT01_PriorityMessages() {

    	log().info("It's gonna be ok.", Loggable.prompt);

    	// some test activities ...
        log().warn("Warn me!", Loggable.prompt);

        // some test activities ...
        log().error("Tell me more!", Loggable.prompt);
    }
}

As a result the information, warnings and errors are visible on the priority message panel in the center of the test detail page:

Priority Messages

9.12.2. For global context

Priority messages can also be used to enrich the dashboard with test run specific information (e.g. the environment it was executed in. In this case log the priority message as following:

    @Test(groups = "LOGS")
    public void test_promptOutSideMethodContext() {
        new Thread(() -> log().info("Prompt outside method context", Loggable.prompt)).start();
    }

The priority message is displayed on the priority message panel above the Test Classes bar chart:

Priority Message on Dashboard

9.13. Layout Checks

In case you are using Layout Checks Report-NG provides special features to analyze failed layout tests.

9.13.1. Comparison by Image Size

When a layout test failed because there is a difference of the image size between the actual screenshot and the reference image, Report-NG will add a failure aspect to the test details panel:

Wrong Image Size Failure Aspect

9.13.2. Comparison by Pixel Distance

When a layout test failed because the percentage of the pixels that are different is too high (pixel distance) Report-NG will also add a failure aspect to the test details panel:

High Pixel Distance Size Failure Aspect

Furthermore, Report-NG provides a special dialog to compare the actual screenshot and the reference image to actually identify the mismatching pixels. It can be opened by clicking one of the three thumbnail images. The pixel comparison dialog provides a diamond shaped slider to change the size of the left and right part of the pixel comparison area. You can also select what is displayed on the left and right part, the actual screenshot, the reference image or the pixel difference:

Pixel Comparison Dialog

9.14. More on Assertions

9.14.1. Collected assertions

When Collected assertions are used and some of its assertions fail the test is not aborted, but it will be marked as failure in Report-NG:

Failed collected assertions
Assert Collector Details

The failures are marked in the step view:

Assert Collector Steps

9.14.2. Optional assertions

When Optional assertions fail, the test will also be fully executed and will be marked as passed.

Optional Assert Pass
Optional Assert Details

The failed optional asserts will be displayed as warnings:

Optional Assert Steps

9.15. Report customizing

Some aspects of ReportNG can be customized. Have a look for the complete list of all properties.

9.15.1. Code snippet in method details

In case of using helper methods for assert test aspects it could be useful to get the source code line of the calling method, not the helper method itself.

You can exclude packages or classes with a regex definition:

test.properties
# Exclude all packages or classes which names contain 'abstract' or 'foo', ignoring case sensitives
tt.report.source.exclusion.regex=(?i:abstract|foo)

The following image shows the differences between default (left) and tt.report.source.exclusion.regex=AbstractSourceCodeTests (right).

Exclusion of packages or classes for source code snippet
Setting this value as a local test property will be ignored because source code snippets are only collected at the end of the whole test.

9.16. Report of aborted test runs

In case of unexpected abortion of a test execution the report is generated, too. Due to abortion of still running test methods some information can be missing, like result states or execution time.

Testerra additionally provides a mechanism to implicitly create a test report when an unexpected exit of the current execution occurred. All existing information created until the moment of the abortion are collected to generate a report. This might lead to missing as information like the execution time, result states and screenshots might be abandoned if running test methods have been interrupted.

Report-NG states missing
Report-NG execution time missing

A hint indicating the possibly incomplete state of the report is shown as a warning in the section 'priority messages'. The report generation in case of abortion of the test execution is described in detail in the paragraph JVMExitHook.

Report-NG Aborted Exection

9.17. Print preview

If you want to consolidate all report information into a single PDF file or print it directly, the "Print Report" feature can be accessed by clicking the button located at the bottom of the navigation drawer.

On the left side, a preview shows how the printed report might appear. It includes an indicator displaying the estimated page count as you scroll through the content.

On the right side, you will find various settings that allow you to customize the contents of the preview. By clicking the "Print" button, the final version will be sent to the browser’s print dialog, where you can print the report or download it as a PDF file.

Print Preview

10. Modules

10.1. Layout Check

Layout tests always mean checking whether a GUI is designed according to the guidelines.

10.1.1. Introduction

Are the position, size, color and shape of the elements correct? Are distances maintained? Perhaps even certain elements are missing? In test automation, functionality is primarily tested. Here it is actually irrelevant how a button is designed and where it is positioned. The main thing is that the element can be uniquely recognized via XPath or CSS selectors.

However, in some frontends such as content management systems with a high level of relevance to end users (certain portal solutions, shops) the management is also extremely important. However, testing this with Selenium’s previous means is not in any relation between effort and benefit. Manual visual inspection is usually still the fastest way to do this.

Although, manual inspection can never be pixel-accurate. An inspection of perhaps more than 100 elements is too costly. Smaller shifts or colour deviations are easily overlooked.

At this point an automatic comparison of reference screenshots can help. This test is to be seen in addition to the functional tests, but cannot replace them. In a test case, it would also be conceivable to combine the check of function and layout.

10.1.2. How it works

A layout test with the Testerra utilities is actually a comparison between a reference and the actual state. This is done via the screenshot functionality of Selenium, in which both screenshots (reference and actual) are compared pixel by pixel from top to bottom. In this comparison, a third image is created in which differences (ReferencePixel != ActualPixel) are marked in red.

Visualization of a layout check

There are two different ways in which Testerra handles different sized images. Using the property tt.layoutcheck.pixel.count.hard.assertion, you can customize the handling of such cases.

If the property is set to false (default) Testerra reports this as an Optional assertion. Only the common pixels of both images as considered as 100% for the calculation. Pixels that are outside of one or the other image are not included in the error calculation.

With the property set to true, the size difference of the two images is included in the calculation so that all pixels that appear in only one image counted as incorrect ones.

In both scenarios, all pixels outside one or the other image are marked in blue in the generated distance image.

Visualization of a layout check with different sized images
Prerequisites

The following prerequisites must be met

  • Concrete execution environment: Layout tests should run in a concrete browser environment to ensure the comparability of the screenshots.

  • Size of browser window: Define fixed size to exclude different sizes of the images at different screen resolutions.

  • Screen resolution: Make sure you have the same screen resolution on each presentation device.

  • Scaling: The scaling should always be set to 100%. Using a different scaling causes variations in the positions of the elements on the website. For more details, see this Selenium issue.

10.1.3. Configuration

In order to get the layout check to work, you need at least a reference image and a place where it’s located.

test.properties
tt.layoutcheck.reference.path=src/test/resources/layout-references/{tt.browser}
tt.layoutcheck.reference.nametemplate=%s.png

# Highly recommended to disable full screen for browser
tt.browser.maximize=false

# Highly recommended to switch of the demo mode for layout tests
tt.demomode=false

The directory for reference image may result in src/test/resources/layout-references/chrome/WelcomePage.png for example.

The comparison is generally carried out over the entire reference image. In this case, it is a prerequisite that the reference screenshot and the screenshot created during the test runtime have the same resolution (height and width are identical).

10.1.4. Check the whole page

// 1% of the pixels are allowed to be different
float pixelDistanceToleranceThresholdPercent = 1.0;

page.expect().screenshot().pixelDistance("WelcomePage").isLowerThan(pixelDistanceToleranceThresholdPercent);
To organize the reference screenshots in subfolders you can add the relative structure to the name like my/additional/folder/WelcomePage.

10.1.5. Check a single UiElement

To check the layout of a single UiElement only, you can use the standard asserts implementation.

// 10% of the pixels are allowed to be different
float pixelDistanceToleranceThresholdPercent = 10.0;

uiElement.expect().screenshot().pixelDistance("HeaderComponent").isLowerThan(pixelDistanceToleranceThresholdPercent);

10.1.6. Check images with RGB deviation

The property tt.layoutcheck.pixel.rgb.deviation.percent controls, which color difference in percentage of a single pixel is count as a match. You can use that value to handle slightly changes in color or alpha values.

Especially for photo checks it should be helpful in case of different kinds of image formats and compression algorithms.

tt.layoutcheck.pixel.rgb.deviation.percent=0.0 tt.layoutcheck.pixel.rgb.deviation.percent=2.0
Screenshot of the report visualization of a layout check
Screenshot of the report visualization of a layout check

10.1.7. Check image files

In case you don’t want to check a screenshot of the browser but rather a pre-existing image, there is also a way to do that. The assertImage method requires the File object of the actual image and executes a layout check.

// 5% of the pixels are allowed to be different
float pixelDistanceToleranceThresholdPercent = 5.0;

File absoluteFile = FileUtils.getResourceFile("images/actual.png");
LayoutCheck.assertImage(absoluteFile, "reference", pixelDistanceToleranceThresholdPercent);

10.1.8. Take reference screenshots on first run

When you have configured the reference screenshots location and implemented the tests, you can now set the option

tt.layoutcheck.takereference=true

to enable taking automatically screenshots based on the current browser and size configuration and storing them to the reference image’s location.

All concrete distance values in this tt.layoutcheck.takereference-mode will return 0 (zero) and always pass the tests.

10.1.9. Reporting

Report-NG provides a good presentation of the results of layout checks. See here for more details.

10.2. Localization

Websites come in many languages, where the functionality may not change, but labels of buttons, input or other interactive elements. Testerra provides an easy and simple way to support the localization of UiElements based on Locale.

10.2.1. Enable Unicode for your project

In build environments, where Unicode is not default, you should force to use it in your project by setting the Java compile options.

build.gradle
compileJava.options.encoding = 'UTF-8'
compileTestJava.options.encoding = "UTF-8"
pom.xml
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>

For Gradle, you should also set your system-wide or local gradle.properties.

gradle.properties
systemProp.file.encoding=utf-8

10.2.2. Add localization resource bundle

The localization is based on Unicode .properties files loaded during runtime. To add such files, create a new resource bundle lang in src/main/resources and add all required locales.

lang_en.properties
BTN_LOGIN=Login
lang_de.properties
BTN_LOGIN=Anmelden
You can change the default encoding for .properties files in IntellJ at File → Settings → File Encodings
intellij properties encoding

10.2.3. Access localization text

Now you can instance UiElements by localized strings.

import eu.tsystems.mms.tic.testframework.l10n.SimpleLocalization;

UiElement loginBtn = find(By.linkText(SimpleLocalization.getText("BTN_LOGIN")));

SimpleLocalization uses Locale.getDefault() by default, but you can switch the default locale the following way.

LocalizedBundle defaultBundle = SimpleLocalization.setDefault(Locale.GERMAN);

10.2.4. Session based localization

For thread-safe localization, you can use session based localization by initializing your localized bundles based on the session’s locale.

Locale sessionLocale = WebDriverManager.getSessionLocale(WebDriver).orElse(Locale.getDefault());

LocalizedBundle sessionBundle = new LocalizedBundle("testdata", sessionLocale);
sessionBundle.getString("TEST_KEY");

When the SUT locale changes, you should also set the session’s locale:

public class LocalizedPage extends Page {

    public void switchLocale(Locale locale) {
        // Implement your language switch here
        // ...

        // Don't forget to set the sessions locale
        WebDriverManager.setSessionLocale(getWebDriver(), locale);
    }
}

The best way to change the locale for your tests is, to pass the language property as command line argument.

gradle test -Duser.language=de

For Maven, you need some extra configuration

pom.xml
<project>
    <properties>
        <user.language>de</user.language>
        <user.country>DE</user.country>
        <argLine>-Duser.language=${user.language} -Duser.country=${user.country}</argLine>
    </properties>
</project>

before running the command

mvn test -Duser.language=de -Duser.country=DE

10.2.6. Change locale from property

When you want to change the locale from a property. Do the following

String testLanguage = PROPERTY_MANAGER.getProperty("test.language", "de");
Locale.setDefault(Locale.forLanguageTag(testLanguage));

10.2.7. Change locale for the user agent

The WebDriver API official doesn’t support changing the language of a browser session. But there exists non-standard or experimental ways on Stackoverflow to change your browser locale.

Anyways, it’s currently the better way to visit your SUT with a predefined language and change it there (with a language switcher).

10.3. Mail connector

The module mail-connector allows you to send and receive/read emails as well as solve mail surrounding tasks like encoding or encrypting.

The Mail connector uses Jakarta Mail 2.

10.3.1. Project setup

Gradle
// build.gradle
compile 'io.testerra:mail-connector:2.11'
Maven
<!-- pom.xml -->
<dependencies>
    <dependency>
        <groupId>io.testerra</groupId>
        <artifactId>mail-connector</artifactId>
        <version>2.11</version>
    </dependency>
</dependencies>
Configuration Parameters

All MailConnector classes need parameters to connect to the corresponding server. These are set within the mailconnection.properties file or by system properties. The following parameters must be set.

mailconnection.properties
#SMTP
SMTP_SERVER=smtp.mailserver.com
SMTP_SERVER_PORT=25
SMTP_USERNAME=user
SMTP_PASSWORD=password

#POP3
POP3_SERVER=pop.mailserver.com
POP3_SERVER_PORT=110
POP3_FOLDER_INBOX=inbox
POP3_USERNAME=user
POP3_PASSWORD=password

# IMAP
IMAP_SERVER=imap.mailserver.com
IMAP_SERVER_PORT=143
IMAP_FOLDER_INBOX=INBOX
IMAP_USERNAME=user
IMAP_PASSWORD=password

# TIMING for POP3 and IMAP
# sets the timeout between polls, default = 50s
POLLING_TIMER_SECONDS=10
# sets the maximum number of polls, default = 20
MAX_READ_TRIES=20

If you want to use an SSL-encrypted connection, you need to set SMTP_SSL_ENABLED / POP3_SSL_ENABLED to true. The ports must then be adjusted according to the server. The actual connection to the mail server is implicitly opened within each MailConnector class.

10.3.2. Common notes

Please note that Jakarta Mail 2 changed some packages. If you update to the latest Mail connector you have to change your imports:

// Jakarta Mail 1.6.x
import javax.mail.Message;
import javax.mail.internet.MimeBodyPart;
import javax.mail.search.AndTerm;
import javax.mail.search.SearchTerm;
...

// Jakarta Mail 2.x
import jakarta.mail.Message;
import jakarta.mail.internet.MimeBodyPart;
import jakarta.mail.search.AndTerm;
import jakarta.mail.search.SearchTerm;
...

10.3.3. POP3MailConnector

The POP3MailConnector provides access to a POP3 server. Emails are read and processed by using methods of the superclass AbstractInboxConnector. Specific mails can be extracted with serverside filtering by matching given criteria expressed with SearchTerm (see Jakarta Mail API)

Reading and Deleting Mails
POP3MailConnector pop3MailConnector = new POP3MailConnector();

// wait until the email with subject 'test' arrived in the InboxFolder
SubjectTerm subjectTerm = new SubjectTerm(subject);
EmailQuery query = new EmailQuery().setSearchTerm(subjectTerm);
Email email = pop3MailConnector.query(query).findFirst().get();

// delete all emails with this subject from the server while setting timeout and max number of polls explicitly
query.setRetryCount(5);
query.setPauseMs(2000);

Email email = pop3MailConnector.query(query).findFirst().get();

// delete mails matching the given criteria
String recipient = email.getRecipients().get(0));

SearchTerm searchTerm = new AndTerm(new SearchTerm[]{
        subjectTerm,
        new RecipientTerm(RecipientType.TO, new InternetAddress(recipient))
});

pop3MailConnector.deleteMessage(searchTerm);
Working with attachments
// wait until the email with subject 'test' arrived in the InboxFolder
EmailQuery query = new EmailQuery().setSearchTerm(new SubjectTerm(subject));
Email email = pop3MailConnector.query(query).findFirst().get();

try {
    Multipart content = (Multipart) email.getMessage().getContent();
    int contentCnt = content.getCount();
    String attachmentFileName = null;

    for (int i = 0; i < contentCnt; i++) {
        Part part = content.getBodyPart(i);

        // Retrieve attachment
        if (part.getDisposition().equals(Part.ATTACHMENT)) {
            attachmentFileName = part.getFileName();
        }
    }
} catch (IOException e) {
    e.printStackTrace();
} catch (MessagingException e) {
    e.printStackTrace();
}

10.3.4. SMTPMailConnector

This entity allows sending emails via the SMTP protocol.

Sending Mails
SMTPMailConnector smtpMailConnector = new SMTPMailConnector();

// send a created message
MimeMessage createdMessage = new MimeMessage(session);
try {
    msg.addRecipients(RecipientType.TO, RECIPIENT);
    msg.addFrom(new Address[]{new InternetAddress(SENDER)});
    msg.setSubject("testerra");
    msg.setText("mail text");
} catch (MessagingException e) {
    LOGGER.error(e.toString());
}
smtpMailConnector.sendMessage(createdMessage);

// send an existing message
MimeMessage existingMessage = MailUtils.loadEmailFile("test-mail.eml");
smtpMailConnector.sendMessage(existingMessage);

10.3.5. ImapMailConnector

The ImapMailConnector operates like the POP3MailConnector with an additional method to mark all mails as seen.

Working with Mails using ImapMailConnector
ImapMailConnector imapMailConnector = new ImapMailConnector();

EmailQuery query = new EmailQuery().setSearchTerm(new SubjectTerm(subject));
imapMailConnector.query(query).findFirst().ifPresent(email -> {
    // EMail found
});

// mark all mails in inbox as seen
imapMailConnector.markAllMailsAsSeen();

// delete all mails in inbox
imapMailConnector.deleteAllMessages();

10.3.6. Get simply the message count

You can get the message count for the inbox, of a specified folder name.

connector.getMessageCount();
connector.getMessageCount("FolderName");

10.3.7. SSL settings and trusting hosts for self-signed certificates

SSL is enabled per default for POP3 and IMAP and can be configured via. properties.

IMAP_SSL_ENABLED=false
POP3_SSL_ENABLED=false
SMTP_SSL_ENABLED=true

The MailConnector uses Certificate Utilities for trusting hosts.

10.3.8. Custom configuration

You can set properties to the JavaMail framework like:

connector.configureSessionProperties(properties -> {
    properties.put("mail.imaps.auth.ntlm.disable", true);
});

See the original documentation for more information:

10.3.9. Debugging the MailConnector

Enable the debug mode programatically

connector.getSession().setDebug(true);

or via Properties

DEBUG_SETTING=true

10.3.10. Best Practices

Combine search terms

You can combine search terms the following way

EmailQuery query = new EmailQuery();

query.withAllOfSearchTerms(
    new SubjectTerm("My Subject"),
    new ReceivedDateTerm(DateTerm.EQ, new Date())
);

// or
SearchTerm oneOf = new OrTerm(
    new SubjectTerm("My Subject"),
    new SubjectTerm("PetsOverNight.com"),
);
query.setSearchTerm(oneOf);

// or
List<SearchTerm> searchTerms = new ArrayList<>();
searchTerms.add(oneOf);
searchTerms.add(new ReceivedDateTerm(DateTerm.EQ, new Date()));
query.withAllOfSearchTerms(searchTerms);
Find emails by specified date

To find emails for a specified date, you should combine the SentDateTerm and an explicit filter, because the internal library is not able to filter by exact date with the IMAP protocol.

EmailQuery query = new EmailQuery();
Date now = new Date();

// Query emails that arrived today
query.setSearchTerm(new SentDateTerm(ComparisonTerm.GE, now));

connector.query(query)
    .filter(email -> email.getSentDate().after(now))
    .forEach(email -> {
        // EMail found
    });

10.3.11. MailUtils

This helper class contains methods which facilitate reoccurring tasks when working with mails, e.g. encrypting, decrypting and comparing mails.

Encryption, Decryption and Comparison
String pahtKeyStore = "your/path/to/cacert.p12";
String password = "123456";
String subject = "test";
String sentContent = "Testerra Testmail"

SMTPMailConnector smtpMailConnector = new SMTPMailConnector();
Session session = smtpMailConnector.getSession();

MimeMessage sentMessage = new MimeMessage(session);
sentMessage.setText(sentContent);
sentMessage.setSubject(subject);

// encrypt message
MimeMessage encryptedMsg = MailUtils.encryptMessageWithKeystore(sentMessage, session, pahtKeyStore, password);

smtpMailConnector.sendMessage(encryptedMsg);
Email receivedMsg = waitForMessage(subject);

// compare Mails and verify difference due to encryption
boolean areContentsEqual = MailUtils.compareSentAndReceivedEmailContents(sentMessage, receivedMsg);
Assert.assertFalse(areContentsEqual);

// decrypt message
MimeMessage decryptedMsg = MailUtils.decryptMessageWithKeystore(encryptedMsg, session, pahtKeyStore, password);
// verify receivedContent is equal to sentContent
String receivedContent = ((Multipart) decryptedMsg.getContent()).getBodyPart(0).getContent().toString();
Assert.assertEquals(receivedContent, sentContent);

10.4. PropertyManager

The PropertyManagerProvider interface provides a PROPERTY_MANAGER instance of IPropertyManager

String property         = PROPERTY_MANAGER.getProperty("my.property", "default");
Long longProperty       = PROPERTY_MANAGER.getLongProperty("my.long", 200L);
Double doubleProperty   = PROPERTY_MANAGER.getDoubleProperty("my.double", 200.0);
Boolean booleanProperty = PROPERTY_MANAGER.getBooleanProperty("my.boolean", false);

It will look for the property definition appearance in the following order:

When it’s still not defined, it will fall back to the given default value.

10.4.1. Property files

Testerra supports to load .properties files located under test/resources.

The following files are loaded automatically at Testerra start (in that order):

  • test.properties

  • system.properties

The test.properties is called the Test Configuration and contains everything required by the test, like Browser setup, SUT credentials or Layout test thresholds.

You can load additional properties files at any time.

PROPERTY_MANAGER.loadProperties("my.property.file");
This will override already defined properties.

10.4.2. System properties

When a system.properties exists, Testerra loads and sets the given properties via System.setProperty() if they are not present already.

This will not override already defined system properties given by the command line.

The path of this file can be changed by tt.system.settings.file and will be automatically loaded with the following message

common.PropertyManager - Load system properties: /path/to/your/system.properties

10.4.3. Test local properties

If you want to set properties for a test method only, you can use

PROPERTY_MANAGER.setTestLocalProperty("myproperty", "myvalue");

This property will be removed after every test method.

10.5. Connectors

While using the framework some external connectors could be useful. The maintainers provide some of them in separate repositories. Please take a look at the following list if there is a connector mapping your need.

Name Description

Appium Connector

Uses the open source standard Appium to run web tests based on Testerra on mobile devices.

Azure DevOps Connector

Automatic test result synchronization for Microsoft AzureDevOps platform.

Cucumber Connector

Provides the opportunity to use Cucumber and Gherkin to specify .feature files and combine it with the advantages of Testerra.

HPQC Connector

Automatic test result synchronization to HP Application Lifecycle Management, former called HP QualityCenter.

Selenoid Connector

Using a Selenium Grid based on Selenoid this module provides access to videos and VNC streams.

TeamCity Connector

A simple notification service for Jetbrains TeamCity.

Xray Connector

Automatic test result synchronization Jira XRay.

11. Utilities

11.1. Assert Utilities (deprecated)

This class has been replaced by the Assertion interface, therefore marked as @deprecated and should not be used anymore.

This class provides some extra assertion methods for TestNG:

AssertUtils.assertContains("Hello World", "Martin", "Greeting");
// Greeting [Hello World] contains [Martin] expected [true] but found [false]

AssertUtils.assertContainsNot("Hello World", "World", "Greeting");
// Greeting [Hello World] contains [World] expected [false] but found [true]

AssertUtils.assertGreaterEqualThan(new BigDecimal(2), new BigDecimal(4), "High number");
// High number [2] is greater or equal than [4] expected [true] but found [false]

AssertUtils.assertLowerThan(new BigDecimal(2), new BigDecimal(-1), "Low number");
// Low number [2] is lower than [-1] expected [true] but found [false]

11.2. Certificate Utilities

You can trust specified hosts in a whitespace separated list like

tt.cert.trusted.hosts=example.com google.com

Or just trust any (not recommended)

tt.cert.trusted.hosts=*

You can also configure this programatically like

CertUtils certUtils = CertUtils.getInstance();
String trustedHosts[] = {"t-systems.com"};
certUtils.setTrustedHosts(trustedHosts);
certUtils.setTrustAllHosts();

This will configure accepting certificates where possible:

  • User agents ACCEPT_INSECURE_CERTS capability

  • All internal APIs that use HttpsURLConnection like FileDownloader or MailConnector

You can also set defaults for all created SSLSocketFactory and HostnameVerifier.

// Trust all certificates in the current Java VM instance.
certUtils.makeDefault();

11.3. Desktop WebDriver utilities

This utility class provides some additional methods to interact with web elements.

Please consider this utility class as a fallback solution.

It could be useful if elements are hidden or not reachable by Selenium.

11.3.1. Supported actions

A short clickJS example
UiElement element = find(By.id("label"));
DesktopWebDriverUtils utils = new DesktopWebDriverUtils();

utils.clickJS(element);

The following methods are supported:

  • clickJS()

  • rightClickJS()

  • doubleClickJS()

  • mouseOverJS()

  • clickAbsolute()

  • mouseOverAbsolute2Axis()

  • clickByImage()

  • mouseOverByImage()

See Mouse over for more details about the mouseOverAbsolute2Axis method.

11.3.2. mouseOver vs. mouseOverAbsolute2Axis

The mouseOverAbsolute2Axis method does not move the mouse pointer relativly from the last position.

The following graphic shows 2 different mouse pointer paths beginning at the upper right image to the button 1.

absolute2Axis
Figure 5. Two possible mouse paths

The standard path (green) goes over a menu with hover effects. This could hide your target element. The blue path variant always goes first to the point (0,0) of the viewport, then in x and y direction the the target element.

11.3.3. By-image-utils

The clickByImage and mouseOverByImage methods use the image recognition technology of sikuli-api to locate a part of the current web page using a screenshot placed in the src/test/resources/ directory. After the referenced element was found, the operation is performed (either click or mouseOver) at the corresponding location, the center of the found screen region, using Selenium actions.

These methods are useful, if some elements can not be accessed in the usual way, for example if they are inside a canvas or svg element.

The following graphic shows the report-ng threads view with a selected testcase in the method filter, which is visible on the page but cannot be accessed since the chart is a canvas element.

by image threads example
Figure 6. A screenshot used to find the selected testcase

With a screenshot of this method, in this case "test_expectedFailed.png", the by-image-utils can be used to find this element and perform a mouseOver or click action in the center of the test method bar.

11.4. JavaScript Utilities

JSUtils provide some helper methods to inject and execute JavaScript code via WebDriver.

11.4.1. Execute JavaScript

Sometimes in automated testing for web applications you want to access your system under test by JavaScript or you just want to implement a code snippet to run custom validations. For example, Testerras Demo mode will use this behaviour to highlight elements while asserting or typing to visualize current actions.

Executing JavaScript
UiElement hiddenUploadField = findById("hidden-upload-field");
hiddenUploadField.expect().present(true);

hiddenUploadField.findWebElement(webElement -> {
    // will change style to display a hidden element
    JSUtils.executeScript(hiddenUploadField.getWebDriver(), "arguments[0].setAttribute('style', 'display:block; opacity: 1')", webElement);
});

11.4.2. Inject JavaScript (deprecated)

Content Security Policies disallow injection JavaScript into the DOM. Therefore, the following feature is @deprecated.

For executing more than a single line of JavaScript code it is recommended to write a JavaScript file and store it in src/main/resources or src/test/resources directory. Then you can inject the full JavaScript file with following method.

Implementing own JavaScript code
// WebDriver, Path to resource file, id of the script-tag
JSUtils.implementJavascriptOnPage(driver, "js/inject/custom.js", "customJs");

Testerra will then inject your resource file into the current DOM of your WebDriver instance according to the template.

<script id="customJs" type="text/javascript">
    // your javascript code here.
</script>

11.5. Download Utilities

11.5.1. FileDownloader

The FileDownloader utility provides methods for downloading resources from web sites. It uses the default proxy configuration.

Basic example how to use the FileDownloader
// define the url where to download the resource from
String downloadUrl = "https://upload.wikimedia.org/wikipedia/commons/thumb/0/0a/"+
	"S%C3%A5g_med_arm_av_valn%C3%B6t_-_Skoklosters_slott_-_92588.tif/"+
	"lossy-page1-1024px-S%C3%A5g_med_arm_av_valn%C3%B6t_-_Skoklosters_slott_-_92588.tif.jpg";

// construct the downloader
FileDownloader downloader = new FileDownloader();

// perform the download
File downloadFile = downloader.download(downloadUrl);

System.out.println("downloaded file exists:" + downloadFile.exists());
File downloading settings.
CertUtils certUtils = new CertUtils();
certUtils.setTrustAllHosts(true);

DefaultConnectionConfigurator connectionConfigurator = new DefaultConnectionConfigurator();
connectionConfigurator.useCertUtils(certUtils);
connectionConfigurator.immitateCookiesFrom(WebDriver)

FileDownloader downloader = new FileDownloader();
downloader.setConnectionConfigurator(connectionConfigurator);
downloader.setDownloadLocation("/some/directory");
downloader.setProxy(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("123.0.0.1", 8080)));
Delete all downloaded files.
downloader.cleanup();

11.6. PDF Utilities

This is a utility class for handling and retrieving information from PDF files, such as text content or the amount of pages.

11.6.1. Extracting text

It is possible to extract text content of whole PDF documents or just one specific page for verifications.

// reads text from a file given by a filename (absolute path of file).
String textFromPdf = PdfUtils.getStringFromPdf(String pdfFileLocation);
//reads text from only one page of a pdf given as input steam.
String textFromPdf = PdfUtils.getStringFromPdf(InputStream stream, int pageNumber);

11.6.2. Exporting images

It is also possible to use methods of this class to render an image for one specific page of the PDF or export the entire document to multiple images. That way you can save reference images and use them for visual comparison of PDF files by using the assertImage method of Layout Check.

A short example of how to export a single page from a pdf file as an image
int dpi = 150;
String absoluteFilePath = FileUtils.getAbsoluteFilePath("testfiles/TestDocument.pdf");
int pageNumber = 2;

/*
 The image files are saved under the name of the PDF-document with the corresponding page number appended,
 in this case the second page will be saved as "TestDocument.pdf_page2.png"
*/
File savedImage = PdfUtils.getImageFromPdf(absoluteFilePath, dpi, confidenceThreshold);

In the following example all pages of the locally saved PDF "TestDocument.pdf" are testet with a layout check.

int dpi = 150;
int confidenceThreshold = 5;
String absoluteFilePath = FileUtils.getAbsoluteFilePath("testfiles/TestDocument.pdf");

List<File> actualImages = PdfUtils.getImageFromPdf(absoluteFilePath, dpi);

for (File image : actualImages) {
    String referenceName = FilenameUtils.removeExtension(image.getName());
    LayoutCheck.assertImage(image, referenceName, confidenceThreshold);
}

Using the PdfUtils, images of all pages are rendered with a resolution of 150 dpi. The received list of files allows to iterate through it and call the assertImage method to visually check each page. In this case, the reference images are named in the same way as the actual images, so that the whole document can be easily checked in a simple loop.

11.7. Proxy Utilities

This is a static helper class based for reading the proxy configuration via PropertyManager.

For specifying proxy settings in system.properties see also an example here. See details about handling of system.properties here.

import java.net.URL;
import eu.tsystems.mms.tic.testframework.utils.ProxyUtils;

// Format e.g.: http://{http.proxyHost}:{http.proxyPort}
URL httpProxyUrl = ProxyUtils.getSystemHttpProxyUrl();
URL httpsProxyUrl = ProxyUtils.getSystemHttpsProxyUrl();
You can overwrite the values in system.properties with JVM parameter like
-Dhttps.proxyHost=yourproxy.com.

11.8. UITest Utilities

The UITestUtils supports you to generate additional screenshots.

You can add the screenshots into the report to the method steps.

Use the WebDriverManager within a test method
@Test
public void testDemo() {
    UITestUtils.takeScreenshot(WebDriverManager.getWebDriver(), true);
}
Within a page you can use the driver object.
class ExamplePage extends Page {
    public void doAnything() {
        UITestUtils.takeScreenshot(this.getWebDriver(), true);
    }
}
Take screenshots of all WebDriver instances of current test.
@Test
public void testDemo() {
    WEB_DRIVER_MANAGER.getWebDriver("session1");
    WEB_DRIVER_MANAGER.getWebDriver("session2");
    // Screenshots from 'session1' and 'session2' are created
    UITestUtils.takeScreenshots();
}
UITestUtils.takeScreenshots() takes only screenshot from WebDriver instances you have used in your current test.

You can also store a simple screenshot to your project directory.

Save a screenshot as a simple file.
@Test
public void testDemo() {
    ...
    // FileSystems.getDefault() returns the current working directory
    File path = FileSystems.getDefault().getPath("screen.png").toFile();
    UITestUtils.takeWebDriverScreenshotToFile(WebDriverManager.getWebDriver(), path);
    ...
}
Screenshots are always saved in the PNG image format.

11.9. Timer Utilities

The timer utilities provide some useful time related classes.

11.9.1. Sleepy interface

The Sleepy interface provides for proper sleep logging.

import eu.tsystems.mms.tic.testframework.utils.Sleepy;

class MyWorkflow implements Sleepy {
    public void doSomething() {
        sleep();        // Sleeps for an internal default timeout
        sleep(long);    // Sleeps for milliseconds
    }
}

This will log something like

MyWorkflow - sleep(200ms) on MyWorkflow@62b635fe

11.9.2. TimerUtils

If you want to pause your current test execution, because you may have to wait for something or need a hard timeout you can use the TimerUtils.

// Will pause current thread for 1 second.
TimerUtils.sleep(1000);

// You can pass a comment as well
TimerUtils.sleep(5000, "Wait for something to happen.");

Both of these methods calls of sleep() will produce a log message. If you want to sleep without a message, TimerUtils of Testerra provides a method sleepSilent().

11.9.3. Timer (deprecated)

Timings can be controlled using TestController, therefore this class is marked as @deprecated and should not be used anymore.

The Timer provides a basic timer feature, if you have to execute the same snippet of code in a defined interval. It stops after timeout or your sequence code was executed without any exceptions. If your sequence code was not successful the Timer will occur a TimeoutException.

Simple timer sequence
final Timer timer = new Timer(500, 15_000);
timer.executeSequence(new Timer.Sequence<Boolean>() {
    @Override
    public void run() throws Throwable {
        // sequence code here
    }
});

With this approach you will block your current thread, mostly your main thread.

If you want to execute your Sequence in another thread - we got you. Just use the executeSequenceThread method.

You can also return an object. In that case no TimeoutException will occur. Therefor you have to verify your returning object.

public MyObject runTimer() {
    final Timer timer = new Timer(500, 15_000);
    ThrowablePackedResponse<MyObject> myObjectResponse
        = timer.executeSequenceThread(new Timer.Sequence<MyObject>() {
        @Override
        public void run() throws Throwable {
            // sequence code here
            setPassState(boolean);  // exit the sequence if your condition is true before timeout
            setReturningObject(new MyObject());

        }
    });
    return myObjectResponse.getResponse();
}

11.10. WebDriver Utilities

11.10.1. Keep a session alive

Sometimes when interacting with a remote selenium grid provider you will get in touch with the given session timeout of the provider. If your session is idle, the grid provider will close your session due to resource reasons after a timeout set by the provider itself. This is a helpful feature for selenium grid providers, but maybe you just want to set your WebDriver session on hold, because you are interacting with some other WebDriver session in the same thread.

To keep a session alive while processing some other tasks in the main thread you have to send some interactions. For this you can use the managed methods in Testerra framework. This will help you to get your things done and will ensure that you can’t keep sessions alive forever to avoid grid abuse.

Keep a driver alive while interacting with second session
WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
// Do some test stuff with session your session

// Starting a second session for example to test concurrent interactions.
WebDriver driverWithOtherUser = WEB_DRIVER_MANAGER.getWebDriver("Session2");

// Keep alive driver while doing actions on driverWithOtherUser for 90 seconds, while refreshing all 10 seconds
WEB_DRIVER_MANAGER.keepAlive(driver, 10, 90);

// Do your things with your second driver

// NOTE: Please release your WebDriverKeepAliveSequence as you dont need the lock
WEB_DRIVER_MANAGER.stopKeepingAlive(driver);

11.11. JVM Utilities

This is an utility class that gives you access to the jvm system performance indicators.

Example on how to use the JVM utility.
// the jvm's performance indicators
int cpuPercent = JVMUtils.getCPUUsagePercent();
int memoryPercent = JVMUtils.getMemoryUsagePercent();
long usedMemory = JVMUtils.getUsedMemory();
long maximumMemory = JVMUtils.getUsedMemory();

System.out.println("cpu usage: " + cpuPercent + " %");
System.out.println("memory usage: " + memoryPercent + " %");
System.out.println("used memory: " + usedMemory);
System.out.println("maximum memory: " + maximumMemory);

// a typical output of these utility methods would be:
//
// cpu usage: 33 %
// memory usage: 20 %
// used memory: 370
// maximum memory: 370

12. Selenium 4

12.1. Testerra and Selenium 4

Since version 2.4 Testerra includes Selenium 4. It brings some new features like support of Chrome developer tools, but also some breaking changes.

12.2. Important changes

The Selenium devs provide a short migration guide for Selenium 4.

The following subsections describes how the changes affect Testerra.

12.2.1. Custom capabilities

Any capability that not in the standard of W3C needs a vendor prefix.

Otherwise, the session could not created and Selenium returns

...
WARNING: Support for Legacy Capabilities is deprecated; You are sending the following invalid capabilities: [foo]; Please update to W3C Syntax: https://www.selenium.dev/blog/2022/legacy-protocol-support/
...
java.lang.IllegalArgumentException: Illegal key values seen in w3c capabilities: [foo]
...
Example for custom capabilities compatible with Selenium 4
DesktopWebDriverRequest request = new DesktopWebDriverRequest();
MutableCapabilities caps = request.getMutableCapabilities();

// The custom caps have to packaged into an additional map and added to the request with a vendor prefix.
Map<String, Object> customCaps = Map.of("foo", "bar");
caps.setCapability("custom:caps", customCaps);

WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver(request);

12.2.2. Browser version

Be aware of using a browser version if you execute your test against a standalone Selenium 4 server. This is running as a local grid which knows nothing about any local browser versions.

Your test will run into a timeout!

12.3. Support of WebDriver BiDi API

The WebDriver bidirectional protocol allows to communicate via websocket with the browser. Webdriver BiDi gives more control about your session and you are able to access to the developer tools of Chrome, Firefox or Edge.

12.3.1. Listen to console logs with LogInspector

The browser console window shows different types of logging information. You have to know which kind of logs you need.

Type of log entries Class for LogInspector

Console logs

ConsoleLogEntry

JavaScript exceptions

JavascriptLogEntry

Console logs and JS exceptions

LogEntry

The created consumer for all listeners will be executed in an additional thread.
If you use an Assert this will be no impact to the main thread.
Example for listening to all console logs
public class WebDriverBiDiTests extends TesterraTest implements
        Loggable,
        WebDriverManagerProvider,
        UiElementFinderFactoryProvider {

    @Test
    public void testT01_LogListener_ConsoleLogs() throws MalformedURLException {
        DesktopWebDriverRequest request = new DesktopWebDriverRequest();
        request.setBrowser(Browsers.chrome);
        request.setBaseUrl(
                "https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html"
        );
        // Important to activate websocket communication
        request.getMutableCapabilities().setCapability("webSocketUrl", true);
        WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver(request);

        UiElementFinder uiElementFinder
            = UI_ELEMENT_FINDER_FACTORY.create(webDriver);

        // You need the RemoteWebDriver or the ChromeDriver instance
        // Testerra generates a driver instance of type EventFiringWebDriver
        // which does not work for LogInspector
        Augmenter augmenter = new Augmenter();
        RemoteWebDriver remoteWebDriver
            = WEB_DRIVER_MANAGER.unwrapWebDriver(webDriver, RemoteWebDriver.class).get();
        WebDriver remoteDriver = augmenter.augment(remoteWebDriver);
        LogInspector logInspector = new LogInspector(remoteDriver);

        // Look for the type of log entries you want to catch
        List<LogEntry> logEntryList = new ArrayList<>();
        logInspector.onConsoleEntry(logEntryList::add);

        uiElementFinder.find(By.id("consoleLog")).click();
        uiElementFinder.find(By.id("consoleError")).click();
        uiElementFinder.find(By.id("jsException")).click();
        uiElementFinder.find(By.id("logWithStacktrace")).click();

        // To make sure we have at least 4 elements in our list
        CONTROL.retryTimes(5, () -> {
            ASSERT.assertTrue(logEntryList.size() >= 4);
            TimerUtils.sleepSilent(1000);
        });

        logEntryList.forEach(logEntry -> {
            AtomicReference<GenericLogEntry> genericLogEntry = new AtomicReference<>();
            // 'LogEntry' is only a container for different types of logs.
            logEntry.getConsoleLogEntry().ifPresent(genericLogEntry::set);
            logEntry.getJavascriptLogEntry().ifPresent(genericLogEntry::set);

            log().info("LOG_ENTRY: {} {} {} {} - {}",
                    genericLogEntry.get().getTimestamp(),
                    genericLogEntry.get().getType(),
                    genericLogEntry.get().getLevel(),
                    genericLogEntry.get().getType(),
                    genericLogEntry.get().getText()
            );
        });

        ASSERT.assertEquals(logEntryList.size(), 4, "LogEntry list");
    }

}

12.4. Support of Chrome developer tools (CDP)

Selenium 4 supports the access to the Chrome developer tools. Testerra provides a simple API to interact with CDP.

  • As the name says, it only works in Chrome browser. :-)

  • This Testerra CDP utility uses the native Chrome implementation. It is the most flexible usage of CDP, but the API could change with further Selenium versions.

  • If you’re running your tests against a grid, check the CDP compatible Chrome versions in Selenium release notes

12.4.1. Set basic authentication

import org.openqa.selenium.UsernameAndPassword;
...

public class ChromeDevToolsTests extends TesterraTest implements
        ChromeDevToolsProvider,
        WebDriverManagerProvider,
        UiElementFinderFactoryProvider {

    @Test
    public void test_CDP_BasicAuthentication() {
        WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver();
        UiElementFinder uiElementFinder = UI_ELEMENT_FINDER_FACTORY.create(webDriver);

        // Credentials are used for all further website calls in your webdriver session
        CHROME_DEV_TOOLS
            .setBasicAuthentication(webDriver, UsernameAndPassword.of("admin", "admin"));

        webDriver.get("https://the-internet.herokuapp.com/basic_auth");
        uiElementFinder.find(By.tagName("p"))
            .assertThat().text().isContaining("Congratulations");
    }

}

You can restrict the basic authentication to special hosts.

CHROME_DEV_TOOLS
    .setBasicAuthentication(
            webDriver,
            UsernameAndPassword.of("admin", "admin"),
            "herokuapp.com",
            "example.com");

12.4.2. Change geolocation

public class ChromeDevToolsTests extends TesterraTest implements
        ChromeDevToolsProvider,
        WebDriverManagerProvider,
        UiElementFinderFactoryProvider {

    @Test
    public void test_CDP_GeoLocation() {
        WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver();
        UiElementFinder uiElementFinder = UI_ELEMENT_FINDER_FACTORY.create(webDriver);

        CHROME_DEV_TOOLS.setGeoLocation(
                webDriver,
                52.52084,      // latitude
                13.40943,      // longitude
                1);            // accuracy

        webDriver.get("https://my-location.org/");
        uiElementFinder.find(By.xpath("//button[@aria-label = 'Consent']"))
            .click(); // Cookie disclaimer
        uiElementFinder.find(By.id("latitude"))
            .assertThat().text().isContaining("52.52084");
        uiElementFinder.find(By.id("longitude"))
            .assertThat().text().isContaining("13.40943");
    }

}

12.4.3. Listen to browser console

The browser console window shows different types of logging information. You have to know which kind of logs you need.

The created consumer for all listeners will be executed in an additional thread.
If you use an Assert this will be no impact to the main thread.
JavaScript logging information
import org.openqa.selenium.devtools.events.ConsoleEvent;
...

public class ChromeDevToolsTests extends TesterraTest implements
        ChromeDevToolsProvider,
        Loggable,
        WebDriverManagerProvider,
        UiElementFinderFactoryProvider {

    @Test
    public void test_CDP_LogListener_JsLogs() throws MalformedURLException {
        DesktopWebDriverRequest request = new DesktopWebDriverRequest();
        request
            .setBaseUrl("https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html");
        WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver(request);
        DevTools devTools = CHROME_DEV_TOOLS.getRawDevTools(webDriver);

        // Create a list for console events
        List<ConsoleEvent> consoleEvents = new ArrayList<>();
        // Create a consumer and add them to a listener
        Consumer<ConsoleEvent> addEntry = consoleEvents::add;
        devTools.getDomains().events().addConsoleListener(addEntry);

        UiElementFinder uiElementFinder = UI_ELEMENT_FINDER_FACTORY.create(webDriver);
        uiElementFinder.find(By.id("consoleLog")).click();
        uiElementFinder.find(By.id("consoleError")).click();

        consoleEvents.forEach(event ->
                log().info(
                        "JS_LOGS: {} {} - {}",
                        event.getTimestamp(),
                        event.getType(),
                        event.getMessages().toString())
        );
    }
}
JavaScript exception logs
import org.openqa.selenium.JavascriptException;
...

public class ChromeDevToolsTests extends TesterraTest implements
        ChromeDevToolsProvider,
        Loggable,
        WebDriverManagerProvider,
        UiElementFinderFactoryProvider {

    @Test
    public void testT_CDP_LogListener_JsExceptions() throws MalformedURLException {
        DesktopWebDriverRequest request = new DesktopWebDriverRequest();
        request
            .setBaseUrl("https://www.selenium.dev/selenium/web/bidi/logEntryAdded.html");
        WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver(request);
        DevTools devTools = CHROME_DEV_TOOLS.getRawDevTools(webDriver);

        // Create a list for JS exceptions
        List<JavascriptException> jsExceptionsList = new ArrayList<>();
        Consumer<JavascriptException> addEntry = jsExceptionsList::add;
        devTools.getDomains().events().addJavascriptExceptionListener(addEntry);

        UiElementFinder uiElementFinder = UI_ELEMENT_FINDER_FACTORY.create(webDriver);
        uiElementFinder.find(By.id("jsException")).click();
        uiElementFinder.find(By.id("logWithStacktrace")).click();

        jsExceptionsList.forEach(jsException ->
                log().info(
                        "JS_EXCEPTION: {} {}",
                        jsException.getMessage(),
                        jsException.getSystemInformation()
                )
        );
    }

}
'Broken' page resources
import org.openqa.selenium.devtools.v137.log.model.LogEntry;
...

public class ChromeDevToolsTests extends TesterraTest implements
        ChromeDevToolsProvider,
        Loggable,
        WebDriverManagerProvider {

    @Test
    public void test_CDP_LogListener_BrokenImages() {
        WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver();
        DevTools devTools = CHROME_DEV_TOOLS.getRawDevTools(webDriver);
        devTools.send(Log.enable());

        List<LogEntry> logEntries = new ArrayList<>();
        Consumer<LogEntry> addedLog = logEntries::add;
        devTools.addListener(Log.entryAdded(), addedLog);

        webDriver.get("http://the-internet.herokuapp.com/broken_images");
        TimerUtils.sleep(1000);     // Short wait to get delayed logs

        logEntries.forEach(logEntry ->
                log().info(
                        "LOG_ENTRY: {} {} {} - {} ({})",
                        logEntry.getTimestamp(),
                        logEntry.getLevel(),
                        logEntry.getSource(),
                        logEntry.getText(),
                        logEntry.getUrl()
                )
        );
    }

}

12.4.4. Listen to Network logs

import org.openqa.selenium.devtools.v137.network.Network;
import org.openqa.selenium.devtools.v137.network.model.RequestWillBeSent;
import org.openqa.selenium.devtools.v137.network.model.ResponseReceived;
...

public class ChromeDevToolsTests extends TesterraTest implements
        ChromeDevToolsProvider,
        Loggable,
        WebDriverManagerProvider {

    @Test
    public void test_CDP_NetworkListener() {
        WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver();
        DevTools devTools = CHROME_DEV_TOOLS.getRawDevTools(webDriver);
        devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));

        // Create lists for requests and responses
        List<ResponseReceived> responseList = new ArrayList<>();
        List<RequestWillBeSent> requestList = new ArrayList<>();

        devTools.addListener(Network.responseReceived(), response -> responseList.add(response));
        devTools.addListener(Network.requestWillBeSent(), request -> requestList.add(request));

        webDriver.get("https://the-internet.herokuapp.com/broken_images");

        requestList.forEach(request ->
                log().info(
                        "Request: {} {} - {}",
                        request.getRequestId().toString(),
                        request.getRequest().getMethod(),
                        request.getRequest().getUrl()
                )
        );

        responseList.forEach(response ->
                log().info(
                        "Response: {} {} - {}",
                        response.getRequestId().toString(),
                        response.getResponse().getStatus(),
                        response.getResponse().getStatusText()
                )
        );
    }

}

12.4.5. Set device emulation

There is a simple implementation to emulate mobile devices.

public class ChromeDevToolsTests extends TesterraTest implements
        ChromeDevToolsProvider,
        WebDriverManagerProvider {

    @Test
    public void test_CDP_GeoLocation() {
        WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver();

        CHROME_DEV_TOOLS.setDevice(
                webDriver,
                new Dimension(400, 900),    // resolution
                100,                        // Scale factor
                true);                      // it's a mobile device

        webDriver.get("...");
    }

}

If you need some more impact on device settings, you can use the origin method

WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver();
DevTools devTools = CHROME_DEV_TOOLS.getRawDevTools(webDriver);
devTools.send(Emulation.setDeviceMetricsOverride(...);

12.4.6. Manipulate browser requests

Change the web requests of your browser:

public class ChromeDevToolsTests extends TesterraTest implements
        ChromeDevToolsProvider,
        UiElementFinderFactoryProvider,
        WebDriverManagerProvider {

    // https://weatherstack.com/ uses your client IP address to find out your location.
    // There is a REST api call to https://weatherstack.com/ws_api.php?ip=<ip> to get
    // the local weather information.
    // This test updates the REST api call with a static public IP address
    @Test
    public void testCDP_Network_changeRequest() {
        WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver();
        DevTools rawDevTools = CHROME_DEV_TOOLS.getRawDevTools(webDriver);
        final String location1 = "213.136.89.121";  // free German proxy server in Munich

        rawDevTools.send(Fetch.enable(Optional.empty(), Optional.empty()));
        rawDevTools.addListener(Fetch.requestPaused(), requestConsumer -> {
            Request request = requestConsumer.getRequest();
            String currentUrl = request.getUrl();
            if (currentUrl.contains("ws_api.php?ip=")) {
                String updatedUrl = currentUrl.substring(0, currentUrl.indexOf("?"))
                        + "?ip=" + location1;
                rawDevTools.send(
                        Fetch.continueRequest(
                                requestConsumer.getRequestId(),
                                Optional.of(updatedUrl),
                                Optional.empty(),
                                Optional.empty(),
                                Optional.empty(),
                                Optional.empty()));
            } else {
                // All other requests will be sent without any change
                rawDevTools.send(
                        Fetch.continueRequest(
                                requestConsumer.getRequestId(),
                                Optional.of(currentUrl),
                                Optional.empty(),
                                Optional.empty(),
                                Optional.empty(),
                                Optional.empty()));

            }

        });


        webDriver.get("https://weatherstack.com/");

        UiElementFinder uiElementFinder = UI_ELEMENT_FINDER_FACTORY.create(webDriver);
        uiElementFinder.find(By.xpath("//div[@id = 'cookiescript_accept']")).click();
        UiElement weatherLocation = uiElementFinder
                .find(By.xpath("//span[@data-api = 'location']"));
        weatherLocation.assertThat().text().isContaining("Munich");
    }
}

13. JVM

13.1. JVMMonitor

The JVMMonitor is the Observer of the hardware utilization for memory and cpu. With the start of a test while using the TesterraListener the latter implicitly starts the JVMMonitor. Thus a concurrent thread for monitoring purposes only is initiated next to the actual test execution. Every ten seconds the following parameters are logged at DEBUG Level.

  • JVM Memory usage in MB

  • JVM Memory reserved in MB

  • JVM CPU usage in per cent

The JVMMonitor is automatically terminated after the test execution and a graph showing the memory consumption is put into the report.

13.2. JVMExitHook

The JVMExitHook was integrated to achieve the generation of a report with Report-NG in case an aborted test run. With the start of a test execution the JVMExitHook is added as a shutdown hook to the JVM.

When the JVM is about to stop, shutdown hooks are always called by default. This includes cases of abortion, all other system errors and normal finish. In the latter case a flag indicating an already created report is set true once in the process of the report generation. To avoid overriding the already existing report, the JVMExitHook only triggers when this flag is false, as this is the only indicator of an unexpected exit and a missing report. It then sends the ExecutionAbortEvent, which is then caught by the corresponding listeners creating a report.


Extending Testerra

14. Extending Testerra

Testerra provides several ways for extensions.

14.1. Modules

To register your module, you need to create a module injection configuration based on the Google Guice framework.

package io.testerra.myproject.mymodule;

import com.google.inject.AbstractModule;

public class ConfigureMyModule extends AbstractModule {
    @Override
    protected void configure() {
        // inject dependencies here
    }
}
Please be aware, that the package namespace prefix io.testerra is required in order to find your module configuration.
All module configuration instances are loaded in alphabetical order.

14.2. Hooks

When you need more module features, like registering to Events or perform setup and teardown functionality, you can implement the ModuleHook interface.

package io.testerra.myproject.mymodule;

import eu.tsystems.mms.tic.testframework.hooks.ModuleHook;

public class ConfigureMyModule extends AbstractModule implements ModuleHook {
    @Override
    public void init() {
        //
    }

    @Override
    public void terminate() {
        //
    }
}

While the init() method is called on startup of Testerra as one of the earliest steps, you are able to make your customizations as soon as possible.

The terminate() method is called at the most latest point for Testerra right before terminating the execution. You should use this method to cleanup your module, if necessary, for example closing database connections in a database module.

14.3. Events and Listeners

Testerra provides a Google Guava event bus with custom events for test execution. The following events are currently implemented:

Event Description

MethodStartEvent

Called on start of every test method annotated by TestNG @Test annotation.

MethodEndEvent

Called at the end of every test method annotated by TestNG @Test annotation.

TestStatusUpdateEvent

Called after the final result of every test method annotated by TestNG @Test annotation. A final result of a test method can be PASSED, RETRIED, RECOVERED, SKIPPED, FAILED or EXPECTED_FAILED.

This event can be used to send test results to a test management system or issue tracker.

ExecutionFinishEvent

Called at the end of test run to trigger report generation and other output worker.

ExecutionAbortEvent

Called on test run abortion due to unclear circumstances like hanging sessions, JVM exit or other. Used to create report with existing execution information.

InterceptMethodsEvent

Called before suite execution. The events methods list provides a list of tests to execute. Read more about this in Intercept test method execution

ContextUpdateEvent

Called every time the internal context data has been changed significantly.

FinalizeExecutionEvent

Called on the very end of the test execution when the execution model has been finalized. Use this event to generate a report.

14.3.1. Create custom event listeners

The simplest way to get in touch with the event bus is to write and register your own implementation of the event’s Listener interface and add the @Subscribe annotation.

Simple event listener based on TestStatusUpdateEvent
import com.google.common.eventbus.Subscribe;
import eu.tsystems.mms.tic.testframework.events.TestStatusUpdateEvent;
import eu.tsystems.mms.tic.testframework.report.model.context.MethodContext;
import eu.tsystems.mms.tic.testframework.logging.Loggable;

public class MyStatusListener implements TestStatusUpdateEvent.Listener, Loggable {

    @Override
    @Subscribe
    public void onTestStatusUpdate(TestStatusUpdateEvent event) {
        MethodContext methodContext = event.getMethodContext();
        log().info(
                String.format("%s has the status %s", methodContext.getName(),
                        methodContext.getStatus())
        );
    }
}

If you want to react to some more events, you can just implement multiple interfaces.

14.3.2. Register custom event listener

After you defined your first custom listener you now have to register it to the TesterraListener.

Registering your listener
Testerra.getEventBus().register(new LogEventListener());

14.3.3. Fire events by yourself

While Implementing your own module you may reach a point, where you want to inform other components or modules about an important event. You can achieve this by just posting this event to the bus.

For example, if your module changes some data in the underlying data model, you have to inform all other "participants" about your change by firing an ContextUpdateEvent event.

// Update some data in data model...
methodContext.name = "new_Test_Method_Name";

Testerra.getEventBus().post(new ContextUpdateEvent().setContext(methodContext));

14.3.4. Create custom event types

If you want to provide some custom events to other modules. You can implement these by implementing any kind of class for the event bus.

Creating custom event types
public class CustomEvent {
    public interface Listener {
        void onCustomEvent(CustomEvent event);
    }
}

With your CustomEvent created, you now can fire these events or react to them in the way described in the sections Fire events by yourself and Create custom event listeners.

CustomEvent Listener Example
@Override
@Subscribe
public void onCustomEvent(CustomEvent event) {
   log().info("Custom Event started!");
}

14.3.5. Intercept test method execution

With the test InterceptMethodsEvent, you are able to modify the list of tests being executed before execution.

import eu.tsystems.mms.tic.testframework.events.InterceptMethodsEvent;

public class MyTest extends TesterraTest {

    public class MyListener implements InterceptMethodsEvent.Listener {

        @Override
        @Subscribe
        public void onInterceptMethods(InterceptMethodsEvent event) {
            event.getMethodInstances().removeIf(iMethodInstance -> true);
        }
    }

    @BeforeTest
    public void setupListener() {
        Testerra.getEventBus().register(new MyListener());
    }
}

14.3.6. Listen to TestNG events

Since the TesterraListener listens to TestNG events, it also forwards some of these events the same way like any other events.

import eu.tsystems.mms.tic.testframework.logging.Loggable;
import com.google.common.eventbus.Subscribe;
import org.testng.ISuite;
import org.testng.ISuiteListener;

class MySuiteListener implements ISuiteListener, Loggable {

    @Subscribe
    @Override
    public void onStart(ISuite suite) {
        log().info("Suite started");
    }
}

14.4. Provide Properties

When you want to provide some properties, you can use the IProperties interface within your module.

package eu.tsystems.mms.tic.mymodule;

import eu.tsystems.mms.tic.testframework.common.IProperties;

public class MyModule {

    public enum Properties implements IProperties {
        GREETING("greeting", "hello world"),
        ENABLED("enabled", false),
        ANSWER("answer", 42),
        ;
        private final String property;
        private final Object defaultValue;

        Properties(String property, Object defaultValue) {
            this.property = property;
            this.defaultValue = defaultValue;
        }

        @Override
        public String toString() {
            return String.format("tt.mymodule.%s",property);
        }

        @Override
        public Object getDefault() {
            return defaultValue;
        }
    }
}

Override the default values in a .properties file.

tt.mymodule.greeting=hello planet
tt.mymodule.enabled=true

And access them in your code like

MyModule.Properties.GREETING.toString();    // tt.mymodule.greeting
MyModule.Properties.GREETING.asString();    // hello planet
MyModule.Properties.ENABLED.asBool();       // true
MyModule.Properties.ANSWER.asLong();        // 42

14.5. Create a report

When you want to create a custom report, you have to add the report-model module as a dependency to your module and listen to the FinalizeExecutionEvent.

import com.google.common.eventbus.Subscribe;
import eu.tsystems.mms.tic.testframework.events.FinalizeExecutionEvent;
import eu.tsystems.mms.tic.testframework.logging.Loggable;

public class GenerateReportListener implements FinalizeExecutionEvent.Listener, Loggable {

    @Override
    @Subscribe
    public void onFinalizeExecution(FinalizeExecutionEvent event) {
        log().info("Generate report");
    }
}

14.5.1. Generate the protobuf report model

Testerra ships Google Protobuf model adapters for the internal context model. You can automatically generate all the models during the execution, when you register the AbstractReportModelListener in your module hook.

import com.google.common.eventbus.EventBus;
import eu.tsystems.mms.tic.testframework.listeners.AbstractReportModelListener;
import eu.tsystems.mms.tic.testframework.hooks.ModuleHook;
import eu.tsystems.mms.tic.testframework.report.TesterraListener;

public class CustomReportModuleHook implements ModuleHook {

    @Override
    public void init() {
        Report report = Testerra.getInjector().getInstance(Report.class);
        EventBus eventBus = Testerra.getEventBus();
        eventBus.register(new GenerateReportModelListener(report.getReportDirectory()));
    }
}

This will generate Protobuf models in test-report/models.


Appendix

15. Best Practices

This section contains several articles for best practices for the usage and implementation of tests with Testerra.

15.1. Handling of WebDriver instances

We strongly recommend the following rules for dealing with WebDriver instances:

Create or call your WebDriver instances only within the context of test or setup methods.

✅ Good practice

@BeforeMethod
public void before() {
    // Context of a setup method
    WebDriver driver = WebDriverManager.getWebDriver();
}

@Test
public void myDemoTest() {
    // Context of a test method
    WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();
    ...
}

❌ Bad practice - don’t do this

public class MyTest extends TesterraTest
        implements WebDriverManagerProvider, PageFactoryProvider {

    // This may cause side effects!
    // The session information are missing in the report because
    // your new session cannot link to the method context
    WebDriver driver = WEB_DRIVER_MANAGER.getWebDriver();

    @Test
    public void myDemoTest() {
        StartPage startPage = PAGE_FACTORY.createPage(StartPage.class, driver);
        ...
    }
}

Only store the names of WebDriver instances, not the whole object.

✅ Good practice

public class MyTest extends TesterraTest implements WebDriverManagerProvider {

    private String exclusiveSessionId = "";
    private String normalSesion = "mySession"

    @BeforeClass
    public void beforeClass() {
        WebDriver driver1 = WEB_DRIVER_MANAGER.getWebDriver();
        exclusiveSessionId = WEB_DRIVER_MANAGER.makeExclusive(driver1);

        // Create a session with a custom name
        WebDriver driver2 = WEB_DRIVER_MANAGER.getWebDriver(this.normalSesion);
        ...
    }

    @Test
    public void myDemoTest() {
        // Always use the WEB_DRIVER_MANAGER
        WebDriver driver1 = WEB_DRIVER_MANAGER.getWebDriver(this.exclusiveSessionId);
        WebDriver driver2 = WEB_DRIVER_MANAGER.getWebDriver(this.normalSesion);
        ...
    }
}

15.2. Browser specific knowledge

15.2.1. Firefox

Prevent Firefox download confirmation dialog
WEB_DRIVER_MANAGER.setUserAgentConfig(Browsers.firefox, (FirefoxConfig) options -> {
    // You have to add every mimetype you want no confirmation for
    options.addPreference("browser.helperApps.neverAsk.saveToDisk", "application/zip");
    options.addPreference("browser.download.manager.showAlertOnComplete", false);
    options.addPreference("browser.download.manager.showWhenStarting", false);
});

15.2.2. Chrome

Chrome in a container

If using Selenium with Chrome in a Docker container it may comes to some random WebDriverException, because of some internal container limits.

If you are getting random selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash, may this code snippet will solve your problem, by disabling the usage of dev/shm memory and enabling the usage of /tmp instead. This will may slow down your execution time, because you are using disk instead of memory. The reason is, that chrome / chrome driver leads to errors when /dev/shm is too small.

Java snippet for test classes
WEB_DRIVER_MANAGER.setUserAgentConfig(Browsers.chromeHeadless, (ChromeConfig) options -> {
    options.addArguments("--disable-dev-shm-usage");
});

15.2.3. Internet Explorer

Skip certificate warning page

When testing websites with own certificates you may encounter issues and warning pages in each browser you use. For Chrome or Firefox these warnings can be skipped by setting properties, but for the Internet Explorer you have to define a small helper method, that you can call right after opening the base url.

public void skipInternetExplorerSecurityWarning(WebDriver driver, boolean handleAlert) {
    driver.navigate().to("javascript:document.getElementById('overridelink').click()");
    if (handleAlert) {
        driver.switchTo().alert().accept();
    }
}

15.3. Working with HTML elements

15.3.1. Radio buttons

Since radio buttons share the same name, as the following example shows

<input type="radio" name="beverage" value="tee">
<input type="radio" name="beverage" value="coffee">

It’s not a good practice to select it By.name. It’s better to select both options separately.

// Good practice
PreparedLocator locator = LOCATE.prepare("//input[@name='%s' and @value='%s']");
UiElement teeOption = find(locator.with("beverage", "tee"));
UiElement coffeeOption = find(locator.with("beverage", "coffee"));

// Bad practice
UiElement options = find(By.name("beverage"));

15.3.2. Shadow Roots

Modern web applications are allowed to use some third-party components, which can be integrated with Shadow DOM. This is the modern art of an iframe, because the components will be loaded via asynchronous JavaScript.

Each embedded Shadow DOM component will have its own shadow root. To work with shadow root elements Testerra provide the method shadowRoot() on the UiElement class.

Testerra uses the Selenium 4 function webElement.getShadowRoot() to find child elements of a shadow root.

Given the following HTML code snippet it will be easier how to get the corresponding UiElement of the Shadow DOM component.

HTML Code
<body>
    <div id="wrapper">
    <!-- HTML code-->
    <my-custom-shadow-root-element>
    <!-- #shadowRoot -->
        <div class="custom-component">
            <input id="custom-component-login-name" name="name">
        </div>
    </my-custom-shadow-root-element>
    </div>
    <!-- HTML code-->
</body>
Java Code
UiElement shadowRootElement = find(By.cssSelector("my-custom-shadow-root-element")).shadowRoot();
// You have to use CSS selectors for child elements.
UiElement input = shadowRootElement.find(By.cssSelector("#custom-component-login-name"));
To access child elements of shadow roots Selenium 4 only allows By.cssSelector. Using any other By Testerra occurs an exception.

15.4. Support multiple profiles

Supporting multiple profiles is useful, for different environments or test suites.

build.gradle
test {
    def profiles = [
        "mySuite": "suite.xml"
    ]

    def suiteFiles = []
    profiles.each { k, v ->
        if (project.hasProperty("" + k)) {
            def f = 'src/test/resources/' + v
            suiteFiles << f
        }
    }

    useTestNG() {
        suites(suiteFiles as String[])
    }
}
pom.xml
<project>
   <profiles>
        <profile>
            <id>mySuite</id>
            <build>
                <plugins>
                    <plugin>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <configuration>
                            <skip>false</skip>
                            <suiteXmlFiles>
                                <suiteXmlFile>src/test/resources/suite.xml</suiteXmlFile>
                            </suiteXmlFiles>
                        </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>
</project>
Gradle
gradle test -PmySuite
Maven
mvn test -PmySuite

15.5. Test on multiple environments

If you run your tests on multiple test environments, you need for every environment a specific set of settings, for example another tt.baseurl or tt.browser.

15.5.1. Define your properties

Your test.properties describes your default setting. If you want to change some properties, you can define another property file and locate it into the resource folder.

test.properties
tt.browser=chrome
tt.baseurl=http://example.org
test_qa.properties with specific settings for a QA environment
tt.browser=firefox
tt.baseurl=http://qa.example.org

15.5.2. Load specific properties at startup

Load your custom property file at test startup if its necessary.

Example to load properties in any setup method
@BeforeSuite
public void setup() {
    String env = PROPERTY_MANAGER.getProperty("env", "");
    if (!"".equals(env)) {
        PROPERTY_MANAGER.loadProperties("test_" + env + ".properties");
    }
}

15.5.3. What happens?

Testerra is loading the test.properties automatically at its initialization. Loading another property file will overwrite already existing values.

If you add your env property at the Gradle or Maven call, you can control the execution depending on the test environment.

Gradle example
gradle test -Denv=qa

15.6. Project setup

Project setup recommendations.

15.6.1. Normalize file encoding

Always use the same file encoding for all your source file types like .html, .properties, .java etc.

IntelliJ: You can change the default encoding for files at File → Settings → File Encodings

15.6.2. Normalize file endings

Prevent using default line ending CRLF on Windows.

Git: Setup git config --global core.autocrlf input to prevent any commited CRLF.

15.7. Debugging tests

When you want to debug tests in your IDE, you can use the following setup to debug tests after they failed.

// Check if the JVM is in Debug mode
boolean isDebug = java.lang.management.ManagementFactory.getRuntimeMXBean().getInputArguments().toString().indexOf("-agentlib:jdwp") > 0;
if (isDebug) {
    // Disable close windows after test
    WEB_DRIVER_MANAGER.getConfig().setShutdownSessionAfterTestMethod(false);

    // Register to the end event
    EventBus eventBus = Testerra.getEventBus();
    eventBus.register(new MethodEndEvent.Listener() {
        @Override
        @Subscribe
        public void onMethodEnd(MethodEndEvent event) {
            if (event.isFailed()) {
                // Set your breakpoint here
                log().error("Failed");
            }
        }
    });
}

15.8. Access to Chrome dev tools

  • The following examples are working with Selenium 4 server. If you are using another grid like Selenoid the usage could be different.

  • Only Chrome browser supports access to development tools.

15.8.1. Emulate geo location

Local webdriver
WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver();

ChromeDriver chromeDriver = WEB_DRIVER_MANAGER
        .unwrapWebDriver(webDriver, ChromeDriver.class).get();

DevTools devTools = chromeDriver.getDevTools();
devTools.createSession();
devTools.send(Emulation.setGeolocationOverride(
        Optional.of(52.52084),
        Optional.of(13.40943),
        Optional.of(1)));

webDriver.get("https://my-location.org/");      // page gets the new geo location information
Remote webdriver (e.g. of a grid)
WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver();

RemoteWebDriver remoteWebDriver = WEB_DRIVER_MANAGER
        .unwrapWebDriver(webDriver, RemoteWebDriver.class).get();
webDriver = new Augmenter().augment(remoteWebDriver);

DevTools devTools = ((HasDevTools) webDriver).getDevTools();
devTools.createSession();

devTools.send(Emulation.setGeolocationOverride(
        Optional.of(52.52084),
        Optional.of(13.40943),
        Optional.of(1)));

webDriver.get("https://my-location.org/");

15.8.2. Basic authentication

WebDriver webDriver = WEB_DRIVER_MANAGER.getWebDriver();
WebDriver remoteWebDriver = WEB_DRIVER_MANAGER.unwrapWebDriver(webDriver, RemoteWebDriver.class).get();

AtomicReference<DevTools> devToolsAtomicReference = new AtomicReference<>();

remoteWebDriver = new Augmenter()
        .addDriverAugmentation(
                "chrome",
                HasAuthentication.class,
                (caps, exec) -> (whenThisMatches, useTheseCredentials) -> {
                    devToolsAtomicReference.get().createSessionIfThereIsNotOne();
                    devToolsAtomicReference.get().getDomains()
                            .network()
                            .addAuthHandler(whenThisMatches, useTheseCredentials);
                })
        .augment(remoteWebDriver);

DevTools devTools = ((HasDevTools) remoteWebDriver).getDevTools();
devTools.createSession();
devToolsAtomicReference.set(devTools);

// Set credentials and call the page with 'basic authentication' protection
((HasAuthentication) remoteWebDriver).register(UsernameAndPassword.of("admin", "admin"));
webDriver.get("https://the-internet.herokuapp.com/basic_auth");

16. Migration from Testerra 1

The main goal for re-inventing the GuiElement API was at first, to provide a better user experience for implementing tests scripts in the role of a Test Engineer.

Automated testing on web pages is hard, especially on large enterprise environments we address with Testerra. And yes, we know the trouble what happens when tests (suddenly) fails.

As an Test Engineer, you may want expressive info messages for bugs in the System under Test or to keep your tests up to date to the specification. You may need easy to understand test code and a robust extensible Page Object model for large test environments by keeping the full potential of a Software Developer?

The new Testerra API provides it all.

If you’re a developer and want to know how the new API works, you can jump straight to The internals or just proceed with the next chapter to get an overview.

16.1. Page object changes

When implementing the PageFactoryProvider interface, you get an PAGE_FACTORY instance within your class.

import eu.tsystems.mms.tic.testframework.testing.PageFactoryProvider;

class MyTest extends TesterraTest implements PageFactoryProvider, WebDriverManagerProvider {

    @Test
    public void test_MyPageTest() {
        MyPage page = PAGE_FACTORY.createPage(MyPage.class);

        // Or using a different WebDriver
        MyPage page = PAGE_FACTORY.createPage(MyPage.class, WEB_DRIVER_MANAGER.getWebDriver());
    }
}

but you can instantiate pages within pages more easily:

class MyPage extends Page {
    public AnotherPage navigateToAnotherPage() {
        return createPage(AnotherPage.class);
    }
}
Using the static PageFactory is now @deprecated
MyPage page = PageFactory.create(MyPage.class, WebDriverManager.getWebDriver());
Passing variables to the Page constructor is now @deprecated
MyPage page = PageFactory.create(
    MyPage.class,
    WebDriverManager.getWebDriver(),
    new PageVariables()
);
Constructor instantiation of Pages is prohibited!
MyPage page = new MyPage(WebDriverManager.getWebDriver());

16.1.1. Components pattern

The new standard way to implement Sub Pages aka Components is now

public class MyForm extends AbstractComponent<MyForm> {
    public MyForm(UiElement rootElement) {
        super(rootElement);
    }
}

Instantiate components

class MyPage extends Page {
    private MyForm form = createComponent(MyForm.class, find(By.tagName("form")));
}

16.1.2. Implicit Element checks

The standard way of implicit GuiElement checks is now

class MyPage extends Page {
    @Check
    private UiElement uiElement = findById(42);
}
Performing explicit page checks is prohibited!
class MyPage extends Page {
    public MyPage(WebDriver webDriver) {
        super(webDriver);
        checkPage(); (1)
    }
}

MyPage page = PAGE_FACTORY.create(MyPage.class);
page.checkPage(); (2)
1 Calling checkPage() as protected member is prohibited
2 Calling checkPage() as public member is prohibited

16.2. Responsive PageFactory

The responsive page factory features have been removed from the default implementation. To use it anyway, you need to inject in in your Modules configuration.

import eu.tsystems.mms.tic.testframework.pageobjects.internal.ResponsivePageFactory;

public class MyModule extends AbstractModule {
    protected void configure() {
        bind(PageFactory.class).to(ResponsivePageFactory.class).in(Scopes.SINGLETON);
    }
}

16.3. Element creation changes

The new standard way to instantiate GuiElements is now

class MyPage extends Page {
    private UiElement uiElement = findById(42);
    private UiElement uiElement = find(By.xpath("//div[1]"));
}
Constructor instantiation of GuiElements is now @deprecated
class MyPage extends Page {
    private GuiElement guiElement = new GuiElement(driver, By.xpath("//div[1]"));
}

For descendant elements

class MyPage extends Page {
    private UiElement parent = findById(42);
    private UiElement sub = parent.find(By.xpath("//div[1]"));
}
getSubElement is now @deprecated
class MyPage extends Page {
    private GuiElement parent = new GuiElement(By.id(42), driver);
    private GuiElement sub = parent.getSubElement(By.xpath("//div[1]"));
}

List elements

UiElement anchors = find(By.tagName("a"));

anchors.expect().foundElements().is(3); (1)
UiElementList<UiElement> list = anchors.list();
list.first().expect().value(Attribute.TITLE).is("StartPage"); (2)
list.get(1).expect().value(Attribute.TITLE).is("About Us"); (3)
list.last().expect().value(Attribute.TITLE).is("Contact"); (4)

list.forEach(anchor -> anchor.expect().value(Attribute.HREF).startsWith("https")); (5)
GuiElement lists are now @deprecated
GuiElement anchors = new GuiElement(driver, By.tagName("a"));

Assert.assertEquals(anchors.getNumberOfFoundElements(), 3); (1)

List<GuiElement> list = anchor.getList();
list.get(0).asserts().assertAttributeValue("title", "StartPage"); (2)
list.get(1).asserts().assertAttributeValue("title", "About Us"); (3)
list.get(list.size()-1).asserts().assertAttributeValue("title", "Contact"); (4)

list.forEach(anchor -> Assert.assertTrue(anchor.getAttribute("href").startsWith("https"))); (5)

For elements in frames

class MyPage extends Page {
    private UiElement frame = find(By.tagName("frame")); (1)
    private UiElement uiElement = frame.findById(14); (2)
}
Passing frames to the constructor is not supported anymore.
class MyPage extends Page {
    private GuiElement frame = new GuiElement(By.tagName("frame"), driver); (1)
    private GuiElement guiElement = new GuiElement(By.id(14), driver, frame); (2)
}

16.4. Assertion changes

16.4.1. Element assertions

The new standard way to perform assertions on elements like Pages and GuiElements is now

uiElement.expect().displayed(true); (1)
uiElement.expect().value().contains("Hallo Welt").is(true); (2)
Using the GuiElement assertions is now @deprecated
guiElement.asserts().assertIsDisplayed(); (1)
guiElement.asserts().assertAttributeContains("value", "Hallo Welt"); (2)

Perform decisions on occurrence with the waitFor prefix.

if (uiElement.waitFor().displayed(true)) {
    // Optional element became visible
}
Using the GuiElement waits is now @deprecated
if (guiElement.waits().waitForIsDisplayed()) {
}

Support of more features through consistent assertion API

uiElement.expect().css("display").is("none"); (1)
uiElement.expect().text()
    .map(value -> value.toLowerCase()) (2)
    .matches("^hello\\s.orld") (3)
    .is(true);
1 Perform assertions on the element’s CSS properties
2 Map values
3 Regular expression assertions

Custom failure messages

uiElement.expect().displayed().is(true, "Element is displayed");

16.4.2. Page assertions

Assert that a text is visible on a page

page.expect().url().endsWith("index.html").is(true); (1)

class MyPage extends Page {
    public void checkIfPageContainsText(String text) {
        this.getFinder()
            .findDeep(XPath.from("*").text().contains(text))
            .expect().displayed(true); (2)
    }
}
Using the text assertions is now @deprecated
Assert.assertTrue(page.getWebDriver().getCurrentUrl().endsWith("index.html")); (1)
page.assertIsTextDisplayed("You see me"); (2)

16.4.3. Screenshot based Assertions

The new standard way to perform screenshot based assertions is now

uiElement.expect().screenshot().pixelDistance("ElementReference").isLowerThan(1);
page.expect().screenshot().pixelDistance("PageReference").isBetween(0, 10);

Add screenshot to the report

page.screenshotToReport();
Using the static UITestUtils is now @deprecated
UITestUtils.takeScreenshot(page.getWebDriver(), true);

16.4.4. Layout based Assertions

To check if a element is beside another element

UiElement left = find(By.id("left"));
UiElement right = find(By.id("right"));

left.expect().bounds().leftOf(right).is(true);
left.expect().bounds().intersects(right).is(false);

Elements aligned to the same right

UiElement top = find(By.id("top"));
UiElement bottom = find(By.id("bottom"));

top.expect().bounds().fromRight().toRightOf(bottom).is(0);

Element contains another element

UiElement body = find(By.tagName("body"));
UiElement nav = parent.find(By.tagName("nav"));

body.expect().bounds().contains(nav).is(true);

16.5. New Control API

With the TestController API, you are able to control your test flow during runtime. Like timeouts, assertion handling and retry intervals. The Control instance is availabe as soon you implement the TestControllerProvider interface.

import eu.tsystems.mms.tic.testframework.testing.TestControllerProvider;

class MyTest implements TestControllerProvider {
}

16.5.1. Collected Assertions

The new standard way to collect assertions of elements in tests or pages is now

CONTROL.collectAssertions(() -> uiElement.expect().displayed(true));

For many elements or pages

CONTROL.collectAssertions(() -> {
    MyPage page = PAGE_FACTORY.create(MyPage.class);
    page.expect().title().is("TestPage");
    uiElement.expect().value().contains("Hello");
});

For custom assertions using AssertProvider

CONTROL.collectAssertions(() -> {
    String data = loadSomeData();
    ASSERT.assertEquals(data, "Hello World", "some data");
});

For other test methods

@Test
public void test_CollectEverything() {
    CONTROL.collectAssertions(() -> test_TestSomething());
}
Using the static AssertCollector is now @deprecated
AssertCollector.assertTrue(false);
Using the GuiElement’s assert collector is now @deprecated
guiElement.assertCollector().assertIsDisplayed();
Forcing standard assertions is now @deprecated
page.forceGuiElementStandardAsserts();
Setting collected assertions by default is now @deprecated
tt.guielement.default.assertcollector=true

16.5.2. Optional Assertions

The new standard way for optional assertions works like Collected Assertions

CONTROL.optionalAssertions(() -> uiElement.expect().displayed(true));
Using the static NonFunctionalAssert is now @deprecated
NonFunctionalAssert.assertTrue(false);
Using the GuiElement’s non functional asserts are now @deprecated
guiElement.nonFunctionalAsserts().assertIsDisplayed();

16.6. Timeouts and Retry API

16.6.1. @Check timeouts

The new standard way for setting GuiElement timeouts on @Check is now

class MyPage extends Page {
    @Check(timeout = 1)
    private UiElement uiElement;
}
Setting and restoring explicit timeouts on the GuiElement is now @deprecated
guiElement.setTimeoutInSeconds(1);
guiElement.restoreDefaultTimeout();

For the whole Page

@PageOptions(elementTimeoutInSeconds = 1)
class MyPage extends Page {...}
Setting explicit timeouts on the Page is now @deprecated
page.setElementTimeoutInSeconds(1);

Override during runtime

CONTROL.withTimeout(1, () -> uiElement.expect().displayed(true));

For many elements

CONTROL.withTimeout(1, () -> {
    MyPage page = PAGE_FACTORY.create(MyPage.class);
    page.expect().title().is("TestPage");
    uiElement.expect().value().contains("Hello");
});

For other test methods

@Test
public void test_TestSomething_fast() {
    CONTROL.withTimeout(1, () -> test_TestSomething());
}
Setting timeouts using static POConfig was removed!
POConfig.setThreadLocalUiElementTimeoutInSeconds(1);
POConfig.setUiElementTimeoutInSeconds(1);
POConfig.removeThreadLocalUiElementTimeout();

16.7. Modul migration

We want to make Testerra more SOLID. Thats why we finally introduced Dependency Injection via. Google Guice.

To enable you ModuleHook for v2, you need to extend this class from AbstractModule.

import com.google.inject.AbstractModule;
import eu.tsystems.mms.tic.testframework.hooks.ModuleHook;

public class MyModuleHook extends AbstractModule implements ModuleHook {
}

16.8. Removed features

  • The CSVTestDataReader module was removed. Please use OpenCSV library instead. It is more flexible as CSVTestDataReader could ever be.

  • AssertUtils were removed. Please use ASSERT from AssertProvider

  • From old GuiElement the assertLayout() was removed. Please use uiElement.expects().bounds()…​

16.9. The internals

This chapter explains how the new API works internally.

16.9.1. Everything is timed, but once

Every assertions is performed multiple times with a maximum timeout of tt.element.timeout.seconds. If this timeout has reached, the assertion will finally fail.

But there is only one timeout for each assertion now. No more implicit timeouts on sub method calls like getWebElement(), isPresent() etc.

This is what an assertion internally does, when you perform uiElement.expect().text().contains("Something").

  1. Find web element using WebDriver

  2. Check if element is present

  3. Retrieve the text of the element

  4. If the text does not contain "Something", start over with 1.

  5. Otherwise when the timeout has reached, an assertion error message will be displayed that the text of the element you’re looking for doesn’t contain the string "Something".

16.9.2. More consistence, less complexity

There will be only one interface for everything you need in a manner of an easy to read fluent API. It is not too abstract like TestNG Assert, and not to technically like AssertJ.

The new interface will always act exactly like you expect to, no matter in which context you are. You don’t have to decide which method you should use. The standard way will be the best fit for most cases. Let the framework handle the workarrounds for you.

16.9.3. Strict Page Object pattern

Testerra was built with the Page Object pattern in mind. The new API makes it easier for your team, to keep you on track makes it harder to break out, even if your project contains hundreds of Pages and thousands of Tests.

The new components extension allows you to implement page objects like a web developer would do, by separating functionality into reusable components.

16.9.4. Smaller codebase and less boilerplate

The API provides abstract assertion implementations for several properties.

  1. StringAssertion allows you to perform assertions on strings like contains("Something")

  2. QuantityAssertion allows you to perform assertions on quantified values like isBetween(-2,3)

  3. BinaryAssertion allows to assert if an value is boolean or a string that represents a boolean value with is(true)

These generic assertions are used in many other assertions and supports a hierarchical order. This is what the hierarchy looks like when you perform uiElement.screenshot().file().extension().is("png")

  1. Take a screenshot and return a ScreenshotAssertion

  2. Return a generic FileAssertion with the taken screenshot file

  3. Return a generic StringAssertion with the given file name extension

This implementation helps to keep the internal assertion code small, easy extensible and maintainable.

17. Known issues

Because we depend on other frameworks and tools like TestNG and Selenium we may encounter issues that we want to fix, but are bound to releases and fixes in our dependencies.

Every known issue in our dependencies that will lead to an error, an unexpected or unsightly behaviour in Testerra framework will be documented here, as well as a solution or a workaround.

17.1. Issues with Selenium

17.1.1. Close WebDriver sessions without WebDriverManager

Never close WebDriver sessions calling WebDriver.quit(). This may encounter problems or some kind of unexpected issues, because the session is not marked as closed in WebDriverManager 's session store.

17.1.2. Selenium 4 and browser version

Using a standalone Selenium 4 server (no grid) you must not set browser version via tt.browser.setting or tt.browser.version. Your local Selenium 4 server has no information about the versions of your installed browsers and cannot handle version information in your Webdriver request.

18. Overview of all Testerra properties

Properties are managed by the PropertyManager

18.1. Testerra core properties

Property default Description

tt.system.settings.file

system.properties

File name of property file for proxy settings.

tt.cert.trusted.hosts

(empty)

Whitespace separated list of trusted hosts (for SSL sockets)

18.2. WebdriverManager properties

Property default Description

tt.browser.setting

na.

Define the user agent configuration like browser[:version[:platform]] (example: firefox:65:windows). Overrides tt.browser, tt.browser.version and tt.browser.platform (recommended).

The following types of browsers are supported:

  • firefox

  • chrome

  • ie

  • edge

  • safari

  • chromeHeadless

tt.browser

na.

Only defines the browser type, will be overwritten by tt.browser.setting.

tt.browser.version

na.

Only defines the browser version, will be overwritten by tt.browser.setting.

tt.browser.platform

na.

Only defines the browser platform, will be overwritten by tt.browser.setting.

tt.baseurl

na.

URL of the first site called in a new browser session.

tt.webdriver.mode

remote

Sets the webdriver mode. remote uses an external Selenium server (deprecated).

Deprecated: This property is not used anymore. If you want to have local WebDriver sessions, keep tt.selenium.server.url empty.

tt.selenium.server.url

na.

The complete URL to a remote Selenium server (http://localhost:4444/wd/hub).

This setting overrides the following two properties.

tt.selenium.server.host

localhost

The host name of the remote Selenium server (deprecated).

tt.selenium.server.port

4444

The port of the remote Selenium server (deprecated).

tt.browser.maximize

false

Try to maximize the browser window.

tt.browser.maximize.position

self

Screen position for the window to maximize. If you have several screens and want to maximize the window on another screen than your default screen, you can choose between (left, right, top or bottom)

tt.window.size

1920x1080

Default window size for all new sessions (when tt.browser.maximize is false).

tt.display.resolution

1920x1080

Deprecated: Renamed to tt.window.size

tt.wdm.closewindows.aftertestmethods

true

If true, after every test method all open browser windows are closed.

tt.wdm.closewindows.onfailure

true

If true, after failed test methods all open browser windows are closed

tt.wdm.timeouts.seconds.window.switch.duration

5

Maximum duration to wait for on a WebDriverUtils.findWindowAndSwitchTo() in seconds.

webdriver.timeouts.seconds.pageload

120

Defines the Selenium timeout for page load seconds.
(driver.manage().timeouts().pageLoadTimeout())

webdriver.timeouts.seconds.script

120

Defines the Selenium timeout for execution of async scripts in seconds.
(driver.manage().timeouts().setScriptTimeout())

tt.selenium.webdriver.create.retry

10

Waiting time in seconds for a retry of creating a webdriver session in case the first attempt fails.

tt.selenium.remote.timeout.connection

10

Defines the read timeout in seconds for a remote webdriver session.

tt.selenium.remote.timeout.read

90

Defines the connection timeout in seconds for a remote webdriver session.

18.3. PageFactory properties

Property default Description

tt.page.factory.loops

20

The loop detections prevents endless recursive creation of new page instances. This property defines the max count of loops.

18.4. UiElement properties

Property default Description

tt.element.timeout.seconds

8

Default timeout for UiElement actions and assertions

tt.guielement.default.assertcollector @deprecated

false

Sets the behavior of @deprecated GuiElement.asserts():
true asserts() reacts like assertCollector() (Continue at FAIL)
false asserts() reacts like default assert (Stop at FAIL)

tt.guielement.checkrule

CheckRule.IS_DISPLAYED

Rule for Page objects validation of UiElements
(see Check Annotations)

tt.delay.after.guielement.action.millis

0

Waits in milliseconds after an action on a UiElement.

tt.delay.before.guielement.action.millis

0

Waits in milliseconds before an action on a UiElement.

18.5. Execution properties

Property default Description

tt.demomode

false

Visually marks every UiElement that is being processed by click, type or assert. This may break layout checks.

tt.demomode.timeout

2000

Timeout in ms for visual marks of UiElements.

tt.dryrun

false

All testmethods are executed with ignoring all steps. Also all setup methods (before, after) are ignored.
This is useful to check your TestNG suite without executing real testcode.

tt.on.state.testfailed.skip.shutdown

false

If true all browser sessions are left open.

tt.on.state.testfailed.skip.following.tests

false

If true, all follwoing tests are skipped in case a test failed.

tt.failed.tests.if.throwable.classes

na.

Failed tests condition: Throwable Class(~es, devided by ',').

tt.failed.tests.if.throwable.messages

na.

Failed tests condition. Throwable Message(~s, devided by ',').

tt.failed.tests.max.retries

1

How often tests should be retried by the Testerra RetryAnalyzer.

tt.watchdog.enable

false

Enables/Disables the WebDriverWatchDog.

tt.watchdog.timeout.seconds

300

Sets the timeout in seconds after the WebDriverWatchDog terminates the test execution (with System.exit(99) terminated).

tt.failure.corridor.active

true

Activate the failure corridor.

tt.failure.corridor.allowed.failed.tests.high

0

Number of test methods with weighting high allowed to fail to still mark the suite as passed.

tt.failure.corridor.allowed.failed.tests.mid

0

Number of test methods with weighting mid allowed to fail to still mark the suite as passed.

tt.failure.corridor.allowed.failed.tests.low

0

Number of test methods with weighting low allowed to fail to still mark the suite as passed.

tt.perf.test

false

If true, activates performance test related behaviour sets default values for the performance test.

tt.perf.page.thinktime.ms

0

Sets a thinktime in ms for each page load.

tt.perf.generate.statistics

false

If true, activates generation of graphs for the performance measurements.

18.6. Report properties

Property default Description

tt.report.dir

/target/surefire-reports/report/

Creates the report in the specified directory below the working directory.

tt.report.name

na.

Names the report (e.g. the project where Testerra is used)

tt.report.history.maxtestruns

50

Defines the maximum number of test runs that will be retained in the test history

tt.runcfg

na.

Set a run configuration to use different variations (test sets) of a test scope within a build task.

tt.screenshotter.active

true

If true, screenshots are fetched and added to the report.

tt.screenshot.on.pageload

false

If true, screenshot after page is loaded will be taken

tt.screencaster.active

true

If true, all screencasts are fetchted and added to the report depending on the enabled test method states by tt.screencaster.active.on.failed and tt.screencaster.active.on.success.

tt.screencaster.active.on.failed

true

If true, all screencasts for failed tests are fetched and added to the report.

tt.screencaster.active.on.success

false

If true, all screencasts for successful tests are fetched and added to the report.

tt.report.activate.sources

true

If true, adds source information to report

tt.report.source.root

src

Root directory for searching source info.

tt.report.source.lines.prefetch

5

Amount of lines taken into account before the actual error occurred (print lines between error line and error line minus tt.report.source.lines.prefetch)

tt.report.source.exclusion.regex

Defines a regular expression for package and/or class names which will ignored for source code snippets of error details.

18.7. Layout Check properties

Property default Description

tt.layoutcheck.reference.path

src/test/resources/screenreferences/reference

Path where the reference screenshots where saved

tt.layoutcheck.reference.nametemplate

Reference%s.png

Prefix for ReferenceScreenshots

tt.layoutcheck.takereference

false

Determines whether reference images where taken in the current run

tt.layoutcheck.use.ignore.color

false

Specifies whether the upper left pixel in the reference image defines an "ignore color". If true, then every pixel with this color will be ignored during the later comparison.

tt.layoutcheck.use.area.color

false

Specifies whether the upper left pixel in the reference image defines an "area color". If true, then every area surrounded by pixels with this color will be used for later comparison, other areas are dismissed. Opposite of tt.layoutcheck.use.ignore.color.

tt.layoutcheck.actual.nametemplate

Actual%s.png

Filename scheme for saving current screenshots. The value must contain a '%s' which is replaced by the specified target file name during test execution.

tt.layoutcheck.distance.nametemplate

Distance%s.png

Filename scheme for saving distance images. The value must contain a '%s' which is replaced by the specified target file name during test execution.

tt.layoutcheck.distance.path

src/test/resources/screenreferences/distance

Directory path under which the calculated distance images are stored.

tt.layoutcheck.actual.path

src/test/resources/screenreferences/actual

Directory path under which the current screenshots for the comparison are saved

tt.layoutcheck.pixel.rgb.deviation.percent

0.0

Max allowed difference in rgb values between actual and reference image in percentage. If on of Red, Green, and Blue percentages are higher than the given value, the corresponding pixel is marked as false (means red color in distance image)

tt.layoutcheck.pixel.count.hard.assertion

false

Specifies the handling of different sized images. If false, only the common pixels of both images are used for the error calculation and pixels that are outside one of the images are ignored. If true, the pixels outside of one or the other image are included in the error calculation and are counted as incorrect pixels.

19. Architecture

architecture

20. Previous Testerra versions

Glossary

SUT

System under test