6. Mark59 Selenium Scripting

Writing Java/Selenium Scripts using Mark59

To Start..

The best way to get a good idea of how to script using the Mark59 framework is to look at and run the samples provided in the dataHunterPVTest project:

  • com.mark59.datahunter.performanceTest.scripts.DataHunterLifecyclePvtScript

  • com.mark59.datahunter.performanceTest.scripts.DataHunterLifecycleIteratorPvtScript

  • com.mark59.datahunter.performanceTest.scripts.DataHunterBasicSampleScript

DataHunterLifecyclePvtScripthas been written to give an example of the general functionality available. The 'Iterator' version DataHunterLifecycleIteratorPvtScript is a more specialised, showing how to write a script that may need to repeat a workflow several times. DataHunterBasicSampleScript is a simple script (it doesn't use the DSL project), so is a good way to see what the Mark59 is providing, particularly if you are not familiar with Selenium.

Javadoc

The classes associated with scripting have been documented in a fashion to be usable and hopefully helpful from a script writer's perspective. The Javadoc and supplied samples are the primary sources of documentation for scripting. This chapter is written to provide an overview of how the Mark59 classes have been structured, and a summary of functionality.

The Structure of the Mark59 Selenium Scripting Classes

The core class of Mark59 for selenium scripting is com.mark59.selenium.corejmeterimpl.SeleniumAbstractJavaSamplerClient.

This is the class that your script needs to extend in order to invoke Mark59 functionality. For the special case of repeatedly iterating over a workflow you extend SeleniumIteratorAbstractJavaSamplerClient - but that class is just an extension of SeleniumAbstractJavaSamplerClient with a bit of extra functionality to allow iteration.

SeleniumAbstractJavaSamplerClient is itself an extension of the JMeter class AbstractJavaSamplerClient. This is the class that need to be implemented in order to create a 'Java Request' Sampler in a JMeter test plan. From JMeter's perspective, a Mark59 selenium script is just an AbstractJavaSamplerClientimplementation that produces the expected SampleResultoutput (in Mark59 the SampleResultusually contain 'subResults' that correspond to the script startTransaction()/endTransaction()methods).

The following sections discuss the methods of SeleniumAbstractJavaSamplerClient which can or need to be implemented to write a script.

The script logic: runSeleniumTest()

This method should contain the main scripting logic. It is passed a selenium WebDriver(setup options discussed in the additionalTestParameters()section). It is also passed a JmeterFunctionsForSeleniumScripts object ('jm'), which contains startTransaction/endTransaction methods used to capture transaction timings.

jm.startTransaction("DH-lifecycle-0001-gotoDeleteMultiplePoliciesUrl");
driver.get(dataHunterUrl + TestConstants.DELETE_MULTIPLE_POLICIES_URL_PATH + "?application=" + application);
jm.endTransaction("DH-lifecycle-0001-gotoDeleteMultiplePoliciesUrl");

endTransaction() methods exist to set the transaction explicitly as a PASS or a FAIL, you can also use the setTransaction() method to explicitly set labels and timings. You can also set datapoints. These are just named tags and with a value. The same datapoint name tag can be used many times, to see how some value has changed over time during a test.

jm.userDataPoint(application + "_Total_Unused_Policy_Count", countPolicies);

You also have control over the script logging output either via defaults based on the log4j level, or they can be explicitly set off, or to write, or to buffer (at the start and/or end of transactions). The logs available at script level are:

  • screenshots (jpg files). Hint: occasionally the WebDriverwill fail to write an image - in which case a stack trace is written to the 'jpg' file, which can be opened using a text editor.

  • html source

  • (Chrome only) the Performance Log. Basically a textual version of Chrome's Developer Tools Network Panel

When logs are 'buffered' (held in-memory), they can be written by calling jm.writeBufferedArtifacts(), otherwise they will simply be cleared at the end of script execution.

The logs, and a text file with any available stack trace will always be output on a script execution Exception or AssertionErrorbeing caught.

Logging fine-grain control available is indicated in the DataHunterLifecyclePvtScript sample.

// jm.logScreenshotsAtStartOfTransactions(Mark59LogLevels.WRITE);
// jm.logScreenshotsAtEndOfTransactions(Mark59LogLevels.WRITE);
// jm.logPageSourceAtStartOfTransactions(Mark59LogLevels.WRITE);
// jm.logPageSourceAtEndOfTransactions(Mark59LogLevels.WRITE );
// jm.logPerformanceLogAtEndOfTransactions(Mark59LogLevels.WRITE);
// jm.logAllLogsAtEndOfTransactions(Mark59LogLevels.BUFFER);

Log-level control is detailed in full in the JmeterFunctionsForSeleniumScriptsand Mark59LogLevelsjavadoc.

A note about debug level and the mark59-selenium-sample-dsl project.

The implementation of how log4j DEBUG level is treated in the sample DSL project is worth noting because it involves a significant change to the code execution path, in order to capture additional information what would otherwise not be available. In the DSL core class, com.mark59.seleniumDSL.core.Elemental, the method runUsingFluentWaitInDebugMode() is invoked rather than the standard getFluentWait() method. This allows detailed logging of the reason for a re-try, but means re-try logic needs to be controlled manually. Obviously it's up to you if you want to use this technique to debug - it is mentioned here at it's not obvious unless you dig a bit. See the Elemental.runUsingFluentWaitInDebugMode() and FluentWaitFactory.getFluentWaitDebugMode() javadoc for full details.

Script Setup Parameters: additionalTestParameters()

Parameter values can be set in the script using the additionalTestParameters() method. Some parameters are pre-defined with default values, that can be overridden in this method.

Pre-defined parameter

default value

notes

DRIVER

"CHROME"

FIREFOX is the alternative

HEADLESS_MODE

"true"

PAGE_LOAD_STRATEGY

"NORMAL"

NONE or EAGER are alternative (NONE is useful in complex pages, we have not tested with EAGER)

PROXY

(no proxy)

two general formats are available, specifying a PAC script or a httpProxy / sslProxy URL. Javadoc at SeleniumDriverBuilder.setProxy()

ADDITIONAL OPTIONS

(no options)

see https://peter.sh/experiments/chromium-command-line-switches/ for a plethora of Chrome options available

WRITE_FFOX_BROWSER_LOGFILE

"false"

stops the rather verbose output written to log when using Firefox

all logging settings:

"default"

for default, logging behaviour will depend on log4j setting. Javadoc at JmeterFunctionsForSeleniumScripts and Mark59LogLevels

Restrict_To_Only_Run_On_IPs_List

"" (empty)

intended for Distributed testing, where a script thread is only to run on the listed IP address(es). Javadoc at IpUtilities.localIPisNotOnListOfIPaddresses()

BROWSER_EXECUTABLE

no default

used to set a path to an alternate browser executable.

SeleniumIteratorAbstractJavaSamplerClient

only:

see Javadoc at SeleniumIteratorAbstractJavaSamplerClient

ITERATE_FOR_PERIOD_IN_SECS

0

ITERATE_FOR_NUMBER_OF_TIMES

1

ITERATION_PACING_IN_SECS

0

STOP_THREAD_AFTER_TEST_START_IN_SECS

0

STOP_THREAD_ON_FAILURE

"false"

Further descriptions of the built-in parameter values are available via the 'See Also' links in the Javadoc for SeleniumAbstractJavaSamplerClient.

Parameter values set in additionalTestParameters(), or by default are available during script execution via the JavaSamplerContext object

protected Map<String, String> additionalTestParameters() {
:
jmeterAdditionalParameters.put("FORCE_TXN_FAIL_PERCENT", "20");
:
protected void runSeleniumTest(JavaSamplerContext context, JmeterFunctionsForSeleniumScripts jm, WebDriver driver) {
:
int forceTxnFailPercent = new Integer(context.getParameter("FORCE_TXN_FAIL_PERCENT").trim());
:

Once deployed to JMeter, these parameters are available and can be overwritten in the Java Request panel for the script. See the DataHunterSeleniumTestPlan.jmx in the dataHunterPVTest project for, including some basic examples of how to use the parameters in a more dynamic fashion.

Script parameters in JMeter

Script properties: mark59.properties

A mark59 selenium script needs to be know two things in order to execute: the location of the WebDriver executable it is going to use (which may be for Firefox or Chrome), and where to write log files. It looks for a mark59.properties file to get the information. It can also pick up the properties from system properties, which will override anything in mark59.properties if both exist. Assuming mark59.properties is placed on the root directory of your Maven project, along with the WebDriver executable(s), then the file would typically contain values similar to this table:

property

sample value

mark59.selenium.driver.path.chrome

./chromedriver.exe

mark59.selenium.driver.path.firefox

./geckodriver.exe

mark59.screenshot.directory

C:/JmeterTestErrorLogs/DataHunter/

The script will actually execute without themark59.screenshot.directory property being set but is not advised (a message will be an output warning the property is not set), and failure will occur when the script attempts to write a log. For deployment purposes to a server where you may have multiple instances of JMeter, you could consider placing the WebDriver in a common location: mark59.selenium.driver.path.chrome=C:/_chrome/chromedriver.exe

An example of using system properties instead of having a mark59.properties file (or over-riding an existing one) is given in DataHunterSeleniumTestPlan.jmx in the dataHunterPVTest project.

Using system properties for Mark59

User Error Handling: userActionsOnScriptFailure()

By default whenever a script fails (a thrown Exceptionor AssertionError ) logs are written including any stack trace information and the script ends. In the situation where a user action may needed, it is possible to do so by creating a userActionsOnScriptFailure() method. A typical use case would where a application user log-out may be needed to allow the next script attempt to run ok. DataHunterLifecycleIteratorPvtScriptcontains a sample usage.

The methods of SeleniumIteratorAbstractJavaSamplerClient

As the idea of this class is to allow iteration over a work flow, the methods to override are slightly different to SeleniumAbstractJavaSamplerClient :

  • initiateSeleniumTest() - allows for user action before iteration.

  • iterateSeleniumTest() - logic to be iterated over.

  • finalizeSeleniumTest() - allows for user action after iteration.

Iteration is controlled by additional parameters as documented in the "SeleniumIteratorAbstractJavaSamplerClient only" part of the parameters table (details and example settings are available in the class Javadoc).

For an example of usage review sample script DataHunterLifecycleIteratorPvtScript.

Running a script from Eclipse: main()

We consider the ability to run and test a script directly from Eclipse an important part of Mark59 functionality. Three main options have been provided to give some flexibly in testing the script:

  • simple single thread execution

  • multi-thread execution

  • multi-thread execution with thread parameterization

A Log4jConfigurationHelperclass has been written to assist setting the log4j level. Further details available in the sample scripts.

A Mark59 selenium script Main()