10. Trend Analysis Database Load

Uploading JMeter or LoadRunner test results into the Trend Analysis MySQL database

This chapter assumes you have performed or reviewed the sample load described for the install:

Basically, the Trend Analysis load program, Runcheck, takes the results given to it from a JMeter or LoadRunner test, and creates a summary of the test which can be viewed in the Trend Analysis Web Application. As the test results processed, they are compared to values on SLA (Service Level Agreement) tables, which are set using the Trend Analysis Web application. Warning or Error messages are output if the expected SLA ranges are not met -which can be parsed and colour-coded when running from a Jenkins server. The severity will be Error (Red) for SLA failures which mean the test cannot be considered valid (out-of-range Txn PASS Counts or Txn Fail %), and Warning (Yellow) when a SLA has failed, but that has not invalidated the test (90th % response times, any metric SLA failure).

To view the parameters available for Runcheck, just start the program without parameters, and the options will be listed.

C:\gitrepo\mark-5-9\mark59-wip\metricsRuncheck\target>java -jar metricsRuncheck.jar
Starting runcheck ..
Parsing failed. Reason: Missing required options: a, i
usage: Runcheck
-a,--application <arg> Application Id, as it will appear in the
Trending Graph Application dropdown selections
-c,--captureperiod <arg> Only capture test results for the given number
of minutes, from the excluded start period
(default is all results except those skipped
by the excludestart parm are included)
-h,--mysqlserver <arg> Server hosting the 'pvmetrics' mySql database
where results will be held (defaults to
localhost)
-i,--input <arg> The directory or file containing the
performance test results. Multiple
xml/csv/jtl results files allowed for Jmeter
within a directory, a single .mdb file is
required for Loadrunner
-p,--mysqlPort <arg> Port number for the 'pvmetrics' mySql database
where results will be held (defaults to 3306)
-r,--reference <arg> A reference. Usual purpose would be to
identify this run (possibly by a link). Eg <a
href='http://ciServer/job/myJob/001/HTML_Repor
t'>run 001</a>
-t,--tool <arg> Performance Tool used to generate the results
to be processed { JMETER (default) |
LOADRUNNER }
-u,--dbUsername <arg> Username 'pvmetrics' mySql database (defaults
to admin)
-w,--dbpassWord <arg> Password for the 'pvmetrics' mySql database
-x,--eXcludestart <arg> exclude results at the start of the test for
the given number of minutes (defaults to 0)
-z,--timeZone <arg> (Loadrunner only) Required when running
extract from zone other than where Analysis
Report was generated. Also, internal raw
stored time may not take daylight saving into
account. Two format otions 1) offest againt
GMT. Eg 'GMT+02:00' or 2) IANA Time Zone
Database (TZDB) codes. Refer to
https://en.wikipedia.org/wiki/List_of_tz_datab
ase_time_zones. Eg 'Australia/Sydney'
Sample usages
------------
1. Jmeter example
Process Jmeter xml formatted result in directory C:/jmeter-results/BIGAPP (file/s ends in .xml)
The graph application name will be MY_COMPANY_BIG_APP, with a reference for this run of 'run ref 645'.
The pvmetrics database is hosted locally on a MySql instance assigned to port 3309 (default user/password of admin/admin) :
java -jar metricsRuncheck.jar -a MY_COMPANY_BIG_APP -i C:/jmeter-results/BIGAPP -r "run ref 645" -p 3309
2. Loadrunner example
Process Loadrunner analysis result at C:/templr/BIGAPP/AnalysisSession (containing file AnalysisSession.mdb).
The graph application name will be MY_COMPANY_BIG_APP, with a reference for this run of 'run ref 644'.
The pvmetrics database is hosted locally on a MySql instance assigned to port 3309 (default user/password of admin/admin) :
java -jar metricsRuncheck.jar -a MY_COMPANY_BIG_APP -i C:/templr/BIGAPP/AnalysisSession/AnalysisSession.mdb -r "run ref 644" -p 3309 -t LOADRUNNER

The application id (-a) and input file or directory (-i) are the only mandatory parameters. Obviously you need to add parameters where the defaults as listed above are not appropriate.

When you are wanting to load the results for the first run of a new application, you do not need to do anything special (it will appear as a single run graphic in Trend Analysis). However, you will need to set transactional SLAs , and new metric SLAs, if you want SLA checking for the application..

The Sample usages section incorrectly has the jar name as runcheck.jar-1.0 instead of metricsRuncheck.jar (edited above) - to be fixed next release..

Loading JMeter Results

The general format will be similar to this DataHunter application sample load :

java -jar metricsRuncheck.jar -a DataHunter -i C:\Jmeter_Results\DataHunter
-h pv-report-server -p 3308 -t JMETER
-r "<a href='http://myjenkinsserver/job/DataHunter_Result/543/HTML_Report'>run 543</a>"

For JMeter, the input is a directory. It can process multiple files in the directory, considering them part of a single run . The -h and -p parameters refer to the location of the target MySQL database. The -r parameter is just a tag to help identify the run. In this example the data load run was from a Jenkins server, and the parameter was built in a way that allows a link back to Jenkins when displaying the run from in the Trending Analysis Web application

Loading LoadRunner Results

For LoadRunner, the input is all taken from the Access ".mdb" file created during Analysis. Note that this file contains data for the entire test (you can use the run -x and -c parameters to capture a target test period). As an example of the format , assuming the database is local on port 3306 with default user/password, and the access db file is at C:/templr/BIGAPP/AnalysisSession/AnalysisSession.mdb:

java -jar metricsRuncheck.jar -a MY_COMPANY_BIG_APP -i C:/templr/BIGAPP/AnalysisSession/AnalysisSession.mdb -r "run ref 666" -t LOADRUNNER

A few notes about the data extract process:

  • 'metrics' are identified by Event_map events of type 'SiteScope' (data on Monitor_meter)

  • 'datapoints' are identified by Event_map events of 'DataPoint' (data on DataPoint_meter)

  • 'transactions' are identified by Event_map events of 'Transaction' (data on Event_meter table)

Filtering of which metrics and datapoints are to be captured is done using the Trending Analysis Web Application. To assist, the Runcheck program outputs the full Event-map table, indicating what events have been mapped.

The LoadRunner functionality is considered 'legacy' as our own testing is now exclusively done using JMeter. However we will retain the functionality in the code base on future releases.

Note that the raw .mdb file does not seem to allow for Daylight Savings or when the Analysis and run occur in different time zones. You can use the -z parameter to compensate.