Automate Performance Testing With STAF and JMETER - Part 2 / 4

Step 2. Develop a Performance Scenario in JMeter

Apache JMeter is a 100% pure Java desktop application (offers both GUI and command line interface) designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions. Lots of good information can be found here: Download it, install it, and start building a test plan!.

Here we offer some tips on helping you get started quickly (if you couldn't find helpful info on the JMeter site, you will find our info here very handy). For example, how to parameterize number of concurrent users, how to generate a performance report, and how to run it through the command line interface (after a test plan or scripts are done in GUI).

Once JMeter is running, you start its GUI - it should look similar to the following

A test plan in JMeter basically consists of test groups (roles or logical groups of concurrent users), operations (in the example above they are basically HTTP GET requests, each with a series query strings. Query strings are in the forms of "parameter=value" pairs, like in Google's search page - http://www.google.com/search?hl=en&q=jmeter&btnG=Google+Search - one of the query string is "q=jmeter"), think time (how long in-between clicks a user waits), and other HTTP functions such as Cookies.

After your scenario is defined, you want to generate workloads with increasing number of users, i.e., you start with 10 users, then 20, 50, 100, 200, so on and so forth, while the operation mix does not change. So you need to parameterize the "Thread Properties" in the "Thread Group" panel in JMeter, as shown below:

Once these properties are parameterized, you may specify their values in jmeter.properties file or as command line parameters, so tests can be run without changing the script itself (which is in XML).

JMeter also provides fairly good reports. See one sample below:

Of course, for this article, the most important feature of JMeter to performance automation is its command line interface. With it, one can remotely execute performance tests without having to display the GUI, and the report can be generated from the log files from JMeter.

To specify 1 user per user group for 5 groups (total 5 users), reamup in 1 second, 
and 25 loops each group iterates over the same operation set:

./bin/jmeter.lucene -t MyWebPerfScenario.jmx \\
-Jgroup1.threads=1 -Jgroup1.loops=25 -Jgroup1.rampup=1  \\
-Jgroup2.threads=1 -Jgroup2.loops=25 -Jgroup2.rampup=1  \\
-Jgroup3.threads=1 -Jgroup3.loops=25 -Jgroup3.rampup=1  \\
-Jgroup4.threads=1 -Jgroup4.loops=25 -Jgroup4.rampup=1  \\
-Jgroup5.threads=1 -Jgroup5.loops=25 -Jgroup5.rampup=1

To specify 10 user per user group for 5 groups (total 50 users), reamup in 40 seconds, 
and 50 loops each group iterates over the same operation set:

./bin/jmeter.lucene -t MyWebPerfScenario.jmx \\
-Jgroup1.threads=10 -Jgroup1.loops=50 -Jgroup1.rampup=40  \\
-Jgroup2.threads=10 -Jgroup2.loops=50 -Jgroup2.rampup=40  \\
-Jgroup3.threads=10 -Jgroup3.loops=50 -Jgroup3.rampup=40  \\
-Jgroup4.threads=10 -Jgroup4.loops=50 -Jgroup4.rampup=40  \\
-Jgroup5.threads=10 -Jgroup5.loops=50 -Jgroup5.rampup=40

Now with a performance measuring tool in place, we next talk about what tools to use to automatically collect system resource usage information - CPU, Memory, Network, and Disk IO.