Executing a JMeter testplan from Rational Quality manager

About this time last year I wrote a couple of posts on using JMeter against  a Jazz server. A few days ago Gary posted a comment about “making it possible to kick off JMeter tests from RQM and get results sent back so that they can be managed inside RQM” and it stuck in my head, so here’s how it can be done.

RQM allows arbitrary tools to be executed as “test tools” via one facility : the Command Line Adapter. It’s pretty well documented in the Infocentre and Article 809 on Jazz.net . So it ended up being a (mostly:-) straightforward exercise. My requirements were simple: execute a JMeter testplan from RQM and get the appropriate result (PASSED, FAILED,…) to be set for the RQM Test Result. The result would depend on the result of executing a HTTP Request and I’d consider it a bonus if I could set a Custom Property on the RQM result with the value of the HTTP Response Message.

First I needed to get the Command Line Adapter running and “healthy” in the RQM Adapter Console. Easy. I looked through the Infocentre, decided I didn’t need to fiddle with the commandline.properties file for now, opened a Command Prompt and fired up the CLA following the instructions inSetting up and starting the command-line adapter.

RQM gave the Adapter the green light.

I turned my attention to the JMeter Test Plan. I first needed to verify that I could even run a test plan from RQM so I setup a simple JMeter Test Plan that accessed jmeter.apache.org.

Then I followed the steps in the ” Create and Execute” section of Article 809 on Jazz.net. The Command Line Script Details part of the Test Script looked like this:

Running the Test Case gave me the expected result, with an attachment to the Test Result capturing the output from JMeter.

Now I needed to see if the test failed the appropriate status would be set in RQM. In this case, my definition of “failure” was that the HTTP Response Code would be something other than “200”. So I changed the Server Name in the HTTP Request to jmeter.apache.org1 and ran the test again.

It passed again. Though http://jmeter.apache.org1/ doesn’t exist.

Baffled me, until I went through some message boards on the topic of JMeter exit codes and found that JMeter always exits with “0” (zero) unless it encounters a fatal error that isn’t related to the test plan itself. Which meant that I needed to add something to the test plan that set the exit status to one of the codes RQM expected.

After more research and a lot of experimentation, I found that a Bean Shell Sampler script wrapped by an If Controller plus a little tweak to the “jmeter.bat” startup file did the trick.

The modified JMeter testplan looked like this:

First the If Controller:

Basically the Condition

!(${JMeterThread.last_sample_ok})

only allows the If Controller’s child(ren) to execute if the HTTP Response from the last HTTP Request was anything other than “OK“.

The only child I have for the IF Controller is a Bean Shell Sampler:

The script does the following:

String qmcpf = props.get("QMCPF");

This line gets the value of a JMeter property named QMCPF (my shorthand for Quality Manager Custom Properties File) which the RQM CLA sets in the environment and is passed in as an argument to JMeter using “-JQMCPF=%qm_CustomPropertiesFile% “.

SampleResult prev_result=ctx.getPreviousResult();

This line uses the JMeterContext variable to retrieve the Sampler Result from the previous/last Sample, which in my case is the HTTP Request.

f = new FileOutputStream(qmcpf, true);
 p = new PrintStream(f);
 p.println("ResponseMessage=" + prev_result.getResponseMessage() );
 p.close();
 f.close();

These lines open the Quality Manager Custom Properties File, print the Response Message as a name value pair to it and close it. RQM will pick this up and show in the custom properties section of the Test Result like so:

Finally

System.exit(40);

This line sets the exit code for the process to “40” which is what RQM uses to signify an “Error”.

The last thing I needed to do was tweak the jmeter.bat startup script. The batch file has the following lines in it after the JMeter invocation:

rem If the errorlevel is not zero, then display it and pause
 if NOT errorlevel 0 goto pause
 if errorlevel 1 goto pause
 goto end
 :pause
 echo errorlevel=%ERRORLEVEL%
 pause
 :end

which looks fine, except that a) the ERRORLEVEL is only displayed to standard output and not set as the value of the batch file’s exit and b) the “pause” if the error level is set causes the execution from RQM to “hang”  as it waits for someone to “Press any key to continue“.

So I replaced these lines with

exit /B %ERRORLEVEL%

which passes whatever value I use in the System.exit() call in the JMeter Bean Shell Sampler back to RQM for the Test Result to be set appropriately.

I should really figure out the same if running JMeter on non-Windows platforms and add error handling to the Bean Shell script, but that’s for another time.