Simple collection of notes begun that will be organized down the road if there's
any need to do so.
In order to execute JMeter, I had to use real Sun Java and not OpenJDK. I modified
the last line of bin/jmeter thus:
/home/russ/dev/jdk1.6.0_38/bin/ java $ARGS $JVM_ARGS -jar `dirname $0`/ApacheJMeter.jar "$@"
Miscellaneous notes
Limit the number of threads. The harder JMeter is forced to work (i.e.: more
threads), the more inflated timing gets because threads are waiting to gain
access to the CPU. See
Best
Practices .
The test plan. This consists of:
list of thread groups
logic controllers
sample generating controllers
listeners
timers
assertions
configuration elements
See an example of a test plan here .
Output. JMeter reports warnings and error to jmeter.log .
Test files. The JMeter load tests are simply XML files, so you can check
them in to your version-control system.
Running JMeter—options
...passed to shell script bin/jmeter :
-H http://web-proxy.austin.acme.com
-P 8080
-u username
-a password
JMeter links
The Apache documentation isn't very good. It's poorly written with bad grammar
and mechanics, and the tutorials really aren't. There's no "JMeter for Dummies"
step-by-step, start off simple documentation at all. You're better off looking
elsewhere, like here:
Other useful links...
My First JMeter Test
This is the short tutorial referred to in the Links section on this page. I don't
illustrate this fully because those illustrations are at the link.
Launch JMeter.
Right-click Test Plan
, do Add -> Threads (Users)
Thread Group
.
This allows us to control the number of threads (and hence virtual users),
the ramp-up period, the loop count (how many times each virtual user will
execute a test script), and the schedule (start- and end times and days).
Right-click Test Plan
, do Add -> Config Element ->
HTTP Request Defaults
.
Server Name or IP: localhost Port Number: 8000
Later, if we want to run against some other server and port number, this is
where we change them.
Right-click Thread Group
,* do Add -> Sampler ->
HTTP Request
.
Name: Stats Request Path: /stats/stats
* The original tutorial had this as right-click Test Plan -> Add
-> Sampler
, but of course, that's impossible.
Right-click HTTP Request (Status Request), do Add ->
Assertions -> Response Assertion
.
Then name this response assertion appropriately (it's for testing a 200 Okay
return), click Response Code and Matches , then add 200 as
a Patterns to Test by click Add , then scrolling up in the
field and typing "200".
Now we need some way of visualing the results, both Graph Results and
Summary Report
.
Right-click Thread Group
, do Add -> Listener ->
Graph Results
.
(bis) Do the same to get a Summary Report . Name the file it will
produce summary-report.
Run the test.
Choose Run -> Start
.
This will yield an alert saying, "You should save your test plan before
running." Yes, you want to do this.
Note in the Summary Report pane that the test was execited 25 times,
averaging 2ms per request with a minimum response time of 1ms and a maximum
of 14ms. There were errors 100% of the time (at very least because there was
nothing at localhost:8000/stats/stats ).
There is nothing of interest in the Graph Results pane because there
was no server to hit.
Choose Run -> Clear
or Clear All
after making
changes (like increasing the number of threads to run) before restarting.
My own first JMeter test: pinging my ReST application
This is my first actual test I set up on my own. I ping my server application,
a restlet.
Look-up: my first JMeter ReST test attempt
Even after setting up the header content, which JMeter supports, I never am
able to penetrate Jersey from JMeter (see log which reports a 404. Conclusion:
I'll have to checkout some special ReST sampler plug-ins for JMeter.
I thought specifying the header content would be all that's different:
Look-up: my second attempt
The purpose of these notes is to document failure and success. Here's how I
achieved a working GET with notes on what better to do:
The HTTP Defaults aren't added to with each HTTP
Request as I dimly surmised. Instead, they're the default overwriten
by the latter. So, I had to put the whole path in the GET .
I can see a lot more of what's going wrong by adding a listener, the
Simple Data Writer : right-click Thread Group (here,
Look-up ), do Add -> Listener -> Simple Data Writer
.
Click Configure and ensure what's noted in the image below. As for
what comes out, it's ugly, but useful:
~/dev/apache-jmeter-2.7/test-plans $ cat ../bin/lookup.log
<?xml version="1.0" encoding="UTF-8"?>
<testResults version="1.2">
<httpSample t="13" lt="13" ts="1358467601684" s="true" lb="GET" rc="200" rm="OK" tn="Look-up 1-1" dt="text" by="774">
<assertionResult>
<name>Status 200 Okay</name>
<failure>false</failure>
<error>false</error>
</assertionResult>
<responseData
class="java.lang.String">{"oid":"000000000000000000000992", \
"identities":"[email protected] ", \
"partneroid":"000000000000000000000222", \
"username":"venkat-partnerx", \ "firstname":"Venkat", \
"lastname":"Mynampati","fullname":"Venkat Mynampati", \
"phone":"510-987-9874","mobile":"510-987-9874", \
"fax":"510-987-9874","birthdate":"1973-01-01T05:12:12-07:00", \
"language":"en","country":"US", \
"accountdata":{"tosaccepted":"true", \
"favoritecolor":"white","menuscheme":"tandy", \
"nationalanthem":"String of Pearls","bestchoice":"selection7", \
"favoritefriend":"Levi Miller","bestbeach":"sandy"}, \
"loggedin":"false","created":"2013-01-16T11:36:06.110-07:00", \
"forgotten":"false"}\
</responseData>
<java.net.URL>http://localhost:8000/accountmgr/api/v1/[email protected] </java.net.URL>
</httpSample>
</testResults>
The HTTP Header Manager is different too; I had messed up the
first time.
JMeter run script (sample)
#!/bin/sh
echo "Starting tests on remote JMeter Service..."
DIR=/usr/local/jmeter2/jmeter
${DIR}/bin/jmeter.sh -n -t test-plan.jmx -l logfile.jtl -R 192.168.0.103 -Gtest.hostname=192.168.0.110 \
-Gtest.port=443 -Gtest.protocol=https -Gtest.threads=10 -Gtest.ramp=10 -Gtest.loop=1 -X
JMeter run script for Jenkins (sample)
#!/bin/sh
echo "Starting tests on remote JMeter Service..."
DIR=/opt/jmeter/jmeter
${DIR}/bin/jmeter.sh -n -t jmeter/test-plan.jmx -l build/reports/jmeter/performance.jtl $REMOTE_JMETER \
$HOST_VALS $THREADS $MISC -X
JMeter configuration (sample)
# jmeter - JMeter Performance Testing Server
#
# JMeter Server provides a remote connection point to do performance testing.
description "JMeter Server"
start on filesystem or runlevel [2345]
stop on runlevel [!2345]
respawn
respawn limit 3 5
umask 022
env JAVA_HOME=/usr/java/jdk6
env JMETER_HOME=/opt/jmeter/jmeter
env OPTS="-H web-proxy.acme.com -P 8080"
script
PATH="$JAVA_HOME/bin:$JMETER_HOME/bin:$PATH"
echo "PATH=$PATH"
exec start-stop-daemon --start --chdir ${JMETER_HOME} --exec ${JMETER_HOME}/bin/jmeter-server -- $OPTS
end script
Here are notes/steps to installing Jmeter for remote use
Modify /etc/hosts and add the real IP address for the host. JMeter
requires this at startup.
Download JMeter (latest version) and unzip it.
Move it to the location /opt/jmeter
Create a symbolic link (ln -s <jmeter_version> jmeter
).
This will create a link so that JMeter can always be found at
/opt/jmeter/jmeter . When JMeter needs to be updated, just download
the new version and change the symbolic link. If all apps use the
/opt/jmeter/jmeter , nothing should break.
Copy jmeter.conf from jmeter to the new JMeter server at
/etc/init.d . This will create an upstart script.
Now start jmeter-server using the upstart script using
sudo start jmeter
.
user.properties (sample)
user.properties is a file that resides on the remote JMeter that allows
each instance to use unique values that aren't supplied by the client startup
script.
It must be placed on the remote server in the ${JMETER_HOME}/bin directory.
This allows us to specify unique client id so that multiple instances don't stomp
on each other (ex: id.start). Each instance must have unique values, so don't just
copy the properties file to every instance without modifying the values.
# Sample user.properties file
#
## Licensed to the Apache Software Foundation (ASF) under one or more
## contributor license agreements. See the NOTICE file distributed with
## this work for additional information regarding copyright ownership.
## The ASF licenses this file to You under the Apache License, Version 2.0
## (the "License"); you may not use this file except in compliance with
## the License. You may obtain a copy of the License at
##
## http://www.apache.org/licenses/LICENSE-2.0
##
## Unless required by applicable law or agreed to in writing, software
## distributed under the License is distributed on an "AS IS" BASIS,
## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
## See the License for the specific language governing permissions and
## limitations under the License.
#---------------------------------------------------------------------------
# Classpath configuration
#---------------------------------------------------------------------------
#
# List of paths (separated by ;) to search for additional JMeter extension classes
# - for example new GUI elements and samplers
# These are in addition to lib/ext. Do not use this for utility jars.
#search_paths=/app1/lib;/app2/lib
# Users can define additional classpath items by setting the property below
# - for example, utility jars or JUnit test cases
#
# Use the default separator for the host version of Java
# Paths with spaces may cause problems for the JVM
#user.classpath=../classes;../jars/jar1.jar
#---------------------------------------------------------------------------
# Logging configuration
#---------------------------------------------------------------------------
#log_level.jorphan.reflect=DEBUG
# Warning: enabling the next debug line causes javax.net.ssl.SSLException: Received fatal alert: unexpected_message
# for certain sites when used with the default HTTP Sampler
#log_level.jmeter.util.HttpSSLProtocolSocketFactory=DEBUG
#log_level.jmeter.util.JsseSSLManager=DEBUG
# Enable Proxy request debug
#log_level.jmeter.protocol.http.proxy.HttpRequestHdr=DEBUG
id.start=1
id.max=9999
This said, it's possible to ignore user.properties and set up defaults as
if the values were there. This is useful because you want to commit your test
plans to your source-code repository and not have to worry about what is or isn't
in user.properties (or other properties files outside your source base).
In the illustration below, id.start is defined such that it has a default
value of 1. The fields in
${__partner( field1 , field2 , field3 )}
are:
variable name
value assigned to (don't know what this is)
default value
(Put elsewhere in this document:)
The ${__P(var, value)} variable is defined usually on the command line or
defaulted as shown above for variables HOSTNAME , PORT
and PROTOCOL . The second field is the default used when the definition
cannot be found.
Redefining what's already been defined...
In the following illustration, HTTP Header Manager defines a number of
states and expectations. These are what must appear in the HTTP request header
passed to each invocation in the test.
However, to kick off our test, we've got to draw an OAuth token from the OAuth
service. We must not pass Authorization in the header because that's
not recognized (or tolerated) by OAuth. Therefore, we redefine it not to exist
simply by "defining" it as not containing anything. We do this separately and
specifically in the HTTP Get OAuth Token step. We also redefine Content-Type
for that step because OAuth requires application/x-www-form-urlencoded
instead of application/json .
Practical advice...
When running against your local web application, you might not want to leave
some of the real load test variables the same. For example, I didn't want to
run a test that created 100,000 users, used SSL and the wrong port number on
my local development host. So I changed these values as shown below to 10 users,
port 8000 and HTTP instead of HTTPS .
But, don't save these changes unless you mean to.
Also, don't run your web application in Eclipse in debug mode unless you want
to babysit it against stopping on a breakpoint. On the other hand, running
in debug mode with a breakpoint set would be a good way to track down something
that only happens under load, in a big, long test, etc.
Some very practical examples...
This shows creating test steps that draw on the defaults and definitions discussed
in earlier sections.
Add a new action, i.e.: a new test step in a restlet, here, one called "Account
Manager". We right-click Users here. Most any action or test step we would
want to define would be a Sampler , here specifically, an HTTP Request .
We want to specify creating a new account. Under that action, as it's going to be
a POST operation, we can define the request payload. Notice that we
prescribe the URL (path) and the JSON payload. JMeter excels in setting up load
testing; so we use a sort of variable, ${USER_ID} , already defined, that
will be incremented each time a new account is created so that we can create any
number of account names (identities). This was handled by the User Incrementer
higher up. Here, the password for each account is also varied.
After the operation is conducted, we are interested in the result. So, we add a
Response Assertion .
In the response, we expect a response payload and, inside that payload which we
know to be JSON, will be an "oid". So, we tell JMeter that the response must
contain the text "oid".
Next, we want the value of that oid because it's concocted live by the underlying
database. We cannot predict its value. So, we extract it via a Regular Expression
Extractor :
We configure the extracto to look in the body (of the JSON response payload), for
the oid key-value pair and extract it. This involves knowledge of regular expressions:
we match the pattern enclosed between parens and say to use (match) pattern number 1.
The template is something I haven't grok'd yet: see the JMeter documentation. (Sorry.)
Arbitrarily, it's decided to stuff this value into a variable named acctOid ,
since this is the oid of an account. This variable is herewith defined: it did not
need to be defined higher up.
Last, we show how acctOid is used by creating HTTP Find Account .
This is a GET operation. Right-click Users and add this new
HTTP Request action filling it out as below. The URI syntax dictates that
we can find an existing user (we want to find the one just added) by adding its
oid to the command line. This is the oid we just saved.
Last, after all the messing around we could do, we want to remove the account
created as part of the test. This is so that the next time we run JMeter, the
application won't complain that the user already exists. For this application,
this isn't usually a permitted operation, so accommodate must be made to use
the superuser's entity key, something that will force it to do the delete
anyway. Notice acctOid again in the command line.
Here's how to inject a change into the HTTP header: we add a new Config
Element , HTTP Manager with the superuser's key. Everything from
the original HTTP Manager is upheld as it's inherited, but this one
aspect is changed by us and only for the delete operation.
Here's the HTTP Header Manager change (the entitykey):
Getting results...
This is where the View Results Tree and Summary Report come in.
You need them so you can see what is going on or happened.
Illustration of JMeter in Jenkins...
This illustrates how JMeter is set up in Jenkins for an application named
accountmgr . This is only for the purpose of illstrating what needs to be
or can be set up in Jenkins configuration.
Things not to do with JMeter scripts...
Don't make the script too heavy. Reduce the number of validations, logic,
response parses, debuggers, log-in information that weigh down the
script.
Consider real-world use of what you're testing. Will users have cookies
that need to be erased? Do you need to parameterize the headers to make
the script more flexible? What response assertions will be needed?
Don't use absolute filesystem paths, so the script can be run on other hosts
without modification. Gather filenames only; make the paths the object needs
either relative or configurable.
Don't leave URLs that don't really matter or are less readable, stuff from
Google Analytics, plug-ins, Windows Update, etc.