Grinder Load Testing Framework

This page expands on grinder.sourceforge.net/g3/getting-started.html as a step-by-step "classroom" approach to installing, configuring, running, and making use of this free tool for performance testing. Along the way, I hope to point out internals and insights gained from experience.

Topics this page:

Site Map  
About this site  

Introduction

Grinder Java load testing framework, downloaded from sourceforge.net/projects/grinder. It's described in J2EE Performance Testing with BEA WebLogic Server (Expert Press and now by APress 2002) by Peter Zadrozny, Philip Aston and Ted Osborne.

Grinder3 (in Beta as of Dec 2006) uses the Jython scripting engine based on Python.

Phil Aston now tirelessly answers questions on these forums:

Grinder-use forum

Grinder-Use mail list on sourceforge

grinder-development archive

gmane.comp.java.grinder.user searchable archives.

his article dated 11/11/2002 describes how the net.grinder.Console in CLASSPATH=/opt/grinder/lib/grinder.jar controls net.grinder.Grinder agents running on client machines.

Download and Installation

At the time you read this, the version number may have likely moved on from "grinder-3.0-beta32" created December 14, 2006. But you get the idea...

Get announcements of new releases by subscribing to the low-volume mailing list grinder-announce@lists.sourceforge.net.

If it's not already on your system, download and install:

• Java 5 or 1.4+ SDK. Specify the full path of its jar files in your machine's CLASSPATH environment variable.
• Since files are in zip (compressed) format, Winzip or $20 Power Archiver. You can also use the jar program that comes with Java.

Download from grinder.sourceforge.net the package "Grinder 3":

• grinder-3.0-beta32-src.zip
• grinder-3.0-beta32.zip

Both these zips contain folder "grinder-3.0-beta32", so you need to unzip them to the same folder (such as C:\ root), and accept the overwrite of common files.

• grinder-3.0-beta32.zip contains folders contrib, examples, lib
  In the "lib" sub-folder are compiled bytecode jar files grinder.jar and jython.jar.
  In the "examples" sub-folder are sample Python scripts processed by classes in jython.jar.
• grinder-3.0-beta32-src.zip contains folders docs/javadoc, etc, native, src, src-j2se5, tests-src
  In its "src" folders are ".java" source code files (and associated CVS flags) buried within a library hierarchy:
  
  src/org/syntax/jedit
  src/org/python/core/ClonePyInstance.java
  src/"HTTPClient" -- the Java plug-in for testing HTTP services, which is why you need this even though you may not need/want to change the Grinder product's source code.
  src/net/grinder/ contains java source for each of the Grinder programs: "Console", "Grinder" agent, and "TCPProxy".
  src-j2se5/net/grinder/util contains "NanoTimeTimeAuthority.java"

  Folders within "tests-src" parallel the "src" folder:

  "src/net/grinder/"
  "HTTPClient", which contains file "TestCookie.java"

  Files in the "docs/javadoc" folder are now blank. Previously, they contained files generated by Sun's javadoc utility reference css and properties files in the "etc" folder, where the all-important " grinder.properties" file.

• The "J2EE Performance Testing Source Files 1.0" package dated March 27, 2004:
  • source.zip contains folder "performance-book-1.0" containing files mentioned in the book "J2EE Performance Testing".
    Unzip it within C:\ since it's not specific to any release of the Grinder.
  • chapter_code.zip contains folders Ch04, ch03, ch06, ch07
    unzip it within performance-book-1.0
  • spreadsheets.zip contains files "CH04.xls" and "CH03.zip"
    unzip it within performance-book-1.0
    which creates subfolder Sample_Analysis_Spreadsheets.
    Unzip "CH03.zip" within it or move it to performance-book-1.0/ch03.

• grinder-documentation-20060409.zip is no longer available for download. Previously:
  Its "development" folder
  Its "g2" folder contains a copy of internet pages such as this what's New in Grinder 2 page.
  Its "g3" folder contains a copy of internet pages such as this What's New in Grinder 3 page.
  The "images" folder stores jpg files.
  Its "skin" folder...

• If you have beta-29 or newer, install jython from the lib folder. Or you can download Jython 2.2a and follow these install instructions.

Jakarta Commons HttpClient

Agents read from a common network a share grinder.properties file which specifies the number of worker processes, threads, plug-ins, etc.

Configuring Grinder

1. Add the lib folder to the java CLASSPATH. On Windows, these two are equivalent:
   C:\grinder-3.0-beta32/lib/grinder.jar;C:\grinder-3.0-beta32/lib/grinder-j2se5.jar
   C:\GRINDE~1.0-B/lib/grinder.jar;C:\GRINDE~1.0-B/lib/grinder-j2se5.jar

   If you create a custom jar file, add its path (with the jar file name) to the CLASSPATH environment variable.

2. Create a folder such as "mygrinder" within folders containing source and other assets for your project.
   Do this instead of the recommendation on Grinder's website, which places custom Grinder files within a "project" folder among other Grinder files.

   Avoid putting your own files within the default folder name (e.g., "C:\grinder-3.0-beta32"). Inevitably, when versions change, you would need to figure out which files need to be saved or end up spending frustrating hours debugging why new versions don't work.
   Also, if you leave the examples folder untouched, you'll have working examples to reference.

3. Use my instructions to define an environment variable such as "GRINDERENV" to specify that location.
4. Create in your grinder project folder a grinder.properties file.

   Copy the sample grinder.properties file from the Grinder download examples folder.
   But open it using Wordpad (not Notepad).

   The common convention is to put properties files within the "etc" folder.
   However, you may find it more convenient to have it in the same folder as your python scripts.

5. Edit the grinder.properties file contents:

   The "#" in "#grinder.useConsole=false" comments out a line to be ignored.
   Without this specification, the default of "true" is automatically applied.
   But I prefer an explicit specification, even if it is the same as the default.

   grinder.useConsole=true
   grinder.consolePort=6372
   

   The consolePort specifies the port through which the Console listens for communications with Grinder agents (not the port used by the application under test).

6. Verify that the Java Runtime Environment was installed correctly and available for use at the location specified by the CLASSPATH enviornment variable.
   java -version

Invoke the Modules

1. Establish the "current" directory holding user-specific project files.
   cd \mygrinder
2. Run the Console and Grinder agent in separate command windows (by clicking Start, Selecting "Run...", typing "cmd", and clicking "OK"),

   Facility Desc.	Invocation command in Windows	Result	src Folder
   Console	java net.grinder.Console	-	Console
   Agent	java net.grinder.Grinder %GRINDERPROPERTIES%	waiting for console signal	"Grinder"
   Proxy	java net.grinder.TCPProxy -console -http > grinder.py	-	"TCPProxy"

   Its java net.grinder.TCPSniffer proxy creates test scripts from HTTP traffic captured from the browser.

The agent will finish immediately if the Console is not already invoked.

Note: Ironically, the "grinder.py" default python script file is not provided by default.

The Grinder's "Console" is like LoadRunner's "Controller". It is the central program (on a "main" machine) which starts and stops load generator ("injector") worker processes which interpret and carry out Jython test scripts. During a test run, it collates the "samples" it receives, and displays statistics summarizing the activity.

Unlike LoadRunner (which presents activity of all agents on a single screen), the status of each Grinder agent is presented on a separate window. Also, I haven't see where Grinder calculates all the analysis that LoadRunner provides (graphs of standard deviation, 90th percentile, median, etc.).

Console Invocation

There are several ways to specify the scripts to be distributed by the Console and executed by the Grinder agent.

1. Point the default folder to where the grinder.properties file is located, then invoke the Console:

   c:
   cd c:\mygrinder
   java net.grinder.Console

2. Set an environment variable "GRINDERPROPERTIES" to hold the grinder properties file path, then invoke the Console with that variable:

   
   set GRINDERPROPERTIES=c:\grinder-3.0-Beta32\examples\grinder.properties
   java net.grinder.Console %GRINDERPROPERTIES%

   This invokes Grinder using the "grinder.properties" file in the current directory. The Task Manager sees a new "java.exe" process appear. Exit.

   If the Console comes up with a pop-up message "Failed to bind to console address, check options", the properties files may not have been recognized.

   There are several reasons why you get this message:

   Exception in thread "main" java.lang.NoClassDefFoundError: net/grinder/console
   • The "C" in "Console" is not in upper case. If you type the lower case "console", expect the message.
   • You did not establish the CLASSPATH correctly (as explained above).

   If you don't see see the Console, maybe your "grinder.properties" file contains "grinder.useConsole=false".

   Adding # in front of this line (commenting it out) is usually preferred to changing "false" to "true".

   
   Closing the command window will make the Console go away.
   Closing the console will make the command window go away.

   My custom Windows command file was adapted from sample "startConsole.cmd" Windows command script:

   SET GRINDERCLASS=C:\grinder-3.0-beta32
   call %GRINDERCLASS%\setGrinderEnv.cmd
   java -cp %CLASSPATH% net.grinder.Console
   pause Press Ctrl-C
   

   This example uses the GRINDERENV environment variable and invokes the JRE specified in the CLASSPATH environment variable.

   More about the "setGrinderEnv.cmd" file later.

   To make this execute every time you boot up, create a link to this within your "Startup" folder.

   Click the "Distribute" menu option and "Set directory...". In the pop-up "Set the root directory for script distribution" look in the path where your python scripts are stored. For example: "C:\grinder3.0-beta32\examples".
   Note: Unfortunately, the list of files do not appear.
   Press "Set directory" button on faith that the files are in the path selected.

   When you press the "Distribute files" menu item, notice the messages appearing in the Grinder agent command window.
   Also notice that a folder named "...-file-store" is created on the agent machine.

   To confirm, click the "Script" tab and click on the little light blue "lightbulb" icon to the left of the desired folder path.

Grinder Agent Remote Worker Processing

Until an "agent" is active, the Console GUI "Action" menu options are greyed out.

If you need multiple agents:

1. Find the IP addresses. On each machine:
   ipconfig /all

   Look for IP Address: . . . . . . . . . . . . : 70.7.129.255

2. Change your grinder.properties file to specify the IP address of your Console machine:
   grinder.consoleHost=127.0.0.125
3. Make sure the Console machine can connect to the agent. Run a command prompt and type:
   ping 10.1.1.197:6372

   Note: Port 6372 is the default port for the agent and worker processes to contact the Console.

   telnet 10.1.1.197 6372 quit
4. Run another command Window and invoke the Grinder agent on the agent machine:
   java   net.grinder.Grinder

   "waiting for console signal" should appear.

   Under the Console's "Processes" tab, a new process (identified by the machine name) should have State "Connected"

Double-click on the script file to load it on the script frame.

Right-click on the script file you want to run and select "Set Script" from the context menu. For example, "console.py".

To start load testing, select "Action" menu option "Start processes".
At the center left panel, "Collecting Samples:" should appear in green.

Click on the "Processes" tab to make sure it says "Connected".

Note: The console.py script results in just samples being generated.

Once the bar passes by, click "Stop Processes".

Click "Results" to see that the "Description" column contains a value such as "Log method" defined in "hello.py" script code "test1 = Test(1, "Log method")".

Click "Collection".

Click "Collect Statistics" (from agents).

Now use Windows Explorer to take a look at the logs.

Use Wordpad to view the logs, which were created with UNIX type lines rather than Windows type lines.

Clicking "Stop Collecting" terminates the agent process, requiring you to start it again from the command line.

Properties Configuration

Script files in the grinder.properties file are only referenced if a script references script names (such as the helloworld.py, goodbye.py) with code

scripts.values(): 

in sequence or in parallel

First, define

grinder.processes=1
grinder.threads=2
grinder.runs=10

To create a "log" folder to hold logs:

grinder.logDirectory=logs
grinder.numberOfOldLogs=2
grinder.logProcessStreams=false
		

To ...

grinder.jvm.classpath=build/classes;build/test
grinder.jvm.arguments=-Dpython.home=/applications/jython-2.1
		

grinder.script=helloworld.py

To configure Grinder for automatic redirection ...

"performance-book-1.0"

The "EPizza" folder contains "test_scripts.zip"

The "Pet Store" folder contains files "grinder.properties" and "http-plugin-sniffer-post-340"

Recording Scripts

Scripts can be created by recording actions of a real user using the TCP Proxy described in this tutorial.
1. Start the TCPProxy:
   java net.grinder.TCPProxy -http -localPort 8001 > myscript.py

   These are the defaults (with no keystore) if it were started without any parameters. Note that parameters -httpplugin and -newhttpplugin have been deprecated.

   Add the "-console" parameter if you are using cygwin that do not allow Java processes to be interrupted cleanly.

2. Configure an internet browser (such as IE or Mozilla) to be proxied through port 8001:

   Download Mozilla Firefox or another alternate internet browser and set the proxy there so that you don't have to keep changing browser settings when you want to use your regular browser to check emails, etc.

   Alternately, if you stay with Firefox, you can use Jeremy Gillick's SwitchProxy Firefox extension to quickly switch proxy settings.

Each recording session creates two files: httpscript.py and httpscript_test.py

The latest beta records scripts with tests grouped into pages, making it easier to comment out whole pages at once.

3. Entries in the grinder.properties file:
   grinder.test0.parameter.url=http://www.google.com

Introduction to Jython Part 1 and Part 2 from IBM DeveloperWorks.

Common Script Edits

Default script files that are installed were created with Unix/Mac format of just "\n" rather than PC-DOS "\r\n" format. So on PC's open script files with Wordpad rather than Notepad.

Every Grinder script defines "TestRunner" class which instantiates each worker thread. That is repeatedly called for each test (iteration) of that thread.

class TestRunner:
# This method is called for every test run iteration:
def __call__(self):
logWrapper("Hello World")

Because these are Python scripts, each request must begin with two (and only two) tabs.

The HTTPclient plug-in is used through the facade obtained


from net.grinder.script import Test
from net.grinder.script.Grinder import grinder
from net.grinder.plugin.http import HTTPPluginControl
from HTTPClient import NVPair
from java.net import InetAddress

The "Test" class imported above references "test0", "test1", etc. in the grinder.properties files referenced in these statements:

tests = {
"News01" : Test(1, "News 1 posting"),
"Sport01" : Test(2, "Sport 1 posting"),
"Sport02" : Test(3, "Sport 2 posting"),
"Trading01" : Test(4, "Trading 1 query"),
"LifeStyle01" : Test(5, "LifeStyle 1 posting"),
}

Because test sequences are defined manually, inserting tests within a list would require the list to be manually renumbered

To specify logging:
log = grinder.logger.output;

To reset:

HTTPPluginControl.getConnectionDefaults().setProxyServer("localhost",8001)

To ...

test = Test(1, "getByName")
instrumentedGetByName = test.wrap(InetAddress.getByName)

class TestRunner:
def __call__(self):
instrumentedGetByName("myhost")

To us a regular expression:

pattern = re.compile("(" + stringName + "=[\w&#;]+)&"
# ...
result2 = pattern.search(target, 0)
value = result2.group(1) # everything captured between the parens

Grinder scripting tutorial on looping through an XML request file for values to use in HTTP requests.

Running Scripts

Each "virtual user" test context runs within its own thread.

A single worker process can manage several worker threads.

At the Console:

• Select files to download to each agents' <path>
• Press the "Send changed files to worker processes" button. This transfers scripts to each agent machines' folder named < machine name >-file-store\incoming
• When the agent runs, logs are sent to the terminal if the script has (before the interface) defined:

  out = grinder.logger.TERMINAL

  If the script has:

  out = grinder.logger.LOG

  then Grinder creates (if it doesn't already exist) a folder named according to the grinder.properties file entry

  grinder.logDirectory=logs

  Within this folder the agent stores several files for each test, named using the agent's machine name:

  out_<machine name>-<test number>.log
  data_<machine name>-<test number>.log

  But before that, files left over from a previous run are renamed with a sequence number:

  
  out_<machine name>-<test number>.log00001
  error_<machine name>-<test number>.log00001

  The number of previous runs kept this way is controlled in grinder.properties file entry

  grinder.numberOfOldLogs=2