Scripting Rational Robot SQA Scripts

Here are my notes on writing scripts run by the automated software testing tool Robot from IBM Rational. This summarizes stacks of manuals and web pages.

Topics this page: Just as with C and other programming languages, Robot scripts make use of several constructs: shell scripts, Modules, Functions, Constants, Library Files, HeaderFiles, Variables, Data Types, and Branching Logic, explained below:

Site Map  
About this site  

Different Files

Within Robot, when you pull down menu File > New , you see the different types of files testers might create:

SQABasic Files contain procedural (programming) code to recognize objects and use variables and constants in if-then-else, do-while, and other logical constructs. In larger projects, script code is modularized for reusability. Reusable code is organized into functions called from a script library header file. The name of such header files are included in scripts that use those functions.

Project Header Files structurally ensure that both function libraries and scripts that use them declare those functions and global constants the same way. Header files declare constants, variables, custom sub procedures, and custom functions which are common across an organization's scripts. This allows for code reuse in many scripts, therby also standardizing programming practices.

GUI Shell Scripts call script procedures (stored using a .rec file extension). Unlike functions, procedures are executed without parameters.

“Test automation must be approached as a full-blown software development effort in its own right. Without this, it is most likely destined to failure in the long term.” — Carl Nagle

Raw Scripts

If you use Rational Robot using just a single script file, here is an example of the SQABasic commands you will likely use:

1. Dimension counter variables and constants. This could be done with an include statements.
2. Launch test support utilities (such as performance loggers, keyboard macro, or screen capture programs) that run in parallel with the test, such as these DDE Driver Debug commands:
   Breakpoints "ON"
   CommandDebug "OFF"
   TestDebug "ON"
   RecordsDebug "OFF"
3. Initialize run variables with DDE Counter commands such as GetSystemDateTime and
   StartTestCase, Version,
   StartProcedure, StartCycle, StartSuite
   StartCounter
4. Define handling of unexpected screens (such as Windows AutoComplete and application pop-ups).
5. Create temporary work files being reused from prior tests or copy in files needed for the test:
   GetAttr, ChDrive, ChDir, FileAttr, FileCopy, Kill, RmDir,
6. Connect to the Test Results database to signal start of test:
   SQLOpen, SQLGetSchema, SQLError
7. Start the application under test:
   LaunchApplication - StartApplication "MyVBApp.exe"
   or
   StartWebBrowser - StartBrowser "http://www.yahoo.com/", "WindowTag=WEBBrowser"
   or in RobotJ:
   startApp("ClassicsJavaA");
8. SetApplicationMap - linking application objects to internal identifiers:
   SQAGetProperties
9. Invoke reusable functions to perform actions:
   CallCycle, CallSuite, CallStep, CallScript
10. Delay/Sleep/Pause so the application is ready to respond to user requests:
    WaitForWebPage, WaitForGUI, WaitForGUIGone, WaitForPropertyValue, WaitForPropertyValueGone
11. SetFocus to a foreground process with a Window handle (HWND).
12. SetContext to specifically recognize an object. Web Example:
    Window SetContext, "WindowTag=WEBBrowser", ""
13. Exercise application functionality operations by using the application's Navigational, GUI and other Controls to select menu item, import file, input/change/delete data, save, print, export, etc.
    InputKeys to send text to the specified component
    InputCharacters to send character(s) to the specified component
14. Analyze the results from verification points and SQAScriptCmdFailure command to simulate script failures. If Robot's Options | Error Recovery menu option "On Script Command Failure" is set to "Continue Execution", any failure in a subsidiary script called will return to the shell script. A global result variable returns all manner of status info to the shell script, including the last step or phase executed.
    GetGUIImage to save the screen shot of a component to a bmp or jpg file
15. Log results at key points in the script using DDE Log Commands or these SQABasic commands:
    SQALogMessage sqaNone,"Status is " & SQLQueryResult(1,I) & "" & SQLQueryResult(2,I),""
16. Update the Test Results database:
    SQAVpLog, SQLRequest. SQLError
17. CloseApplication and databases:
    Window CloseWin, "", ""
    SQLClose
18. Do other clean-up to return to a starting point common to all test runs.
19. Stop timer.

    The Rational LogViewer is automatically displayed after the script is done.

From Rational's Documentation CD:

SQABasic Language Reference

Documents describing some Guidelines for writing Robot SQABasic scripts (moved from Yahoo on 10/02):

SQABasic Code Conventions
Testability Guidelines
Robot Header & Library Files.html

Comment Blocks

Comments for human consumption can be helpful. So most organizations require within each file:
• Description (what this artifact is expected to do)
• Dependencies (other artifacts who expects this artifact)
• Creation (who, including phone and email address)
• Change history (By, Date, Change)

Because all this information are required inputs into Change and Configuration management (CCM) systems, including them at the beginning of each source file seems redundant and indeed can breed conflicting information.

One piece of comment that typically do not overlap with CCM information is a summary of the major functions performed within the file. These become major headings within the file.

Rather than using menu Insert > Comment, just type a quote mark ('), the Basic language code for lines to be ignored by the compiler.

Include Statements

Instead of rewriting the same code in every script: The '$Include compiler directive metacommand is added in scripts to instruct the compiler which header files need to be expanded during compile time. An example is this line that enables scripts to use Rational-supplied utilities to control datapools:
'$Include "sqautil.sbh"

It may be confusing to some that this statement begins with an apostophe ('), which is Basic language syntax for a comment line ignored by the compiler. Also notice that the file name is encased around double quotation marks and is not case sensitive. The file extension is .sbh.

This file lives in Robot's sqabas32 folder within Program Files, which makes it available to all scripts. This file declares:

1. the parameters of functions that can be called through the library.
2. variable names and values for constants referenced by calling scripts.




• Declarations inside a header have a different syntax than in a script
• The location of '$Include statements determine its scope of reference. When the command is located before the first procedure in the module, the header file declarations apply to all the procedures in the module.
• Not iterative references: a library file cannot contain a header file which references that same library file. A library file also cannot have the same name as the script file (.rec) that calls it. For instance, myscript.rec cannot call a function in myscript.rec.

If you are first getting started with Rational Robot, I suggest that you begin by studying the Test Automation Frameworks developed by Carl Nagle of SAS Institute, Inc. and improved upon by Paul Downes and others under Open Source.

Download their Software Automation Framework Support (SAFS) library containing keyword (data) driven execution engine for automated application-independent tests running under Rational Robot.

This product's .msi installs about 5 MB (261 files) into default folder Program Files \Rational \Rational Test \sqabas32\ (where Rational put "sqautil.sbh"). Files in that location is available to all files in the repository.

Their setup instructions asks that you create a DDE_RUNTIME environment variable to that location.

After, do File > Batch Compile.

Includes Enable Modularization

Script files can simply call library functions when include files modularize test suites for reuse:
• a Library (.sbl) file containing functions named to encapsulate script commands that exercise the application under test;
• a Function Header (.sbh) file that use Declare function commands which define the parameters scripts use to call the library's functions.
• a Constants Header (.sbh) file that use Const commands to define values for variable names referenced by calling scripts.

  A good convention is to name constants header files by appending “_x” or “_c” to the filename of its associated function header file.

  One good use of a constants is the amount of time to delay before attempting to recognize objects on each screen that appears. Over time, as accumulated data put a load on the system or as performance improvements are implemented, the delay amount may change.

Object Map Files

The major reason for using Robot is that it can recognize objects and inject actions in other applications.

But SQARobot assumes that testers are willing and able to write programming code to specify how objects can recognize a particular object or attribute, such as.

HTMLImage Click, "Name=txtLogin", "Coords=35,6"

If the application's programmer changes the name "txtLogin", the playback script will no longer work. Adapting to this means testers have to hunt through every script where the name is used.

An alternative is to have the tester define tester's variable name in an include file with this technical information as its value, then use that variable in all scripts. Such a file is called a Map file because it maps names assigned by testers to the technical references Robot needs to recognize objects. With map files, changes will only have to be made in one place (the include file), which is automatically percolated to all scripts.

RobotJ creates a map file automatically during script recording. RobotJ also assigns a weighting value to each object to specify how heavily RobotJ will rely on that object for recognition.

Map files can be generated by the DDE Process Container utility.
• Creating an Application Map for the DDE Engine, such as this Sample Application Map for the DDE Engine

  Robot can recognize objects based on object name, label, index value, or ID value.

Shell Scripts

In RobotVB, a script file that calls another script file (procedure) cannot pass values.

A call to a function can pass values as part of the call.

Shell scripts play back other scripts in sequence. This sample call executes a DOS batch file in the C:\ root drive:

SQAShellExecute "cleanup.bat", "C:\", ""

Using them allow results from several scripts to be stored in the same Log file (which simplies results analysis and data management)

To avoid problems, shell scripts should not manipulate the environment.

Using the SAFS/DDE Library

To use this library, create a GUI script based on the DDNotepad.txt sample file.

You will not see a file if you simply copy it into your project folder. You must use File > New > Script and paste the code within Robot so that Robot can add it to its database of scripts in your project.

All DDE assets are defined with one line:

'$Include "DDEngine.sbh"

The include enables use of DDE functions such as this to enable "LogMessage" functionality:

InitLogFacility msgId, MainLog, logFile

The include also defines global variables (such as MainLog) and constants such as these stored in variable msgID used to determine when logging occurs:

(TEXTLOG_ENABLED OR SQALOG_ENABLED OR CONSOLE_ENABLED)

Additional constants and variables can be defined, such as the location of the test logfile and DDE's test summary report:

Const SaveASFile = "C:\TEMP\MyNotes.txt"
logFile = = SQAGetDir(SQA_DIR_PROJECT) &"MyTestLog1.txt"

The SQABasic command SQAGetDir(SQA_DIR_PROJECT) returns the file path of your test library. But beware that the DDE library assumes the presence of some folders that newer versions of Robot no longer create. So, you must create these folders before using the library.

To invoke a data-driven test using the SFAS/DDE library, the user script invokes one of the drivers functions from the DDE Library along with the file that controls it:

SDStepDriver "StepFile.txt", ";", MainLog
This script contains DD Driver Commands that control Driver functions:

Before all this, dimension variables and constants and delete any preexisting files output from prior tests:

on error resume next
Kill logfile
Kill SaveASFile
on error goto 0

If there is no file to kill (such as when the script is run the first time), on error resume next tells Robot it's OK.

Afterword, invoke DDE's clean-up routines:

AUTerminateAllApps
CloseTextLog MainLog, 1
Reset

DDE Step Files

The SAFS/DDE library has a sample StepDriver file: NotePadStepFile.txt. It contains records such as this:
c, SetApplicationMap, "NotepadMap.txt"

Parameters are position dependent. The first parameter position (which is not case sensitive) defines the type of request:

"S" or "s" records are skipped (treated as comments) by the driver.
"C" or "c" (command) records specify actions affecting the test environment, such as "ExitCycle" to abort processing.
"CW" expects a Warning result
"CF" expects a Failure result
"T" or "t" (Test) control records specify test actions using "component functions" of the SAFS/DDE library.
"TW" expects a Warning result
"TF" expects a Failure result
"B" or "b" records define where blocks of commands begin and end.
"BP" or "bp" records define breakpoints recognized during executions when Debug Mode is enabled and BREAKPOINTS has been turned "ON".

Other examples:

T, WindowID, WindowID, AWindowAction, Param1, ... 
t, AddUserAccount,  , ^user=Name
t, LoginWindow , UserIDField , VerifyProperty , "Text" , "userid"
t, LoginWindow , UserIDField , SetTextValue   , ^USER = "MyUserID"
t, LoginWindow ,   OKButton  ,   Click
t, MainWindow  , MainWindow  , VerifyProperty , "Caption", ^USER

The ^ caret identifies variables from literal strings and is not actually part of the variables' names.

Blocks of commands

To set the block named AExitHandler to execute if/when ExitTable is executed:
C, SetExitTableBlock, AExitHandler
To reset so no special handler is invoked if/when ExitTable is executed:
C, SetExitTableBlock, ""

The DDE library also provides several Flow Control commands to exit to default script blocks:

SetExitTableBlock
SetNoScriptFailureBlock
SetScriptWarningBlock
SetGeneralScriptFailureBlock
SetInvalidFileIOBlock
SetScriptNotExecutedBlock

Different Drivers

The SFAS/DDE library also has two higher level driver functions:

The SFAS/DDE SuiteDriver works with user-defined lists of StepDriver tables that test an entire area of AUT functionality. Examples of test suites:

1. A build viability test suite that walks down the menu to verify that every application module can be invoked.
2. A performance measurement test suite that exercises time-critical functionality of the application to detect how long users need to wait for their critical requests to be processed.
3. A functional viability test suite that inputs the minimum amount of data to exercise processing functions in the application. Examples: Define a Guide and guide package, Create Order, Search Order, Create a Forecast, Adjust Forecast, Release Forecast, etc.
The steps within a suite may be segregated by the criticality of functionality, such as online search vs. data entry vs. creating reports.

The CycleDriver (at the highest-level) invokes sets of suites. For example, there may be different control files to verify Integration Readiness, Systems Integrity, Functional Regression Stability, etc. Additionally, different cycles could correspond to each version or language of the AUT.

Sample commands to call these drivers:

CDCycleDriver "Cycle1.txt", ";", MainLog
Within the Cycle1.txt file are these statements:
STSuiteDriver "Suite1.txt", ";", Suite1Log
STSuiteDriver "Suite2.txt", ";", Suite2Log

• Flow Control Commands for the DDE Engine

A sample script

declare sub CheckSave    'allocate name defined below.
'$Include "sqautil.sbh"   'used for datapool scripts.
'$Include "global.sbh"    'used by all scripts.
Sub Main
	Dim Result As Integer
	'Explicitly declare variable:
		dim strOpSys As String, strFileName As String
		dim strScriptUsed As String, strReturn As Integer
		const SCRIPTUSED As String = "Using Script 5/8/99"

	'Assuming that the application is opened by the calling module:
	'Press Alt-File, Save As to open dialog box:
		InputKeys %"FA"
		'    MenuSelect "File->Save As..." not used

	'Analyze pop-up window to detect and handle unexpected dialog:
		'Instead of Window SetContext, "Caption=Save As", "" 
		'which is not generic:
		Window SetContext, "Current Window" 
		'Get object name...
		'Is it save as? ...

		'Any other errors?
		Assert ... 
		'If an error is found, stop! ...
		'Push a button that says "Save"
		PushButton Click, "Text=Save"

	SQADataPoolOpen
	Do while not EOF(1)
		'Fetch the next record from the datapool:
			SQADataPoolFetch
		'Retrieve a value from the datapool:
			SQADataPoolValue
		Input #1 strOpSys, strFileName
		if strOpSys = strOpSysUT then 'good, 
			numReturn = CheckSave()
			'Set field needed for either good or bad:
				(strScriptUsed = SCRIPTUSED)
			if numReturn> 0 then 'good, 
				'Write good record to Verification Log:
				SQAVpLog (strScriptUsed)
			else
				'Write goon record to Verification Log:
				SQAVpLog (strScriptUsed)
			end if
		end if
		Loop
	End Do
	SQADataPoolClose

	'Exit application:
	Window SetContext, "Current Window" 
	MenuSelect "File->Exit"

End Sub

Sub CheckSave
	InputKeys strFileName
	'Detect to see if an unexpected screen.
		Assert ...
	'Otherwise:
		EditBox Click, "name=Save*"
		InputKeys strFileName

	'Detect to see if file was really saved:
		GetAttr, ChDrive, ChDir, FileAttr, FileAttr,
	'Set return signal: if good:
			Pos1 = 1
		else
			Pos1 = 0
		end if
End Sub

Function Libraries

The Robot product comes with a few functions in its global library source file named Global.sbl which compiles into the Global.sbx file.

Defining functions in a common source library allow them to be used by others. The DDE Library does this on such a massive scale that most web applications do not need custom functions.

To define a custom function:

Function
...
End Function When a script wants to use a function, it first has to know what parameters to exchange with that function. Here is the format for declaring a custom procedure (MySub) in an SQABasic library file (MyLib.sbl):

Declare Sub MySub BasicLib "MyLib" (arg1 As String, arg2 As Integer)

To call a function:

SQAGetProperty
Pos1 = instr(clipBoardString, "find me")
Call instr(clipBoardString, "find me")
instr acton:="clipBoardString", value:="find me"

The last statement uses named arguments. A function's argument become parameters controlling the function or sub-procedure called. Double parentheses around an argument indicate that it cannot be changed by the procedure called.

Ideally, each function should return a result code indicating success or failure. The value of that result should use constants SUCCESS or FAILURE so that it can be customized. Some prefer zero to indicate failure. Microsoft prefers numeric "-1" to indicate failure.

Functions return a string or numeric value to the module which called it. But Sub Procedures do not return values to its caller.

The pointer to Library Files is stored in the Windows Registry key CommonDir under HKEY_LOCAL_MACHINE/SOFTWARE/Rational Software/Common   (from Luke Goodwin)

One advantage of using the SAFS/DDE library is that it performs additional functionality not provided in native SQABasic commands. For example, SAFS/DDE String Utilities mimic the SQABasic string Trim$ functions. But it also trims TAB characters.

Dimensioning Variables and Constants

Both variables and CONSTANTS can be passed from a calling function to a script using that function. CONSTANTS are defined in header files using a phrase like this:

Const TESTID As String = "Test B.3.a"

Constants are in all upper case by convention.

Within a function library, a custom data type is defined by a phrase such as this:

Type CustData
CustName As String
CustID As Long
End CustData
To reference a custom data type:
dim Customer As CustData
Customer.CustName = "Jennifer Farriday"
Customer.CustID = 533128
• Arrays are multi-dimensional (table) data. dim varray() sets the maximum dimensions to 8 rather than 60 if you use the Redim command.

  Redim	Declare dynamic arrays and reallocate memory.
  Erase	Reinitializes the contents of an array.
  LBound	Return the lower bound of an array subscript.
  UBound	Return the upper bound of an array subscript.
  Option Base	Declare the default lower bound for array subscripts.

• A Variant can take on different data types.

  The script name Prefix (up to 30 characters) isassigned automatically in the "GUI Record Options" dialog box. Robot will append incremented numbers to this text.

  Shell "Notepad.exe",3 InputKeys "Hello, world.{enter}Notepad is maximized."

  Define variables to read data files

• If Not EOF(iFileNum) Then
  
  dosomething
  
  End If

Variables and Their Use

DD Variables

Getting & Keeping Focus

To activate a specific window: AppActivate WindowTitle

If more than one window has that title, a window is chosen at random. And the comparison is NOT case-sensitive!

Robot waits for the time value defined in the global value ______

• Establishing Focus:
  • By default, if Robot cannot find a window during playback, it waits for 2 seconds and then looks for it again. If it still cannot find the window after 30 seconds, it times out and returns a command failure indication. Script execution then continues or stops based on the On script command failure setting in the Error Recovery tab of the GUI Playback Options dialog box.

• Test for Windows which "Does Not Exist" (Unknown Objects)

• Wild cards
  • Window SetContext, "Caption={Customer No: *}"
  • Window SetContext, "Current Window"
    tells the script to enter actions on whatever window already has the focus.
  For more information on using wildcards or the partial caption matching feature, refer to the topics "Matching Window Captions with Wildcards" or "Partial Window Caption Matching Defaults" (respectively) in Robot's on-line Help.

Handling Delays and Errors

Insert delay values with SetTime(milliseconds) and ResetTime commands
1000 milliseconds = 1 second
60000 milliseconds = 1 minute

Handling unexpected Windows and messages

Data Pool Utilities

To open a datapool, specifying all parameter values avoids problems:
dp=SQADatapoolOpen("NameOfDatapool", TRUE, SQA_DP_SEQUENTIAL, TRUE)

The nSequence parameter above uses these Constants from DpConst.sbh:

SQA_DP_RANDOM = &H00000001
SQA_DP_SEQUENTIAL = &H00000010
SQA_DP_SHUFFLE = &H00000100

To retrieve the value of the specified datapool column.

SQADatapoolValue

To move the datapool's cursor to the next row:

SQADatapoolFetch

To reset the cursor:

SQADatapoolRewind

To close a datapool:

SQADatapoolClose

Rational provides header file dbconst.sbh (in its sqabas32 folder) to guarantee that scripts calling utility functions use the same definitions the utility scripts use. The constants for return values from SQADatapool functions:

sqaDpSuccess = 0
sqaDpUnitialized = -1
sqaDpFailure = -2
sqaDpEOF = -3
sqaDpInvalidArgument = -998
sqaDpExtendedError = -999

Using Data Pools

Verification Points

Verification points (VPs) inserted into scripts determine whether the AUT performed as expected.

From the Robot Tools menu, select Inspector....

This gathers information about every object (seen or unseen) of every process running on the test machine.

Scroll through the Object Hierarchy to find the Caption= for your application under test. Click the [+] to expose objects for that window. Expand the "Generic,Class=Shel DocObject View;ClassIndex=1" Object, then the "HTMLDocument,HTMLTitle=..." displayed by your app.

Click each of these objects to view the value of each object's properties, such as width, height, innerText and outerHTML. Multiple instances of the same type are differentiated with an Index=value.

The name of Verification Points must be 20 characters or less and contain no special characters such as periods and dashes. Individual VPs are stored in the project's vp folder. Robot automatically creates VP files with the script name and file extension .obp. If you right-click on a VP item and select copy, you can paste it into another script's VP list.

Remember (in Robot world) you must right-click on "Verification Points" in order to paste. Robot automatically deletes and re-creates the .obp files.

Robot allows VP's to be created on the fly with these SQABasic commands:

SQAVpGetCurrentBaselineFileName,
SQAVpGetBaselineFileName,
SQAVpGetActualFileName

DDE General Verification Functions:
VerifyProperty 'Verify a single property value
VerifyPropertyContains 'Verify property value contains a string
VerifyValueContains 'Verify variable value contains a string
VerifyPropertyToFile 'Verify a large string property value with a file benchmark
VerifyClipboardToFile 'Verify clipboard contents with a file benchmark
VerifyClipboardVP 'Perform a Robot ClipboardVP on clipboard contents
GUIDoesExist 'Test for the visual existence of a Window or Component
GUIDoesNotExist 'Test for the visual non-existence of a Window or Component
AssignPropertyVariable 'Assign a property value to a specific DDVariable
VerifyTabOrder 'Verifies the Tab Order of child components
VerifyFileToFile 'Performs an ASCII comparison of two files (identical to VerifyTextFileToFile)
VerifyTextFileToFile 'Performs an ASCII comparison of two files (identical to VerifyFileToFile)
VerifyBinaryFileToFile 'Performs a binary comparison of two files
VerifyGUIImageToFile 'Verify the screen shot of a component with a benchmark bmp or jpg file
VerifyObjectDataToFile 'Verify object data with a file benchmark
CaptureObjectDataToFile 'Capture object data into a specified file

Error Messages

Procedures communicate their runtime completion status by setting the Err global variable. It is a number between 1 and 32,767. So User-defined error codes should be larger than these values. This is initialized to value 0 to indicates no error occured. Error$ provides the message text.

To simulate an error use the SQABasic command Error.

Debugging

When debugging a script,
• If a module fails to load, it may be because a global variable or constant with the same name as an existing one has a different data type.
• use Animation Play Mode to see each script line as it executes. The yellow cursor points to the next script line to be executed.
• Check "Acknowledge results" in "GUI Playback Options".
• There are several ways you could temporarily pause execution to examine the value of a variable or to check the state of an object before it is modified by a subsequent command.
  • Step Into a script to execute it one step at a time, pressing F11 after each script statement. Or Step Over individual script commands and stop after every called subprogram is executed.
  • Set Breakpoints to programmatically stop excution at designated points in the script.
• Issue log event messages from scripts with
  SQALogMessage Result, "Message text"

• Issue messages to the console window with
  SQAConsoleClear and SQAConsoleWrite "Start of Playback"
• Robot uses the Trap utility to detect General Protection Faults (GPF's) and store information about the state of the Windows session that was running. That data placed in file Sqatrap.log is in Common Object File Format (COFF).

• Up to 20 timers can be running with utility commands
  StartTimer "020"
  StopTimer "020"

• Put error handling code in a separate section of the procedure
  On Error
  On Error

• Settings for user toolbar settings are stored by Robot in the sqa.ini file [SQA ROBOT TOOLBAR SETTINGS] section.