Multi-Tier Reference Apps for Benchmarking
This series lists the benchmarking applications available, providing an analysis of the installation and operation of key benchmark apps.
|
|
|
|
| 
|
|
|
|
|
|
|
|
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Repeat | Input Screen Change (LR Transaction Name) | T | DB Use | Notes | |
|---|---|---|---|---|---|
| n/a | 
| Configuration | - | - | - |
| Once for Init. (Entry) | 
| I01_InvokeSAPGUI | - | - | - |
| I02_Logon_SAP | - | - | - | |
| I03_EasyAccess_menu | - | - | - | |
| (15 posting steps) |
| R01_VA01_Invoke (Create Sales Order) | VA01 | - | - |
| R01a_Order_Type_Lookup | - | 30% | - | |
| R01b_Sales_Org_Lookup | - | 30% | - | |
| R01c_Dist_Channel_Lookup | - | 30% | - | |
| R01d_Sales_Division_Lookup | - | 30% | - | |
| R01e_Sales_Office_Lookup | - | 30% | - | |
| R01f_Sales_Group_Lookup | - | 30% | - | |
| R02_Second_screen_5_items | - | - | - | |
| - | R03_Second_screen_Save | - | - | - | |
|
| R04_VL01N_invoke (Create Outbound Delivery with Order Reference) | VL01N | - | - | |
| - | R05_First_screen | - | - | - | |
| R06_First_screen_Save | - | - | - | |
| R07_VA03_invoke (Display Sales Order: Initial Screen) | VA03 | - | - | |
|
| R08_Display_customer_order_Enter | - | - | - | |
| R09_VL02N_invoke (Change Outbound Delivery) | VL02N | - | - | |
| - | R10_Choose_[F20]_Post_goods_issue | - | - | - | |
| R11_VA05 (List of Sales Orders) | VA05 | - | - | |
| - | R12_List_orders_Enter | - | - | - | |
| - | R13_VF01_invoke (Create_invoice) | VF01 | - | - | |
| - | R14_Create_invoice_Save | - | - | - | |
| Once to Exit | - | X01_Call_end | - | - | - |
| - | X02_log_off | - | - | - | |
| X03_log_off_Confirm | - | - | - | |
So 400 x 15 = 6,000 posting dialog steps are processed per baseline hour (excluding entry and exit steps, which are not counted in the metric).
With 10 seconds of user "think time" between user actions and an average 2 second response time, a baseline benchmark run takes 12 seconds x 6,000 steps = 72,000 seconds or 72,000 / 360 sec./hr. = 200 user hours, which means 200 concurrent active users in a given hour.
My assumption is that all users are logged in and actively working
(although realistically there should be some).
Will faster servers or more RAM enable the environment to support more simultaneous users and process more transactions per hour by a corresponding larger number of emulated users?
On May 24, 2007, Oracle announced that its 10g RAC database running SAP ERP 2005 (2-tier) on IBM AIX machines with two dual-core 4.7 GHz POWER6 processors achieved 20,120 SAPS by generating 402,330 fully processed order line items/hour using 1,207,000 posting dialog steps/hour. 4,010 users were emulated, so each user generated 301 posting dialog steps per hour or 301 / 60 = 5 steps per minute, which is 60 / 6 = one every 12 seconds, since the average dialog response time was 1.96 seconds — 10 seconds is added to emulate user "think time" between each step.
These results were optained with the CPU at or near 100% utilization,
so these numbers should be considered maximum possible values.
Results for Sales and Distribution (SD), run by IBM in Beaverton, OR, using these configuration defaults, are:
2-tier, where a single central server runs the database and all application instances.
3-tier, where a single central server accesses the database on another server.
3-tier parallel, where several central servers accesses the database on another server,
with users equally distributed across all database nodes (using a round-robin-method).
The test transactions access the main tables of the SAP Sales & Distribution SAP application (SD):
Combining the SAPS metric with the SPECInt benchmark rating for the server hardware used for the test
enables capacity comparisons and estimation.
Although not all
SAP modules (applications)
are modeled,
SAP's IDES (Internet Demonstration and Evaluation System)
contains master, transaction, and customizing data
for real-life business scenarios that can be run in the SAP System
of integrated Financials, HR, and Logistics applications for
a model international company with
subsidiaries in several countries producing products using several
processes (engineer-to-order, make-to-order, and mass production;
repetitive and process chemical/pharmaceuticals manufacturing, retail).
With each major product release SAP updates IDES for download from the SAP Service Marketplace (with a valid SAP S-User ID). IDES is used in training exercises and SAP's system demonstration portal.
|
|
|
|
In Sep 2005, for
BEA 9.0, Medrec
core J2EE components were redesigned to use a
download of the
open-source Spring framework 1.2.5 based on
|
|
|
|
|
|
|
|
|
| Task | Application |
|---|---|
| 1. Startup | OS |
| 2. Duplicate 500MB file | |
| 3. Open Multiple Folders | |
| 4. Create Zip Archive of 1GB folder | |
| 5. Unzip 1GB file archiver | |
| 6. Word Scroll | Microsoft Office X with Rosetta to run on Intel-based Macs |
| 7. Word Search/Replace | |
| 8. Download E-mail | |
| 9. Convert AAC files to MP3 from Hard Driver | iTunes 6.0.3 |
| 10. Export to QuickTime for Email | iMovie HD |
| 11. Apply Aged video effect | |
| 12. Import 100 photos from hard driver | iPhoto 6.0.1 |
| 13. Multiple Page Loading Test | Camino 1.0 |
| 15. Render | Cinema 4DXL 9.1 |
| 16. Antalus Botmatch at 1024x768 Max Settings with sound and graphics enabled—Average Frames Per Second | Unreal Tournament 2004 |
Charles Gaba's System shootouts
SPECjAppServer2004The $2000 jAppServer2004 multi-tier benchmark application (at v1.05) measures the performance of a single J2EE v1.3 application server running all major J2EE technologies:
The app simulates Dealer, Manufacturing, Supplier and Corporate domain logical entities.
SPECjAppServer2004
result reports use the performance metric of the number of JOPS
(jAppServer Operations Per Second) completed during the Measurement Interval.
JOPS is composed of the total number of business transactions completed in the Dealer Domain,
added to the total number of workorders completed in the Manufacturing Domain, normalized per second.
The app includes a Supplier Emulator Java Servlet that can run inside any Java enabled web server to emulate the sending and receiving of orders to/from suppliers.
The app includes a client driver (run on a separate machine) that exercises all parts of the underlying infrastructure that make up the application environment:
However, "SPECjAppServer2004 strives to stress the middle-tier rather than the client tier or the database server tier."
The Standard Performance Evaluation Corporation (SPEC) is a non-profit corporation formed to establish, maintain and endorse a standardized set of relevant benchmarks that can be applied to the newest generation of high-performance computers. SPEC develops suites of benchmarks and also reviews and publishes submitted results from their member organizations and other benchmark licensees.
TPC-AppThe TPC-App benchmark web services app simulates activities of a distributor operating business-to-business transactional application servers operating in a 24x7 environment. TPC-App showcases the performance capabilities of application server systems.
The workload was published August 2005 to exercise commercially available application server products, messaging products, and databases associated with such environments, which are characterized by:
TPC-App
result reports use the performance metrics of the number of SIPS
(Service Interactions Per Second) completed by each application server
during the Measurement Interval.
"Total" SIPS refers to the entire cluster of servers in the entire configuration (SUT).
The lone report on 6/21/05 measured 174.9 SIPS per server.
The distinctiveness of TPC benchmarks is that the SIPS metric is associated with
dollar costs, such as the $327.41/SIPS published for an IBM eServer xSeries x366
using a Intel Xeon DP 3.60GHz CPU running
MS.NET 1.1 and Microsoft SQL Server 2000 on Windows 2003 Standard Edition.
Important Notes:
"The workload was designed specifically to stress the Application Server. As such, the work to be performed by the database was purposely minimized."
"TPC-App does not benchmark the logic needed to process or display the presentation layer (for example, HTML) to the clients."
The non-profit Transaction Processing Performance Council is based in San Francisco, California, USA.
|
|
|
|
Related:
| Your first name: Your family name: Your location (city, country): Your Email address: |
Top of Page
Thank you! | |||
Operating system software vendors:
Instead of remoting with stateless session EJBs,
"MedRec-Spring" exposes service beans via Spring’s HTTP Invoker architecture of
POJOs (Plain Old Java Objects).
Spring's Inversion of Control (IoC) features
containers that inject dependencies into configured components.
