Introduction

The Faban driver framework is a framework is a Java™ platform-based framework that allows developers to easily define new benchmarks using the Java Programming Language. It makes use of the Java 2 Standard Edition (J2SE) 5.0 features, primarily annotations as documented in

        http://java.sun.com/j2se/1.5.0/docs/relnotes/features.html#annotations

to ease benchmark development. The benchmark developer will only need to focus on the benchmark driver logic. The timing and selection of the operations, including recording of the metrics is handled by the driver framework. 

Design Goals and Features

The goal of the Faban driver framework is to allow for extremely easy and efficient creation of new benchmark drivers for N-Tier benchmarks driven from network-connected driver systems.  New and added operations should be no more complex than a script, which, in its very basic form can be captured. Specifications for the driver is given in descriptive form. The driver framework supports the following features:

1. Rich stochastic model/policy for driving the benchmark
2. Clear separation between driver and timing controls, data generation, protocol handling, measurements, and post-processing of the measurements
3. Clear separation between spec and configuration
4. Inter-operation data sharing (same thread, same driver)
5. Descriptive driver specification
6. Controlling rampup, steady state, and ramp down times or number of iterations
7. Implicit, automated response time measurements
8. Calibrated sleep or think times
9. Synchronized ramp up, steady state, and ramp down times between driver systems
10. Arrival rates or think times
11. Random value generation
12. Rich, extensible set of transaction mixes
13. Extensible network protocol support
14. HTML response verification/parsing
15. Standardized, extensible statistic collection
16. Standardized, extensible driver report
17. Extensive error or failure management
18. Managed premature benchmark termination in case of error

Definition of Terms

The System Under Test (SUT) comprises all components being tested and varies from benchmark to benchmark. The driver framework and the driver is usually not part of the SUT. Some non client-server benchmarks can however not differentiate between the SUT and the driver making the driver implicitly part of the SUT.

An Operation is a single unit of work executed by the user or the driver and timed, and managed by the driver.

An Operation Mix defines how operations are executed by a simulated user or driver thread.

An Operation Cycle defines the timing characteristics from one operation to the next.

A Metric is the resulting data of the operations collected over the steady state time of the benchmark operation.

Cycle Time defines the elapsed time between the begin of the previous operation to the begin of the current operation.

Think Time defines the elapsed time between the end of the previous operation to the begin of the current operation.

The Critical Section defines the portion of an operation in which the response time is measured.

Data Preparation Section defines the portion of an operation before the critical section where the data gets prepared.

Validation Section defines the portion of an operation after the critical section where the results get validated.