Runtime Reporting

Module: Driver Framework


For long running, monitored benchmarks, it is desirable to learn early on whether the run meets the expectations. Faban has a runtime reporting feature that calculates the results of the run based on a time period immediately preceding the report as well as the overall results up to the time of the report. The metrics reported in the runtime report generally converge towards the final report in a steady (non-load varying) run.

Currently, if enabled, the runtime metrics are reported into the run log, one entry each time, for each benchmark driver type. The reports lend themselves very well to runtime graphing. This is, however, not available at this time.

Once enabled and configured, Faban will write a report as in the following example to the run log for each time period:

65.00s - WebDriver: MyOperation1/MyOperation2/MyOperation3

The following describes each of the output fields:

    65.00s - WebDriver: MyOperation1/MyOperation2/MyOperation3

The displayed stats are displayed for each operation and are labeled as follows:

Cthru current thruput (as of the last interval period)
Othru overall measured steady state thruput, not available before steady state
CErr current error rate
CResp current response time (avg of the last interval period)
OResp overall avg steady state response time, 0 before steady state
CSD current standard deviation of response time
OSD overall standard deviation of response time, not available before steady state
C90%Resp current 90th% response time as of last interval period, not available before steady state
O90%Resp overall steady state 90th% response time, not available before steady state


To enable runtime reporting, simply set the runtime stats to true in your runConfig part of the configuration file:

<fa:runConfig definition="sample.driver.WebDriver"
    <runtimeStats enabled="true">

Once the run ramp up starts, a stats record will be written to the log every number of seconds specified by the interval.


There are no restrictions or limitations for enabling the runtime reporting. However, there may be performance implications. With the runtime reporting disabled, there is absolutely no crosstalk between the agents. Enabling the runtime reporting causes each stat from each thread to be read for each of the given interval. This will create some crosstalk, not much - but may be enough to impact scalability of large scale runs with very many driver agents. increasing the inteval time should reduce the impact.