rel="stylesheet" type="text/css"> Previous BeginningNext

Benchmark Deployment


Lets first go through our checklist. At this point we should have the following in our project structure:

  1. The benchmark class
  2. The configuration file
  3. The submission form

You're almost there. There is just one file missing before packaging it up - the benchmark deployment descriptor.

The Deployment Descriptor

The benchmark deployment descriptor is a file named benchmark.xml in the benchmark deploy sub-directory. It is a very simple XML file that tells Faban about all the benchmark resources. This file is optional for certain cases wth the following criteria:

  1. The benchmark is implemented using the Faban driver framework
  2. DefaultFabanBenchmark is sufficient as the benchmark class
  3. All other settings follow the default, i.e. using run.xml as the config file name and config.xhtml as the submission form name

Your first few benchmarks will likely fit into this criteria, making it simple to use default values. However, once you start implementing your own benchmark class and use any non-default settings, the deployment descriptor comes in handy. Let us take a look at the full deployment descriptor, with its default values:

<?xml version="1.0" encoding="UTF-8"?>
<benchmark>
    <name></name>
    <version></version>
    <config-form>config.xhtml</config-form>
    <config-stylesheet></config-stylesheet>
    <config-file-name>run.xml</config-file-name>
    <benchmark-class>com.sun.faban.harness.DefaultFabanBenchmark2</benchmark-class>
    <result-file-path>summary.xml</result-file-path>
    <metric></metric>
    <scaleName></scaleName>
    <scaleUnit></scaleUnit>
    <banner-page>bannerPage.html</banner-page>     
</benchmark>

The name element lists the long name describing the benchmark. This element is mandatory. If the benchmark is implemented using the Faban driver framework, the name of the benchmark as defined in the @BenchmarkDefinition annotation becomes the default name. If the name element is specified, it will override the @BenchmarkDefinition annotation. 

The version element describes the benchmark version. This is a string field so versions with dots and alphanumerics - such as 1.01a - are allowed. This element is mandatory. If the benchmark is implemented using the Faban driver framework, the version of the benchmark as defined in the @BenchmarkDefinition annotation is used as the default version. If this element is specified, it will override the @BenchmarkDefinition annotation.

The config-form element contains the file name for the submission form. This element is optional and defaults to config.xhtml.

The config-stylesheet element references the stylesheet used for displaying the submission form. If this element has any value, the corresponding stylesheet must be in the benchmark's META-INF directory. This field is used only in case the default display of the configuration form is inadequate. It is very rarely used.

The config-file-name element contains the file name for the configuration file. This element is optional and defaults to run.xml.

The benchmark-class element contains the fully qualified class name of the benchmark class to use for this benchmark. This element is mandatory. If and only if the benchmark is implemented using the Faban driver framework, this field defaults to com.sun.faban.harness.DefaultFabanBenchmark. Otherwise there is no default value for this field.

The result-file-path element contains the name of the benchmark result file other than summary.xml. This is used for benchmark integrations that produce their own result file and do not wish to use the summary.xml file format. A minimal summary.xml must still be generated to provide the minimim result metadata that Faban needs to understand. This element must not be used for benchmarks implemented using the Faban driver framework.

The metric element describes the metric of the benchmark results as it should be reported and listed in the result list. This element is mandatory . If the benchmark is implemented using the Faban driver framework, the version of the benchmark as defined in the @BenchmarkDefinition annotation is used. This metric element, if exists, overrides the metric from the Faban driver framework.

The scaleName element describes the name of the scale to use, such as "concurrent users". The scaleUnit element provides a unit for the scale. The only difference between these two elements is that the scaleName is formatted as a prefix and the scaleUnit is formatted as a suffix. At least one of these two elements must be provided. If the benchmark is implemented using the Faban driver framework, the scaleName or scaleUnit is described in the @BenchmarkDefinition annotation. These elements, if they exist, override the @BenchmarkDefinition annotation in the driver framework.

The banner-page element containes the name of the banner page. For now we just allow custom banner(html file) with height not greater than 80 pixels. The banner page needs to be placed under BenchmarkDir/public_html/. This element is optional and defaults to custom banner.

Building and Packaging

Now we have all the pieces we need for packaging. The next step is to build and package the benchmark into a deployable unit called the deployable jar file. The ant build script, build.xml in the top-level benchmark directory can be used for this. The build.properties file controls how the build script builds, packages, and deploys the benchmark. Here is the file from the web101 sample.

bench.shortname=web101
faban.home=../../..
faban.url=http://localhost:9980/
deploy.user=deployer
deploy.password=adminadmin
deploy.clearconfig=false
compiler.target.version=1.5

The bench.shortname field names the generated libraries and deployable file name. It is also used throughout Faban and referred to as the short name. Please note that the bench.shortname value CANNOT have any dot characters in the name.The faban.home field references the path where the Faban kit is installed on your system. This path can be either relative or absolute and is used for the build script to find all the necessary Faban compile-time and deploy-time libraries. The compiler.target.version controls how the classes are compiled. Faban supports both 1.5 and 1.6. A benchmark build with 1.6 can use the JavaSE 6 features. However, it cannot be used on J2SE 1.5. Other fields are used for deployment and will be discussed in the next section.

Once we have all these fields configured properly, we can build our deployable jar file by just calling ant from the base directory of our project. This will build the default target: deploy.jar.

$ ant

Or

$ ant deploy.jar

To call ant from an IDE, just have the IDE build the deploy.jar target directly. Ant will create a deployable jar file and place it into the project's build directory.

The deploy.jar target builds and packages the deployable jar file according to the Faban benchmark deployable jar file structure described in the followings:

<bench.shortname>.jar
META-INF
benchmark.xml
config.xhtml
run.xml
bin
lib
<other_libs>.jar
<bench.shortname>.jar
public_html
resources

The structure of the deployable jar file is a subset of the project structure described in the development overview with the content of the deploy directory being placed in the jar's META-INF directory. The scripts and binaries are archived directly into the jar's bin directory. The lib directory is also archived directly with the addition of the jar of the benchmark classes. This jar is also named after the benchmark's short name but is located in the lib directory. The public_html and resources directories are archived directly from their corresponding directories in the project struture.

Deployment

At this point, the benchmark is ready to deploy. Edit the build.properties file to address the entries used for deployment. Here is the file once again:

bench.shortname=web101
faban.home=../../..
faban.url=http://localhost:9980/
deploy.user=deployer
deploy.password=adminadmin
deploy.clearconfig=false
compiler.target.version=1.5

The faban.url field refers to the URL of the Faban harness/master we want to deploy to. The deploy.user and deploy.password fields are required if and only if security is enabled in the Faban master. By default they are not needed. If security is enabled, the deploy user and password is NOT the same as the user password provided by a security provider in order not to expose a person's password in a file. The deploy user and password can be configured in the config/harness.xml file and defaults to deployer and adminadmin, subsequently. please change this if security is a concern. And finally, the deploy.clearconfig field tells the Faban master to clear and rename old configuration files in the profiles on the master. When the new submission form is no longer compatible with the old configuration file saved in the profiles, the flag will need to set to true. After redeployment, the submitted run will use the default configuration file bundled in the benchmark instead of the saved ones from previous runs in the profile.

Once these fields are set correctly, just run

$ ant deploy

The ant deploy target will internally contact the Faban master specified by faban.url, upload the deployable jar file, and execute the deployment process on the master. Note that the deploy target will automatically call the deploy.jar target to build the deployable jar before deploying, if not already done so. There is no need to build the deploy.jar target prior to the deploy target.

Now, use your browser to point to your Faban master system at port 9980, select the "Schedule Run" hyperlink and your submission form should show up. If you have an active rig with multiple benchmarks deployed, your session may have been set to a different benchmark. In this case, select the "Switch Profile" hyperlink and the newly deployed benchmark should be listed in the list of benchmarks.

Alternate Deployment Methods

The section above describes the deployment method that is most intuitive for benchmark developers. However, there are two other deployment methods that may be used:

    1. Using the deploy form. Point your browser to http://<faban_master>:9980/deploy and select the deployable jar file. Enter the deploy user name and password if security is enabled on the master. Check the "Clear previous benchmark configuration" checkbox, if applicable. Then press the deploy button. The deployable jar file will get uploaded and deployed on the Faban harness. This method is also shown in the Deploying Benchmarks section of the Faban Harness Users Guide.

    2. Copy the deployable jar file directly into the benchmarks directory in the Faban master installation. DO NOT unjar the file. Faban will automatically pick up this jar file the next time you use your browser to access Faban. Note that this method does not have anything equivalent to the deploy.clearconfig field to automatically clear and rename old, incompatible configuration files. If needed, you have to go into the config/profiles directory in the Faban master installation and manually remove or rename the applicable saved configuration files.

Troubleshooting Deployment

Once the benchmark is deployed, it should show up in the "Switch Profile" screen (if you have multiple benchmarks deployed) and the corresponding submission form should show up when scheduling a run. If not, there are two places to look for information:

    1. If the benchmark cannot be seen by Faban, something went wrong during or after deployment. The problems are logged in the Faban log file at logs/faban.log.xml. Check the contents of this file for exceptions and error messages, and correct your deploy image accordingly.

    2. If the benchmark can be seen but the submission form refuses to show up or behaves incorrectly when you select the different tabs, there might be issues with form display or variable bindings. These issues will be shown in the Chiba log file at master/logs/chiba.log. Check the contents of this file for incorrectly bound elements and display issues and verify the submission form. Also note that changing the configuration file structure without deploying with deploy.clearconfig set may also have caused the issue. Verify the deploy.clearconfig flag (or it's equivalent checkbox in the deploy form) and redeploy will remedy this issue.

Previous BeginningNext