Previous Beginning

Services/Tools Deployment


Lets first go through our checklist. At this point we should have the followings in our project structure:

  1. The service classes, if any
  2. The tools classes, if any
  3. The deployment descriptor services-tools.xml
  4. The toolsets configurations toolsets.xml

A set of services and tools can be deployed as part of a benchmark as well as deployed standalone in its own package to be used by multiple benchmarks.

  

The Service/Tools Deployment Descriptor

The services/tools deployment descriptor is named services-tools.xml. For a Faban benchmark or services/tools project, it is located in the deploy directory. Once packaged, this file will be in the META-INF directory of the deployable jar file. It is a simple xml file that declares a set of services and tools. It also binds tools to services the tool would observe. Below is an example of a services-tools.xml deployment descriptor:

<?xml version="1.0" encoding="UTF-8"?>
<services-tools>
    <service id="MySQLService">
        <class>com.sun.services.MySQLService</class>
    </service>
    <tool id="mysqlstats" service="MySQLService>
        <class>com.sun.tools.MySQLStats</class>
    </tool>
    <service id="MemcachedService">
        <class>com.sun.services.MemcachedService</class>
    </service>
</services-tools>

A services/tools deployment can contain multiple services and multiple tools. Consequently, the services-tools root element can have multiple service child elements and multiple tools child elements.

The service element declares a service. It must contain an id attribute specifying the service id. The service id must be unique across all service deployments. The service id is the one and only way a benchmark run would specify the needed services in the run configuration file.

The tool element declares a tool. It must contain an id attribute specifying the tool id. The tool id is the way a benchmark run would specify a tool to run in the run configuration file. The tool id is also referenced in the toolsets.xml, discussed later. Also it may contain a service attribute linking the tool to a service. If the tool is linked to a service, it will be able to obtain the service configuration through its ToolContext. Note that the service id or name is global. A tool can reference a service even though it is deployed in a different jar and deployed at a different time (as long as it is deployed on the master). This allows users to add tools referencing a service without being the same implementor or bundling in the same services/tools bundle.

The class element is used as a child element to either the service and tool elements. This is a required element linking the service, by id, to the fully qualified name of the service class or tool class the Faban harness needs to invoke when specified in the run configuration.

Toolset Configuration

The toolset configuration file - toolsets.xml - is located the deploy directory of a Faban project. It resides in the META-INF directory for a jar. Note that the project's deploy directory maps to the META-INF directory at the time we build the jar. Also, for OS toolsets, a similar file with the same name may also reside in the FABAN_HOME/config/<OS> directory. It defines the toolset for the operating system in question. Below is an example of a toolset configuration file:

<?xml version="1.0" encoding="UTF-8"?>
<toolsets>
    <toolset>
        <service>MySQLService</service>
        <name>default</name>
        <includes>mysqlstat</includes>
        <includes>vmstat 10</includes>
        <includes>iostat -xn 10</includes>
        <includes>mpstat 10</includes>
    </toolset>
    <toolset>
        <service>MySQLService</service>
        <name>detail</name>
        <base>default</base>
        <includes>cpustat</includes>
        <includes>trapstat</includes>
    </toolset>
    <toolset>
        <service>MySQLService</service>
        <name>lite</name>
        <base>default</base>
        <excludes>iostat -xn 10</excludes>
    </toolset>
</toolsets>

Each toolset element defines a toolset which is identified by the toolset name.

The service element binds the toolset to a service. For OS toolsets, the service element shall be omitted. It will implicitly refer to the OS as the services.

The name element sets the name the toolset is to be referred. The toolset name is unique in the scope of the service it refers. Therefore, the same toolset name can be used for multiple services and the OS. The reserved name "default" is used to specify default toolsets for a service or for the OS. If no tools are specified, the tools specified in the default toolset are getting invoked.

The includes element includes a tool command or name into the toolset.

The excludes element excludes an previously included tool command or name from the toolset.

The base element includes all the tools included in another toolset. In conjunction with include and exclude, it allows forming a toolset from another toolset and making minor modification to the set of tools specified without having to specify the whole list of tools over and over again.

Tool Output

Any arbitrary tools can be configured  either in the toolset configuration file as described above or manually during every run. Faban has in-built support for tools like vmstat, iostat, mpstat, nicstat (for network statistics), etc. These tools will be run at the start of the steady-state period and stopped at the end of steady-state. The tool output will automatically appear in the Statistics tab in the Run result view. You can add any custom tools but for their result to show up in the Statistics tab, the tool output log should follow this naming convention: <tool>.log.<host> where <tool> refers to the tool name and <host> identifies the host it was collected from. The host name should be one of the hosts in the benchmark configuration. For example, when iostat is run on a host called "dbserver", it's output will appear in the log file iostat.log.dbserver.


Building and Packaging

Now we have all the pieces we need for packaging. Next step is to build and package up the services into a deployable unit called the deployable jar file.  This may be part of a benchmark jar or a separate service/tools jar. The structure is the same. We usually use the build.xml, our ant build script, to package both benchmarks and services. The build.properties file controls how the build script builds, packages, and deploys the jars. For service packages, lets look at the file from our MySQL sample. Other samples use the exact same structure:

service.shortname=MySQLService
faban.home=../../..
faban.url=http://localhost:9980/
deploy.user=deployer
deploy.password=adminadmin
compiler.target.version=1.5

The service.shortname (in case of a benchmark deployment this property is called bench.shortname) field names the generated libraries and deployable file name. It is also used throughout Faban and referred to as the short name. Please note that the bench.shortname value CANNOT have any dot characters in the name.The faban.home field references the path where the Faban kit is installed on your system. This path can be either relative or absolute and is used for the build script to find all the necessary Faban compile-time and deploy-time libraries. The compiler.target.version controls how the classes are compiled. Faban supports both 1.5 and 1.6. A benchmark build with 1.6 can use the JavaSE 6 features. However, it cannot be used on J2SE 1.5. Other fields are used for deployment and will be discussed in the next section.

Once we have all these fields configured properly, we can build our deployable jar file by just calling ant from the base directory of our project. This will build the default target: deploy.jar.

$ ant

Or

$ ant deploy.jar

To call ant from an IDE, just have the IDE build the deploy.jar target directly. Ant will create a deployable jar file and place it into the project's build directory.

The deploy.jar target builds and packages the deployable jar file according to the Faban benchmark deployable jar file structure described in the followings:

<service.shortname>.jar
META-INF
services-tools.xml
toolsets.xml
bin
lib
<other_libs>.jar
<service.shortname>.jar
public_html
resources

The structure of the deployable jar file is a subset of the project structure described in the development overview with the content of the deploy directory being placed in the jar's META-INF directory. The scripts and binaries are archived directly into the jar's bin directory. The lib directory is also archived directly with the addition of the jar of our benchmark classes. This jar is also named after the benchmark's short name but is located in the lib directory. The public_html and resources directories are archived directly from their corresponding directories in the project struture.

Deployment

At this point we should be ready to deploy. Lets go back to our build.properties file and address the entries used for deployment. Here is the file once again:

service.shortname=MySQLService
faban.home=../../..
faban.url=http://localhost:9980/
deploy.user=deployer
deploy.password=adminadmin
compiler.target.version=1.5

The faban.url field refers to the URL of the Faban harness/master we want to deploy to. The deploy.user and deploy.password fields are required if and only if security is enabled in the Faban master. By default they are not needed. If security is enabled, the deploy user and password is NOT the same as the user password provided by a security provider in order not to expose a person's password in a file. The deploy user and password can be configured in the config/harness.xml file and defaults to deployer and adminadmin, subsequently. please change this if security is a concern.

Once these fields are set correctly, just run

$ ant deploy

The ant deploy target will internally contact the Faban master specified by faban.url, upload the deployable jar file, and execute the deployment process on the master. Note that the deploy target will automatically call the deploy.jar target to build the deployable jar before deploying, if not already done so. There is no need to build the deploy.jar target prior to the deploy target.

For a service that is deployed as part of a benchmark, only that particular benchmark may refer to the service and invoke the service. For services in their own service bundle, however, all benchmarks may refer to the newly deployed services from their configuration files. The service will get invoked as specified in the benchmark configuration file.

Alternate Deployment Methods

The section above describes the deployment method that is most intuitive for developers. However, there are two other deployment methods that may be used:

    1. Using the deploy form. Point your browser to http://<faban_master>:9980/deploy and select the deployable jar file. Enter the deploy user name and password if security is enabled on the master. The "Clear previous benchmark configuration" checkbox does nothing when deploying a service. Once all is done press the deploy button. The deployable jar file will get uploaded and deployed on the Faban harness. This method is also shown in the Deploying Benchmarks section of the Faban Harness Users Guide. The same interfaces is used for both deploying benchmarks and services.

    2. Copy the deployable jar file directly into the services directory in the Faban master installation. DO NOT unjar the file. Faban will automatically pick up this jar file the next time you use your browser to access Faban.

Troubleshooting Deployment

Once the services are deployed, benchmarks can refer to them via the configuration file. If a particular service is deployed in the benchmark deployable jar file of the benchmark, it can only be used by that benchmark. Service deployables can be referred to by both the benchmark and the service.

If the service cannot be used by a benchmark, the errors will get logged into the run log for the benchmark run. The error messages indicate the problem with the service. Check the contents of the run log to diagnose service issues and correct your deploy image accordingly.

PreviousBeginning