com.sun.faban.driver
Class HttpTransport

java.lang.Object
  extended by com.sun.faban.driver.HttpTransport
Direct Known Subclasses:
ApacheHC3Transport, SunHttpTransport

public class HttpTransport
extends Object

The HttpTransport provides initialization services and utility methods for using the HTTP and HTTPS protocols. The convention for the method names in this class are as follows:

Currenly, the HttpTransport class does not provide a way to keep binary data for further analysis. This function can and will be added if there is a use case for keeping such binary data.

Author:
Akara Sucharitakul

Field Summary
static int BUFFER_SIZE
           
 
Constructor Summary
HttpTransport()
          Deprecated. Replaced by the newInstance() method
 
Method Summary
 void addTextType(String texttype)
          Add a MIME type to the list of text types.
 byte[] downloadURL(String url)
          Retrieve large response from the URL and returns the data read.
 String dumpResponseHeaders()
          Utility class to get responseHeaders as a string.
 StringBuilder fetchPage(String page, String[] images, String postRequest)
          Makes a POST request, fetches the main page and all other image or resource pages.
 StringBuilder fetchResponseData(InputStream stream)
          Fetches the data from the stream, converts to char, and returns it as a StringBuilder.
 StringBuilder fetchResponseData(Reader reader)
          Fetches the data from the reader and returns it as a StringBuilder.
 StringBuilder fetchURL(String url)
          Reads data from the URL and returns the data read.
 StringBuilder fetchURL(String url, Map<String,String> headers)
          Reads data from the URL and returns the data read.
 StringBuilder fetchURL(String url, String postRequest)
          Makes a POST request to the URL.
 StringBuilder fetchURL(String url, String postRequest, Map<String,String> headers)
          Makes a POST request to the URL.
 StringBuilder fetchURL(URL url)
          Reads data from the URL and returns the data read.
 StringBuilder fetchURL(URL url, Map<String,String> headers)
          Reads data from the URL and returns the data read.
 StringBuilder fetchURL(URL url, String postRequest)
          Makes a POST request to the URL.
 StringBuilder fetchURL(URL url, String postRequest, Map<String,String> headers)
          Makes a POST request to the URL.
 StringBuilder fetchURL(URL page, URL[] images, String postRequest)
          Method not implemented.
 int getContentSize()
          Obtains the size of the last read page or resource.
 org.apache.commons.httpclient.Cookie[] getCookies()
          Obtains the list of all cookies
 String[] getCookieValuesByName(String name)
          Obtains the list of cookie values by the name of the cookies.
 StringBuilder getResponseBuffer()
          Obtains the reference of the current response buffer.
 int getResponseCode()
          Obtains the response code of the last request.
 String[] getResponseHeader(String name)
          Obtains the header fields of the last request's response.
 boolean isFollowRedirects()
          Checks whether the connections managed by this transport follows redirects or not.
 boolean matchResponse(InputStream stream, String regex)
          Matches the regular expression against the data read from the stream.
 boolean matchResponse(Reader reader, String regex)
          Matches the regular expression against the data read from the reader.
 boolean matchResponse(String regex)
          Maches the regular expression against the data in the current buffer.
 boolean matchURL(String url, String regex)
          Matches the regular expression against the response fetched from the URL.
 boolean matchURL(String url, String regex, Map<String,String> headers)
          Matches the regular expression against the response fetched from the URL.
 boolean matchURL(String url, String postRequest, String regex)
          Mathces the regular expression against the response fetched from the post request made to the URL.
 boolean matchURL(String url, String postRequest, String regex, Map<String,String> headers)
          Mathces the regular expression against the response fetched from the post request made to the URL.
 boolean matchURL(URL url, String regex)
          Matches the regular expression against the response fetched from the URL.
 boolean matchURL(URL url, String regex, Map<String,String> headers)
          Matches the regular expression against the response fetched from the URL.
 boolean matchURL(URL url, String postRequest, String regex)
          Mathces the regular expression against the response fetched from the post request made to the URL.
 boolean matchURL(URL url, String postRequest, String regex, Map<String,String> headers)
          Mathces the regular expression against the response fetched from the post request made to the URL.
static HttpTransport newInstance()
          Creates an instance of HttpTransport with a fresh state.
 int readURL(String url)
          Reads data from the URL and discards it, keeping just the size of the total read.
 int readURL(String url, byte[] postRequest)
          Makes a POST request to the URL without encoding the data (the header type is application/octet-stream).
 int readURL(String url, Map<String,String> headers)
          Reads data from the URL and discards it, keeping just the size of the total read.
 int readURL(String url, String postRequest)
          Makes a POST request to the URL.
 int readURL(String url, String postRequest, Map<String,String> headers)
          Makes a POST request to the URL.
 int readURL(URL url)
          Reads data from the URL and discards it, keeping just the size of the total read.
 int readURL(URL url, byte[] postRequest, Map<String,String> headers)
          Makes a POST request to the URL without encoding the data (the header type is application/octet-stream).
 int readURL(URL url, Map<String,String> headers)
          Reads data from the URL and discards it, keeping just the size of the total read.
 int readURL(URL url, String postRequest)
          Makes a POST request to the URL.
 int readURL(URL url, String postRequest, Map<String,String> headers)
          Makes a POST request to the URL.
 void setDownloadSpeed(int kbps)
          Set the download speed for this HTTP transport object
 void setFollowRedirects(boolean follow)
          Sets the http connections managed by this transport to follow or not follow HTTP redirects.
static void setProvider(String newProvider)
          Sets the provider class.
 void setUploadSpeed(int kbps)
          Set the upload speed for this HTTP transport object
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

BUFFER_SIZE

public static final int BUFFER_SIZE
Constructor Detail

HttpTransport

public HttpTransport()
Deprecated. Replaced by the newInstance() method

Constructs a new HttpTransport object.

Method Detail

setProvider

public static void setProvider(String newProvider)
Sets the provider class. NewInstance() will instantiate the given provider.

Parameters:
newProvider - The fully qualified name of the provider class

newInstance

public static HttpTransport newInstance()
Creates an instance of HttpTransport with a fresh state. The actual instance created may be a subclass of HttpTransport fitting to the desired transport mechanism. You can select the transport provider by calling setProvider() before calling newInstance().

Returns:
A new instance of HttpTransport or a subclass thereof
See Also:
setProvider(java.lang.String)

setFollowRedirects

public void setFollowRedirects(boolean follow)
Sets the http connections managed by this transport to follow or not follow HTTP redirects.

Parameters:
follow - True if HTTP redirects should be automatically followed, false otherwise

addTextType

public void addTextType(String texttype)
Add a MIME type to the list of text types. If the response is of this type the fetchULR() methods will return the response data.

Parameters:
texttype - The content type of a HTTP response that contains text.

isFollowRedirects

public boolean isFollowRedirects()
Checks whether the connections managed by this transport follows redirects or not.

Returns:
True if redirects are followed, false otherwise

getResponseBuffer

public StringBuilder getResponseBuffer()
Obtains the reference of the current response buffer.

Returns:
The response buffer

readURL

public int readURL(URL url,
                   Map<String,String> headers)
            throws IOException
Reads data from the URL and discards it, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis.

Parameters:
url - The URL to read from
headers - The request headers
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(URL url)
            throws IOException
Reads data from the URL and discards it, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis.

Parameters:
url - The URL to read from
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(String url,
                   Map<String,String> headers)
            throws IOException
Reads data from the URL and discards it, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis.

Parameters:
url - The URL to read from
headers - The request headers
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(String url)
            throws IOException
Reads data from the URL and discards it, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis.

Parameters:
url - The URL to read from
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(URL url,
                   String postRequest)
            throws IOException
Makes a POST request to the URL. Reads data back and discards the data, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis.

Parameters:
url - The URL to read from
postRequest - The post request string
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(URL url,
                   String postRequest,
                   Map<String,String> headers)
            throws IOException
Makes a POST request to the URL. Reads data back and discards the data, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis. Note that the POST request will be URL encoded.

Parameters:
url - The URL to read from
postRequest - The post request string
headers - The request headers
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(URL url,
                   byte[] postRequest,
                   Map<String,String> headers)
            throws IOException
Makes a POST request to the URL without encoding the data (the header type is application/octet-stream).

Parameters:
url - The URL to read from
postRequest - The binary data to send
headers - The request headers
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(String url,
                   byte[] postRequest)
            throws IOException
Makes a POST request to the URL without encoding the data (the header type is application/octet-stream).

Parameters:
url - The URL to read from
postRequest - The binary data to send
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(String url,
                   String postRequest)
            throws IOException
Makes a POST request to the URL. Reads data back and discards the data, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis.

Parameters:
url - The URL to read from
postRequest - The post request string
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(String url,
                   String postRequest,
                   Map<String,String> headers)
            throws IOException
Makes a POST request to the URL. Reads data back and discards the data, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis.

Parameters:
url - The URL to read from
postRequest - The post request string
headers - The request headers
Returns:
The number of bytes read
Throws:
IOException

fetchURL

public StringBuilder fetchURL(URL url,
                              Map<String,String> headers)
                       throws IOException
Reads data from the URL and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types.

Parameters:
url - The URL to read from
headers - The request headers
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

fetchURL

public StringBuilder fetchURL(URL url)
                       throws IOException
Reads data from the URL and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types. Use getContentSize() to obtain the bytes of binary data read.

Parameters:
url - The URL to read from
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

fetchURL

public StringBuilder fetchURL(String url,
                              Map<String,String> headers)
                       throws IOException
Reads data from the URL and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types. Use getContentSize() to obtain the bytes of binary data read.

Parameters:
url - The URL to read from
headers - The request headers
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

fetchURL

public StringBuilder fetchURL(String url)
                       throws IOException
Reads data from the URL and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types. Use getContentSize() to obtain the bytes of binary data read.

Parameters:
url - The URL to read from
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

downloadURL

public byte[] downloadURL(String url)
                   throws IOException
Retrieve large response from the URL and returns the data read. Use this method for any arbitrary return data type e.g. file downloads. This method will only download upto 1 MB to conserve memory. However, it will read all of the response and update contentSize appropriately.

Parameters:
url - The URL to read from
Returns:
The byte array containing the resulting data
Throws:
IOException
See Also:
getContentSize()

fetchURL

public StringBuilder fetchURL(String url,
                              String postRequest)
                       throws IOException
Makes a POST request to the URL. Reads data back and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types. Use getContentSize() to obtain the bytes of binary data read.

Parameters:
url - The URL to read from
postRequest - The post request string
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

fetchURL

public StringBuilder fetchURL(String url,
                              String postRequest,
                              Map<String,String> headers)
                       throws IOException
Makes a POST request to the URL. Reads data back and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types. Use getContentSize() to obtain the bytes of binary data read.

Parameters:
url - The URL to read from
postRequest - The post request string
headers - The request headers
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

fetchURL

public StringBuilder fetchURL(URL url,
                              String postRequest,
                              Map<String,String> headers)
                       throws IOException
Makes a POST request to the URL. Reads data back and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types. Use getContentSize() to obtain the bytes of binary data read.

Parameters:
url - The URL to read from
postRequest - The post request string
headers - The request headers
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

fetchURL

public StringBuilder fetchURL(URL url,
                              String postRequest)
                       throws IOException
Makes a POST request to the URL. Reads data back and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types. Use getContentSize() to obtain the bytes of binary data read.

Parameters:
url - The URL to read from
postRequest - The post request string
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

fetchURL

public StringBuilder fetchURL(URL page,
                              URL[] images,
                              String postRequest)
                       throws IOException
Method not implemented. Makes a POST request. Fetches the main page and all other image or resource pages based on the given URLs.

Parameters:
page - The page URL
images - The image or other resource URLs to fetch with page
postRequest - The post string
Returns:
The buffer of the main page
Throws:
IOException - If an I/O error occurred

fetchPage

public StringBuilder fetchPage(String page,
                               String[] images,
                               String postRequest)
                        throws IOException
Makes a POST request, fetches the main page and all other image or resource pages.

Parameters:
page - The page URL
images - The image or other resource URLs to fetch with page
postRequest - The post string
Returns:
The buffer of the main page
Throws:
IOException - If an I/O error occurred

getContentSize

public int getContentSize()
Obtains the size of the last read page or resource. The result is in bytes for non-decoded content and in characters for decoded content. All binary content is not decoded. Text content is decoded only using the fetch or match commands.

Returns:
The size, in bytes, of the last page read

fetchResponseData

public StringBuilder fetchResponseData(InputStream stream)
                                throws IOException
Fetches the data from the stream, converts to char, and returns it as a StringBuilder.

Parameters:
stream - The stream to read from
Returns:
The resulting data
Throws:
IOException

fetchResponseData

public StringBuilder fetchResponseData(Reader reader)
                                throws IOException
Fetches the data from the reader and returns it as a StringBuilder.

Parameters:
reader - The reader to read from
Returns:
The resulting data
Throws:
IOException

matchResponse

public boolean matchResponse(String regex)
Maches the regular expression against the data in the current buffer.

Parameters:
regex - The regular expression to match
Returns:
True if the match succeeds, false otherwise

matchResponse

public boolean matchResponse(InputStream stream,
                             String regex)
                      throws IOException
Matches the regular expression against the data read from the stream.

Parameters:
stream - The source of the data
regex - The regular expression to match
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchResponse

public boolean matchResponse(Reader reader,
                             String regex)
                      throws IOException
Matches the regular expression against the data read from the reader.

Parameters:
reader - The source of the data
regex - The regular expression to match
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(String url,
                        String regex)
                 throws IOException
Matches the regular expression against the response fetched from the URL.

Parameters:
url - The source of the data
regex - THe regular expression to match
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(String url,
                        String regex,
                        Map<String,String> headers)
                 throws IOException
Matches the regular expression against the response fetched from the URL.

Parameters:
url - The source of the data
regex - The regular expression to match
headers - The request headers
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(URL url,
                        String regex)
                 throws IOException
Matches the regular expression against the response fetched from the URL.

Parameters:
url - The source of the data
regex - The regular expression to match
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(URL url,
                        String regex,
                        Map<String,String> headers)
                 throws IOException
Matches the regular expression against the response fetched from the URL.

Parameters:
url - The source of the data
regex - The regular expression to match
headers - The request headers
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(URL url,
                        String postRequest,
                        String regex)
                 throws IOException
Mathces the regular expression against the response fetched from the post request made to the URL.

Parameters:
url - The source of the data
postRequest - The post request string
regex - The regular expression to match
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(URL url,
                        String postRequest,
                        String regex,
                        Map<String,String> headers)
                 throws IOException
Mathces the regular expression against the response fetched from the post request made to the URL.

Parameters:
url - The source of the data
postRequest - The post request string
regex - The regular expression to match
headers - The request headers
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(String url,
                        String postRequest,
                        String regex)
                 throws IOException
Mathces the regular expression against the response fetched from the post request made to the URL.

Parameters:
url - The source of the data
postRequest - The post request string
regex - The regular expression to match
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(String url,
                        String postRequest,
                        String regex,
                        Map<String,String> headers)
                 throws IOException
Mathces the regular expression against the response fetched from the post request made to the URL.

Parameters:
url - The source of the data
postRequest - The post request string
regex - The regular expression to match
headers - The request headers
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

getCookieValuesByName

public String[] getCookieValuesByName(String name)
Obtains the list of cookie values by the name of the cookies.

Parameters:
name - The cookie name
Returns:
An array of non-duplicating cookie values.

getCookies

public org.apache.commons.httpclient.Cookie[] getCookies()
Obtains the list of all cookies

Returns:
array of Cookie objects

getResponseHeader

public String[] getResponseHeader(String name)
Obtains the header fields of the last request's response.

Parameters:
name - The response header field of interest
Returns:
An array of response header values

dumpResponseHeaders

public String dumpResponseHeaders()
Utility class to get responseHeaders as a string. The formatting is not localized

Returns:
responseHeaders

getResponseCode

public int getResponseCode()
Obtains the response code of the last request.

Returns:
responseCode

setDownloadSpeed

public void setDownloadSpeed(int kbps)
Set the download speed for this HTTP transport object

Parameters:
kbps - desired speed in kilobytes per second
Throws:
UnsupportedOperationException - if the underlying HTTP transport doesn't support bandwidth-metered (throttled) sockets

setUploadSpeed

public void setUploadSpeed(int kbps)
Set the upload speed for this HTTP transport object

Parameters:
kbps - desired speed in kilobytes per second
Throws:
UnsupportedOperationException - if the underlying HTTP transport doesn't support bandwidth-metered (throttled) sockets