com.sun.faban.driver.transport.sunhttp
Class SunHttpTransport

java.lang.Object
  extended by com.sun.faban.driver.HttpTransport
      extended by com.sun.faban.driver.transport.sunhttp.SunHttpTransport

public class SunHttpTransport
extends HttpTransport

The SunHttpTransport provides initialization services and utility methods for using the HTTP and HTTPS protocols through the JDK's java.net.HttpUrlConnection facility. The convention for the method names in this class are as follows:

Currenly, the SunHttpTransport class does not provide a way to keep binary data for further analysis. This function can and will be added if there is a use case for keeping such binary data.

Author:
Akara Sucharitakul

Field Summary
 
Fields inherited from class com.sun.faban.driver.HttpTransport
BUFFER_SIZE
 
Constructor Summary
SunHttpTransport()
          Constructs a new SunHttpTransport object.
 
Method Summary
 void addTextType(String texttype)
          Add a MIME type to the list of text types.
 byte[] downloadURL(String url)
          Retrieve large response from the URL and returns the data read.
 String dumpResponseHeaders()
          Utility class to get responseHeaders as a string.
 StringBuilder fetchPage(String page, String[] images, String postRequest)
          Makes a POST request, fetches the main page and all other image or resource pages.
 StringBuilder fetchResponse(HttpURLConnection connection)
          Fetches http response data from an already established connection.
 StringBuilder fetchResponseData(InputStream stream)
          Fetches the data from the stream, converts to char, and returns it as a StringBuilder.
 StringBuilder fetchResponseData(Reader reader)
          Fetches the data from the reader and returns it as a StringBuilder.
 StringBuilder fetchURL(String url)
          Reads data from the URL and returns the data read.
 StringBuilder fetchURL(String url, Map<String,String> headers)
          Reads data from the URL and returns the data read.
 StringBuilder fetchURL(String url, String postRequest)
          Makes a POST request to the URL.
 StringBuilder fetchURL(String url, String postRequest, Map<String,String> headers)
          Makes a POST request to the URL.
 StringBuilder fetchURL(URL url)
          Reads data from the URL and returns the data read.
 StringBuilder fetchURL(URL url, Map<String,String> headers)
          Reads data from the URL and returns the data read.
 StringBuilder fetchURL(URL url, String postRequest)
          Makes a POST request to the URL.
 StringBuilder fetchURL(URL url, String postRequest, Map<String,String> headers)
          Makes a POST request to the URL.
 StringBuilder fetchURL(URL page, URL[] images, String postRequest)
          Method not implemented.
 int getContentSize()
          Obtains the size of the last read page or resource.
 String[] getCookieValuesByName(String name)
          Obtains the list of cookie values by the name of the cookies.
 StringBuilder getResponseBuffer()
          Obtains the reference of the current response buffer.
 int getResponseCode()
          Obtains the response code of the previous request.
 String[] getResponseHeader(String name)
          Obtains the header fields of the last request's response.
 boolean isFollowRedirects()
          Checks whether the connections managed by this transport follows redirects or not.
 boolean matchResponse(InputStream stream, String regex)
          Matches the regular expression against the data read from the stream.
 boolean matchResponse(Reader reader, String regex)
          Matches the regular expression against the data read from the reader.
 boolean matchResponse(String regex)
          Maches the regular expression against the data in the current buffer.
 boolean matchResponse(URLConnection connection, String regex, Map<String,String> headers)
          Matches the regular expression against the data read from the connection.
 boolean matchURL(String url, String regex)
          Matches the regular expression against the response fetched from the URL.
 boolean matchURL(String url, String regex, Map<String,String> headers)
          Matches the regular expression against the response fetched from the URL.
 boolean matchURL(String url, String postRequest, String regex)
          Mathces the regular expression against the response fetched from the post request made to the URL.
 boolean matchURL(String url, String postRequest, String regex, Map<String,String> headers)
          Mathces the regular expression against the response fetched from the post request made to the URL.
 boolean matchURL(URL url, String regex)
          Matches the regular expression against the response fetched from the URL.
 boolean matchURL(URL url, String regex, Map<String,String> headers)
          Matches the regular expression against the response fetched from the URL.
 boolean matchURL(URL url, String postRequest, String regex)
          Mathces the regular expression against the response fetched from the post request made to the URL.
 boolean matchURL(URL url, String postRequest, String regex, Map<String,String> headers)
          Mathces the regular expression against the response fetched from the post request made to the URL.
 int readURL(String url)
          Reads data from the URL and discards it, keeping just the size of the total read.
 int readURL(String url, byte[] postRequest)
          Makes a POST request to the URL without encoding the data (the header type is application/octet-stream).
 int readURL(String url, Map<String,String> headers)
          Reads data from the URL and discards it, keeping just the size of the total read.
 int readURL(String url, String postRequest)
          Makes a POST request to the URL.
 int readURL(String url, String postRequest, Map<String,String> headers)
          Makes a POST request to the URL.
 int readURL(URL url)
          Reads data from the URL and discards it, keeping just the size of the total read.
 int readURL(URL url, byte[] postRequest, Map<String,String> headers)
          Makes a POST request to the URL without encoding the data (the header type is application/octet-stream).
 int readURL(URL url, Map<String,String> headers)
          Reads data from the URL and discards it, keeping just the size of the total read.
 int readURL(URL url, String postRequest)
          Makes a POST request to the URL.
 int readURL(URL url, String postRequest, Map<String,String> headers)
          Makes a POST request to the URL.
 void setFollowRedirects(boolean follow)
          Sets the http connections managed by this transport to follow or not follow HTTP redirects.
 
Methods inherited from class com.sun.faban.driver.HttpTransport
getCookies, newInstance, setDownloadSpeed, setProvider, setUploadSpeed
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

SunHttpTransport

public SunHttpTransport()
Constructs a new SunHttpTransport object.

Method Detail

setFollowRedirects

public void setFollowRedirects(boolean follow)
Sets the http connections managed by this transport to follow or not follow HTTP redirects.

Overrides:
setFollowRedirects in class HttpTransport
Parameters:
follow - True if HTTP redirects should be automatically followed, false otherwise

addTextType

public void addTextType(String texttype)
Add a MIME type to the list of text types. If the response is of this type the fetchULR() methods will return the response data.

Overrides:
addTextType in class HttpTransport
Parameters:
texttype - The content type of a HTTP response that contains text.

isFollowRedirects

public boolean isFollowRedirects()
Checks whether the connections managed by this transport follows redirects or not.

Overrides:
isFollowRedirects in class HttpTransport
Returns:
True if redirects are followed, false otherwise

getResponseBuffer

public StringBuilder getResponseBuffer()
Obtains the reference of the current response buffer.

Overrides:
getResponseBuffer in class HttpTransport
Returns:
The response buffer

readURL

public int readURL(URL url,
                   Map<String,String> headers)
            throws IOException
Reads data from the URL and discards it, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis.

Overrides:
readURL in class HttpTransport
Parameters:
url - The URL to read from
headers - The request headers
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(URL url)
            throws IOException
Reads data from the URL and discards it, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis.

Overrides:
readURL in class HttpTransport
Parameters:
url - The URL to read from
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(String url,
                   Map<String,String> headers)
            throws IOException
Reads data from the URL and discards it, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis.

Overrides:
readURL in class HttpTransport
Parameters:
url - The URL to read from
headers - The request headers
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(String url)
            throws IOException
Reads data from the URL and discards it, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis.

Overrides:
readURL in class HttpTransport
Parameters:
url - The URL to read from
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(URL url,
                   String postRequest)
            throws IOException
Makes a POST request to the URL. Reads data back and discards the data, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis.

Overrides:
readURL in class HttpTransport
Parameters:
url - The URL to read from
postRequest - The post request string
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(URL url,
                   String postRequest,
                   Map<String,String> headers)
            throws IOException
Makes a POST request to the URL. Reads data back and discards the data, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis. Note that the POST request will be URL encoded.

Overrides:
readURL in class HttpTransport
Parameters:
url - The URL to read from
postRequest - The post request string
headers - The request headers
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(URL url,
                   byte[] postRequest,
                   Map<String,String> headers)
            throws IOException
Makes a POST request to the URL without encoding the data (the header type is application/octet-stream).

Overrides:
readURL in class HttpTransport
Parameters:
url - The URL to read from
postRequest - The binary data to send
headers - The request headers
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(String url,
                   byte[] postRequest)
            throws IOException
Makes a POST request to the URL without encoding the data (the header type is application/octet-stream).

Overrides:
readURL in class HttpTransport
Parameters:
url - The URL to read from
postRequest - The binary data to send
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(String url,
                   String postRequest)
            throws IOException
Makes a POST request to the URL. Reads data back and discards the data, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis.

Overrides:
readURL in class HttpTransport
Parameters:
url - The URL to read from
postRequest - The post request string
Returns:
The number of bytes read
Throws:
IOException

readURL

public int readURL(String url,
                   String postRequest,
                   Map<String,String> headers)
            throws IOException
Makes a POST request to the URL. Reads data back and discards the data, keeping just the size of the total read. This is useful for ensuring receival of binary or text data that do not need further analysis.

Overrides:
readURL in class HttpTransport
Parameters:
url - The URL to read from
postRequest - The post request string
headers - The request headers
Returns:
The number of bytes read
Throws:
IOException

fetchURL

public StringBuilder fetchURL(URL url,
                              Map<String,String> headers)
                       throws IOException
Reads data from the URL and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types.

Overrides:
fetchURL in class HttpTransport
Parameters:
url - The URL to read from
headers - The request headers
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

fetchURL

public StringBuilder fetchURL(URL url)
                       throws IOException
Reads data from the URL and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types. Use getContentSize() to obtain the bytes of binary data read.

Overrides:
fetchURL in class HttpTransport
Parameters:
url - The URL to read from
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

fetchURL

public StringBuilder fetchURL(String url,
                              Map<String,String> headers)
                       throws IOException
Reads data from the URL and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types. Use getContentSize() to obtain the bytes of binary data read.

Overrides:
fetchURL in class HttpTransport
Parameters:
url - The URL to read from
headers - The request headers
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

fetchURL

public StringBuilder fetchURL(String url)
                       throws IOException
Reads data from the URL and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types. Use getContentSize() to obtain the bytes of binary data read.

Overrides:
fetchURL in class HttpTransport
Parameters:
url - The URL to read from
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

downloadURL

public byte[] downloadURL(String url)
                   throws IOException
Retrieve large response from the URL and returns the data read. Use this method for any arbitrary return data type e.g. file downloads. This method will only download upto 1 MB to conserve memory. However, it will read all of the response and update contentSize appropriately.

Overrides:
downloadURL in class HttpTransport
Parameters:
url - The URL to read from
Returns:
The byte array containing the resulting data
Throws:
IOException
See Also:
getContentSize()

fetchURL

public StringBuilder fetchURL(String url,
                              String postRequest)
                       throws IOException
Makes a POST request to the URL. Reads data back and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types. Use getContentSize() to obtain the bytes of binary data read.

Overrides:
fetchURL in class HttpTransport
Parameters:
url - The URL to read from
postRequest - The post request string
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

fetchURL

public StringBuilder fetchURL(String url,
                              String postRequest,
                              Map<String,String> headers)
                       throws IOException
Makes a POST request to the URL. Reads data back and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types. Use getContentSize() to obtain the bytes of binary data read.

Overrides:
fetchURL in class HttpTransport
Parameters:
url - The URL to read from
postRequest - The post request string
headers - The request headers
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

fetchURL

public StringBuilder fetchURL(URL url,
                              String postRequest,
                              Map<String,String> headers)
                       throws IOException
Makes a POST request to the URL. Reads data back and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types. Use getContentSize() to obtain the bytes of binary data read.

Overrides:
fetchURL in class HttpTransport
Parameters:
url - The URL to read from
postRequest - The post request string
headers - The request headers
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

fetchURL

public StringBuilder fetchURL(URL url,
                              String postRequest)
                       throws IOException
Makes a POST request to the URL. Reads data back and returns the data read. Note that this method only works with text data as it does the byte-to-char conversion. This method will return null for responses with binary MIME types. The addTextType(String) method is used to register additional MIME types as text types. Use getContentSize() to obtain the bytes of binary data read.

Overrides:
fetchURL in class HttpTransport
Parameters:
url - The URL to read from
postRequest - The post request string
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

fetchURL

public StringBuilder fetchURL(URL page,
                              URL[] images,
                              String postRequest)
                       throws IOException
Method not implemented. Makes a POST request. Fetches the main page and all other image or resource pages based on the given URLs.

Overrides:
fetchURL in class HttpTransport
Parameters:
page - The page URL
images - The image or other resource URLs to fetch with page
postRequest - The post string
Returns:
The buffer of the main page
Throws:
IOException - If an I/O error occurred

fetchPage

public StringBuilder fetchPage(String page,
                               String[] images,
                               String postRequest)
                        throws IOException
Makes a POST request, fetches the main page and all other image or resource pages.

Overrides:
fetchPage in class HttpTransport
Parameters:
page - The page URL
images - The image or other resource URLs to fetch with page
postRequest - The post string
Returns:
The buffer of the main page
Throws:
IOException - If an I/O error occurred

fetchResponse

public StringBuilder fetchResponse(HttpURLConnection connection)
                            throws IOException
Fetches http response data from an already established connection. If the response data is binary, null is returned. Use getContentSize() for the bytes read in this case. The addTextType(String) method is used to register additional MIME types as text types.

Parameters:
connection - The connection to fetch from
Returns:
The StringBuilder buffer containing the resulting document
Throws:
IOException
See Also:
addTextType(String), getContentSize()

getContentSize

public int getContentSize()
Obtains the size of the last read page or resource. The result is in bytes for non-decoded content and in characters for decoded content. All binary content is not decoded. Text content is decoded only using the fetch or match commands.

Overrides:
getContentSize in class HttpTransport
Returns:
The size, in bytes, of the last page read

fetchResponseData

public StringBuilder fetchResponseData(InputStream stream)
                                throws IOException
Fetches the data from the stream, converts to char, and returns it as a StringBuilder.

Overrides:
fetchResponseData in class HttpTransport
Parameters:
stream - The stream to read from
Returns:
The resulting data
Throws:
IOException

fetchResponseData

public StringBuilder fetchResponseData(Reader reader)
                                throws IOException
Fetches the data from the reader and returns it as a StringBuilder.

Overrides:
fetchResponseData in class HttpTransport
Parameters:
reader - The reader to read from
Returns:
The resulting data
Throws:
IOException

matchResponse

public boolean matchResponse(String regex)
Maches the regular expression against the data in the current buffer.

Overrides:
matchResponse in class HttpTransport
Parameters:
regex - The regular expression to match
Returns:
True if the match succeeds, false otherwise

matchResponse

public boolean matchResponse(URLConnection connection,
                             String regex,
                             Map<String,String> headers)
                      throws IOException
Matches the regular expression against the data read from the connection.

Parameters:
connection - The source of the data
regex - The regular expression to match
headers - The request headers
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchResponse

public boolean matchResponse(InputStream stream,
                             String regex)
                      throws IOException
Matches the regular expression against the data read from the stream.

Overrides:
matchResponse in class HttpTransport
Parameters:
stream - The source of the data
regex - The regular expression to match
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchResponse

public boolean matchResponse(Reader reader,
                             String regex)
                      throws IOException
Matches the regular expression against the data read from the reader.

Overrides:
matchResponse in class HttpTransport
Parameters:
reader - The source of the data
regex - The regular expression to match
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(String url,
                        String regex)
                 throws IOException
Matches the regular expression against the response fetched from the URL.

Overrides:
matchURL in class HttpTransport
Parameters:
url - The source of the data
regex - THe regular expression to match
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(String url,
                        String regex,
                        Map<String,String> headers)
                 throws IOException
Matches the regular expression against the response fetched from the URL.

Overrides:
matchURL in class HttpTransport
Parameters:
url - The source of the data
regex - The regular expression to match
headers - The request headers
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(URL url,
                        String regex)
                 throws IOException
Matches the regular expression against the response fetched from the URL.

Overrides:
matchURL in class HttpTransport
Parameters:
url - The source of the data
regex - The regular expression to match
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(URL url,
                        String regex,
                        Map<String,String> headers)
                 throws IOException
Matches the regular expression against the response fetched from the URL.

Overrides:
matchURL in class HttpTransport
Parameters:
url - The source of the data
regex - The regular expression to match
headers - The request headers
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(URL url,
                        String postRequest,
                        String regex)
                 throws IOException
Mathces the regular expression against the response fetched from the post request made to the URL.

Overrides:
matchURL in class HttpTransport
Parameters:
url - The source of the data
postRequest - The post request string
regex - The regular expression to match
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(URL url,
                        String postRequest,
                        String regex,
                        Map<String,String> headers)
                 throws IOException
Mathces the regular expression against the response fetched from the post request made to the URL.

Overrides:
matchURL in class HttpTransport
Parameters:
url - The source of the data
postRequest - The post request string
regex - The regular expression to match
headers - The request headers
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(String url,
                        String postRequest,
                        String regex)
                 throws IOException
Mathces the regular expression against the response fetched from the post request made to the URL.

Overrides:
matchURL in class HttpTransport
Parameters:
url - The source of the data
postRequest - The post request string
regex - The regular expression to match
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

matchURL

public boolean matchURL(String url,
                        String postRequest,
                        String regex,
                        Map<String,String> headers)
                 throws IOException
Mathces the regular expression against the response fetched from the post request made to the URL.

Overrides:
matchURL in class HttpTransport
Parameters:
url - The source of the data
postRequest - The post request string
regex - The regular expression to match
headers - The request headers
Returns:
True if the match succeeds, false otherwise
Throws:
IOException

getCookieValuesByName

public String[] getCookieValuesByName(String name)
Obtains the list of cookie values by the name of the cookies.

Overrides:
getCookieValuesByName in class HttpTransport
Parameters:
name - The cookie name
Returns:
An array of non-duplicating cookie values.

getResponseHeader

public String[] getResponseHeader(String name)
Obtains the header fields of the last request's response.

Overrides:
getResponseHeader in class HttpTransport
Parameters:
name - The response header field of interest
Returns:
An array of response header values

dumpResponseHeaders

public String dumpResponseHeaders()
Utility class to get responseHeaders as a string. The formatting is not localized

Overrides:
dumpResponseHeaders in class HttpTransport
Returns:
responseHeaders

getResponseCode

public int getResponseCode()
Obtains the response code of the previous request.

Overrides:
getResponseCode in class HttpTransport
Returns:
responseCode The response code