Getting Started Guide

Platforms:

Shell/Console Mode

Shell/Console (command line) mode runs on all platforms where Python 2.5+ can be installed. Tested on Windows XP, Vista, Cygwin, Ubuntu, Eee PC, Mac OS.

GUI Mode

Eventually I hope the Pylot GUI will run on all platforms that support Python and wxWidgets. The GUI has mostly been developed and tested on Windows XP. The application code is pure Python and it uses a cross-platform toolkit. Nothing should preclude it from running on other platforms., but my first trial on Linux (Ubuntu) core dumped the Python interpreter and I haven't investigated further yet

Installing Pylot

Step 1: Download and unzip the latest Pylot release

Get the latest release here: Download Pylot

Step 6: Run Pylot

GUI Mode:

> python pylot.py -g

Shell/Console Mode - Command Line Options:

> python pylot.py

usage: pylot.py [options] args
  -a, --agents=NUM_AGENTS  :  number of agents
  -r, --rampup=RAMPUP      :  rampup in seconds
  -i, --interval=INTERVAL  :  interval in milliseconds
  -d, --duration=DURATION  :  test duration in seconds
  -l, --log_responses      :  log responses
  -g, --gui                :  start GUI

Using Pylot

Step 1: Create Test Cases

Test cases are declared in an XML file named "testcases.xml". This is the format that the test engine understands. Editing XML may seem natural to some people, but awkward to others. The nice thing about this structure is that it will be very easy to create a more friendly user interface [sometime in the future] for defining test cases.

A test case is defined using the following syntax. Only the URL element is required.

<case>
  <url>URL</url>
  <method>HTTP METHOD</method>
  <body>REQUEST BODY CONTENT</body>
  <add_header>ADDITIONAL HTTP HEADER</add_header>
  <verify>STRING OR REGULAR EXPRESSION</verify>
  <verify_negative>STRING OR REGULAR EXPRESSION</verify_negative>
</case>

Below is an example of the simplest possible test case file. It contains a single test case which will be executed continuously during the test run. The test case contains a URL for the service under test. Since no method or body defined, it will default to sending an HTTP GET to this resource. Since no verifications are defined, it will pass/fail the test case based on the HTTP status code it receives (pass if status is < 400).

<testcases>
  <case>
    <url>http://www.example.com/foo</url>
  </case>
</testcases>

We can add positive and negative verifications. A positive verification is a string or regular expression that must be contained in the response body. A negative expression is a string or regular expression that must not be contained in the response body.

<case>
    <url>http://www.goldb.org/foo</url>
    <verify>Copyright.*Corey Goldberg</verify>
    <verify_negative>Error</verify_negative>
<case>

Example: Yahoo! Search Web Services (REST API)

Yahoo offers various REST Web Services to access search results. In this example, I will show how to create Pylot test cases to interact with the REST API.

Here is a simple GET request against the service:

http://search.yahooapis.com/WebSearchService/V1/webSearch?appid=YahooDemo&query=foo

A Pylot test case for this request would look like this:

<case>
  <url>http://search.yahooapis.com/WebSearchService/V1/webSearch?appid=YahooDemo&amp;query=foo</url>
</case>

Notice that the ampersand (&) in the URL was escaped with the code: "&"
This is done becasue certain characters ("<" and "&") are illegal in XML documents. Since we are definig test cases within an XML doc, we must either escape these with ampersand codes, or place them within a CDATA section.

Yahoo also allows the query parameters to be passed in the POST data block. In this case we must also change the "Content-type" HTTP header to: "application/x-www-form-urlencoded". (Pylot defaults to "text/xml")

Here is a POST request against the service:

<case>
  <url>http://search.yahooapis.com/WebSearchService/V1/webSearch</url>
  <method>POST</method>
  <body><![CDATA[appid=YahooDemo&query=webinject]]></body>
  <add_header>Content-type: application/x-www-form-urlencoded</add_header>
</case>

Now that we know how to create individual cases, we can create a test case file containing several of these. In this example, our test case file contains Yahoo web search queries for: "foo", "bar", "baz"

<testcases>
  <case>
    <url>http://search.yahooapis.com/WebSearchService/V1/webSearch?appid=YahooDemo&amp;query=foo</url>
  </case>
    <case>
    <url>http://search.yahooapis.com/WebSearchService/V1/webSearch?appid=YahooDemo&amp;query=bar</url>
  </case>
    <case>
    <url>http://search.yahooapis.com/WebSearchService/V1/webSearch?appid=YahooDemo&amp;query=baz</url>
  </case>
</testcases>

Example: SOAP API

We can model our test cases to talk to any HTTP API. This example shows how you could send requests to a SOAP service. The SOAP envelope we need to send will be enclosed in the HTTP POST body.

<case>
  <url>http://www.example.org/StockPrice</url>
  <method>POST</method>
  <add_header>Content-Type: application/soap+xml; charset=utf-8</add_header>
  <body><!
    [CDATA[
    
      <!-- This is the SOAP Envelope  -->  
      <?xml version="1.0"?>
      <soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope"
        soap:encodingStyle="http://www.w3.org/2001/12/soap-encoding">
        <soap:Body xmlns:m="http://www.example.org/stock">
          <m:GetStockPrice>
            <m:StockName>IBM</m:StockName>
          </m:GetStockPrice>
        </soap:Body>
      </soap:Envelope>
      
    ]]>
  </body>
</case>

Step 2: Model Workload Scenario

Define a workload using the controls on the UI. Using the options below. you can create a steady-state or increasing load test.

Agents: number of agents (virtual users) to run
Rampup: time span over which agents are started. They will be evenly distributed throughout this time span. (see note below)
Interval: interval at which each user sends requests. The requests from each user agent are paced at even intervals (unless the respone time is slower thean the interval deifined)
Duration: time span of the test

Step 3: Execute and Monitor

During the test, you can view real-time stats on the UI. At the end of a test run, an HTML report is automatically generated, showing test results.

Step 4: View Results

When a test is finished, a results directory is created and a report is automatically generated to summarize the test results. It includes various statistics and graphs for response times and throughput. A sample of the results report can be seen here:

Sample Report

Pylot also writes results to delimited text files so you can import them into your favorite spreadsheet to crunch numbers, generate statistics, and create graphs.

Pylot - Web Performance Tool