Tuesday, November 12, 2013

Performance testing on forms



    HTML form is an important part of internet, without it, we will not be able to interact with web server, like placing an order, booking a hotel/flight etc.

    In previous blog, we talked about performance testing on search forms.   But it didn't cover the more complex cases such as select, radio button etc which are typical on HTML forms.  In this blog, we are going to focus on these complex cases and show that the netgend javascript function "getFormField()" can easily take care of them.

   First let's warm up with something simple, this can be seen as a review of previous blog:   suppose we have a form with some text fields to be filled, like "firstName", "lastName", "city", we can generate http request like the following:

 action(http, "http://www.example.com/");  
 a = getFormField(http.replyBody, "registerationForm");  
 a.firstName.value = "john";  
 a.lastName.value = "smith";  
 a.city.value  = "chicago";  
 http.POSTData = combineHttpParam(a);  
 //post data will look like "firstName=john&lastName=smith&city=chicago"
 action(http, "http://www.example.com/register?");  
    In the above code snippet, second parameter to getFormField (in our case "registerationForm") is used to identify the right form in a HTML page.  When HTML page only contains one form, you can leave this parameter empty.

   Of course, this will be more interesting if the fields are read from a csv file.   Suppose we have a file called "users.csv", each of its rows contains first name, last name and city.
 function userInit() {  
  var list = readCSVFile("users.csv");   
  var i = 0 ;  
 }  
 function VUSER () {  
  item = list[i];   
  //here item is an array variable that contains the fields of the row in csv file  
  i ++;   
  action(http, "http://www.example.com/");  
  a = getFormField(http.replyBody, "registrationForm");  
  a.firstName.value = item[0];  
  a.lastName.value = item[1];  
  a.city.value  = item[2];  
  http.POSTData = combineHttpParam(a);  
  action(http, "http://www.example.com/register?");   
 }   

     Note that it will work for both textfield and text area.  The difference from user's point of view is that text field typically has one line, text area can have multiple lines.  In HTML, text field and text area are quite different. Luckily, our users don't need to worry about it -- they are taken care of by netgend test platform.

    What if the form has radio button?   In performance testing we would like to randomly pick a option. Let's suppose, we have radio button with name "vehicle" and it has 4 options: "bike", "car", "pickup", "truck",

 a = getFormField(http.replyBody, "registrationForm");  
 a.vehicle.value = randNumber(1, a.vehicle.size);  
 //...some more assignments to a...  
 http.POSTData = combineHttpParam(a);  
   Here the variable "a.vehicle.size" holds the number of items for a.vehicle, in this case, it's 4.  A good news here:  the above code snippet not only works for radio button, it works for "select" field (a.k.a dropdown box) of a form too.

    What about checkbox. Checkbox is different from radio button in that user can check zero or more items.  So in performance testing, we need to emulate selecting a subset of checkboxes.  How do we do this? randSequence(min, max) function can help.  It will return a subset of the sequence (min, min+1, ...max).  Suppose we have a checkbox group called "area", with items: "n", "nw", "w", "sw", "s", "se", "e", "ne", "c".

 a = getFormField(http.replyBody, "registrationForm");  
 a.area.value = randSequence(1, a.area.size);  
 //...some more assignments to a...  
 http.POSTData = combineHttpParam(a);  


    Finally we put all of them together,  suppose we have a form that has
  • select field called "color", 
  • radio button group called "vehicleType" and 
  • checkbox group called "make".   
Here is code snippet to do performance testing:

 action(http, "http://www.example.com/vehicles");  
 a = getFormField(http.replyBody, "findVehicleForm");  
 a.color.value = randNumber(1, a.color.size);  
 a.vehicleType.value = randNumber(1, a.vehicleType.size);  
 a.make.value = randSequence(1, a.make.size);  
 http.POSTData = combineHttpParam(a);  
 action(http, "http://www.example.com/vehicles");  

    Not bad for a "complex" form, right?



Monday, November 11, 2013

Syntax comparison between JMeter and Netgend

Jmeter is a great open source performance testing platform.  It's used by a large community of users and gain popularity at the cost of loadrunner, as shown in this report thanks to the job site Indeed.

    The Jmeter GUI is nice. But to gain performance advantage, one may have to run it on command line as suggested by this tip.  Also, you may have to be careful on how many threads to have, because  hundreds of active threads may affect performance. This creates a problem:  when HTTP transaction latency is high, say, hundreds of milliseconds, one needs to have a large number of VUsers (threads) to bring up the transaction rate, but a large number of threads may affect performance.

    These are not the toughest problems with JMeter,  the toughest problem is with parameterization or correlation:  user needs to be really good at Java programming to do it.   For many users with only basic programming background, this can be a serious challenge.

    Netgend javascript makes things simpler:
  • Don't have to worry about the number of threads,  one system can support about 50,000 VUsers.
  • Complex parameterization/correlation can be simple. 
  • Can view real time statistics on GUI while many VUsers are running.

    The following question asked on a forum provides an excellent example.   The question is on processing the following "complex" JSON message and extract the values of two fields "equipmentPartId",  "compositePartName".


 [  
   {  
    "equipmentPart":{  
      "allAttributes":{  
       "compositePartName":"SRW224G4P-K9-AU ",  
       "equipmentPartPhysicalSiteId":"73411",  
       "equipmentPartSiteId":73414,  
       "equipmentPartId":542024,  
       "productPartName":"SRW224G4P-K9-AU",  
       "equipmentPartName":"SRW224G4P-K9-AU"  
      }  
    },  
    "updatedTimeStamp":1337838775000,  
    "solutionEntityId":542227,  
    "solutionId":13959179,  
    "status":"validation ok",  
    "id":95509,  
    "createdTimeStamp":1337838666000,  
    "messages":", Non Configurable"  
   },  
   {  
    "equipmentPart":{  
      "allAttributes":{  
       "compositePartName":" UC560-T1E1-K9 ",  
       "equipmentPartPhysicalSiteId":"73411",  
       "equipmentPartSiteId":73412,  
       "equipmentPartId":542027,  
       "productPartName":"UC560-T1E1-K9",  
       "equipmentPartName":"UC560-T1E1-K9"  
      }  
    },  
    "updatedTimeStamp":1337838775000,  
    "solutionEntityId":542230,  
    "solutionId":13959179,  
    "status":"validationok",  
    "id":95510,  
    "createdTimeStamp":1337838666000,  
    "messages":", Non Configurable"  
   }  
 ]  
    Here is a solution given for JMeter.
 import org.json.JSONArray;  
 import org.json.JSONObject;  
 String jsonString = prev.getResponseDataAsString();  
 JSONArray equipmentParts = new JSONArray(jsonString);  
 print("parsed json");  
 JSONArray parts = new JSONArray();  
 for(int i=0;i<equipmentParts.length();i++ ){  
      JSONObject equipmentPart = equipmentParts.getJSONObject(i).getJSONObject("equipmentPart");  
      print(equipmentPart.toString());  
      JSONObject allAttributes = equipmentPart.getJSONObject("allAttributes");  
      print(allAttributes.toString());  
      JSONObject part = new JSONObject();  
      print(allAttributes.getLong("equipmentPartId"));  
      part.put("partId",allAttributes.getLong("equipmentPartId"));  
      print(allAttributes.getString("compositePartName"));  
      part.put("partNumber",allAttributes.getString("compositePartName"));  
      // add more here  
      parts.put(part);  
 }  
 vars.put("jsonResponse", parts.toString());  
   
    An ordinary user have to really focus his/her attention to be able to follow it.  With Netgend javascript, it's as simple as the following:

 a = fromJson(data); 
 parts = [];
 for (i = 0; i < getSize(a); i++) {
      part.partId   = a[i].equipmentPart.allAttributes.equipmentPartId;
      part.partName = a[i].equipmentPart.allAttributes.compositePartName;
      push(parts, part); 
 }
    Here the compound variable "a[i].equipmentPart.allAttributes.equipmentPartId" is based on the JSON message structure: at highest level, this JSON message is an array,  the each element in the array can be represented by a[i],  in each element, we have hierarchy like equipmentPart -- allAttributes --equipmentPartId.  Now you understand where the compound variable comes from.

   We believe parameterization/correlation should be made so simple that one doesn't need to be a guru on programming to do performance testing!   

Sunday, November 10, 2013

Performance testing on search form


    Searching is as important to an e-commerce web site as google is to the internet.  With a good, responsive search form, user can quickly find the item/product to purchase or book.

   So how should we do performance testing on search form?  simply sending HTTP POST or GET request with something like "search_block=examples" may not work, since there may be some hidden fields present in the form data like "search_block=examples&cpId=12001&_sessionId=100012". That's because the form may look like
 <form action="blabla">  
 <input type="text" name="search_block"/>  
 <input type="hidden" name="cpId" value="12001"/>  
 <input type="hidden" name="_sessionId" value="100012"/>  
 </form>  


   How to send search request and keep the hidden fields?   With netgend javascript, we can use the function "getFormField".  Here is an example.
 function VUSER() {  
      action(http, "http://www.example.com/");  
      a = getFormField(http.replyBody);  
      a.search_block.value = "examples";  
      http.POSTData = combineHttpParam(a);  
      action(http, "http://www.example.com/");  
 }  
   In the above, "getFormField(http.replyBody)" will parse the form in the http reply body and produce a compound variable with multiple fields, each field has a name and value (if present) from the form. We will assign a value to the "search_block" field and "combine" the fields in the compound variable "a" and create a string like
"search_block=examples&cpId=12001&_sessionId=100012".

   Of course, to make it more effective, we can read a list of query phrases from a csv file. This is covered in previous blogs and will be left as an exercise for the reader.

   In addition to query string, sometimes we need to create a pair of random dates when doing search.  This happens, for example, when we reserve a hotel room or air plane tickets.   It's quite easy with netgend javascript.  Here is an example that shows a random start date is picked between now and 7 days later, an end date date that's 2 days later.

 t = time();  
 t += randNumber(0, 86400 *7);  
 a.startDate = toDate(t);  
 //variable "a" will contain "11/18/2013"
 a.endDate = toDate(t + 86400 * 2);  
 //...
 http.POSTData = combineHttpParam(a);
 //http action
    For web site that's particular about format of date, you can easily change it using the optional second parameter of "toDate" function.
 a.startDate = toDate(t, "%Y-%m-%d");  
 //will produce something like 2013-12-16  


    Last thing we cover in this blog is related to "suggestions" in search form. We frequently see some suggestions when we are typing queries.  How did the suggestions get to browser?  Turns out that behind the scene, there are ajax calls to server to fetch suggestions as user types the characters in the text box.

   How do we emulate clients talking to server for suggestions?  It's only a few lines on netgend's javascript.

 query = "computer";  
 for (i = 1; i < length(query); i ++) {  
      partial = substr(query, 0, i);  
      //variable "partial" will contain the first i characters of the query string
      action(http,"http://www.example.com/query?searchTerm=${partial}");  
      sleep(500);
 }  
  Here we use the variable "partial" to hold progressively more characters in the query string and use it to do transaction with server.

   There are lots of interesting questions on performance testing on search forms, we will cover  them in future blogs.



Thursday, November 7, 2013

Performance comparison between Javascript, python and perl


    One of our projects has heavy processing on json string.  Python was the language of choice, but I was curious on how the other languages perform, so I did a simple comparison against  2 other languages (actually their interpreters): Javascript, Perl.

   We are not surprised that Javascript is the fastest because 1) JSON is a native data format in javascript  2)  we had heard V8 engine (which Node is based on) is the fastest interpreter of all scripting languages.   We know this doesn't mean we have to change our  choice of language to be javascript right away, but it does give some idea how much performance gain we may get.

    Disclaimer:  this is just one test and focus on one type of message processing,   so it's far from being comprehensive.  Don't want to start a war here :-)

Setup

  • CPU:   Intel(R) Core(TM) i3 CPU M 380  @ 2.53GHz
  • OS: Ubuntu 11.04 desktop 64bit
  • Node (for javascript):  v0.4.9
  • Python 2.7.2+,  json module version 2.0.9.
  • Perl 5.12.4,  JSON module version 2.53

Test benchmark


  • Take the following json string, convert it to a object (specific to that language/intepreter under test)
    {"addr": {"city": "austin", "zip": 12345}}
  • replace the "city" field of the object to be "houston"
  • encode the object into json string.
  • do the above 1 million times and measure the total time it takes.

Result

LanguageTime taken
Python17.03s
Javascript (Node)2.85s
Perl4.38s


Here are the code snippets for each of the three languages.

Python

 import time  
 import json  
 import os  
 #purpose: test performance of python.  
 a = '{"addr": {"city": "austin", "zip": 12345}}'  
 ts = time.time()  
 for i in range(0,1000000):  
   x = json.loads(a)  
      x["addr"]["city"] = "houston"  
      y = json.dumps(x);  
 print time.time() - ts  

Javascript

 //==========process a simple json string in nodejs, took 2.845seconds=============  
 a = '{"addr": {"city": "austin", "zip": 12345}}';  
 ts = new Date().getTime()  
 for (i=0; i<1000000; i++) {  
   b = JSON.parse(a);  
      b.addr.city = "houston"  
      JSON.stringify(b);  
 }  
 console.log(new Date().getTime() - ts)  


Perl

 #==========process a simple json string in perl, took 4.377seconds=============  
 use JSON;  
 use Time::HiRes qw( usleep ualarm gettimeofday tv_interval nanosleep  
            clock_gettime clock_getres clock_nanosleep clock  
            stat );  
 $json = JSON->new->allow_nonref;  
 $a = '{"addr": {"city": "austin", "zip": 12345}}';  
 $start = getTS();  
 for ($i=0; $i<1000000; $i++) {  
   $b = $json->decode ($a);  
      $b->{addr}->{city} = "houston";  
      $c = $json->encode($b);  
 }  
 print getTS() - $start,"\n";  
 sub getTS {  
      my ($seconds, $microseconds) = gettimeofday;  
      return $seconds + (0.0+ $microseconds)/1000000.0;  
 }  


Monday, November 4, 2013

Extracting fields from server responses


    In performance testing, we frequently have to process server responses.  One part of the processing  is extracting fields from the response messages and saving them to parameter(s) to be used later.
 
    In Loadrunner, the most common way of extraction is by using boundary method:  you provide the left-boundary of a field and a right-boundary.  It returns the value contained in-between. This could be a good choice when the response data is in free format, but if the response data has a structure, like a HTML form, Netgend scripting provides an easier and more intuitive way for data extraction.

   The first example here is based on an example from this blog (thanks for it!). We need to extract the value for "ID1" field from the following message:
 <form>  
 <input type="hidden" name="ID1" value="11111">  
 <input type="hidden" name="ID2" value="22222">  
 ... (more lines) ...  
 </form>  
    Of course, one can extract the value in this case as
 Left-boundary: ID1\" value=\"  
 Right-boundary: "  
 
    In netgend javascript script, we can do it as
 a = getFormField(str);  
 #now variable a.ID1.value will contain the value "11111".
    Much cleaner, right?

    In case you really want to use boundary method, netgend javascript supports it for feature compatibility :-)
 a = substring(str, "ID1\" value=\"", "\"");  
 println(a[0]);  
    Note that in netgend javascript, boundary can potentially be dynamically changed, while in the case of Loadrunner, it's static (fixed in test creation time).
 left  = ... ; #may be extracted from previous server response   
 right = ... ; #may be extracted from previous server response   
 a = substring(str, left, right);   
 println(a[0]);   

    The second example involves CAPTCHA. We as a human being,  can "see" the characters better than a machine/program.   However, there are some web sites that use a much weaker format:   In their registration forms, there is a question like  "what is sum of 14 + 9:  [      ]":
This may deter some casual hackers, but it will not deter sophisticated hackers from registering lots of accounts.

    Suppose we need to test this,  we can use the boundary method mentioned earlier, but it will be very tedious:  we will extract both numbers into their parameters, do the arithmetic operation on the two parameters and write it back to a parameter.

    In netgend javascript script, it's very trivial:
 contains(http.replyBody,  /what is sum of (\d+)\s+\+\s+(\d+)/);  
 answer = g1 + g2;  
 //now you can send POST data like "...userAnswer=${answer}"  
     Note that in the above, the two variables "g1", "g2" will contain the two numbers. We just add them up and assign the sum to the variable "answer".  In case you wonder,  the values in "g1", "g2" are strings,  can you add them as if they are numbers?   The answer is yes! If they are really numbers,  like "14" and "9" in our case, you can performance the arithmetic operation.

     Of course, we are not hackers :-)  We are the proud performance testing engineers!

    What if we need extract some values from a response that looks like xml? More precisely, suppose we have a response that looks like the following, how do we extract name of the first employee in the response?

 <employees>  
 <employee>  
      <id>1001</id>  
       <name>John Smith</name>  
 </employee>  
 <employee>  
      <id>1002</id>  
       <name>Jane Dole</name>  
 </employee>  
 </employees>  

    You may be tempted to use regular expression, it may work, but it may get you the wrong answer (for instance, get you the name of the second employee).   Here is a much simpler solution.
 a = fromXml(str);  
 //now a.employees.employee[0].name.value will contain the value "John Smith" 
    Here the compound variable "a.employees.employee[1].name.value" come from the xml structure, reader can quickly see the easy mapping here.

   Different test scenarios may require extraction of data differently. Based on the above examples, we can tell they are not going to be too hard for netgend javascript :-)


Saturday, November 2, 2013

JSON message processing in performance test


    We all know that JSON as a message format has become more popular,  databases such MongoDB, CouchDB support it, so do many web services.  So it's not a surprise that performance test tools need to deal with it.    However,  it's not easy to process JSON messages on many existing test platforms,  in some cases, users are desperate enough to use regular expressions!

    Here we are going to show how easy it is to process json messages in netgend javascript, even for user with modest coding background.  For those who have read previous blogs which touched on the subject of processing JSON messages,  this blog is a continuation and it will cover some more examples on how to use netgend javascript to process JSON messages in performance testing.

    Before diving into the examples,  the author would like to thank the sites where examples/questions were taken from. Readers of this blog are encouraged to read more at those sites and try to solve questions/challenges raised there.

Example 1

    This example is based on a stackoverflow question:   given a JSON string like the following, find the id's whose value is not null.
 [{"id":"123","value":"abc","description":"something"},
 {"id":"456","value":null,"description":"something else"},
 {"id":"789","value":"def","description":"something more"}]
    The solution is fairly simple.  In the following solution, we assume variable "str" contains the above JSON string.
 a = fromJson(str);  
 i = 0;  
 while (i < getSize(a)) {
      if(a[i].value != "") {  
           println(a[i].id);  
      }  
      i ++; 
 } 
    Note that function "fromJson" parses the JSON string and assign to variable "a",   "a" will contain an array of json objects, each containing "id",  "value" and "description" fields.  The variable "a[id].value" means the "value" field of id'th item of the array  "a".   Finally, as a reminder, all the variables here are local to the VUser.


Example 2

    Based on another stackoverflow question (there have lots of questions, don't they?):  given a JSON string like the following, print all id's of the rows whose type is "B".
[{"type":"A","id":"2093638401"}
{"type":"B","id":"20843301"}
{"type":"A","id":"20743305"}
{"type":"B","id":"20645303"}
{"type":"C","id":"20564501"}]
    The solution is very similar to example 1.
 #assume str has the content of above json string
 a = fromJson(str);   
 id = 0;  
 cnt = 0; 
 while (id < getSize(a)) {  
    if (a[id].type === "B") {   
      cnt ++;   
    }  
    id ++;   
 }   
   


Example 3



      Suppose we have the following JSON string as HTTP response,  we need to get value of certain fields.
 {  
   "response_time": 0.014376163482666016,  
   "applications": [  
     {  
       "api_key": "blted0e7982e1cf62a8",  
       "name": "gta",  
       "uid": "gta",  
       "account_name": "jack"  
     },  
     {  
       "api_key": "blt1423c40d23e4a423",  
       "name": "cellapp",  
       "uid": "cellapp",  
       "account_name": "max"  
     }  
   ]  
 }  
    The JSON structure is a little more complex, so are the compound variables like "resp.applications[0].api_key".  However, it should be very easy to map the structure of the variable to the fields in JSON string.
 resp = fromJson(http.replyBody);  
 //now you can access fields in the json string.  
 //In the following expressions, "application" is an array of two item, first item has index 0  
 resp.applications[0].api_key    
 //The above will have value "blted0e7982e1cf62a8"  
 resp.applications[1].name  
 //the above will have value "cellapp"  


Example 4

     Again based on an example in stockoverflow.  The JSON string in this example is long and complex.  Here we need to find the names of two nodes after "CURRENT", in our case, we need to find  "36735928", "37851136". 
 {  
 "payload": {  
       "userName": {  
             "firstName": "Peter",  
              "lastName": "Pan"  
              },  
       "reservationLists": {  
               "CURRENT": {  
                    "36735928": {  
                           "startDate": "2013-10-03",  
                            "destination": "Balearen",  
                            "mainProductType": "Mietwagen",  
                           "allProductTypes": [  
                                    "Mietwagen"  
                                     ]  
                           },  
                    "37851136": {  
                           "startDate": "2013-10-14",  
                           "destination": "Nevada, Las Vegas",  
                           "mainProductType": "Camper",  
                          "allProductTypes": [  
                                     "Extra",  
                                     "Extra"  
                                     ]  
                           }  
                      },  
                 "PAST": {  
                     "12061210": {  
                           "startDate": "2012-05-05",  
                           "destination": "Loire / Nivernais",  
                            "mainProductType": "Boot",  
                            "allProductTypes": [  
                                     "Boot",  
                                     "Extra"  
                                     ]  
                           }  
                     }     
                  }  
          }  
     }  
    Despite the long JSON string, the solution is quite short and easy to understand.
 //assume variable "jsonstr"contains the complex json string in the question  
 a = fromJson(jsonstr);  
 b = keys(a.payload.reservationLists.CURRENT);  
 //now b[0] will contain first node name ("36735928" in this case)   
 //and b[1] will contain second node name ("37851136" in this case)  


Example 5.

    Finally an example JSON from Alex Ersenie's blog,  we need to print all the child categoryId's.
 {  
   "code":"200",  
   "description":"Ok",  
   "categoryTree":{  
    "categories":[  
      {  
       "categoryId":3,  
       "parentId":1,  
       "name":"Root Catalog",  
       "isActive":1,  
       "includeInMenu":0,  
       "position":3,  
       "level":1,  
       "children":[  
         {  
          "categoryId":45,  
          "parentId":3,  
          "name":"Featured",  
          "isActive":1,  
          "includeInMenu":0,  
          "position":2,  
          "level":2,  
          "children":[ ]  
         },  
         {  
          "categoryId":46,  
          "parentId":3,  
          "name":"Might Like",  
          "isActive":1,  
          "includeInMenu":0,  
          "position":4,  
          "level":2,  
          "children":[ ]  
         },  
         {  
          "categoryId":47,  
          "parentId":3,  
          "name":"Recent Additions",  
          "isActive":1,  
          "includeInMenu":0,  
          "position":5,  
          "level":2,  
          "children":[       ]  
         },  
         {  
          "categoryId":48,  
          "parentId":3,  
          "name":"Hero Products",  
          "isActive":1,  
          "includeInMenu":0,  
          "position":6,  
          "level":2,  
          "children":[ ]  
         }  
       ]  
      }  
    ]  
   }  
 }   

     The solution, as usual, is fairly simple.
 a = fromJson(str);  
 b = a.categoryTree.categories[0].children;  
 for (id = 0; id < getSize(b); id ++) { 
      println(b[id].categoryId);  
 } 

    Now that we have shown processing complex JSON structure can be so simple,  I am sure you may want to apply this technique to more challenging issues encountered in your performance testing!



Friday, November 1, 2013

Statistics in Performance testing


    In performance testing, statistics is very important.  For instance, the following statistics will give user an idea how the server under test is performing,
  • max concurrent connections
  • transaction rate
  • latency
  • bandwidth
  • ...
     In this blog, we are going to cover some "new" features in statistics that netgend javascript supports,

User-defined Counter

    For HTTP transactions, sometimes it's enough to see the HTTP requests get responses,  sometimes we need to see whether the HTTP responses are all 200 OK.   There are cases when even the 200 OK responses are not enough, for example,  when you try to submit a login form, you got 200 OK but the response text tells you login is unsuccessful.  How can you catch this by watching statistics charts?

   This is where user-defined counter can help.  In netgend javascript,  you can define your own success counter.  
 http.POSTData = "user=${username}&pass=${password}";  
 action(http,"http://www.example.com/login?");  
 if (contains(http.replyBody, /Welcome/)) {  
   stats("success");  
 }  
    The above simple logic checks whether a response contains "Welcome" and increment the counter only when it does.  You can add it to your script and watch the statistics chart in GUI.  It will make you happy if you see the curve for user-defined counter matches with the request rate curve.  

   This can be very useful if you are testing a non-HTTP protocol, where there is no well defined transaction. You can use the user-defined counter to find out how well the server is responding to the requests. 

User-defined Latency

   In addition to counter, we measure different latencies, including 
  • TCP connection latency,  
  • First HTTP response latency (the time between request and first packet in response)
  • Last HTTP response latency (the time between request and last packet in response)
    However, there are cases when we may want to measure a range of operations.  For example, certain web portal may need multiple HTTP transactions,  what if we want to measure the total time?     
     Our user-defined-latency can help here.  You just need to add two lines of script around your operations.
 start = currentTick;  
 //do a bunch of operations here  
 stats("latency", currentTick - start);  
     Note that the variable "currentTick" will contain the latest timestamp.
 
     With these extra lines,  statistics window will show the user-defined latency in addition to the other latencies. 

Dynamic Rate Adjustment

     When running a performance test, we sometimes start with a low transaction rate,  set up increment schedule like:   increment the rate by x for every y seconds.  While this will work,  there are cases where it's better to adjust the rate manually first, especially when we are not familiar with the system capacity yet:
  • The rate increment is too slow, based on the feed back from latency and other statistics, we would like to increment the increment rate. We are torn between restarting the test (with faster increment) or being more patient and watch the rate slowly creep up. 
  • The rate increment is too fast,  latency starts to grow bad while rate is still increasing.  
     In Netgend platform, we support dynamic adjustment of rate. You can start with certain rate and increment the rate manually based on the state of the system, if the latency is healthy, transaction success rate is good (remember we have user-defined counter for success),  you can increment the rate by a user-defined amount, otherwise, you can decrease the rate to find out the tipping point of the system under test.

     Of course, after you have gotten an idea on the capacity of the system, you are welcome to set up a test with automatic rate increment (increment by x for every y seconds), which is also supported.