Thursday, October 31, 2013

Syntax comparison between Loadrunner and Netgend javascript



    In previous blogs, we know that Netgend platform, built on javascript, can support 50,000 VUsers on one system. But performance testing is not only about supporting a large number VUsers,  it's also about flexibility - how easy it is to emulate client.

    To make the test platform flexible, we have to deal with per-VUser variables,  these are variables that are only accessible (read and write) by a VUser.  This is the basis for many VUsers to run independently.  We will use "pvar" to denote per-VUser variable in this blog.

    Loadrunner offers flexible support on pvar,  you can create a pvar and assign value to it and use its value later to do some operations.   Down side is that it requires some good C background to use it.  In netgend javascript, a variable is naturally a pvar.  So user doesn't need much programming background. Here is a quick comparison.
LoadrunnerNetGend Javascript
Assign to a pvarlr_save_string("John Doe", "fullName")

//generate a random number
//between 20 and 30
randomnumber = rand() % 11
randomnumber += 20
lr_save_int(randomnumber, "age")
fullName = "John Doe";
age  = randNumber(20,30);
get value from pvarname = lr_eval_string("fullName");
age  = lr_eval_string("age");
char msgbuffer[100];
sprintf(msgbuffer, "my name is %s, age is %s", name, age);
lr_save_string(msgbuffer, "msgbuffer")
msgbuffer = "my name is ${fullName}, my age is ${age}";
compare pvar with a string if(strcmp(lr_eval_string("{pMonth}"),"JAN")==0) if (pMonth == "JAN") {
    ......
}
     Note that in the above, loadrunner syntax would require user to be aware of the type of variable and use different functions to access them (lr_save_string vs lr_save_int).   To make matter worse, you may have to use "lr_save_var" in some cases.  In netgend javascript, all is transparent to user, who never has to worry about all these: is it an integer, is it a string, what is the size of the string.

     The above is about scalar pvar, it only holds one value.  What about array variable?   We need it to hold a list of values, for instance, a list of links within a HTML page.  In netgend javascript, a pvar can be array variable,  you just need to use an index to get the value of an item.  The following will give you a good idea of the syntax.

    Here is is an example (taken from Google group discussion),  that shows how to grab a list of values (in string format) from a http response and print them out in Loadrunner.
 web_reg_save_param ("IDValues", "LB=value=\"", "RB=\"", "Ord=All", LAST);  
 // get number of matches  
 nCount = atoi(lr_eval_string("{IDValue_count}"));  
 for (i = 1; i <= nCount; i++) {  
   // create full name of a current parameter  
   sprintf(szParamName, "{IDValue_%d}", i);  
   // output a value of current parameter  
   lr_output_message("Value of %s: %s",szParamName, lr_eval_string(szParamName));  
 }  
    In netgend javascript, this is super easy, thanks to the simple syntax for array variable.
 IDValues = substring(http.replyBody, "value=\"", "\"", "all");  
 for (i=0; i < getSize(IDValues); i++) {  
   logMsg(sprintf("Value of %d is %s\n", i, IDValues[i]));  
 }  
    Note that in the above, "substring" function will grab all the strings between "value=\"" and "\"" and return a list, "IDValues" variable will hold this list.

   Lastly, we look at syntax related to HTTP.  To access information about a HTTP transaction,  Loadrunner requires you to do the following
 HttpRetCode = web_get_int_property(HTTP_INFO_RETURN_CODE);  
 downloadSize = web_get_int_property(HTTP_INFO_DOWNLOAD_SIZE);
 downloadTime = web_get_int_property(HTTP_INFO_DOWNLOAD_TIME);

   In netgend javascript, we can easily access the information from following variables
 http.respCode   
 http.request  
 http.replyHeader  
 http.replyBody 
 http.finalRespTime :  The time it took beween http request and entire http response is received.
    With them, we can easily do many interesting operations, such as logging all the requests and replies when the respCode is >= 500.
if (http.respCode >= 500) { 
   logMsg(http.request);  
   logMsg(http.replyHeader);  
   logMsg(http.replyBody);  
}

    There will be more to come on Loadrunner syntax vs netgend javascript. Stay tuned.

Wednesday, October 30, 2013

HTTP Performance testing made simple (II)



    In blog HTTP Performance testing made simple , we mainly talked about how to generate HTTP request dynamically.  While it is important, it is only half the story,  the other half is:  how to process the HTTP response sent by server.  This is what we will focus on in this blog.

   The simplest means of processing HTTP response is by regular expression.  For example, when you send a HTTP request to query all the shoes in a online store, you can use regular expression to grab the number of pages (of shoes) in the reply.
action(http, "http://www.example.com/search?product=shoes"); 
//the above action will save http replyBody in variable http.replyBody
contains(http.replyBody,  /there are (\d+) pages/); 
pages = g1;  
    Note that "g1" is the global variable that holds first grabbed number. We need to save it to a local variable so we don't lose the value when another instance does the regular expression match.
    What do you do with the (grabbed) number of pages? You can emulate a user clicking through the pages by having a simple loop.
while (id = 2; id <= pages; id ++) {  
    action(http,"http://www.example.com/product=shoes&page=${id}"); 
    sleep(3000);
}  
    Did you notice the loop starts from page 2?  That's because the search reply page typically contains page 1.

    How do we handle json message in HTTP reply?  Some of us may try to use regular expression.  While that may work in some cases, it's error-prone, cumbersome and not very easy for typical users.
Let's suppose HTTP reply contains the following:
 {"type": "contractor", "id": 1234, "name": "Jsmith" }  
    How do we extract "id"?   You just call fromJson() function and now it's easy to access any fields, including "id".
reply = fromJson(http.replyBody);  
id = reply.id;  
if (reply.type == "contractor") {
    action(http,"http://www.example.com/searchContractor/${id}");  
} else {
    action(http, "http://www.example.com/searchOthers/${id}");  
}

     What if the json structure is a little complex like the following and you want to extract some fields?
 { "store":   
 {"book": [{ "category": "reference","author": "Nigel Rees","title": "Sayings of the Century","price": 8.95},  
 { "category": "fiction","author": "Evelyn Waugh","title": "Sword of Honour","price": 12.99},  
 { "category": "fiction","author": "Herman Melville","title": "Moby Dick","isbn": "0-553-21311-3","price": 8.99},  
 { "category": "fiction","author": "J. R. R. Tolkien","title": "The Lord of the Rings","isbn": "0-395-19395-8","price": 22.99}],  
 "bicycle": {"color": "red","price": 19.95}} }  
    Luckily we can use json path.  Here is an example to show how to extract the author of the first book.
a = fromJson(str, '$.store.book[0]');  
author = a.author;  
action(http,"http://www.example.com/search?product=books&author=${author}"); 
    If you need to get the price of the second book, just make a little change
a = fromJson(str, '$.store.book[1]');   
price = a.price;  

    Sometimes server reply is in xml format, so can we deal with that?  On some platform, it's a non-trivial task to deal with xml messages. It's very easy with netgend javascript script.
 //suppose http reply is a xml message.
 <employee>  
 <id>1</id>  
 <firstName>John</firstName>  
 <lastName>Smith</lastName>  
 </employee>  

 //now let's grab a field in the xml message.
 reply = fromXml(http.replyBody)  
 firstName = reply.employee.firstName.value
 //now the variable "firstName" will contain "John" 

    As we mentioned in one of the earlier blogs, HTTP has gotten more complex, so there will be more interesting scenarios coming and we will handle it.

Tuesday, October 29, 2013

Replaying web server log


    It's no secret that testing teams in many companies have attempted to test their servers by replaying web server log (in particular Apache server log).  It sounds simple enough to replay a web server log,  but it actually can be quite complex and can take multiple forms.   Here we are going to show how netgend javascript script can support all forms of replaying.

    In the simplest case, each row in a web server log has a field called URL (let's say, it's the 3rd field),  and we just want to send HTTP GET request based on the URL field.
function userInit() {
 var URLs = readCSVFile("access.log", "\n"); 
 var numOfURLs = getSize(URLs);
 var id = 0;
}
function VUSER () {
 while (id < numOfURLs) { 
  //assume in each row (record), 3rd field is URL.  
  url = URLs[id][3];  
  id ++; 
  action(http,url);
 }
}

    Sometimes, the web server log doesn't have URL field in a record, instead, each record has "protocol", "hostname",  "URI" and "query" fields.  Suppose they occupy fields 2,3, 4, 5  (typically the first two fields in each row are date/time and client IP).  We can concatenate them to format a URL.
function userInit() {
 var URLs = readCSVFile("access.log", "\n"); 
 var numOfURLs = getSize(URLs);
 var id = 0;
}
function VUSER () {
 while (id < numOfURLs ) {
  //assume in each row (record), 4th field is URL.  
  r = URLs[id]; 
  //r[2], r[3], r[4], r[5] are  "protocol", "hostname",  "URI" and "query" respectively
  id ++;  
  action(http,"${r[2]}://${r[3]}${r[4]}${r[5]}");  
 }
}
   
     The above can get a little more complicated in some web server logs:  when a field is null,  , it's sometimes shown as "-".  Let's take care of that.
....
function VUSER () {
 while (id < numOfURLs) {  
  r = URLs[id];  
  id ++;  
  query = r[5];  
  if (query == "-") { 
   query = "";  
  } 
  action(http,"${r[2]}://${r[3]}${r[4]}${query}");  
 }
}  
 

    Some users need to replay web server log with POST data when it's available, unfortunately it's not supported in many test platforms.  It's fairly easy to do on Netgend platform.
function userInit() {
 var URLs = readCSVFile("access.log", "\n"); 
 var numOfURLs = getSize(URLs);
 var id = 0;
}
function VUSER () {
 while (id < numOfURLs) {  
  r = URLs[id];  
  //Suppose r[2] is URL field, r[3] is post data (or "-" if it's null)
  id ++;  
  if (r[3] != "-") {
   http.POSTData = r[3];  
  }  
  action(http,r[2]);
 }
} 


     Some user needs to replay the access log and observe the time-stamp in the log.   In other word, they want to send each http request with the same interval between them as in the access log.
 function userInit() {
 var URLs = readCSVFile("access.log");   
 var numOfURLs = getSize(URLs);   
 var id = 0;  
 var startTS = currentTick;  
 var startTSInLog = numOfURLs[0][0];  
}
function VUSER() {  
 while (id < numOfURLs) {   
  r = URLs[id];   
  id ++;   
  timeElapsedInLog = r[0] - startTSInLog;  
  timeElapsed = currentTick - startTS;  
  ms2sleep = timeElapsedInLog * 1000 - timeElapsed;   
  if (ms2sleep > 0) {   
   sleep(ms2sleep);  
  }  
  action(http,r[2]); 
 }
}  
     Here when a VUser gets a URL, it wil calculate the time that it needs to wait before it sends the HTTP request.   Note that currentTick is in millisecond while the timestamp in the server log is in seconds, that's why we have timeElapsedInLog * 1000.

    As you can see, there are many different requirements on replaying web server log.  netgend platform is flexible enough to handle them all.

HTTP Performance testing made simple


    HTTP is the most popular protocol because it's the foundation of any web site.  Over the years,  it has gotten more complex because people tweak it all the time.   So has performance testing.   There are lots of options on many platforms when testing HTTP/Web server.  As we shall see,  performance testing on HTTP can actually be fairly simple, even when you are emulating 50,000 HTTP clients (VUsers).

    This is what a pair of basic HTTP transactions look like in javascript on Netgend platform.
 action(http,"http://www.example.com");  

 #HTTPS transaction very simple too, just change "http://" to "https://"
 action(http,"https://www.example.com)  
     Piece of cake!

    A little more advanced example is to emulate a user clicking through a sequence of pages.
 for (id = 1; id < 5; id ++) {  
   action(http,"http://www.example.com/docs/tutorial${id}.html"); 
   sleep(10000);
 } 
    Here the "sleep 10000" simulates a user spending 10 seconds (10000ms) reading the content.  You can make the time to be random, i.e. replace the "sleep 10000" with the following two lines to make it more realistic.
   x = randNumber(2000, 30000);  
   sleep(x);  


     HTTP POST is important because it's used in submitting a form, login, registrations and uploading..
This example shows how easy it is
 http.POSTData = "username=jsmith&pass=letmein";  
 action(http,"http://www.example.com/login?");  
    With the support for variables in netgend javascript, we can add some variable part to the POST data.
 name = randString(5,10);
 http.POSTData = "username=${name}&pass=letmein";  


     HTTP header is absolutely important,  with netgend javascript, you can add any HTTP header,  in particular, you can add

  • REFER header (which is useful in server security protection against CSRF) 
  • User-Agent header so that your client will emulate an IE, or Firefox or Chrome.

 httpHeader.REFER = "http://www.example.com/support";  
 httpHeader."User-Agent" = "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)";  
 name = "John";
 http.POSTData = "username=${name}&pass=letmein";  
 action(http,"http://www.example.com/login?");  
     Note that in the scripting, "User-Agent" header is enclosed with double quote, this is to prevent the "-" in the name from being mis-interpreted (as a minus sign).

     It's common to have a test reading data from a csv file and uses that data to "fill" a form and submit to server.  Here is an example:
function userInit() {
     var db = readCSVFile("data.csv");  //global variable
     var id = 0;                        //global variable
}
function VUSER () {
     r = db[id];  
     id ++;  
     info.name = r[0];  
     info.age  = r[1];  
     info.city  = r[2];  
     info.state  = r[3];  
     http.POSTData = toJson(info);  
     action(http,"http://www.example.com/register"); 
}  
    In this example, each VUser (client) will read a line/row from the csv file.  Each row of csv file may have multiple fields:   "name",  "age, "city", "state".
    For instance,  one row may look like  "John,23,Chicago,IL".   The "toJson" function will turn a dictionary variable ("info" in our case) into a json record like
{"name": "John", "age": "23",  "city": "Chicago", "state": "IL" }

     As a final example in this blog, we show how to generate xml message as HTTP Post data.
 function VUSER () {  
      name = "John";   
      age = 23;   
      city = "Chicago";   
      state = "IL";   
      http.POSTData = q|<user>   
      <name>${name}</name>   
      <age>${age}</age>   
      <city>${city}</city>   
      <state>${state}</state>   
 </user>|;   
      action(http,"http://www.example.com/register");  
 }  
    It looks very intuitive, doesn't it?   By the way, you can expand this example so that it will read data from the csv file just like the previous example.   Just don't forget to change the variables in the xml template string from "${name}"  to "${info.name}",   "${age}" to  "${info.age}" etc.

   There are many more interesting examples in performance testing on HTTP/web servers, we will cover them in other blogs, stay tuned.


Monday, October 28, 2013

Performance testing with binary data



    In performance testing, most of the time we are dealing with ASCII data, that's because html messages are in ASCII.  There are cases when we have to deal with binary data, either within HTTP payload or in a protocol that involves binary message.   There are two areas of processing:

  1. Process incoming message (from server) that contains binary data
  2. Encode message in binary format before sending it to server.

    For point 1, we need to have a way to convert a binary data to string. As a hypothetical example, let's suppose an incoming message in the form of "\x00\x01\x00\x04\x00\x00\x03\xe8" (8 bytes in total).  We need to grab the last 4 bytes (which may vary with messages) and change it to decimal format, in this case, 1000  (0x3e8 is 1000).

    For point 2,  we need a simple way to convert numbers in ASCII format into its binary format.

    These issues seem simple, and they can happen in the real world (think of each VUser being a sensor, reporting temperature info to master server),   but sadly many performance test platforms don't support it well.   With netgend platform,  it's fairly easy.

    Here is sample script to handle incoming message.

 //get incoming (binary) message and put it in variable "msg"  
 x = subtr(msg, 4,4);  
 //this will grab 4 bytes from 4th byte in msg  
 x = unpack("N", x);  
 //now x will contain the decimal value corresponding to the 4 bytes  
 //now you can process the value, say, sleep for x millisecond  
 sleep(x); 

    Creating binary messages and sending them are equally easy.
 //suppose we want to send "\x10\x00\x00\0x04" followed by 4 bytes of   
 //temperature in float format  
 temp = rand() * 100;  
 msg = "\x10\x00\x00\0x04"  
 x = pack("f", temp);  
 msg = "${msg}${x}"; 
 //Now we can send it  
 send(msg);  

    What about if we need to send a message that has both binary part and ASCII part? Not a problem.
 msg = "John Smith";  
 buff = pack("n", length(msg));  
 //the above line will pack number "1" in two byte big-endean format  
 buff = combine(buff, msg);  
 send(buff);  

    The above are quite trivial in the eyes of many scripting languages, however,  it becomes less so when you think about many concurrent instances are all receiving and sending messages independently.  Of course, the test scenario can quickly get more complicated when VUsers need to check many binary fields, do sleep or send messages in nested conditions/loops.

The power of a csv file


    In performance testing, one of the most important things is to do parameterization.  Sometimes, parameterization is done by loading data from a csv file.

   The simplest example in my view is a csv file where each row contains a username and password.  How can we emulate user logging in from these username/password pairs?
 function userInit() {
  var list = readCSVFile("users.csv");  
  var i = 0 ;
 }

function VUSER () {
  item = list[i];  
  //here item is an array variable that contains the fields of the row in csv file
  i ++; 
  http.POSTData = "username=${item[0]}&pass=${item[1]}";  
  action(http,"http://www.example.com/login?");  
}  
Notes.

  • variable with a "var" in front of it is a global variable -- it can be accessed/operated by all VUSERs.
  • "list[i]" in VUSER() function means i'th element of the list, in another word, it's i'th row of the csv file. Since  each row contains 2 elements, so "list[i]" (hence "item") is itself an array. 
  • "item[0]", "item[1]" are the first two fields of that row. In our case, they are username and password.


    Sometimes, when data is uniform, we don't need a csv file. For example, if we want to emulate test users like userxxxxx, where xxxxx ranges from 00000, 00001, ... to 99999, we don't have to populate a csv file with these usernames and load it into test.
function userInit() {
  var i = 0 ;
}

function VUSER () {
  user = sprintf("%05d", i);  
  i ++; 
  http.POSTData = "username=${user}&pass=letmein";  
  action(http,"http://www.example.com/login?");  
}
     Sadly, there are some test platform that requires you to create a csv file even in this case!

    what if your csv file consists of a list of products and you just want to randomly pick one and query your e-commerce web site?

function userInit() {
  var list = readCSVFile("products.csv");   
  var listLen = getSize(list);
}

function VUSER () {
  i = randNumber(0, listLen - 1);  
  item = list[i];   
  action(http,"http://www.example.com/query?product=${item[0]}");  
}   
 

    There are some questions asked on stackoverflow where each VUsers needs to go through all the rows in  csv file.  This is so simple that it's sad that it's not supported well on some other platforms.
function userInit() {
  var list = readCSVFile("products.csv");   
  var listLen = getSize(list);
}

function VUSER () {
  for (id = 0; id < listLen; id ++ ) {
    item = list[id][0];  
    action(http,"http://www.example.com/query?product=${item[0]}");  
  }  
}

    Of course, you can make it fancier,  by adding a think-time in the csv file every other line.  In another word, after user does query, he pauses for the specified millisecond before proceeding.  The script is essentially the same as above. The difference is, for the odd numbered lines, the item is the number of milliseconds to sleep.

function userInit() {
  var list = readCSVFile("products.csv");   
  var listLen = getSize(list);
}

function VUSER () {
  for(id = 0; id < listLen; id ++) {
    item = list[id][0]; 
    odd = id % 2;
    if (odd) {
      sleep(item);
    } else { 
      action(http,"http://www.example.com/query?product=${item}");  
    }   
  }  
}


    You may have recognized the power of csv file by now and wonder if you could make it a "CSV-driven test".  Absolutely!  This will work well where a good test script is written by you and other user can vary his csv file to control the behavior of the test.

Networking made easy


    Server performance testing is based on client emulation.  At a high level, all the client emulation boils down to some simple networking operations:
  • client connects to server, 
  • get into a sequence of message exchanges with server, i.e. get into a loop of
    • receive a message from server
    • send one or more messages to server
  • close the connection.
    With Netgend EP scripting, networking is just as easy as it should be!

Make a connection:

 //syntax  
 connect(<hostname|ip>, <port>);

 //example  
 connect("example.com", 25);

Receive a message:

 //syntax  
 recv(<varName>);

 //example  
 recv(msg); 
 //this will put the received message in variable "msg" 

To send a message,

 //syntax  
 send(<msg>);  

 //example 1
 send("Hello my name is John"); 
 //example 2   
 name = "John";  
 msg = "Hello my name is ${name}";  
 send(msg);  

To close a connection,

 //syntax  
 close();  

    What about SSL/TLS based sessions?  Yes, we support them.  In fact, you can start SSL/TLS at any time when the connection is setup.
 starttls();  

    After that, all the messages you send will be encrypted, all the message you receive will be decrypted.  You can process messages as if they are clear text.  This makes it very easy to debug a SSL/TLS based transaction: you can simply print the message you sent or message you received.  This is invaluable since packet captured by sniffers like wireshark may not help (all packets are encrypted!).


    Now combined with variables, conditions supported in EP scripting, we can emulate any complex TCP/IP based protocols.
As a simplified real world example,  let's emulate a simple SMTP client.
 //read from a file and populate variables email and receipant  
 connect("example.com", 25);  
 //wait for server greating  
 recv(msg);  
 send("HELO myhost");  
 recv(msg);  
 send("MAIL FROM: <${email}>"); 
 recv(msg);  
 send("RCPT TO: <${receipant}@example.com>");  
 recv(msg);  
 send("DATA");  
 recv(msg); 
 send("Subject: Test Message\nTest Body\n.");  
 recv(msg);  
 send("QUIT");  
 recv(msg); 
 close();

    You may notice that we don't do any error checking here,  that's because we want to make the basic example look as simple as possible.  Adding error checking and handling is not hard with an "if" condition. For instance,  we can do the following to check if the server greeting begins with "220",
 //read from a file and populate variables email and receipant  
 connect("example.com", 25);  
 //wait for server greating  
 recv(msg); 
 if (! contains(msg, /^220/)) {  
   totalFailure ++;  
   return;
 }  
    Of course, adding error checking for all received messages is going to make the script much longer.

    Our platform can emulate a large number of clients - 50,000!  If I were a server, I would be worried :-)


Sunday, October 27, 2013

Using random functions for performance testing



    When doing performance testing, we may need to use random functions to emulate client behavior.  This blog use the ep scripting (can be seen as pesudo code here) to show some interesting applications of random functions.

    The simplest example is the function that returns the random number between min and max.
 randNumber(0,100);  
 
    Just with this simple function, we can do some fascinating client emulations. For example, suppose we want to emulate lots of sensor clients who will send temperature info periodically to a server.  Each client will start with a random temperature between 50 and 90 degrees and adjust randomly by 2 degrees, after reporting temperature, it will sleep for 10 seconds.
function VUSER() {
   temp = randNumber(50, 90);  
   while (1) {
       temp += randNumber(-2,2);  
       send(temp);
       sleep(10000);  
   }
}

    Along with a random number, we may want to generate a random string.  This can be used to generate a random user name of length between 4 and 10.
 userName = randString(4,10);  

    The above examples look more like simple exercises,  what about some real world user case?  For example, emulating different browsers?  The following script will emulate a Chrome browser with 30%   chance,  Firefox with 30% and IE with 40%.  We can do it by

  • get a random number between 1 and 100,  this will be the random percentage.
  • if the random percentage is between 71 and 100, then it's Chrome browser.
  • if the random percentage is between 41 and 70, then it's Firefox browser
  • if the random percentage is between 1 and 40, then it's IE browser.

function VUSER() {
    r = randNumber(1,100);  
    if (r > 70) { 
        ua = "Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36";  
    } else if (r > 40 ) {
        ua = "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0"; 
    } else  {
        ua = "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)"; 
    } 
    httpHeader."User-Agent" = ua; 
    action(http,"http://www.example.com");  
}



    This is cumbersome, especially if there are many different browsers to emulate with different percentages.   A easier way to do it would be to use the function "rolldice"
 rolldice(<percentage0>, <percentage1>,<percentage2>...);  
    What this does is, given a list of percentages (total sum is 100%), it will return 0 with <percentage0>,  return 1 with <percentage1>, return 2 with <percentage2> ...

    Now let's look at the browser emulation example implemented with rolldice() function:
 UAs = ["Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36",   
  "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0", 
  "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)"];    
 id = rolldice(30,30,40);  
 httpHeader."User-Agent" = UAs[id];  
 action(http,"http://www.example.com");  
   
 
    Isn't it much simpler?  Similarly we can emulate clients who "click" one of the hyperlinks with certain percentages after visiting main page and sleeping a random time between 2 seconds and 10 seconds.

 action(http,"http://www.example.com/"); 
 thinkTime = randNumber(2000, 10000);  
 sleep(thinkTime);  
 URLs = ["http://www.example.com/page1", 
    "http://www.example.com/page2", 
    "http://www.example.com/page3",   
    "http://www.example.com/page4"];  
 #now roll dice with the percentages  
 id = rolldice(20, 40, 30, 10);  
 action(http,URLs[id]);  

    I am sure there will a lot more examples that can be easily implemented using random function like rolldice().

Wednesday, October 23, 2013

A server performance testing platform that is scalable, flexible and simple




    Loadrunner has long been the leading product as server performance/stress testing tool. It's flexible, supports lots of protocols and features. but it's also

  1. Not scalable - one unit only supports up to a 1,000 VUsers.
  2. Complex to use - one needs very good C programming background to take advantage of it.
  3. Expensive - one system emulating 1,000 VUsers may cost close to $92,000 for 500 VUser license.

    There are many other performance testing platforms in this area, but they are less flexible compared to Loadrunner and suffer from the same scalability limitation.   Netgend has created a new testing platform with breakthroughs in both scalability and flexibility.

  • Scalable - one system can emulate over 50,000 VUsers,
  • Flexible,
    • Complex parameterizations is made easy, 
    • Generates dynamic messages to send to server,
    • Processes server response in a very flexible and easy way,
    • Emulates any networking protocols, open or proprietary,
  • Simple to use - complex parameterizations are made easy on this platform.

The platform runs both in cloud and on hardware such as the following mini PC or a industry server.

The mini PC platform may look small, but it's powerful enough to bring a big server or server farm to its knees.

    Flexibility in client emulation comes from the fact that parameters (a.k.a variables within the scope of a VUSER) are built in.  You can use the value from a parameter and/or do operations on a variable directly.   No need to pull the parameters into code space and do some operations and (optionally) write it back to parameter as on other platforms.

   For example, the following code snippet can emulate many stations who send heartbeat info to master server.
 //javascript syntax is used on Netgend platform
 count = 1;
 stationId = getGlobalId(); 
 //global ID will start from 1, incremented by one with each call to this function.  
 while (1) {  
   send("heartbeat count ${count} on station ${stationId}");    
   count ++;  
   sleep(10000);  
 }  
     As shown in this example,  operation on variable "count" feels natural and creating a dynamic message is easy.

    There are lots of examples in the documentation and will come in future blogs.

    This platform is designed for those users who know some basic programming but don't have complex coding background like seasoned programmers/developers.  These users will be more productive on this platform than experienced programmers/developers on other platforms.

    For more details, please check out Netgend and sign up for free - the cloud based platform is free to use during beta period.

Disclaimer: I work for Netgend.