Replaying web server log

    It's no secret that testing teams in many companies have attempted to test their servers by replaying web server log (in particular Apache server log).  It sounds simple enough to replay a web server log,  but it actually can be quite complex and can take multiple forms.   Here we are going to show how netgend javascript script can support all forms of replaying.

    In the simplest case, each row in a web server log has a field called URL (let's say, it's the 3rd field),  and we just want to send HTTP GET request based on the URL field.

function userInit() {
 var URLs = readCSVFile("access.log", "\n"); 
 var numOfURLs = getSize(URLs);
 var id = 0;
}
function VUSER () {
 while (id < numOfURLs) { 
  //assume in each row (record), 3rd field is URL.  
  url = URLs[id][3];  
  id ++; 
  action(http,url);
 }
}

    Sometimes, the web server log doesn't have URL field in a record, instead, each record has "protocol", "hostname",  "URI" and "query" fields.  Suppose they occupy fields 2,3, 4, 5  (typically the first two fields in each row are date/time and client IP).  We can concatenate them to format a URL.

function userInit() {
 var URLs = readCSVFile("access.log", "\n"); 
 var numOfURLs = getSize(URLs);
 var id = 0;
}
function VUSER () {
 while (id < numOfURLs ) {
  //assume in each row (record), 4th field is URL.  
  r = URLs[id]; 
  //r[2], r[3], r[4], r[5] are  "protocol", "hostname",  "URI" and "query" respectively
  id ++;  
  action(http,"${r[2]}://${r[3]}${r[4]}${r[5]}");  
 }
}

   
     The above can get a little more complicated in some web server logs:  when a field is null,  , it's sometimes shown as "-".  Let's take care of that.

....
function VUSER () {
 while (id < numOfURLs) {  
  r = URLs[id];  
  id ++;  
  query = r[5];  
  if (query == "-") { 
   query = "";  
  } 
  action(http,"${r[2]}://${r[3]}${r[4]}${query}");  
 }
}  

 

    Some users need to replay web server log with POST data when it's available, unfortunately it's not supported in many test platforms.  It's fairly easy to do on Netgend platform.

function userInit() {
 var URLs = readCSVFile("access.log", "\n"); 
 var numOfURLs = getSize(URLs);
 var id = 0;
}
function VUSER () {
 while (id < numOfURLs) {  
  r = URLs[id];  
  //Suppose r[2] is URL field, r[3] is post data (or "-" if it's null)
  id ++;  
  if (r[3] != "-") {
   http.POSTData = r[3];  
  }  
  action(http,r[2]);
 }
} 

     Some user needs to replay the access log and observe the time-stamp in the log.   In other word, they want to send each http request with the same interval between them as in the access log.

 function userInit() {
 var URLs = readCSVFile("access.log");   
 var numOfURLs = getSize(URLs);   
 var id = 0;  
 var startTS = currentTick;  
 var startTSInLog = numOfURLs[0][0];  
}
function VUSER() {  
 while (id < numOfURLs) {   
  r = URLs[id];   
  id ++;   
  timeElapsedInLog = r[0] - startTSInLog;  
  timeElapsed = currentTick - startTS;  
  ms2sleep = timeElapsedInLog * 1000 - timeElapsed;   
  if (ms2sleep > 0) {   
   sleep(ms2sleep);  
  }  
  action(http,r[2]); 
 }
}  

     Here when a VUser gets a URL, it wil calculate the time that it needs to wait before it sends the HTTP request.   Note that currentTick is in millisecond while the timestamp in the server log is in seconds, that's why we have timeElapsedInLog * 1000.

    As you can see, there are many different requirements on replaying web server log.  netgend platform is flexible enough to handle them all.