Friday, May 2, 2014

Reduce the complexity of string formatting



    Performance testing requires more advanced skills than many other testing areas, for example, functional testing.   String formatting is an essential part of the performance testing skill set.  Many beginners think of it as a headache thanks to some examples they saw where testers have to use APIs to move data back and forth between per-instance parameters and  variables in code. As a consequence, even something as simple as “A = B + C” is not simple any more.    
  
   As this blog shows,  many cases the complexity is due to the design of the test platform. With a properly designed test platform, the programming complexity can be abstracted away from the tester and hence scripting can be so simple and clean that it is a pleasure.

    There is a Q&A thread on stackoverflow about adding multiple values to a parameter.  Basically the tester needs to send an HTTP POST with data in the following format, where the numbers are dynamically extracted from previous HTTP response.

 articleIds=["148437", "148720"]  

    This request seems innocent enough, but in fact is not that easy to implement. The author of the question,  who has a good programming background based on his stackoverflow reputation points, must have struggled with it before asking the question in the forum.  The answer comes from a JMeter guru who had answered hundreds if not thousands of questions.  My hat comes off to him!    The code snippet is given here not to ask readers of this blog to understand it, but rather to appreciate its complexity.

StringBuilder sb = new StringBuilder();
sb.append("[");
int count = Integer.parseInt(vars.get("articleID_matchNr"));
for (int i = 1; i <= count; i++) {
    sb.append("\"");
    sb.append(vars.get("articleID_" + i));
    if (i < count) {
        sb.append("\", ");
    }
}
sb.append("\"]");
sampler.addArgument("articleIds", sb.toString());

     The above solution works well,  but it can be hard to program, especially for testers who have not  been doing a lot of programming.    Fortunately, the solution on NetGend is so simple that every test scripter can learn it in a minute.

     To give a more precise context to the test scenario, let us assume the server sends a response like the following, we need to send a HTTP request as required by the author.
....
<preference>148437</preference><preference>148720</preference>
....

     The script on NetGend is quite simple.  The three bold lines in the following basically do what's in the complex code above. The code around the bold lines are there to show how easy it is to make it into a more complete implementation.

1
2
3
4
5
6
7
8
function VUSER () {
    action(http, "http://www.example.com/viewUsers");
    ids = substring(http.replyBody, "<preference>", "</preference>", "all");
    ids = toJson(ids);
    b.articleIds = ids;
    http.POSTData = combineHttpParam(b);
    action(http,"http://www.example.com/viewIDs");
}

  • Line 2:  perform a HTTP transaction, the HTTP response will be in the variable "http.replyBody".
  • Line 3:  extract the value by specifying a left boundary and right boundary and put the result, an array,  in the variable "ids"
  • Line 4:  convert the variable ids who holds the array into a JSON string, in this case, the JSON string should look like  ["148437", "148720"]
  • Line 5:  create a dictionary variable b,  which has a field called "articleIds" and the value of the field is the JSON string "ids"
  • Line 6:  call the function "combineHttpParam" on the dictionary b to create the desired HTTP Post data:   articleIds=["148437", "148720"]. This function will turn a dictionary variable into HTTP parameters like key1=value1&key2=value2 ...
  • Line 7:  do a HTTP transaction with the HTTP POST data.
    As you can see, the complexity of string manipulation (think of the code around StringBuilder in Java) is abstracted from the functions toJson and combinteHttpParam.

   At NetGend, we are driven by a desire to make things as simple and intuitive as possible. We are proud that our platform has the right architecture and technology to actually do it.  If you are interested in learning more, please drop us a line:  info@netgend.com.

Thursday, April 24, 2014

Test Application Monitoring Software



    Today’s blog is addressed to the infrastructure folks who have the job of making sure all of our applications and monitoring systems are functioning effectively. The NetGend load testing tool is awesome for testing applications at scale, but we will discuss another use – how do we generate synthetic yet realistic load to test monitoring, filtering and scanning systems that sit in our infrastructure?

    In today's internet-based economy, it's essential to ensure that servers are running efficiently and as-expected.  It is important to periodically check the health of the servers.  Application monitoring software is a key tool in validating one’s performance and business-readiness.  Another example of non-traditional load testing is with filters and proxies, which also have a role in ensuring the protection and configuration and enforcement of various policies – application blocking, logging, etc… (We will talk about this in more detail in an upcoming blog.)


    How do we check the checkers? In the case of application monitoring software, just like any other software, it is important to load test it during development and after functional testing phase.   A natural question is, how do we emulate a large number of servers while injecting a sufficient/configurable number of abnormal conditions?


   It turns out that NetGend, in addition to being a great performance testing platform,  can also emulate the servers-under-monitoring.  More specifically,  it can emulate the agents running on the servers and emulating the sending of the server statistics such as CPU, memory,  etc to the application monitoring software.

   Here is an example of NetGend script that can be used to emulate thousands of agents.   From the standpoint of TCP/IP, each of the agents is simply a TCP client, sending data to the TCP server (the monitoring system) repeatedly.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
function VUSER () {
    connect("tcp", "monitorserver.example.com", 3000);
    isGoodServer = randNumber(0, 50);
    cpuLimit     = 100;
    memoryLimit  = 100;
    if (isGoodServer) {
        cpuLimit    = 80;
        memoryLimit = 90;
    }
    cpu    = randNumber(1, cpuLimit);
    memory = randNumber(20, memoryLimit);
    while (1) {
        cpu    = randWalk(cpu, 0, cpuLimit, 10);
        memory = randWalk(memory, 0, memoryLimit, 10);
        send("${cpu},${memory}");
        sleep(1000);
    }
}

Here is a brief explanation of the above code:
  • Line 2,  sets up a TCP connection to the server.
  • Line 3,  decides whether the instance is going to be a "bad"/"unhealthy" server, in which case, CPU or memory usage may shoot up to 100%
  • Lines 4-9,  sets the upper limit. For healthy ones, we set the upper limit to be 80% on CPU and 90% on memory.  
  • Lines 10-11, sets the starting value for cpu and memory.
  • Lines 13-16, updates the cpu and memory percentage and sends the information to the server.

   This script is so concise and efficient thanks to our function randWalk(), which gets its name from the phrase "random walk".  This function takes 4 parameters
<currentValue>, <lowerLimit>, <upperLimit>, <step>.  
The returned value can go either up or down from <currentValue> by up to <step> amount.

   This brief example reports only the statistics on CPU and Memory usage, but it can be easily extended to report other statistics as well, such as Disk usage, IO, bandwidth usage etc.  Also, some monitoring software may expect the agents to report the statistics in the form of HTTP/HTTPs requests or other protocols over TCP or SSL/TLS.  Rest assured,  NetGend supports all of the above transport mechanisms.

   At NetGend, we are proud of the flexibility and scalability of our platform, we are especially happy that it can be used to test Application monitoring - a sister software to application performance testing.  Also, if you are interested in how this can be used to generate load for proxies or Big Data analysis, please don’t hesitate to  please drop us a line  info@netgend.com.