I am happy to report that your friendly blogger has arrived safely at the Citrix Summit 06. On my way to the show, I tailed a new Porsche that more than likely belongs to a Citrix sales executive who has been successful selling loads of NetScaler boxes. How can I tell? I can tell because the car’s bumper sticker is quoted in the title of this blog entry, and accelerating Web application is the main reason why IT departments have been buying nearly $1B worth of products from NetScaler and its competitors.
While it is easy to measure the performance of a Porsche, it is not so easy when evaluating a NetScaler. Ascertaining the performance of Web application is exceedingly complex and involves many moving parts, especially when one has to do it without a speedometer. As a result, IT cannot definitely say whether they are buying a German Porsche or a Moroccan Porsche (see photo).
As I indicated in my previous blog entry, we are working with Citrix to address this critical need. Until an easy-to-use speedometer is built into NetScaler, there are a number of techniques that can be used to measure performance enhancements:
- Anecdotal evidence. “The application feels faster.”
- A stopwatch. Run a series of scripted transactions and measure the response time manually using a stopwatch from, say, a branch office,. The technique is not accurate (as the scripted transactions might not be representative of how the application is being used), not statistically significant, and cannot show the improvement NetScaler brings to the edge cases (see my blog entry: “Real, Unreal and Average”)
- A synthetic monitoring service. Subscribe to an external service that will fire off some scripted synthetic transaction every couple of minutes from some fixed reference sites. This technique has the same limitation as the stopwatch approach. The only difference is that this technique involves a third party vendor and costs money.
- Sniffer appliances. Sniff the packets on the network and estimate round-trip time. This technique can only monitor local objects/traffic, and therefore it misses cached pages and objects that come from third-party content suppliers. Sniffers also have difficulty resolving frames and other pages with ambiguous reference headers. Realistically, do you want to buy and manage another box that does nothing but a half-witted job estimating performance for the purpose of evaluating NetScaler?
- Get real. This, I believe, is the only way to measure performance enhancement brought about by NetScaler. Using this technique, the end users’ browser is dynamically and non-intrusively instrumented to report on the performance it is getting. The performance information from real users are collected and summarized to present the most accurate and complete before and after picture.
If you have other tried-and-true techniques that are not covered here, please share with everyone by commenting on this post or email me at hon@symphoniq.com.
