7.6.3 Performance analysis of ADAPTLOAD

This subsection presents a detailed performance analysis of ADAPTLOAD via trace-driven simulation. We selected traces for two consecutive days, June 26 and 27, as representative (see Figure 7.17). During these two days, 52 and 18 million requests were served, respectively. From each trace record we select two values, the request arrival time and the size of the requested file (in bytes).

As described in the previous section, ADAPTLOAD balances the load on the back-end servers using its knowledge of the past request sizes distribution. Specifically, the algorithm of Figure 7.18 schedules the $i^{\rm th}$ batch of $K$ requests according to boundaries computed using the $(i-1)^{\rm th}$ batch of $K$ requests^7.8. As expected, the performance of the policy is sensitive to the value of $K$. For the World Cup 1998 data, we experimented with several values for $K$ and $K = \mbox{50,000}$ appears to be a good choice. Indeed, values between 25,000 and 75,000 results in almost indistinguishable performance, while values in the hundred of thousands are poor due to the high variability over time in the arrival process. In Section 7.6.6, we elaborate on alternative ways to use previous requests to predict the future ones.

We compare ADAPTLOAD against the Join Shortest Weighted Queue (JSWQ) policy, where the length of each queue in the system is weighted by the size of queued requests^7.9. We focus on the following questions:

Can ADAPTLOAD respond quickly to transient overload?   To answer this question, we report performance metrics as a function of time, i.e., we plot the average slowdown perceived by the end user during each time interval corresponding to $K$ requests. Since the system operates under transient overload conditions and is clearly not in steady state, our experiments focus on examining ADAPTLOAD's ability to respond to sudden bursts of arrivals and quickly serve as many requests as possible, as efficiently as possible.

What is the policy sensitivity to different hardware speeds?   To address the common belief that replacing the servers with much faster ones solves the problem of transient overloads, we run our simulations assuming either ``fast'' or ``slow servers''. This is equivalent, in turn, to considering ``low'' and ``high'' load, respectively, and allows us to comment on ADAPTLOAD's ability to quickly flush backed-up queues.

Does the policy achieve equal utilization across servers?   Since ADAPTLOAD bases its boundaries on knowledge of the distribution of requested file sizes, we examine the per-server utilization as a function of time and comment on the policy's ability to distribute load effectively.

Does ADAPTLOAD treat short jobs differently from long jobs?   This question refers to the policy's fairness. To measure the responsiveness of the system, we report the average request slowdown of the classes of requests defined by the request sizes intervals.

Does ADAPTLOAD scale well with respect to the cluster size?   Since ADAPTLOAD's ability to balance the load is a function of an effective mapping of different file sizes to specific servers, we explore the algorithm's scalability by running simulations using the same trace data but on an increased number of back-end servers.

Can we improve ADAPTLOAD's performance with smarter parameterization?   We address this issue by elaborating on alternative ways to use previous requests to predict the future ones. We introduce a new version of the algorithm based on a geometrically discounted history of the request sizes and report on its effectiveness.