The new Eclipse download wizard and RAP performance

The Eclipse Packaging Project is working on a wizard that allows you to build your own customizable download, combining plug-ins from predefined packages and Ganymede. The wizard uses Eclipse runtime technology – namely RAP for the web frontend and Equinox P2 for the installer. Before exposing the wizard to a wider audience, we had one burning question: Does it scale up to the volume of downloads being served by Eclipse.org? Read on to find out.

Load Test – Setup

We conducted a load test on the wizard and would like to share the results with you.

The tests were executed on a modern desktop computer with a Core2Duo 2.8 GHz CPU running Ubuntu Hardy. The JVM used a maximum of 512 MB of heap space and ran the concurrent marksweep garbage collector (-UseConcMarkSweepGC). The tests were run using JMeter 3.2 with 200 concurrent threads. Tomcat 6 was used for the server.

Load Test – Results

The results are promising: with a throughput of 100 requests per second we achieved an average response time of 14 miliseconds. Configuring a download takes up to 80 requests, which means we can serve approximately 100,000 downloads per day with a similar configuration.

Here’s a JMeter screenshot with more details on the results:

The fast average response time is paired with a low standard deviation of 60-80 milliseconds. The maximum response times were caused by full garbage collections, but only a few users should experience these response times (otherwise we would have higher average response times).

The allocation of heap is visualized in the diagram below, after multiple hours of load testing. A heap space of 250 MB was constantly needed for the approximately 1,000 concurrent user sessions that were simulated (250KB per session). Another 200 MB was built up and collected every one to two minutes.

Cloud Tests

To demonstrate performance we also ran tests against the same application running on the smallest available type of Amazon EC2 instance. We ran the same load test as before - but using only 50 concurrent threads. After a period of 16 hours 16,000 sessions had been served, which means a throughput of about 24,000 sessions a day. This is almost equivalent to all the downloads served from Eclipse.org on a busy day. Average response time was approximately 100ms, with network latency accounting for most of the response time. CPU utilization on the server was around 15%.

Conclusion

We’re very positive about the results so far. Even though there is a memory overhead associated with RAP applications, in practice the performance is very solid. Random sampling of the application via a browser in parallel with the load tests showed that responsiveness was very good.

But don’t just take our word for it - try it out for yourself. The application is hosted on an EC2 instance reachable at https://eppwizard.dyndns.org/eppwizard/rap. The load test is still running with 50 concurrent sessions against that same instance.

We plan to beta test the wizard with the “Friends of Eclipse” after seeking initial feedback from the community on both usability and performance. Take it for a spin and tell us what you think.