The three benchmarks do basically the following:
- The add benchmark adds 100 items to an empty collection.
- The iterator benchmark iterates over an collection of 100 items.
- The contains benchmark checks if 20 items are contained in a collection of 100 items. 10 of those items are contained in the collection, and another 10 are not.
The results I report here are rough averages from repeated measurements (~20 times) on a Core 2 Duo 2.4 GHz machine with 4GB ram, running Windows 7 64bit and GWT 2.0.3. They should be accurate with about +/- 10% error.
Chrome 6.0.472.63 beta | Firefox 3.5.13 | Dev Mode (FF 3.5.13) | |
---|---|---|---|
add - ArrayList | 19 ms | 550 ms | 14 ms |
add - HashSet | 200 ms | 680 ms | 30 ms |
add - JsArray | 14 ms | 540 ms | 142000 ms |
iterator - ArrayList | 44 ms | 395 ms | 12 ms |
iterator - HashSet | 138 ms | 1220 ms | 8 ms |
contains - ArrayList | 4700 ms | 8500 ms | 37 ms |
contains - HashSet | 23 ms | 110 ms | 6 ms |
contains - JsArray | 8 ms | 36 ms | 26500 ms |
Several things can be observed based on these results:
Measuring and optimizing performance in the developer mode is useless and can be dangerous. The developer mode is very different from running compiled GWT code in the browser. Code that is executed only within the Java VM tends to be a lot faster. Manipulation of the DOM and calling JavaScript using JSNI, however, is much slower (see also GWT developer guide). I was surprised by the extent of this effect, for example that Java collections are up to 125 times faster than GWT compiled code on Chrome 6 (ArrayList.contains), or that JsArray.contains is 736 times faster when compiled and run on Firefox 3.5 vs. in the developer mode on Firefox 3.5.
Chrome 6 is much faster than Firefox 3.5. This result is pretty obvious given recent browser benchmarks. However, the differences vary depending on the collection type and operation. ArrayList.contains is surprisingly slow on Chrome 6.
The different collection classes have different advantages. This is also pretty obvious given their different requirements and algorithms. The HashSet is slow for add and iterate, but fast for checking the containment of items. It also assures that each element is only contained once. The ArrayList is much faster than the HashSet when calling add and iterate, but also 200 times (Chrome 6) / 77 times (Firefox 3.6) slower when it comes to containment checks, which was surprising.
Potential speed gains when using handcrafted collection classes. The results for the JsArray class indicate that there is a potential for speed improvements by using lightweight collection classes. This finding is different from a benchmark that used bubble sort. The difference might be related to the overhead added by the GWT cross-compilation of ArrayList and HashSet.
In conclusion, when having performance issues in GWT applications that use a large number of collection instances, it might be worth looking at using optimized collection classes that reduce the overhead introduced by ArrayList and HashSet.