23 December 2010

Generic JavaScript Array for GWT objects

GWT supports several wrappers for JavaScript arrays, e.g. JsArray, JsArrayNumber and JsArrayInteger. However, they all require JavaScript primitives or JavaScriptObjects. I needed to pass a JavaScript array containing regular GWT objects to a third party JavaScript library, together with a callback function that resolves specific properties for elements in the array. The skeleton of the array is similar to the JsArray classes provided by GWT (full source code):

import com.google.gwt.core.client.JavaScriptObject;

public class JsGenericArray<T> extends JavaScriptObject {

// typical JsArray methods

}
I initially tried to use this array with the following factory method:

public static native <T> JsGenericArray<T> createGenericArray() /*-{
return [];
}-*/;
However, I ran into the problem that instanceof Array tests on this array in the 3rd party libraries returned false (as debugging showed). This is because GWT runs in an iframe and instanceof does not work across frames. One recommended solution is duck typing, but this was not an option because I did not want to change the 3rd party JavaScript library. To address this issue, I changed the factory method to return an array object from the main frame in which the JavaScript libraries run:

public static native <T> JsGenericArray<T> createGenericArray() /*-{
return new $wnd.Array();
}-*/;
This approach worked with Firefox 3.6 and Chrome 9, but not with Safari (see Webkit Bug 17250). However, I am not a JavaScript expert and would appreciate any feedback on potential shortcomings of this solution, especially concerning cross-browser compatibility.

22 October 2010

Choosel at VisWeek 2010 and at CASCON 2010

Interested in data visualization on the web? There are several Choosel-related events at VisWeek 2010, CSER and CASCON 2010. Choosel is an open-source framework for browser-based data visualization.

Watch video in full screen and HD for better quality

Oct 24-28, I will present the InfoVis poster "Choosel – Web-based Visualization Construction and Coordination for Information Visualization Novices" at VisWeek 2010. Here is a preview:


Oct 31st, there will be a presentation on Choosel and the Work Item Explorer at CSER. The Work Item Explorer (developed by Patrick Gorman, Del Myers and Christoph Treude) is a research prototype built on Choosel that facilitates the flexible, iterative exploration of Jazz data, focusing primarily on work items.

Nov 1-4, there will be CASCON exhibits on Choosel and the Work Item Explorer. The Choosel exhibit (Bradley Blashko, Lars Grammel) will be at booth X2 near the Central Tower, and the Work Item Explorer exhibit (Patrick Gorman, Christoph Treude) will be at booth U5. The exhibits are open 5pm to 7pm on Monday (Nov 1), from 8.30am to 7pm on Tuesday and Wednesday (Nov 2 and 3), and from 8.30am to 1pm on Thursday (Nov 4).

07 October 2010

Attracting the Community’s Many Eyes: an Exploration of User Involvement in Issue Tracking

User input drives the development of software systems by contributing ideas, reporting bugs and clarifiying requirements. To encourage such feedback, open source projects often provide issue tracking systems (e.g. Bugzilla) that are open to the public. In commercial software development, however, this is often not the case. I was interested if opening issue tracking systems of commercial projects to the public would yield similar community input as is the case for open source projects. Together with Holger Schackmann, Adrian Schröter, Christoph Treude and Margaret-Anne Storey, I studied community involvement in issue tracking in the Eclipse and IBM Jazz projects. IBM Jazz is a commercial IDE for collaborative development. The Jazz project has opened its issue tracking system to the Jazz community. Eclipse is a well-known open source IDE with a public issue tracking system.

Our 8 page research paper "Attracting the Community’s Many Eyes: an Exploration of User Involvement in Issue Tracking" was accepted at HAoSE 2010, the Second Workshop on Human Aspects of Software Engineering, co-located with SPLASH 2010. Here is the abstract of our paper:

A community of users who report bugs and request features provides valuable feedback that can be used in product development. Many open source projects provide publicly accessible issue trackers to facilitate such feedback. We compare the community involvement in issue tracker usage between the open source project Eclipse and the closed source project IBM Jazz to evaluate if publicly accessible issue trackers work as well in closed source projects. We find that IBM Jazz successfully receives user feedback through this channel. We then explore the differences in work item processing in IBM Jazz between team members, project members and externals. We conclude that making public issue trackers available in closed source projects is a useful approach for eliciting feedback from the community, but that work items created by team members are processed differently from work items created by project members and externals.

Download Paper

26 September 2010

GWT ArrayList, HashSet and JsArray Benchmark

The performance of GWT applications varies quite a bit between the developer (hosted) mode and the compiled JavaScript code, as well as between different browsers. This blog post describes a benchmark [ Run Benchmark, Source Code ] of two different collection classes (ArrayList, HashSet) in three different environments (Chrome 6, Firefox 3.5, Dev Mode). It tests the add, contains and iterator methods, and also compares the collection classes to a lightweight JavaScript array wrapper.

The three benchmarks do basically the following:
  • The add benchmark adds 100 items to an empty collection.
  • The iterator benchmark iterates over an collection of 100 items.
  • The contains benchmark checks if 20 items are contained in a collection of 100 items. 10 of those items are contained in the collection, and another 10 are not.
Each benchmark is run 2500 times (see GWTBenchmark class).

The results I report here are rough averages from repeated measurements (~20 times) on a Core 2 Duo 2.4 GHz machine with 4GB ram, running Windows 7 64bit and GWT 2.0.3. They should be accurate with about +/- 10% error.
 Chrome
6.0.472.63 beta
Firefox
3.5.13
Dev Mode
(FF 3.5.13)
add - ArrayList19 ms550 ms14 ms
add - HashSet200 ms680 ms30 ms
add - JsArray14 ms540 ms142000 ms
iterator - ArrayList44 ms395 ms12 ms
iterator - HashSet138 ms1220 ms8 ms
contains - ArrayList4700 ms8500 ms37 ms
contains - HashSet23 ms110 ms6 ms
contains - JsArray8 ms36 ms26500 ms

Several things can be observed based on these results:

Measuring and optimizing performance in the developer mode is useless and can be dangerous. The developer mode is very different from running compiled GWT code in the browser. Code that is executed only within the Java VM tends to be a lot faster. Manipulation of the DOM and calling JavaScript using JSNI, however, is much slower (see also GWT developer guide). I was surprised by the extent of this effect, for example that Java collections are up to 125 times faster than GWT compiled code on Chrome 6 (ArrayList.contains), or that JsArray.contains is 736 times faster when compiled and run on Firefox 3.5 vs. in the developer mode on Firefox 3.5.

Chrome 6 is much faster than Firefox 3.5. This result is pretty obvious given recent browser benchmarks. However, the differences vary depending on the collection type and operation. ArrayList.contains is surprisingly slow on Chrome 6.

The different collection classes have different advantages. This is also pretty obvious given their different requirements and algorithms. The HashSet is slow for add and iterate, but fast for checking the containment of items. It also assures that each element is only contained once. The ArrayList is much faster than the HashSet when calling add and iterate, but also 200 times (Chrome 6) / 77 times (Firefox 3.6) slower when it comes to containment checks, which was surprising.

Potential speed gains when using handcrafted collection classes. The results for the JsArray class indicate that there is a potential for speed improvements by using lightweight collection classes. This finding is different from a benchmark that used bubble sort. The difference might be related to the overhead added by the GWT cross-compilation of ArrayList and HashSet.

In conclusion, when having performance issues in GWT applications that use a large number of collection instances, it might be worth looking at using optimized collection classes that reduce the overhead introduced by ArrayList and HashSet.

12 September 2010

Combining Semi-Transparent Window Borders with Opaque Content using CSS RGBa

Developing web applications that support multiple window-like panels in a single browser page is challenging. This blog post describes how I implemented semi-transparent window borders in Choosel to help with window resizing. I used CSS RGBa background colors to combine semi-transparent window borders with opaque window content.

Choosel is a GWT framework that aims at facilitating flexible visual data exploration. It supports multiple windows on the same page. The windows can be dynamically created, closed, moved, resized and brought to the front. Different window content types such as maps, charts & notes are available.

However, one of the findings from a usability study was that resizing the windows was difficult. The main reason was that the window borders were fairly thin (3 pixel). So I decided to increase the border thickness to 7 pixel. However, I found that increasing the thickness of opaque borders often occluded relevant content from underneath, e.g. from other windows. To solve this problem, I implemented semi-transparent borders using CSS. The two screenshots below illustrate the differences between the two designs:

Thin, opaque borders (click thumbnail to enlarge screenshot)


Thick, semi-transparent borders (click thumbnail to enlarge screenshot)


A window in Choosel is basically an HTML div element that contains an HTML table. The table contains cells for the borders, the header and the content. To allow for transparent boundaries and opaque window contents at the same time, I set the background color of the window div to be completely transparent using RGBa:
background-color: rgba(0, 0, 0, 0.0);
Using CSS opacity was not an option, because opacity is inherited and thus combining opaque window content with transparent border would not have been easily possible. The borders have semi-transparent background colors, again set using RGBa. The transparency of the window content depends on the implementation of the content type. For example, notes are slightly semi-transparent when inactive, but the visualization views are opaque.

The semi-transparent window borders were tested and work with Firefox 3.6, Chrome 6 and Safari 5. Choosel is not designed for Internet Explorer (up to version 8).

10 September 2010

Controlling the Z-Index of Map Overlays using the Google Maps Library for GWT

Google Maps is a great way to show location-based data on a map. In Choosel, we implemented a generic map widget which leverages Google Maps. This can for example be used to visualize the locations of recent earthquakes. Choosel also supports multiple coordinated view with selections and highlighting of items across different views (see Video):



However, occlusion in the map becomes a problem when trying to highlight resources across views. For example, when highlighting a particular earthquake in the timeline, it might be hidden by other earthquake overlays which are displayed on top of it in the map. The z-index of the earthquake overlay in the map would need to be adjusted such that the earthquake is displayed on top while being highlighted, but this is not directly possible using the Google Maps API 1.0 for GWT.

I implemented a custom Overlay class that uses a GWT label to display a data item. Using a GWT widget in the custom overlay has several advantages: (1) we can change the CSS styling, including the z-index, (2) we can use standard GWT event handlers, and (3) we don’t need to load images.

Before, we used the MapIconMaker from the gmaps utility library. The CSS based approach has some cross-browser limitation, e.g. rounded corner on IE, but it is faster because no images are loaded any more. However, the z-index changes outlined here are possible with any GWT widget, so switching to an Image widget should be easy.

This are the main elements of the LabelOverlay implementation used in Choosel:
public class LabelOverlay extends Overlay {

private Label label;

private LatLng latLng;

private MapWidget map;

private Point offset;

private MapPane pane;

private Point locationPoint;

public LabelOverlay(LatLng latLng, Point offset,
String text, String styleName) {

this.latLng = latLng;
this.offset = offset;
this.label = new Label(text);
this.label.setStyleName(styleName);
}

public HandlerRegistration addClickHandler(
ClickHandler handler) {

return label.addClickHandler(handler);
}

// ... other mouse handlers (down, move etc.)

@Override
protected final Overlay copy() {
return new LabelOverlay(latLng, offset,
label.getText(), label.getStyleName());
}

@Override
protected final void initialize(MapWidget map) {
this.map = map;

pane = map.getPane(MapPaneType.MARKER_PANE);
pane.add(label);

updatePosition(
map.convertLatLngToDivPixel(latLng));
}

@Override
protected final void redraw(boolean force) {
/*
* We check if the location has changed, because
* Google Maps allows infinite panning along the
* east-west-axis and requires an updated widget
* location in this case, although it will not
* force redrawing.
*/
Point newLocationPoint =
map.convertLatLngToDivPixel(latLng);

if (!force
&& sameLocation(newLocationPoint)) {
return;
}

updatePosition(newLocationPoint);
}

@Override
protected final void remove() {
label.removeFromParent();
}

private boolean sameLocation(Point newLocationPoint) {
assert newLocationPoint != null;
return locationPoint != null
&& locationPoint.getX()
== newLocationPoint.getX()
&& locationPoint.getY()
== newLocationPoint.getY();
}

public void setZIndex(int zIndex) {
// CSS is a class in Choosel
CSS.setZIndex(label, zIndex);
}

// ... other CSS attribute setters

private void updatePosition(Point newLocationPoint) {
assert newLocationPoint != null;
locationPoint = newLocationPoint;
pane.setWidgetPosition(label,
locationPoint.getX() + offset.getX(),
locationPoint.getY() + offset.getY());
}

}
The code uses some convenience methods from the Choosel CSS library class. The overlay are styled by default using this CSS class:
.resourceItemIcon {
width: 16px;
height: 16px;
border: 1px solid darkgray;
text-align:center;
vertical-align:middle;
font-weight:bold;
font-size: 12px;
-moz-border-radius: 10px;
-webkit-border-radius: 10px;
padding: 0px;
}
The rest is pretty straightforward, the overlays are created and added:
overlay = new LabelOverlay(point, Point.newInstance(-10, -10),
label, CSS_RESOURCE_ITEM_ICON);
map.addOverlay(overlay);
In Choosel, this functionality is distributed in the MapItem and MapViewContentDisplay classes.

Using the approach outline in this blog post, you can use GWT widgets in map overlays and control CSS attributes such as the z-Index.

08 September 2010

Choosel Poster at InfoVis 2010

I will present a poster on the Choosel Framework at IEEE InfoVis 2010. The main goal of the Choosel project is to enable software developers and researchers to easily create web-based visual data exploration environments for novices. The poster paper briefly summarizes some of the related work, the features of Choosel, and the results of a prelimary usability evaluation:

Since information visualization has become increasingly vital to experts, it is now important to enable information visualization novices to consume, construct, and coordinate visualizations as well. Choosel is a web-based environment that aims at facilitating flexible visual data exploration for information visualization novices. It supports the iterative construction of multiple coordinated views during the visual data analysis process. A preliminary user study with 8 participants indicated that multiple windows, enhanced drag and drop interaction, and highlighting of items and sets, in particular, support novices in the visual data exploration process in a useful and intuitive way.

Download Poster Abstract (2 pages)

01 September 2010

Why Choosel is based on GWT

I presented Choosel to the Visual Interaction Design (VisID) research group at the University of Victoria. Choosel is an open-source framework for web-based information exploration environments aiming at information visualization novices.

The first part of my presentation focused on several design decision behind Choosel. The framework is targeting information visualization novices - those who are not familiar with information visualization and visual data analysis beyond the graphics encountered in everyday life. Two major design decision we made based on those constraints is choosing the web as the target platform and developing Choosel using GWT.

We assumed that those information visualization novices are more likely to look at smaller data sets (up to 5000 items), but are not willing to spent much time getting started with visual data analysis. This was the main driver behind the decision to develop a web-based environment, because this spares user the burden of installing software. We considered removing this entry barrier more important then scalability beyond several thousand data items. As our main goal was to a provide interactive information exploration environment, responsiveness was important and we decided to use primarily technology that runs on the user's computer and not on the server.

In Choosel, we leverage third party visualization components and toolkits such as the Simile Timeline, Protovis and FlexViz. In order to be able to integrate different technologies such as Flash and JavaScript in the browser, we decided to use a JavaScript based technologies. First, we developed a initial prototype using the dojo toolkit. However, it turned out that because of our software development skills and tool support for unit testing, refactoring, and debugging, we were able to develop the same prototype using GWT in about a quarter of the time. The current version of Choosel is based on GWT.

Here are the slides from my presentation:

22 July 2010

How Information Visualization Novices Construct Visualizations

Visualization for the masses is a topic that has gained a lot of attraction in the InfoVis community in recent years, e.g. in projects such as IBM ManyEyes. The goal is to enable a wide user population to leverage information visualization technology to understand large amounts of data. This could potentially help them make more informed decisions, and is especially promising as more and more data becomes available (see open data). However, there are still many challenges that need to be addressed so that visualization for the masses can become a reality, ranging from limited visual literacy to insufficient tool support.

Together with Melanie Tory and Margaret-Anne Storey, I investigated how information visualization novices construct visualizations in a laboratory setting. Our research paper "How Information Visualization Novices Construct Visualizations" was accepted for presentation at IEEE InfoVis 2010.

Here is the abstract of our paper:

It remains challenging for information visualization novices to rapidly construct visualizations during exploratory data analysis. We conducted an exploratory laboratory study in which information visualization novices explored fictitious sales data by communicating visualization specifications to a human mediator, who rapidly constructed the visualizations using commercial visualization software.

We found that three activities were central to the iterative visualization construction process: data attribute selection, visual template selection, and visual mapping specification. The major barriers faced by the participants were translating questions into data attributes, designing visual mappings, and interpreting the visualizations. Partial specification was common, and the participants used simple heuristics and preferred visualizations they were already familiar with, such as bar, line and pie charts.

From our observations, we derived abstract models that describe barriers in the data exploration process and uncovered how information visualization novices think about visualization specifications. Our findings support the need for tools that suggest potential visualizations and support iterative refinement, that provide explanations and help with learning, and that are tightly integrated into tool support for the overall visual analytics process.

Download Technical Report

19 April 2010

Choosel Mashup Framework Available on Google Code

I released the choosel mashup framework under the Apache 2.0 license on Google Code. It is a research prototype platform that is used in our Bio-Mixer tool, which is developed at the CHISEL group.

Choosel supports the creation of web-based information mashup environments. These mashup environments facilitate the flexible recombination of information in different views such as maps, timelines and graph viewers. Users without any programming expertise can remix information using drag and drop interaction and explore data sets. The workspaces (mashups) can be stored and shared among users.

We are looking for contributors. If you are interested, please post on the choosel mailing list.

14 April 2010

Supporting End Users in Coordinating Multiple Visualizations

As part of my PhD research, I am looking at ways to make visual data analysis more accessible to end users without data analysis expertise. Specifically, I am researching how they can easily coordinate multiple visualizations. This has led to the development of web-based visual analytics research prototype, which I evaluated in a user study. The results indicate that novel concepts such as drop target highlighting, drop previews and using multiple user defined sets are useful and easily usable for end users.

I presented the tool and the results (see poster and presentation below) at the IBM University Days 2010. A version of the visual analytics environment that is tailored to exploring biomedical ontologies is available at: bio-mixer.appspot.com


09 April 2010

Bio-Mixer Early Research Prototype Released

I am happy to announce the early alpha release of Bio-Mixer, a research prototype developed at the CHISEL group at the University of Victoria in collaboration with the National Center of Biomedical Ontology. Bio-Mixer is available at:

bio-mixer.appspot.com

Bio-Mixer is a web-based environment that supports the flexible exploration of biomedical ontologies. The concepts in the ontologies and their mappings can be explored in different views such as graph views, lists and timeline views. Drag-and-drop interaction can be used to show items and collections in different views, to create filtered views and to synchronize selections. Bio-Mixer enhances drag and drop with a new drop target highlighting and preview approach to make working with multiple collections and views easy. Bio-Mixer also provides support for ontology annotation and workspace sharing between collaborators.

Bio-Mixer Overview



Bio-Mixer Features in Detail

18 March 2010

Handling Flex Events in GWT

Tightly integrating Flex components in GWT applications requires that GWT is notified on events triggered in Flex. For example, when the user performs an action in the Flex component, e.g. double-clicking a node in a graph widget, we might want to perform some operations on the GWT side, e.g. updating a dependent view to show information about the double-clicked node. This tutorial shows how UI event from Flex can be forward to and handled by GWT code. The example contains a GWT text box and a Flex text box and buttons for copying the text from one to the other. This tutorial is the third part of a series of blog posts about using Flex components in GWT with the library gwt2swf:

1. GWT Wrapper for Flex components
2. Notifying GWT when a Flex widget is loaded [Demo, Source Code]
3. Handling Flex events in GWT [Demo, Source Code]

This tutorial is based on the previous one, notifying GWT when a Flex widget is loaded, and assumes that you have a similar project setup and that you have created the classes and files with the source code from that tutorial. The main ideas are similar those from the previous tutorial, but we will explicitly register our event handler bridge as a callback in Flex. That way, the Flex component can expose operations that might not be used by our GWT application, but in other applications, e.g. separate JavaScript projects, without requiring the GWT wrapper to provide hooks for those operations.

The Flex component is first extended by adding a 'send' button and changing the text to a text input. Then a method for adding a JavaScript method as event listener is implemented and exposed. When calling registered event listeners, the Flex components adds the content of the text field, so we can use it on the GWT side. Alternatively, one could implement and expose a method for getting the content of the Flex text field from JavaScript.

Text.mxml
<mx:Button id="sendButton" x="200" width="80" label="Send"/>
<mx:TextInput id="textWidget" x="0" width="200"/>

...

public function addSendListener(jsFunctionName:String):void {
sendButton.addEventListener(MouseEvent.CLICK, function(event:MouseEvent):void {
ExternalInterface.call(jsFunctionName, swfID, textWidget.text);
});
}

private function init():void {
...
addJSCallback("addSendListener", addSendListener);
}
The complete Flex component should look like this:

Text.mxml
<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml"
layout="absolute" creationComplete="init()"
backgroundColor="0xffffff" paddingBottom="0" paddingLeft="0"
paddingRight="0" paddingTop="0">

<mx:Script>
<![CDATA[

import mx.events.FlexEvent;

public function displayText(text:String):void {
textWidget.text = text;
}

private function init():void {
addEventListener(FlexEvent.APPLICATION_COMPLETE, onApplicationComplete);
addJSCallback("displayText", displayText);
addJSCallback("addSendListener", addSendListener);
}

private function onApplicationComplete(event:FlexEvent):void {
callLater(function():void {
ExternalInterface.call("_swf_application_complete", swfID);
});
}

public static function addJSCallback(jsFunctionName:String, flexFunction:Function):void {
try {
if (ExternalInterface.available) {
ExternalInterface.addCallback(jsFunctionName, flexFunction);
}
} catch (error:SecurityError) {
trace("Couldn't add javascript callback: " + error);
}
}

public function addSendListener(jsFunctionName:String):void {
sendButton.addEventListener(MouseEvent.CLICK, function(event:MouseEvent):void {
ExternalInterface.call(jsFunctionName, swfID, textWidget.text);
});
}

public function get swfID():String {
return Application.application.parameters.swfid;
}

]]>
</mx:Script>

<mx:Button id="sendButton" x="200" width="80" label="Send"/>
<mx:TextInput id="textWidget" x="0" width="200"/>

</mx:Application>
Now we can build the Flex project and copy the generated .swf file into the GWT project. It should be under the 'public' folder below the folder that contains the .gwt.xml file, e.g. in my case as 'src/de/larsgrammel/blog/flexgwt/public/Test.swf'.

The GWT wrapper keeps track of event handler that are interested in this send event. For that purpose, we add custom event and event handler classes:

TextSentEvent.java
import com.google.gwt.event.shared.GwtEvent;

public class TextSentEvent extends GwtEvent<TextSentEventHandler> {

public static final Type<TextSentEventHandler> TYPE = new Type<TextSentEventHandler>();

private SampleFlexWrapperWidget swfWidget;

private String newText;

public TextSentEvent(SampleFlexWrapperWidget swfWidget, String newText) {
assert swfWidget != null;
this.newText = newText;
this.swfWidget = swfWidget;
}

protected void dispatch(TextSentEventHandler handler) {
handler.onTextSent(this);
}

public String getNewText() {
return newText;
}

public Type<TextSentEventHandler> getAssociatedType() {
return TYPE;
}

public SampleFlexWrapperWidget getSWFWidget() {
return swfWidget;
}
}
TextSentEventHandler.java
public interface TextSentEventHandler extends EventHandler {

void onTextSent(TextSentEvent event);

}
Now, we add a method for registering TextSentEventHandlers to our SampleFlexWrapperWidget:

SampleFlexWrapperWidget.java
...

public HandlerRegistration addTextSentEventHandler(
TextSentEventHandler handler) {

return addHandler(handler, TextSentEvent.TYPE);
}
The next two methods forward the Flex events to the registered event handlers:

SampleFlexWrapperWidget.java
...

public static void _onTextSent(String swfId, String text) {
swfWidgets.get(swfId).onTextSent(text);
}

private void onTextSent(String text) {
fireEvent(new TextSentEvent(this, text));
}
Now we register the _onTextSent method as listener on the Flex component by exposing it as a JavaScript method. The GWT guide recommends that Java methods should be wrapped by $entry() when exposing them as JavaScript callback points. We thus register our wrapped _onTextSent method as JavaScript handle once the Flex component finished loading.

SampleFlexWrapperWidget.java
...

public static void onSwfApplicationComplete(String swfId) {
_registerSwfListeners(swfId);
...
}

private static native void _registerSwfListeners(String swfID) /*-{
var swfWidget = $doc.getElementById(swfID);
swfWidget.addSendListener("_swf_on_text_sent");
}-*/;

private static native void registerCallbackMethods() /*-{
$wnd._swf_on_text_sent=
$entry(@de.larsgrammel.blog.flexgwt.client.SampleFlexWrapperWidget::_onTextSent(Ljava/lang/String;Ljava/lang/String;));
...
}-*/;
Finally, we change our main code to use the new functionality of our wrapper by updating the GWT text field when the Flex component sents over new text:

FlexGWTIntegration.java
...
flexWidget.addTextSentEventHandler(new TextSentEventHandler() {
public void onTextSent(TextSentEvent event) {
textField.setText(event.getNewText());
}
});

The complete FlexGWTIntegration and SampleFlexWrapperWidget should look like this now:

SampleFlexWrapperWidget.java
import java.util.HashMap;
import java.util.Map;

import pl.rmalinowski.gwt2swf.client.ui.SWFWidget;

import com.google.gwt.event.shared.HandlerRegistration;

public class SampleFlexWrapperWidget extends SWFWidget {

private static Map<String, SampleFlexWrapperWidget> swfWidgets = new HashMap<String, SampleFlexWrapperWidget>();

static {
registerCallbackMethods();
}

private static native void _displayText(String swfID, String text) /*-{
$doc.getElementById(swfID).displayText(text);
}-*/;

public static void onSwfApplicationComplete(String swfId) {
_registerSwfListeners(swfId);
swfWidgets.get(swfId).fireSWFWidgetReady();
}

private static native void _registerSwfListeners(String swfID) /*-{
var swfWidget = $doc.getElementById(swfID);
swfWidget.addSendListener("_swf_on_text_sent");
}-*/;

private static native void registerCallbackMethods() /*-{
$wnd._swf_on_text_sent=
$entry(@de.larsgrammel.blog.flexgwt.client.SampleFlexWrapperWidget::_onTextSent(Ljava/lang/String;Ljava/lang/String;));
$wnd._swf_application_complete=
$entry(@de.larsgrammel.blog.flexgwt.client.SampleFlexWrapperWidget::onSwfApplicationComplete(Ljava/lang/String;));
}-*/;

public SampleFlexWrapperWidget(int width, int height) {
super("flexgwtintegration/Test.swf", width, height);
addFlashVar("swfid", getSwfId());
}

public HandlerRegistration addSWFWidgetReadyHandler(
SWFWidgetReadyHandler handler) {

return addHandler(handler, SWFWidgetReadyEvent.TYPE);
}

public HandlerRegistration addTextSentEventHandler(
TextSentEventHandler handler) {

return addHandler(handler, TextSentEvent.TYPE);
}

public static void _onTextSent(String swfId, String text) {
swfWidgets.get(swfId).onTextSent(text);
}

private void onTextSent(String text) {
fireEvent(new TextSentEvent(this, text));
}

public void displayText(String text) {
_displayText(getSwfId(), text);
}

private void fireSWFWidgetReady() {
fireEvent(new SWFWidgetReadyEvent(this));
}

protected void onLoad() {
super.onLoad();
SampleFlexWrapperWidget.swfWidgets.put(getSwfId(), this);
}

protected void onUnload() {
SampleFlexWrapperWidget.swfWidgets.remove(getSwfId());
super.onUnload();
}
}

FlexGWTIntegration.java
import com.google.gwt.core.client.EntryPoint;
import com.google.gwt.event.dom.client.ClickEvent;
import com.google.gwt.event.dom.client.ClickHandler;
import com.google.gwt.event.dom.client.KeyCodes;
import com.google.gwt.event.dom.client.KeyUpEvent;
import com.google.gwt.event.dom.client.KeyUpHandler;
import com.google.gwt.user.client.ui.Button;
import com.google.gwt.user.client.ui.RootPanel;
import com.google.gwt.user.client.ui.TextBox;

public class FlexGWTIntegration implements EntryPoint {

public void onModuleLoad() {
final Button sendButton = new Button("Send to Flex");
final TextBox textField = new TextBox();
textField.setText("from GWT");

final SampleFlexWrapperWidget flexWidget =
new SampleFlexWrapperWidget(300, 50);

RootPanel.get().add(textField);
RootPanel.get().add(sendButton);

RootPanel.get().add(flexWidget);

textField.setFocus(true);
textField.selectAll();

class MyHandler implements ClickHandler, KeyUpHandler {
public void onClick(ClickEvent event) {
displayTextInFlex();
}

public void onKeyUp(KeyUpEvent event) {
if (event.getNativeKeyCode() == KeyCodes.KEY_ENTER) {
displayTextInFlex();
}
}

private void displayTextInFlex() {
flexWidget.displayText(textField.getText());
}
}

MyHandler handler = new MyHandler();
sendButton.addClickHandler(handler);
textField.addKeyUpHandler(handler);

flexWidget.addTextSentEventHandler(new TextSentEventHandler() {
public void onTextSent(TextSentEvent event) {
textField.setText(event.getNewText());
}
});
}
}
This example code allows you to send text from GWT to Flex and from Flex to GWT (demo). Together with the first two parts of this tutorial series, this should enable you to use a Flex component from GWT in a way that resembles the GWT API. There are a couple more details that I will cover in future posts.

02 March 2010

Mashup Environments in Software Engineering

Software developers perform different kinds of analytical activities. For example, they want to find out which code might be affected by a change, which change caused a bug or build failure, or which source code was changing in work items related to performance issues. Similarly, project managers might want to learn from the latest iteration of product development by analyzing produced artifacts, e.g. work items, source code and build results. There are many tasks in software engineering that would benefit from tools that enable the flexible and integrated analysis of information stored in different places such as issue trackers, source code repositories, and requirements documents.

Together with Christoph Treude and Margaret-Anne Storey, I outlined the idea how mashup technology can be leveraged to achieve this. Our 2 page position paper "Mashup Environments in Software Engineering" was accepted at Web2SE, the First Workshop on Web 2.0 for Software Engineering, co-located with ICSE 2010. Here is the abstract of our paper:

Too often, software engineering (SE) tool research is focused on
creating small, stand-alone tools that address rarely understood
developer needs. We believe that research should instead provide
developers with flexible environments and interoperable tools,
and then study how developers appropriate and tailor these tools
in practice. Although there has been some prior work on this, we
feel that flexible tool environments for SE have not yet been fully
explored. In particular, we propose adopting the Web 2.0 idea of
mashups and mashup environments to support SE practitioners in
analytic activities involving multiple information sources.

Download Paper

19 February 2010

Notifying GWT when a Flex widget is initialized

Flex .swf files can take a while to load. When using Flex components from within GWT, this means that the GWT user interface tends to be ready while the Flex widgets are still loading. This can cause problems if we call methods on those Flex widgets before they are completely initialized. This tutorial shows how to notify GWT when a Flex widget is ready, and how to prevent the problem of calling an uninitialized Flex widget. It is the second part of a series of blog posts about using Flex components in GWT with the library gwt2swf:

1. GWT Wrapper for Flex components
2. Notifying GWT when a Flex widget is loaded [Demo, Source Code]
3. Handling Flex events in GWT [Demo, Source Code]

This tutorial is based on the previous one, creating a GWT Wrapper for Flex components, and assumes that you have a similar project setup and that you have created the classes and files with the source code from that tutorial.

The general idea is to listen for the application complete event in Flex, to forward this event to the GWT wrapper and to notify listeners on the GWT side. For this to work, the Flex widget must include it's swfID when forwarding the event to GWT, because we need to know which Flex widget should get notified on the GWT side if we have multiple Flex widgets. For some reason, the Flex application id does not seem to work for that purpose on all browsers, so we pass it in as a Flash var:

1. Add addFlashVar call for passing the swfid into the Flex component to the SampleFlexWrapperWidget constructor:

addFlashVar("swfid", getSwfId());

The whole constructor should look like this:

public SampleFlexWrapperWidget(int width, int height) {
super("flexgwtintegration/Test.swf", width, height);
addFlashVar("swfid", getSwfId());
}

2. In the Flex component, add an accessor to that property:

public function get swfID():String {
return Application.application.parameters.swfid;
}

Now we can use the swf id within Flex. The next step is to listen for the application complete event in Flex and to forward it to GWT.

3. Add the event listener for the application complete event in the init() method of our Flex widget. We use the application complete event, because we want to make sure that the widget is visible and ready (More information on the Flex Application startup event order).

addEventListener(FlexEvent.APPLICATION_COMPLETE, onApplicationComplete);

The whole init method should look like this:

private function init():void {
addEventListener(FlexEvent.APPLICATION_COMPLETE, onApplicationComplete);
addJSCallback("displayText", displayText);
}

4. Add the event listener method to the Flex widget. We use callLater here to make sure the widget is ready when the event is fired. The method calls the method with the javascript identifier _swf_application_complete, which we will implement soon.

import mx.events.FlexEvent;

private function onApplicationComplete(event:FlexEvent):void {
callLater(function():void {
ExternalInterface.call("_swf_application_complete", swfID);
});
}

The complete Flex widget should look like this by now:

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml"
layout="absolute" creationComplete="init()"
backgroundColor="0xffffff" paddingBottom="0" paddingLeft="0"
paddingRight="0" paddingTop="0">

<mx:Script>
<![CDATA[

import mx.events.FlexEvent;

public function displayText(text:String):void {
textWidget.text = text;
}

private function init():void {
addEventListener(FlexEvent.APPLICATION_COMPLETE, onApplicationComplete);
addJSCallback("displayText", displayText);
}

private function onApplicationComplete(event:FlexEvent):void {
callLater(function():void {
ExternalInterface.call("_swf_application_complete", swfID);
});
}

public static function addJSCallback(jsFunctionName:String, flexFunction:Function):void {
try {
if (ExternalInterface.available) {
ExternalInterface.addCallback(jsFunctionName, flexFunction);
}
} catch (error:SecurityError) {
trace("Couldn't add javascript callback: " + error);
}
}

public function get swfID():String {
return Application.application.parameters.swfid;
}

]]>
</mx:Script>
<mx:Text id="textWidget" width="100%"/>
</mx:Application>

5. Build the Flex project and copy the generated .swf file into the GWT project. It should be under the 'public' folder below the folder that contains the .gwt.xml file, e.g. in my case as 'src/de/larsgrammel/blog/flexgwt/public/Test.swf'.

That's it on the Flex side. On the GWT side, we need to keep track of the different widgets and their swf id's so we can forward the event to the right widget.

6. Create a static map that enables us to find the GWT wrapper for a given swf id:

private static Map<String, SampleFlexWrapperWidget> swfWidgets = new HashMap<String, SampleFlexWrapperWidget>();

7. Register the widget in the map when loaded and deregister it when unloaded:

@Override
protected void onLoad() {
super.onLoad();
SampleFlexWrapperWidget.swfWidgets.put(getSwfId(), this);
}

@Override
protected void onUnload() {
SampleFlexWrapperWidget.swfWidgets.remove(getSwfId());
super.onUnload();
}

Now we can add the methods that are called from Flex.

8. Create a class initializer that calls a JSNI method to register the callback methods. That way, the first time one of our wrapper widgets gets created (i.e. the class is loaded), our callback method is registered.

static {
registerCallbackMethods();
}

9. Create the JSNI method that links a Java method to the callback point '_swf_application_complete':

private static native void registerCallbackMethods() /*-{
$wnd._swf_application_complete=
@de.larsgrammel.blog.flexgwt.client.SampleFlexWrapperWidget::onSwfApplicationComplete(Ljava/lang/String;);
}-*/;

10. Add the method that routes the event to the correct wrapper and calls fireSWFWidgetReady on that wrapper:

public static void onSwfApplicationComplete(String swfId) {
swfWidgets.get(swfId).fireSWFWidgetReady();
}

We need a special event and handler on the GWT. So lets create those and the related methods in the wrapper.

11. Create SWFWidgetReadyEvent and SWFWidgetReadyHandler

package de.larsgrammel.blog.flexgwt.client;

import com.google.gwt.event.shared.GwtEvent;

public class SWFWidgetReadyEvent extends GwtEvent<SWFWidgetReadyHandler> {

public static final Type<SWFWidgetReadyHandler> TYPE = new Type<SWFWidgetReadyHandler>();

private SampleFlexWrapperWidget swfWidget;

public SWFWidgetReadyEvent(SampleFlexWrapperWidget swfWidget) {
assert swfWidget != null;
this.swfWidget = swfWidget;
}

@Override
protected void dispatch(SWFWidgetReadyHandler handler) {
handler.onSWFWidgetReady(this);
}

@Override
public Type<SWFWidgetReadyHandler> getAssociatedType() {
return TYPE;
}

public SampleFlexWrapperWidget getSWFWidget() {
return swfWidget;
}

}


package de.larsgrammel.blog.flexgwt.client;

import com.google.gwt.event.shared.EventHandler;

public interface SWFWidgetReadyHandler extends EventHandler {

void onSWFWidgetReady(SWFWidgetReadyEvent event);

}

12. Add a method to register a SWFWidgetReadyHandler in the wrapper class - make sure to return the HandlerRegistration so the
listener can be remove if needed:

public HandlerRegistration addSWFWidgetReadyHandler(
SWFWidgetReadyHandler handler) {

return addHandler(handler, SWFWidgetReadyEvent.TYPE);
}

13. Implement the fireSWFWidgetReady method:

private void fireSWFWidgetReady() {
fireEvent(new SWFWidgetReadyEvent(this));
}

The whole SampleFlexWrapperWidget should look similar to this by now:

package de.larsgrammel.blog.flexgwt.client;

import java.util.HashMap;
import java.util.Map;

import pl.rmalinowski.gwt2swf.client.ui.SWFWidget;

import com.google.gwt.event.shared.HandlerRegistration;

public class SampleFlexWrapperWidget extends SWFWidget {

private static Map<String, SampleFlexWrapperWidget> swfWidgets = new HashMap<String, SampleFlexWrapperWidget>();

static {
registerCallbackMethods();
}

private static native void _displayText(String swfID, String text) /*-{
$doc.getElementById(swfID).displayText(text);
}-*/;

public static void onSwfApplicationComplete(String swfId) {
swfWidgets.get(swfId).fireSWFWidgetReady();
}

private static native void registerCallbackMethods() /*-{
$wnd._swf_application_complete=
@de.larsgrammel.blog.flexgwt.client.SampleFlexWrapperWidget::onSwfApplicationComplete(Ljava/lang/String;);
}-*/;

public SampleFlexWrapperWidget(int width, int height) {
super("flexgwtintegration/Test.swf", width, height);
addFlashVar("swfid", getSwfId());
}

public HandlerRegistration addSWFWidgetReadyHandler(
SWFWidgetReadyHandler handler) {
return addHandler(handler, SWFWidgetReadyEvent.TYPE);
}

public void displayText(String text) {
_displayText(getSwfId(), text);
}

private void fireSWFWidgetReady() {
fireEvent(new SWFWidgetReadyEvent(this));
}

@Override
protected void onLoad() {
super.onLoad();
SampleFlexWrapperWidget.swfWidgets.put(getSwfId(), this);
}

@Override
protected void onUnload() {
SampleFlexWrapperWidget.swfWidgets.remove(getSwfId());
super.onUnload();
}

}

14. In the client code that uses the widget, we can now disable our control widgets by default and then enable them once the Flex widgets are ready:

nameField.setEnabled(false);
sendButton.setEnabled(false);

flexWidget.addSWFWidgetReadyHandler(new SWFWidgetReadyHandler() {
@Override
public void onSWFWidgetReady(SWFWidgetReadyEvent event) {
nameField.setEnabled(true);
sendButton.setEnabled(true);
}
});

I refactored the client code a bit. It now adds two similar Flex wrappers including controls by calling a addFlexWidgetAndGWTControls method. I also removed the automatic text focus - you would need to decide for one of those controls to get the focus for this to work properly. The client code looks like this:

package de.larsgrammel.blog.flexgwt.client;

import com.google.gwt.core.client.EntryPoint;
import com.google.gwt.event.dom.client.ClickEvent;
import com.google.gwt.event.dom.client.ClickHandler;
import com.google.gwt.event.dom.client.KeyCodes;
import com.google.gwt.event.dom.client.KeyUpEvent;
import com.google.gwt.event.dom.client.KeyUpHandler;
import com.google.gwt.user.client.ui.Button;
import com.google.gwt.user.client.ui.RootPanel;
import com.google.gwt.user.client.ui.TextBox;

public class FlexGWTIntegration implements EntryPoint {

public void onModuleLoad() {
addFlexWidgetAndGWTControls();
addFlexWidgetAndGWTControls();
}

private void addFlexWidgetAndGWTControls() {
final Button sendButton = new Button("Send");
final TextBox nameField = new TextBox();
nameField.setText("GWT User");

nameField.setEnabled(false);
sendButton.setEnabled(false);

final SampleFlexWrapperWidget flexWidget = new SampleFlexWrapperWidget(
100, 50);

RootPanel.get().add(nameField);
RootPanel.get().add(sendButton);
RootPanel.get().add(flexWidget);

class MyHandler implements ClickHandler, KeyUpHandler {
public void onClick(ClickEvent event) {
displayTextInFlex();
}

public void onKeyUp(KeyUpEvent event) {
if (event.getNativeKeyCode() == KeyCodes.KEY_ENTER) {
displayTextInFlex();
}
}

private void displayTextInFlex() {
flexWidget.displayText(nameField.getText());
}
}

// Add a handler to send the name to the server
MyHandler handler = new MyHandler();
sendButton.addClickHandler(handler);
nameField.addKeyUpHandler(handler);

flexWidget.addSWFWidgetReadyHandler(new SWFWidgetReadyHandler() {
@Override
public void onSWFWidgetReady(SWFWidgetReadyEvent event) {
nameField.setEnabled(true);
sendButton.setEnabled(true);
}
});
}
}

With this approach, your GWT controls should be disabled until the Flex widgets are completely loaded - and it should work with multiple Flex widgets. More on integrating Flex and GWT will be covered in future posts, so stay tuned.

09 February 2010

GWT Wrapper for Flex components

I am working on a project that integrates a Flex-based graph component into a GWT application. Using Flash / Flex and GWT in combination is tricky when you want to call both Flash/Flex from GWT and GWT from Flash/Flex, because there are two bridges involved: the GWT Java Script Native Interface (JSNI) and the Flex external interface.

This tutorial is the first part of a series of blog posts about using Flex components in GWT with the library gwt2swf:

1. GWT Wrapper for Flex components
2. Notifying GWT when a Flex widget is loaded [Demo, Source Code]
3. Handling Flex events in GWT [Demo, Source Code]

I will explain how to create a GWT adapter around a Flex component that enables calling Flex from GWT. The code will use the library gwt2swf. This post will be pretty strait forward and matches the gwt2swf docs for most part, but will include sample GWT and Flex code. Essentially, I will create a subclass of SWFWidget (part of gwt2swf) that wraps all the Flex specifics and can be used from GWT. This approach enables the use of several swf widgets from GWT at the same time.

To create a sample Flex component, I used the following steps:

1. Create a new Flex Project (e.g. in Flex Builder)

2. Add a creationComplete handler (here: init) to the mxml header

<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml"
layout="absolute"
creationComplete="init()">


3. Remove the padding from the Flex component & set the background color in the mxml header

<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml"
layout="absolute" creationComplete="init()"
backgroundColor="0xffffff"
paddingBottom="0" paddingLeft="0"
paddingRight="0" paddingTop="0">


4. Add the init method and a addJsCallback method with code that I shamelessly stole from my colleague Chris Callendar, who helped me getting this to work and who has an awesome Flex blog, http://flexdevtips.blogspot.com/.

public static function addJSCallback(jsFunctionName:String, flexFunction:Function):void {
try {
if (ExternalInterface.available) {
ExternalInterface.addCallback(jsFunctionName, flexFunction);
}
} catch (error:SecurityError) {
trace("Couldn't add javascript callback: " + error);
}
}


5. Add a text field to the flex component as an example element. We will change the contents of the text field from GWT. The width is set to 100%, so that the widget resizes based on the surrounding HTML element. For other widgets, e.g. a table, you might want to set the height to 100% as well.

<mx:Text id="textWidget" width="100%"/>


6. Add a displayText function and register it as a callback for JavaScript access under the name "displayText" in the init() method.

public function displayText(text:String):void {
textWidget.text = text;
}

private function init():void {
addJSCallback("displayText", displayText);
}


7. The complete .mxml file should look similar to this by now:

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml"
layout="absolute" creationComplete="init()"
backgroundColor="0xffffff" paddingBottom="0" paddingLeft="0"
paddingRight="0" paddingTop="0">
<mx:Script>
<![CDATA[
public function displayText(text:String):void {
textWidget.text = text;
}

private function init():void {
addJSCallback("displayText", displayText);
}

public static function addJSCallback(jsFunctionName:String, flexFunction:Function):void {
try {
if (ExternalInterface.available) {
ExternalInterface.addCallback(jsFunctionName, flexFunction);
}
} catch (error:SecurityError) {
trace("Couldn't add javascript callback: " + error);
}
}
]]>
</mx:Script>
<mx:Text id="textWidget" width="100%"/>
</mx:Application>


8. Build the Flex project and copy the generated .swf file into the GWT project. It should be under the 'public' folder below the folder that contains the .gwt.xml file, e.g. in my case as 'src/de/larsgrammel/blog/flexgwt/public/Test.swf'

That's it on the Flex side. Here is the list of steps I followed on the GWT side:

1. Download gwt2swf from http://sourceforge.net/projects/gwt2swf/ (I use version 0.6.0 in the example presented here)

2. Extract the gwt2swf.jar and add it project's buildpath

3. Add gwt2swf to your project's .gwt.xml file:

<inherits name='pl.rmalinowski.gwt2swf.GWT2SWF' />


4. Create a subclass of SWFWidget (here: SampleFlexWrapperWidget) that sets swf file by specifying the swf source path in the constructor call. The first part of the swf source path is the module name, the rest the path below the 'public' directory including the .swf name.

public SampleFlexWrapperWidget(int width, int height) {
super("flexgwtintegration/Test.swf", width, height);
}


5. Add a displayText method to that class as a public interface on the Java side. This method calls a private static native method that resolves the SWF DOM element and calls the exposed Flex method:

private static native void _displayText(String swfID, String text) /*-{
$doc.getElementById(swfID).displayText(text);
}-*/;

public void displayText(String text) {
_displayText(getSwfId(), text);
}


6. The wrapper should look similar to this by now:

public class SampleFlexWrapperWidget extends SWFWidget {

private static native void _displayText(String swfID, String text) /*-{
$doc.getElementById(swfID).displayText(text);
}-*/;

public SampleFlexWrapperWidget(int width, int height) {
super("flexgwtintegration/Test.swf", width, height);
}

public void displayText(String text) {
_displayText(getSwfId(), text);
}

}


7. The wrapper can now be used in GWT, e.g. in the entry point class:

public class FlexGWTIntegration implements EntryPoint {

public void onModuleLoad() {
final Button sendButton = new Button("Send");
final TextBox nameField = new TextBox();
nameField.setText("GWT User");

final SampleFlexWrapperWidget flexWidget1 = new SampleFlexWrapperWidget(100, 50);
final SampleFlexWrapperWidget flexWidget2 = new SampleFlexWrapperWidget(100, 50);

RootPanel.get().add(nameField);
RootPanel.get().add(sendButton);

RootPanel.get().add(flexWidget1);
RootPanel.get().add(flexWidget2);

nameField.setFocus(true);
nameField.selectAll();

class MyHandler implements ClickHandler, KeyUpHandler {
public void onClick(ClickEvent event) {
displayTextInFlex();
}

public void onKeyUp(KeyUpEvent event) {
if (event.getNativeKeyCode() == KeyCodes.KEY_ENTER) {
displayTextInFlex();
}
}

private void displayTextInFlex() {
flexWidget1.displayText(nameField.getText());
flexWidget2.displayText(nameField.getText() + " 2nd version");
}
}

MyHandler handler = new MyHandler();
sendButton.addClickHandler(handler);
nameField.addKeyUpHandler(handler);
}
}


Using this approach, you can use the SWF wrapper inside GWT like any other GWT widget, except for a small loading delay. I will explain how to deal with delayed loading, events from Flex and others integration issues in future blog posts - stay tuned.