22 July 2010

How Information Visualization Novices Construct Visualizations

Visualization for the masses is a topic that has gained a lot of attraction in the InfoVis community in recent years, e.g. in projects such as IBM ManyEyes. The goal is to enable a wide user population to leverage information visualization technology to understand large amounts of data. This could potentially help them make more informed decisions, and is especially promising as more and more data becomes available (see open data). However, there are still many challenges that need to be addressed so that visualization for the masses can become a reality, ranging from limited visual literacy to insufficient tool support.

Together with Melanie Tory and Margaret-Anne Storey, I investigated how information visualization novices construct visualizations in a laboratory setting. Our research paper "How Information Visualization Novices Construct Visualizations" was accepted for presentation at IEEE InfoVis 2010.

Here is the abstract of our paper:

It remains challenging for information visualization novices to rapidly construct visualizations during exploratory data analysis. We conducted an exploratory laboratory study in which information visualization novices explored fictitious sales data by communicating visualization specifications to a human mediator, who rapidly constructed the visualizations using commercial visualization software.

We found that three activities were central to the iterative visualization construction process: data attribute selection, visual template selection, and visual mapping specification. The major barriers faced by the participants were translating questions into data attributes, designing visual mappings, and interpreting the visualizations. Partial specification was common, and the participants used simple heuristics and preferred visualizations they were already familiar with, such as bar, line and pie charts.

From our observations, we derived abstract models that describe barriers in the data exploration process and uncovered how information visualization novices think about visualization specifications. Our findings support the need for tools that suggest potential visualizations and support iterative refinement, that provide explanations and help with learning, and that are tightly integrated into tool support for the overall visual analytics process.

Download Technical Report

19 April 2010

Choosel Mashup Framework Available on Google Code

I released the choosel mashup framework under the Apache 2.0 license on Google Code. It is a research prototype platform that is used in our Bio-Mixer tool, which is developed at the CHISEL group.

Choosel supports the creation of web-based information mashup environments. These mashup environments facilitate the flexible recombination of information in different views such as maps, timelines and graph viewers. Users without any programming expertise can remix information using drag and drop interaction and explore data sets. The workspaces (mashups) can be stored and shared among users.

We are looking for contributors. If you are interested, please post on the choosel mailing list.

14 April 2010

Supporting End Users in Coordinating Multiple Visualizations

As part of my PhD research, I am looking at ways to make visual data analysis more accessible to end users without data analysis expertise. Specifically, I am researching how they can easily coordinate multiple visualizations. This has led to the development of web-based visual analytics research prototype, which I evaluated in a user study. The results indicate that novel concepts such as drop target highlighting, drop previews and using multiple user defined sets are useful and easily usable for end users.

I presented the tool and the results (see poster and presentation below) at the IBM University Days 2010. A version of the visual analytics environment that is tailored to exploring biomedical ontologies is available at: bio-mixer.appspot.com


09 April 2010

Bio-Mixer Early Research Prototype Released

I am happy to announce the early alpha release of Bio-Mixer, a research prototype developed at the CHISEL group at the University of Victoria in collaboration with the National Center of Biomedical Ontology. Bio-Mixer is available at:

bio-mixer.appspot.com

Bio-Mixer is a web-based environment that supports the flexible exploration of biomedical ontologies. The concepts in the ontologies and their mappings can be explored in different views such as graph views, lists and timeline views. Drag-and-drop interaction can be used to show items and collections in different views, to create filtered views and to synchronize selections. Bio-Mixer enhances drag and drop with a new drop target highlighting and preview approach to make working with multiple collections and views easy. Bio-Mixer also provides support for ontology annotation and workspace sharing between collaborators.

Bio-Mixer Overview



Bio-Mixer Features in Detail

18 March 2010

Handling Flex Events in GWT

Tightly integrating Flex components in GWT applications requires that GWT is notified on events triggered in Flex. For example, when the user performs an action in the Flex component, e.g. double-clicking a node in a graph widget, we might want to perform some operations on the GWT side, e.g. updating a dependent view to show information about the double-clicked node. This tutorial shows how UI event from Flex can be forward to and handled by GWT code. The example contains a GWT text box and a Flex text box and buttons for copying the text from one to the other. This tutorial is the third part of a series of blog posts about using Flex components in GWT with the library gwt2swf:

1. GWT Wrapper for Flex components
2. Notifying GWT when a Flex widget is loaded [Demo, Source Code]
3. Handling Flex events in GWT [Demo, Source Code]

This tutorial is based on the previous one, notifying GWT when a Flex widget is loaded, and assumes that you have a similar project setup and that you have created the classes and files with the source code from that tutorial. The main ideas are similar those from the previous tutorial, but we will explicitly register our event handler bridge as a callback in Flex. That way, the Flex component can expose operations that might not be used by our GWT application, but in other applications, e.g. separate JavaScript projects, without requiring the GWT wrapper to provide hooks for those operations.

The Flex component is first extended by adding a 'send' button and changing the text to a text input. Then a method for adding a JavaScript method as event listener is implemented and exposed. When calling registered event listeners, the Flex components adds the content of the text field, so we can use it on the GWT side. Alternatively, one could implement and expose a method for getting the content of the Flex text field from JavaScript.

Text.mxml
<mx:Button id="sendButton" x="200" width="80" label="Send"/>
<mx:TextInput id="textWidget" x="0" width="200"/>

...

public function addSendListener(jsFunctionName:String):void {
sendButton.addEventListener(MouseEvent.CLICK, function(event:MouseEvent):void {
ExternalInterface.call(jsFunctionName, swfID, textWidget.text);
});
}

private function init():void {
...
addJSCallback("addSendListener", addSendListener);
}
The complete Flex component should look like this:

Text.mxml
<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml"
layout="absolute" creationComplete="init()"
backgroundColor="0xffffff" paddingBottom="0" paddingLeft="0"
paddingRight="0" paddingTop="0">

<mx:Script>
<![CDATA[

import mx.events.FlexEvent;

public function displayText(text:String):void {
textWidget.text = text;
}

private function init():void {
addEventListener(FlexEvent.APPLICATION_COMPLETE, onApplicationComplete);
addJSCallback("displayText", displayText);
addJSCallback("addSendListener", addSendListener);
}

private function onApplicationComplete(event:FlexEvent):void {
callLater(function():void {
ExternalInterface.call("_swf_application_complete", swfID);
});
}

public static function addJSCallback(jsFunctionName:String, flexFunction:Function):void {
try {
if (ExternalInterface.available) {
ExternalInterface.addCallback(jsFunctionName, flexFunction);
}
} catch (error:SecurityError) {
trace("Couldn't add javascript callback: " + error);
}
}

public function addSendListener(jsFunctionName:String):void {
sendButton.addEventListener(MouseEvent.CLICK, function(event:MouseEvent):void {
ExternalInterface.call(jsFunctionName, swfID, textWidget.text);
});
}

public function get swfID():String {
return Application.application.parameters.swfid;
}

]]>
</mx:Script>

<mx:Button id="sendButton" x="200" width="80" label="Send"/>
<mx:TextInput id="textWidget" x="0" width="200"/>

</mx:Application>
Now we can build the Flex project and copy the generated .swf file into the GWT project. It should be under the 'public' folder below the folder that contains the .gwt.xml file, e.g. in my case as 'src/de/larsgrammel/blog/flexgwt/public/Test.swf'.

The GWT wrapper keeps track of event handler that are interested in this send event. For that purpose, we add custom event and event handler classes:

TextSentEvent.java
import com.google.gwt.event.shared.GwtEvent;

public class TextSentEvent extends GwtEvent<TextSentEventHandler> {

public static final Type<TextSentEventHandler> TYPE = new Type<TextSentEventHandler>();

private SampleFlexWrapperWidget swfWidget;

private String newText;

public TextSentEvent(SampleFlexWrapperWidget swfWidget, String newText) {
assert swfWidget != null;
this.newText = newText;
this.swfWidget = swfWidget;
}

protected void dispatch(TextSentEventHandler handler) {
handler.onTextSent(this);
}

public String getNewText() {
return newText;
}

public Type<TextSentEventHandler> getAssociatedType() {
return TYPE;
}

public SampleFlexWrapperWidget getSWFWidget() {
return swfWidget;
}
}
TextSentEventHandler.java
public interface TextSentEventHandler extends EventHandler {

void onTextSent(TextSentEvent event);

}
Now, we add a method for registering TextSentEventHandlers to our SampleFlexWrapperWidget:

SampleFlexWrapperWidget.java
...

public HandlerRegistration addTextSentEventHandler(
TextSentEventHandler handler) {

return addHandler(handler, TextSentEvent.TYPE);
}
The next two methods forward the Flex events to the registered event handlers:

SampleFlexWrapperWidget.java
...

public static void _onTextSent(String swfId, String text) {
swfWidgets.get(swfId).onTextSent(text);
}

private void onTextSent(String text) {
fireEvent(new TextSentEvent(this, text));
}
Now we register the _onTextSent method as listener on the Flex component by exposing it as a JavaScript method. The GWT guide recommends that Java methods should be wrapped by $entry() when exposing them as JavaScript callback points. We thus register our wrapped _onTextSent method as JavaScript handle once the Flex component finished loading.

SampleFlexWrapperWidget.java
...

public static void onSwfApplicationComplete(String swfId) {
_registerSwfListeners(swfId);
...
}

private static native void _registerSwfListeners(String swfID) /*-{
var swfWidget = $doc.getElementById(swfID);
swfWidget.addSendListener("_swf_on_text_sent");
}-*/;

private static native void registerCallbackMethods() /*-{
$wnd._swf_on_text_sent=
$entry(@de.larsgrammel.blog.flexgwt.client.SampleFlexWrapperWidget::_onTextSent(Ljava/lang/String;Ljava/lang/String;));
...
}-*/;
Finally, we change our main code to use the new functionality of our wrapper by updating the GWT text field when the Flex component sents over new text:

FlexGWTIntegration.java
...
flexWidget.addTextSentEventHandler(new TextSentEventHandler() {
public void onTextSent(TextSentEvent event) {
textField.setText(event.getNewText());
}
});

The complete FlexGWTIntegration and SampleFlexWrapperWidget should look like this now:

SampleFlexWrapperWidget.java
import java.util.HashMap;
import java.util.Map;

import pl.rmalinowski.gwt2swf.client.ui.SWFWidget;

import com.google.gwt.event.shared.HandlerRegistration;

public class SampleFlexWrapperWidget extends SWFWidget {

private static Map<String, SampleFlexWrapperWidget> swfWidgets = new HashMap<String, SampleFlexWrapperWidget>();

static {
registerCallbackMethods();
}

private static native void _displayText(String swfID, String text) /*-{
$doc.getElementById(swfID).displayText(text);
}-*/;

public static void onSwfApplicationComplete(String swfId) {
_registerSwfListeners(swfId);
swfWidgets.get(swfId).fireSWFWidgetReady();
}

private static native void _registerSwfListeners(String swfID) /*-{
var swfWidget = $doc.getElementById(swfID);
swfWidget.addSendListener("_swf_on_text_sent");
}-*/;

private static native void registerCallbackMethods() /*-{
$wnd._swf_on_text_sent=
$entry(@de.larsgrammel.blog.flexgwt.client.SampleFlexWrapperWidget::_onTextSent(Ljava/lang/String;Ljava/lang/String;));
$wnd._swf_application_complete=
$entry(@de.larsgrammel.blog.flexgwt.client.SampleFlexWrapperWidget::onSwfApplicationComplete(Ljava/lang/String;));
}-*/;

public SampleFlexWrapperWidget(int width, int height) {
super("flexgwtintegration/Test.swf", width, height);
addFlashVar("swfid", getSwfId());
}

public HandlerRegistration addSWFWidgetReadyHandler(
SWFWidgetReadyHandler handler) {

return addHandler(handler, SWFWidgetReadyEvent.TYPE);
}

public HandlerRegistration addTextSentEventHandler(
TextSentEventHandler handler) {

return addHandler(handler, TextSentEvent.TYPE);
}

public static void _onTextSent(String swfId, String text) {
swfWidgets.get(swfId).onTextSent(text);
}

private void onTextSent(String text) {
fireEvent(new TextSentEvent(this, text));
}

public void displayText(String text) {
_displayText(getSwfId(), text);
}

private void fireSWFWidgetReady() {
fireEvent(new SWFWidgetReadyEvent(this));
}

protected void onLoad() {
super.onLoad();
SampleFlexWrapperWidget.swfWidgets.put(getSwfId(), this);
}

protected void onUnload() {
SampleFlexWrapperWidget.swfWidgets.remove(getSwfId());
super.onUnload();
}
}

FlexGWTIntegration.java
import com.google.gwt.core.client.EntryPoint;
import com.google.gwt.event.dom.client.ClickEvent;
import com.google.gwt.event.dom.client.ClickHandler;
import com.google.gwt.event.dom.client.KeyCodes;
import com.google.gwt.event.dom.client.KeyUpEvent;
import com.google.gwt.event.dom.client.KeyUpHandler;
import com.google.gwt.user.client.ui.Button;
import com.google.gwt.user.client.ui.RootPanel;
import com.google.gwt.user.client.ui.TextBox;

public class FlexGWTIntegration implements EntryPoint {

public void onModuleLoad() {
final Button sendButton = new Button("Send to Flex");
final TextBox textField = new TextBox();
textField.setText("from GWT");

final SampleFlexWrapperWidget flexWidget =
new SampleFlexWrapperWidget(300, 50);

RootPanel.get().add(textField);
RootPanel.get().add(sendButton);

RootPanel.get().add(flexWidget);

textField.setFocus(true);
textField.selectAll();

class MyHandler implements ClickHandler, KeyUpHandler {
public void onClick(ClickEvent event) {
displayTextInFlex();
}

public void onKeyUp(KeyUpEvent event) {
if (event.getNativeKeyCode() == KeyCodes.KEY_ENTER) {
displayTextInFlex();
}
}

private void displayTextInFlex() {
flexWidget.displayText(textField.getText());
}
}

MyHandler handler = new MyHandler();
sendButton.addClickHandler(handler);
textField.addKeyUpHandler(handler);

flexWidget.addTextSentEventHandler(new TextSentEventHandler() {
public void onTextSent(TextSentEvent event) {
textField.setText(event.getNewText());
}
});
}
}
This example code allows you to send text from GWT to Flex and from Flex to GWT (demo). Together with the first two parts of this tutorial series, this should enable you to use a Flex component from GWT in a way that resembles the GWT API. There are a couple more details that I will cover in future posts.