Stopping development on Nomad PIM

I will stop contributing to Nomad PIM. It took me a long time to make this decision, but I am confident that it is the right decision for the following reasons:

First and most importantly, over the last two years I did not have much time to work on Nomad PIM, and I don't think that this will change anytime in the future.

Second, a strong trend towards cloud computing and keeping private data on the web has emerged over the past few years. Furthermore, the devices that are used to access that data, e.g. smart phones & desktop computers, are becoming more diverse. I believe that this trend is fundamental, because it is very convenient to be able to access and modify data, especially personal data such as notes and contacts, from different devices in different situations (e.g. on the bus) without the need for manual synchronization. Nomad PIM is a desktop application for notebooks and desktop computers that stores the data locally. It would need a complete rewrite to make it available as such a cloud-based application, and there would the major obstacle of providing a cloud-based service that is hard to accomplish for an open-source project.

Third, Eclipse technology has changed since I started developing Nomad PIM in early 2005. The first version of Nomad PIM was developed for Eclipse 3.0 - now Eclipse 3.6 is under development, and Nomad PIM is based on Eclipse 3.3 (the development version is based on Eclipse 3.5). The Eclipse Modeling community has created a very good set of technologies and tools in the meantime, especially EMF, and a major part of the framework underlying Nomad PIM would need to be changed to leverage those technologies - which is important to facilitate future development as an Eclipse application, and has been on my todo list for a long time. Unfortunately, this would require a major work investment and I never found myself having the time to do it. Furthermore, e4 is now on the horizon, which would require even further changes.

As I am the main contributor to the project, this will mean the development of Nomad PIM is discontinued. If anybody is interested in taking over as project lead, please let me know by email (lars.grammel@gmail.com). I plan on using Nomad PIM for at least the next two years, and I will still be providing fixes to critical bugs. I will also provide detailed guides how to migrate your data to other services or applications (for the services / applications I migrate my data to). Please be aware that because Nomad PIM stores your data as human-readable XML files, your data is not locked in and you can use freely available tools such as XSLT processors to convert it into formats that can be imported into other services.

It was a great experience developing Nomad PIM. I would like to thank all contributors, especially Frank Ganske and Philip Ritzkopf.

CASCON Workshop on "User Interfaces for Visual Analysis and Monitoring in Business Intelligence".

Are you interested in the future of user interfaces for business intelligence?

Tomorrow (Wednesday, November 4th 2009) there is a CASCON workshop on "User Interfaces for Visual Analysis and Monitoring in Business Intelligence". It is co-organized by Margaret-Anne Storey, Christoph Treude and myself. The workshop starts at 1pm in Markham Ballroom C. This is the agenda:

1:00 pm Introduction (Margaret-Anne Storey, UVic)
1:15 pm Charting and Visualization at Cognos Software (Stephan Jou, IBM)
1:45 pm Mapping the BI Visualization Landscape (Mike McAllister, SAP)
2:15 pm Visualization Construction by Non-Experts (Lars Grammel, UVic)
2:45 pm Visualizing higher-dimensional Pareto fronts for complex decision making (Derek Rayside, MIT)

3:15 pm Coffee break

3:30 pm Dashboards in IBM's Jazz: BI for Software Development (Christoph Treude, UVic)
4:00 pm Business Intelligence on Mobile Devices (Stephan Jou, IBM Cognos)
4:30 pm Wrap-up (Margaret-Anne Storey, UVic)

We hope to see you tomorrow!

Cross-domain data retrieval in client-side mashups

Browsers do not allow dynamic cross-domain data retrieval to prevent cross-site scripting attacks. However, client-based mashups need to access data sources that reside on different servers. There are several solutions to this problem:

• Proxies: requires running proxy on intermediate server, leads to slower responses and additional traffic


• Window.name transport: requires support by the accessed services, because window.name property has to be set


• JSON and dynamic <script> tags: requires that the accessed services expose JSON, which is e.g. not the case for RSS feeds (which use XML)


• Signed JavaScript: only available for Firefox


• W3C Cross-Origin Resource Sharing: needs support on client and accessed service side, only available for
  Firefox 3.5 and Internet Explorer 8


• Flash drop-in: requires Flash on the client and a crossdomain.xml file on the server with the accessed service that support the crossdomain access.

Because all the solutions except the proxy restrict either the browsers or the web services that can be used, I decided to go with the proxy solution. I downloaded the PHP proxy script from Abdul Qabiz Blog and modified it as suggested by the comments on his blog post. I also added a small section to make sure the default content type is XML:

if ($mimeType != "") {
  // set header from mime type
  header("Content-Type: ".$mimeType);
} else {
  // assume web service returns xml
  header("Content-Type: text/xml");
}

The proxy script works for me in a development environment now, but I do not recommend using it in a production environment, because important features such as restrictions of the client domain and forwarding the headers from the original service response are missing.

Google map in dojo FloatingPane

I created a small example that shows how to run Google Maps in a dojo FloatingPane. It can be found here. The code has been tested with Firefox 3.0 and Internet Explorer 8.0 .

The map uses the complete content canvas that is available in the FloatingPane:

<div id="map_container">
<div id="map" style="width:100%;height:100%;"></div>
</div>

In the JavaScript part, a FloatingPane is created for the map_container element, the map widget is created and the resize events from the FloatingPane are forwarded to the map widget using dojo.connect:

var floatingPane = createFloatingPane("map_container");
// ... (some checks etc)
var mapElement = dojo.byId("map");
var map = new GMap2(mapElement);
// ... (map configuration)
function resize() {
  map.checkResize();
}
dojo.connect(floatingPane,"resize", resize);

The example also has function that creates a dojox.layout.FloatingPane from a set of parameters. It contains a minor fix for some problems I had with the dragging of automatically created FloatingPane widgets in Firefox 3.0:

function createFloatingPane(
    divId, title, x, y, width, height) {

  var pane = new dojox.layout.FloatingPane({  
    'title': title,
    'id': divId + "_floater",
    'closeable': true,
    'resizable': true,
    'dockable': false
  }, divId);
 
  // quick fix for positioning, does not seem 
  // necessary in dojo source code test
  // (FloatingPane test), but was necessary with 
  // dojo binaries and Firefox 3.0.10
  pane.domNode.style.left = x + "px";
  pane.domNode.style.top = y + "px";
  pane.resize({ 'w': width, 'h': height });

  pane.startup(); 
 
  return pane;
}

Running Persevere on Amazon EC2

Today I looked into running Persevere on Amazon servers, namely using their Elastic Compute Cloud (EC2). Persevere is a schema-free DB with a JSON/REST interface. It also provides a web front end for easy access.

I found their Amazon Web Services (AWS) Management Console pretty usable - it makes it really easy to configure the running instances, the block storage and the elastic IPs. I ran an instance with the "Basic Fedora Core 8" Amazon Machine Image (AMI). For this purpose, I also created a security group 'test'. Using this Putty for EC2 guide, I was quickly able to connect to the server using SSH.

I created a 1GB elastic block storage (EBS) volume and connected it to the running EC2 instance. The EBS volumes are stored persistently, even if the EC2 instance is terminated (which means all data in the instance is lost, and it can happen due to hardware failures). Furthermore, snapshots can be taken easily using the AWS management console. To use it from the EC2 instance, the volume has to be formatted and mounted (from the SSH console):

mkfs -t ext3 /dev/sdf
mkdir /mnt/volume_1
mount /dev/sdf /mnt/volume_1

I downloaded Java & Persevere to the mounted volume using wget and unpacked them into the /mnt/volume_1/opt/java and /mnt/volume_1/opt/persevere folders (short guide for Java). For real-world usage, it would be better to create a customized AMI that contains & starts them, but I wanted to try things out quickly.

Persevere started up fine (using java -jar startup.jar), but port 8080 was initially blocked by the Amazon firewall. Using the AWS management console, I added an allowed connection for TCP port 8080 (both from and to) and source 0.0.0.0/0. That way, I could access the Persevere web interface (using the Public DNS of the runnning EC2 instance which is available in the management console and appending port 8080).

The next step was securing the access to Persevere (note: before starting with this, I created a Persevere user from the web UI). Persevere uses Jetty, so I tweaked a couple of settings in the Jetty configuration. First, I configured Jetty to use HTTP over SSL. Then I set up the user authentification. For this, I had to configured a user realm (HashUserRealm), which was pretty straightforward. I used plain passwords for testing purposes, but for more serious undertaking encrypted passwords or hash sums and a database storage are more appropriate. After creating the user realm, I modified the Persevere WEB-INF/web.xml to restrict the access to the web UI. I also switched off port 8080 and modified the firewall setting in the AWS management console accordingly. After those changes (and restarting), the Persevere web UI was running on port 8443 over SSH, and required me to log in. Interestingly, I had to use the same user name and password in the user realm and the internal Persevere user, and I got signed in the Persevere web UI automatically. Logging out did not work though.

I also tried out the elastic IP service, which assigns an IP to an EC2 instance. I lost the SSH connection to the EC2 instance after this, I believe it was because the IP address changed. After rebooting the EC2 instance and reconnecting to the new IP address, I had no problems (although I needed to mount the EBS volume and start Persevere again).

Overall, I was pretty impressed how easy it is to set up services on EC2. Also, an initial securing of Persevere turned out to be easy. While I liked the technical side of things pretty much, EC2 is too expensive for my use case (having a private server on the web where I can deploy some customized services and programs). The instance hour of an EC2 instance is 0.10 USD - if you run your server 24/7, this is 876 USD / year, not counting data traffic and storage. If you want to do your own calculation, take a look at the AWS calculator.

Using Dojo DataGrid in Google Gadget

I spent some time last week working with Google Gadgets and Dojo. I had some trouble getting the DataGridrid running in iGoogle at first, so here is a short snippet that shows an example dojo DataGrid running in a Google Gadget.

One mistake I made was calling the initialization methods without using dojo.addOnLoad by registering them just as callbacks for the google onload functionality. That way, the dojo classes were not loaded as expected, which resulted in a lot of weird errors. The solution is calling dojo.addOnLoad from within the callback:

gadgets.util.registerOnLoadHandler(function() {
  dojo.addOnLoad(someExampleInitializationFunction);
});

The complete code for the gadget (using dojo 1.3.1) is below:

<?xml version="1.0" encoding="UTF-8" ?>
<Module>
<ModulePrefs title="Dojo Grid Example" 
   height="400" scrolling="true" />

<Content type="html">
<![CDATA[

<style type="text/css">
    @import url("http://ajax.googleapis.com/
      ajax/libs/dojo/1.3.1/dojo/resources/dojo.css");
    @import url("http://ajax.googleapis.com/
      ajax/libs/dojo/1.3.1/dijit/themes/tundra/
      tundra.css");
    @import url("http://ajax.googleapis.com/
      ajax/libs/dojo/1.3.1/dojox/grid/resources/
      Grid.css");
    @import url("http://ajax.googleapis.com/
      ajax/libs/dojo/1.3.1/dojox/grid/resources/
      tundraGrid.css");
</style>

<script type="text/javascript"
    src="http://ajax.googleapis.com/ajax/
    libs/dojo/1.3.1/dojo/dojo.xd.js"></script>

<script type="text/javascript">

dojo.require("dojo.data.ItemFileReadStore");
dojo.require("dojox.grid.DataGrid");

function init() {

 var dataItems = {
     identifier: 'id',
     label: 'title',
     items: [{ 
      'id': 0,
      'title': 'This blog',
      'url': 'http://lgrammel.blogspot.com/'
     },{ 
      'id': 1,
      'title': 'My homepage',
      'url': 'http://larsgrammel.de/'
  }]
 };
   
 var store = 
   new dojo.data.ItemFileReadStore({data: dataItems});
   
 var structure = [{
     cells: [{
       field: 'title', 
       name: 'Title', 
       width: 'auto' 
     }, {
       field: 'url', 
       name: 'Address', 
       width: 'auto' 
     }]
 }];  
   
 var grid = new dojox.grid.DataGrid({
  'store': store,
  'structure': structure
 }, 'gridNode');

 grid.startup();
};

gadgets.util.registerOnLoadHandler(function() {
 dojo.addOnLoad(init);
});

</script>

<div id="gridNode"></div>

]]>
</Content>

</Module>

iWidgets: Referring to iContext from innerHTML

Recently, I started creating some widgets for IBM mashup center.

For HTML that is added in the iw:content section of the iWidget xml file, the iContext can used to access the widget functionality:

<iw:content mode="view">
<![CDATA[
  <div id="main">
    <input type="button" value="update settings" 
     onclick="iContext.iScope().updateSettings();"/>
  </div>
]]>
</iw:content>

Unfortunately, that does not work if you generate the HTML from inside the javascript and use innnerHTML to insert it into the iWidget. For example, the
following code leads to the javascript error "iContext is not defined".

var node = this.iContext.getElementById("main");
var html = '<input type="button" value="update settings"';
html += 'onclick="iContext.iScope().updateSettings();"/>';
node.innerHTML = html;

The reason for this problem is that Mashup Center rewrites the iContext calls from HTML and replaces iContext with the concrete iContext for the widget (something similar to _ns_0643e0a0e66511ddbf33c9dc98c8dde1_iContext, with the ID of the current widget).

To work around this problem, this identifier has to be used instead of iContext when creating HTML with iContext calls from javascript. The following function returns the complete, resolved iContext identifier:

getCompleteiContextIdentifier : function() {
    return "_" + this.iContext.widgetId + "_iContext";
}

Using this function, the fixed version of the javascript from above would be the following:

var node = this.iContext.getElementById("main");
var html = '<input type="button" value="update settings"';
html += 'onclick="' + this.getCompleteiContextIdentifier();
html += '.iScope().updateSettings();"/>';
node.innerHTML = html;

This code was tested with IBM Mashup Center 1.1. I don't know if the same problem exists in other versions of Mashup Center.