About

I’m an Italian geek living in Lecce, Southern Italy, and my name is Luca Greco (a.k.a. rpl / rply / ripley). At the present time I’m interested in "Javascript as a Real Programming Language", "Mozilla as a Platform"...


Released under CC 3.0 by-nc-sa where not specified.
Copyright © 2009-2011 Luca Greco

CouchDB, ExtJS and Simile Timeline

in 18 Nov 2009 about , ,

In this post I’ll try to summarize my first experiment on using CouchDB in a solution of a real world problem.

Problem

At work we are developing a VOIP application based on Yate (an OpenSource VOIP Engine) and in this kind of application timing is a very important factor, more important than in classic Web applications.

The application is working like a charm but sometimes it encounter exceptional conditions and we feel it can be related to something we’re not handling in the right way or… at the right time ;-)

How can we trace and debug this kind of problems?

We need a complete tracelog of messages received and how our application is handling them… and we need to trace the real-deployed-application, where this exceptional conditions occurs.

Solution

The solution we’re prototyping is based on a simple and not so new architecture:

The application push useful tracing information in a central storage, and we want to search, filter, analize and visualize useful informations from it.

This kind of problem can be solved in a number of ways and technologies but our experimental solution use CouchDB as “tracing-info” storage, and a pure client-side javascript application to access the data.



Some Tech Details

Our application is developed in PHP (sigh!) so we’re using the simpler PHP binding of the CouchDB Rest API:

NOTE: PHP-on-Couch don’t support authentication so I wrote a small patch (and I will setup a github fork asap)

In our current design decision every tracing session became a new different couchdb database:

<?php
// CONNECT TO A DATABASE AND DATABASE CREATION
$client = new couchClient ('http://localhost:5984','alcadialer_tracelogs_'.$start_tracing_datetime); 

if ( !$client->databaseExists() ) {
    $client->createDatabase();
}

// COLLECT TRACING INFO AND STORE IT ON COUCHDB
try {
  $ev->setParam("AlcaDialerCommand",get_class($command));
  $response = $client->storeDoc($ev);
} catch (Exception $e) {
  echo "ERROR: ".$e->getMessage()." (".$e->getCode().")<br>\n"; echo $e;
}
?>

So we’ve tracing logs databases in a CouchDB service…
now we need to navigate them in useful ways and in a timing problem “useful ways” means “visualize events on a timeline”… and the “simile timeline” is so pretty and simple to use:

Now the question is: how can we host all this stuff on CouchDB? we need a separate service?
The answer is: CouchDB is enough :-)

A more technical answer to this question can be: CouchApp is the way :-D

After CouchApp installation we can generate a CouchDB standalone application working tree (as Rails do)

$ couchapp generate myapp

$ ls -l myapp

drwxr-xr-x 4 rpl rpl 4096 2009-11-16 19:02 _attachments
-rw-r--r-- 1 rpl rpl   97 2009-11-16 07:03 couchapp.json
drwxr-xr-x 2 rpl rpl 4096 2009-11-16 07:01 lists
drwxr-xr-x 2 rpl rpl 4096 2009-11-16 07:01 shows
drwxr-xr-x 2 rpl rpl 4096 2009-11-16 07:01 updates
drwxr-xr-x 3 rpl rpl 4096 2009-11-16 07:01 vendor
drwxr-xr-x 2 rpl rpl 4096 2009-11-16 07:01 views

In our usecase we have to put all files of our javascript application in the _attachments dir:

$ ls _attachments/ -l

-rw-r--r-- 1 rpl rpl 1255 2009-11-16 19:02 index.html
drwxr-xr-x 4 rpl rpl 4096 2009-11-16 22:47 js
drwxr-xr-x 2 rpl rpl 4096 2009-11-16 19:01 style

$ ls _attachments/js -l

-rw-r--r-- 1 rpl rpl 5440 2009-11-16 22:47 application.js
drwxr-xr-x 4 rpl rpl 4096 2009-11-16 07:13 extjs
drwxr-xr-x 4 rpl rpl 4096 2009-11-16 14:52 timeline

In “_attachments/js” there is our application.js and two directories, extjs and timeline, where are our application dependencies (I think they can/could be stored under vendor directory… but I will try this in a second experiment)
From extjs and timeline directories we’ve removed unused stuff (examples, documentations etc.).

Let’s see some of the code of application.js (the interesting part of application.js code :-D).

How can we put CouchDB data in an ExtJS grid?
On CouchDB wiki there’s a CouchStore prototype for ExtJS… but it don’t work very well (it have to be reworked, I think), but in our usecase we only need to get from a CouchDB database, so we can use a classic JSONStore from the ExtJS standard lib:

var store = new Ext.data.JsonStore({
        // store configs
        autoDestroy: true,
        url: trace'/tracelogdb/_design/messages/_view/all_by_seqid',
        storeId: 'myStore',
        // reader configs
        root: 'rows',
        idProperty: 'key',
        totalProperty: 'total_rows',
        restful: true,
        fields: [
            {name: 'id'      },  
            {name: 'key'     },  
            {name: 'value'   },
        ]
    });

And attach it to a GridPanel.

Our JsonStore use a design view in the tracelogdb so we need to create it (we’ve not automated the task right now):

{
  "_id":"_design/messages",
  "language":"javascript",
  "views": {
    "all_by_seqid": {
      "map":"function(doc) {  emit(doc.params.AlcaDialerSeqID, doc); }"
    }
  }
}

Now we’ve a local (to the browser) store of CouchDB documents and every row in the view have a timestamp in a sub-property of ‘value’, so we can use that value to insert and visualize events in a simile timeline component:

function TimelineInit(store) {
    var eventSource = new Timeline.DefaultEventSource();
    
    var bandInfos = [
	Timeline.createBandInfo({
            width:          "70%", 
            intervalUnit:   Timeline.DateTime.SECOND, 
            intervalPixels: 100,
            eventSource:    eventSource,
        // THIS BAND ZOOM ON MOUSE WHEEL
	    zoomIndex:      0, 
            zoomSteps: [
              {pixelsPerInterval: 280,  unit: Timeline.DateTime.SECOND},
              {pixelsPerInterval: 140,  unit: Timeline.DateTime.SECOND},
              {pixelsPerInterval:  70,  unit: Timeline.DateTime.SECOND},
              {pixelsPerInterval:  35,  unit: Timeline.DateTime.SECOND},
              {pixelsPerInterval: 400,  unit: Timeline.DateTime.MINUTE},
              {pixelsPerInterval: 200,  unit: Timeline.DateTime.MINUTE},
              {pixelsPerInterval: 100,  unit: Timeline.DateTime.MINUTE}
	    ]
	}),
	Timeline.createBandInfo({
	    overview: true,
	    eventSource:    eventSource,	  
            width:          "10%", 
            intervalUnit:   Timeline.DateTime.HOUR, 
            intervalPixels: 100
	}),
    ];

    timeline_data = { 
	'wiki-url':"http://fake-url/", 
	'wiki-section':"Fake Section", 
	'dateTimeFormat': 'Gregorian',
	'events': []
    };

    var last_timestamp;
    var seq = 0;

    function render_event(record) {
	var description = "";
	// ... FORMAT EVENT DESCRIPTION FROM RECORD CONTENT

	return {
	    durationEvent: false,
	    title: "(" + record.data.key + ") - " + record.data.value.name,
	    description: description,
	    start: new Date(record.data.value.origin*1000),
	    icon: 'js/timeline/timeline_js/images/dark-red-circle.png'
	};
    }

    store.each(function(i) {
	timeline_data.events.push(render_event(i));
    });

    // SYNC TIME BAND
    bandInfos[1].syncWith = 0;
    bandInfos[1].highlight = true;

    // CREATE TIMELINE
    tl = Timeline.create(document.getElementById("my-timeline"), bandInfos);

    // CENTER TO THE FIRST EVENT
    tl.getBand(0).setCenterVisibleDate(Timeline.DateTime.parseGregorianDateTime(timeline_data.events[0].start))
    
    // LOAD ALL DATA
    eventSource.loadJSON(timeline_data, document.location.href); 
    return tl;
}

We can enhance grid/timeline interactions with a listener that move timeline on grid row selection events:

listeners: {
  "rowclick": function(grid,rowIndex,e) {
    var selected = grid.getSelectionModel().getSelected();
    window.timeline_view.getBand(0).setCenterVisibleDate(
    Timeline.DateTime.parseGregorianDateTime(new Date(selected.data.value.origin*1000)));
  }
},

How it looks right now?



ohhh… Yes.. it’s only a prototype but it’s soooo fun, isn’t it? :-)

Happy Hacking,
rpl