Bespin - experimenting with embedded preview

While a browser based text editor or web development environment has many disadvantages compared to the desktop variant, there a also some great possibilities which haven't been explored yet with bespin. Possibilities such as having direct browser DOM access for previewing and possibly manipulating content and that is not limited to text. Images (img tags and background images) can also be edited live (with a bit of canvas magic, as e.g. in pixastic) and server round trips are only required to store the edited images. Traditional editors can only dream of this and adding this possibilities to monster IDEs makes them even more complex.
So here is my proposal (including fully functional early stage prototype) for a live preview, see the screenshots above, just clicking on the eye icon at the toolbar and at the place of the editor canvas with the HML source code, a preview panel gets blended in. Unfortunately there are a few issues. Lets take a look at the style definition of the example. It has defined properties for the "body". But we don't want to change the bespin body properties. So what I am doing to prevent a visual and usability disaster is parsing (not really in the lexical sense) the HTML template, extracting styles and injecting them as filtered CSS rules into the page when rendering the preview and removing the rules when switching off the preview.

So far this is only for HTML templates. But my idea is to have a pluggable template parser system so we can expand PHP, django or other templates. While this is not easy, I have done it before (ErlyDTL, the Django template language for Erlang) and I have used a javascript template system before: dojox.dtl - dojo django rendering. So why not use dojox.dtl for bespin template rendering ? Well, first it is specific for django, second it is text and node based (for performance reasons). That is good for an app where template parts get swapped frequently, but just for preview we don't need a node based template system, at least not for now because it adds a lot of complexity.

And one more thing to consider are the edge cases in which this approach won't work. Hopefully with enough trickery the embedded preview will work good enough in most cases. The bespin toolbar and commando line will anyway prevent a full-browser preview, unless they are blended out as well, which is currently not implemented and I am not even sure whether it would be a good idea to do so.

If you wanna contribute to the technical discussion, please user the google bespin core group.

bespin now dojo-powered

well, at least on my laptop, but I submitted a patch, so if the Mozilla guys like it, I hope it goes into trunk. The screenshot shows bespin running on the brand new Safari 4 beta.

There are still things to do related to the port, for example eliminating global variables outside of dojo and bespin namespace, dojo build profile and fixing the zillons of bugs I probably have introduced (but at least one I fixed, mousewheel works on Firefox as well)

Porting bespin to dojo - 90 percent done

No I try to break the number one rule in software development: the last 10 percent take 90% of the total time. The dashboard works, except the command line sometimes having hiccups. The editor lets me type in text, but anything else breaks. I also replaced all third party libraries (in the external folder) with dojo equivalents. Here my experience in regard to prototype:

• It's an absolutely fantastic framework if you wanna simple things get done quickly and you don't care about JS design philosophies (name spaces, native object extension).
• There are a lot of utility functions for string, array and enumerable manipulation, but some of these functions are "opinionated" as all the great things from the 37systems guys. In dojo you end up cooking together your own one-line solutions for the simple stuff. It is additional work, but results in a customized and optimized solution.
• The huge amount of string utilities suggests that prototype is targeted to developers who prefer not to deal too much with regular expressions.

So what else is missing once the bug fixing is done:

• Name space cleanup: I used a all-lowercase namespace (e.g.: "bespin.editor"), because that is how dojo name space and file hierarchy looks like. I regret that, because it violates the bespin coding guideline and worst of all, requires a lot of slave work to update the documentation, so I will refactor to a name space mostly identical to the current one from bespin trunk (e.g.: "Bespin.Editor")
• There are still some global variables (in the startup scripts), but before I even suggest how to change that I need some feedback from the bespin architects.
• Testing (I introduced lots of bugs ...)
• Build script to create minimized JS/CSS for production, maybe as paver task.

Update: I found a cheat sheet for conversion between dojo <->prototype

Exploring bespin - and porting it to dojo

After some months absent from blogging and open source software involvement, the launch of bespin motivated me to get back to fun-hacking and blogging. Bespin is a browser based code editor which feels and smells like a real editor (vim, emacs, textmate), and not like a form on a webpage as all of the other browser based code editors (including an experimental one I hacked together myself). Internally bespin uses canvas and therefore currently only runs well on Firefox or webkit nightly. As JS lib Protoptype was choosen, but that might change in favor of dojo. So I thought I will give it a try, to port bespin to dojo and learn about the internals of bespin. And it might serve as suggestion in case bespin officially gets ported to dojo.

So where we are so far ? The screenshot at right shows that dashboard canvas rendering halfway works already, but with scaling dimensions something went wrong. Prototype makes the simple things very simple, dojo doesn't care much about the simple things, so I had to hand code a few things which had ready-to-use counterparts in prototype. But the really ugly thing about Prototype based javascript development is the global name space pollution and the extension of native objects, that is so much better handled with dojo and its class system. Of course nobody cares about those details once an app is up and running. Will finish the port and submit it to bespin bugzilla, but it might take a few days to iron out the bugs.

Notapad - the start of a dojo based WebIDE

Niccola Rizzo has been working for a couple of months now on the CodeTextArea dojo widget for in-browser Javascript code editing and he has just launched a public demo and test site named notaPad.

The editor provides Javascript syntax highlighting and additional functionalities with the following key bindings:

CTRL + space - autocomplete
CTRL + l - GOTO LINE
CTRL + b - GOTO MATCHING BRACKET
CTRL + c - COPY
CTRL + x - CUT
CTRL + v - PASTE
right click on the row numbers - BOOKMARKS

Dojo Widget for in-browser editor CodeMirror

CodeMirror, a very impressive in-browser code editor for Javascript, XML/HTML or CSS (or any language, you just have to plug in a your own parser) made some nice progress in the last months. CodeMirror has no dependency on other frameworks or libraries. If you want to use it in a dojo environment as a dojo compatible widget, here I am gonna share here a little tutorial how to write such a thing:

First download CodeMirror and transform CodeMirror.js (the main file which loads the other files into an iframe), into something like this:

dojo.provide("mystuff.widget.CodeMirror");
dojo.require("dijit._Widget");


dojo.declare("mystuff.widget.CodeMirror", dijit._Widget, {

initialized: false,

// currently supported: 'xml' (HTML), 'js' or 'css'
type: 'xml',

options: {
stylesheet: "",
path: "/static/codemirror/js/",
parserfiles: [],
basefiles: ["codemirror_iframe.js"],
linesPerPass: 15,
passDelay: 200,
continuousScanning: false,
saveFunction: function() {
console.log('save');
},
content: " ",
undoDepth: 20,
undoDelay: 800,
disableSpellcheck: true,
textWrapping: true,
readOnly: false,
width: "100%",
height: "100%",
parserConfig: null
},

postMixInProperties: function() {
this.options.stylesheet = "/static/codemirror/css/" + this.type + "colors.css";
this.options.parserfiles = ["parse" + this.type + ".js"];
},

postCreate: function() {
this.inherited(arguments);
},

startup: function() {
if (dijit._isElementShown(this.domNode.parentNode))
this.initialize();
},

initialize: function() {
if (this.initialized)
return;

frame = document.createElement("IFRAME");
frame.style.border = "0";
frame.style.width = this.options.width;
frame.style.height = this.options.height;
// display: block occasionally suppresses some Firefox bugs, so we
// always add it, redundant as it sounds.
frame.style.display = "block";

this.domNode.appendChild(frame);

// Link back to this object, so that the editor can fetch options
// and add a reference to itself.
frame.CodeMirror = this;
this.win = frame.contentWindow;

var _this = this;
var html = ["<html><head><link rel=\"stylesheet\" type=\"text/css\" href=\"" + this.options.stylesheet + "\"/>"];
dojo.forEach(this.options.basefiles.concat(this.options.parserfiles), function(file) {
html.push("<script type=\"text/javascript\" src=\"" + _this.options.path + file + "\"></script>");
});
html.push("</head><body style=\"border-width: 0;\" class=\"editbox\" spellcheck=\"" +
    (this.options.disableSpellcheck ? "false" : "true") + "\"></body></html>");

var doc = this.win.document;
doc.open();
doc.write(html.join(""));
doc.close();

this.initialized = true;
},

getCode: function() {
return this.editor.getCode();
},

setCode: function(code) {
this.editor.importCode(code);
},

focus: function() {
this.win.focus();
},

jumpToChar: function(start, end) {
this.editor.jumpToChar(start, end);
this.focus();
},

jumpToLine: function(line) {
this.editor.jumpToLine(line);
this.focus();
},

currentLine: function() {
return this.editor.currentLine();
},

selection: function() {
return this.editor.selectedText();
},

reindent: function() {
this.editor.reindent();
},

replaceSelection: function(text, focus) {
this.editor.replaceSelection(text);
if (focus) this.focus();
},

replaceChars: function(text, start, end) {
this.editor.replaceChars(text, start, end);
},

getSearchCursor: function(string, fromCursor) {
return this.editor.getSearchCursor(string, fromCursor);
}
});

Then your should concatenate and minifiy (I use YUI compressor) all the JS files:

util.js, stringstream.js, select.js, undo.js, editor.js => codemirror_iframe.js

Also minify the parsers: parsejavascript.js, tokenizejavascript.js, parsecss.js, parsexml.js

At this point you can embed the widget into a test page:

<div
id="html_editor"
dojoType="mystuff.widget.CodeMirror"
type="xml"
style="height:100%;">
</div>

Now comes the tricky part: the editor does not initialize automatically at page load, because with dojo, chances are very high, you are gonna use this widget inside a container, where the editor is hidden at page load, and that would cause trouble with some browsers. So you need to subscribe to an event which triggers the visibility of the editor, so it can be lazy-initialized at first time it becomes visible. For example to use CodeMirror inside a tab container with a tab with id 'html_editor_tab' I do something like this:

dojo.declare("MyApp", null, {
main_tab_selected: function(page) {
if (page.id == 'html_editor_tab'){
dijit.byId('html_editor').initialize();
}
}
});

var myAppInstance = new MyApp();

dojo.subscribe("main_tabs-selectChild", myAppInstance, "main_tab_selected");

Update: Uhh, where was my brain when I wrote this article, I accidentally "misspelled" CodeMirror with CodePress at several places, including the title, it's corrected now ...

Hard to find technical docs about dojo internals

Of course an open source framework has no real untold secrets, but digging into the source code can be time consuming and and without in-deep documentation it's difficult to extract the authors thoughts behind it, so I was pleasantly surprised when I stumbled upon this links about dojo internals (which currently can't be found, or at least not by me, at dojo documentation):

• dojo.dnd 1.1 technical documentation
• dojox.gfx 1.1 documentation

Dojo theming tutorial and more at dojocampus

Ever wanted to know how to create flexible-size background images for rounded corners or other CSS effects specifically for custom-theming Dojo widgets ? Unless you have already intimate knowledge of the Sliding Doors technique and CSS sprites, you should checkout the "Rounded tabs" tutorial on the recently started DojoCampus site. There is also other great stuff there such as a video about the dojo grid and Firebug tips.

Django Template Language - the dojo implementation

Beside of Erlang based ErlyDTL (by Evan Miller and myself) there exists (at least) one other non-Python project which implements the Django Template Language: the dojo Javascript version: dojox DTL (documentation) for creating dynamically and template-based HTML layouts inside the Web browser (opposed to traditional approaches of creating HTML layouts at server-side). Today dojox DTL reached feature completeness, implementing all original Django tags and filters, at least that is what I guess from reading the SVN changes for the README:

--- This aims to be a 1:1 match with the Django Template Language as outlined in
+++ This is a 1:1 match with the Django Template Language as outlined in

While dojox DTL is a very interesting concept and brings new possibilities to separate client-side logic and presentation, I haven't seen yet a killer example. Maybe I just haven't looked far enough. But one thing is sure: if you use a template language at server side and at Javascript client side, and it can be the same template language, that's a huge benefit.

New book "Mastering Dojo" - beta PDF now available

For those who prefer to learn new Javascript stuff by reading a real book, Mastering Dojo (500 pages) has been announced today. It will be relesed in June, but the beta PDF is already available. Alex Russell, one of the initial developers of dojo, is a co-author of that book.

Update: I ordered the PDF version, right after writing this post, but haven't received the e-mail with the download-link yet, so all what is really available right now, are the two example chapters at the publisher's site.

Which Javascript library ?

Update: Simon Willison's Javascript library slides, which where embedded at this place, are not accessible any more, so I took out the embedded slide show.

If it is just about pepping up a webpage with some AJAX and some flash-imitating eye-candy, any of the popular Javascript will serve similarly well. But when it comes to full fledged web applications, the dojo toolkit is leading the pack. Vance Dubberly shares the same vision and today he has provided a detailed answer on the dojo mailing list, why he is choosing dojo:

Because it's an application development framework. Most of the
javascript libraries out there serve some special purpose, Ext is a
gui toolkit, Prototype is a util library, and everything else is
pretty much the same. Dojo holds a special place in the community in
that it provides a unified framework for developing applications in
javascript. What I mean by this is that it provides everything from
simple utility classes to full implementations of design patterns.
Dojo helps me organize my application layout, debug it's output, and
prepare it for distribution.

Drawbacks? The most glaring one is the horrible documentation. It's
gotten a lot better but it's still neither consistent, clear, nor
complete. The documenters are working mostly API style docs. The dojo
book reads like a brick wall. The project is badly in need of some
"Here is how to accomplish (X) style docs" Despite this the pay off is
high for those who invest the time and suffer through the exponetial
increase in 4 letter words coming from between their lips.

The widgets are a nice bonus but I gotta tell you I used it before .9
and wouldn't touch the widget system because it was heavy, slow, and
well, annoying. ( no longer an issue ) It has much to offer beyond
widgets. Take a look through the dojox directory ...

Large file sizes aren't really an issue anymore, and what's better is
that you can if you choose, optimize your builds to include only what
you want in the dojo.js file.

Oh and one last thing, a thing which I think is often overlooked. The
Dojo foundation isn't simply building a javascript library. They spur
and drive innovation in the community often developing or incubating
break through technology. This kind of behavior deserves to be
rewarded.

The Case for the Open Web

Alex Russel, dojo toolkit project leader, presented recently the slides below and I spotted "Web 3.0" on the cover slide ...