Updated ErlyCairo

Did some housekeeping on my growing inventory of Erlang open source projects targeted to web development. I completely rewrote the Erlang part of ErlyCairo, the Erlang bindings for the Cairo 2D graphics library. Previously it had a hard coded node name for the C-node, now it is based on a much more flexible gen-server approach.

Below see how to use this software to create a simple, purple, 100px * 100px PNG image:

start() -> 
 erlycairo:start_link().

%% if you run multiple C-Nodes, to avoid nodename conflicts 
start(CNodeNumber) -> 
  erlycairo:start_link(CNodeNumber).
    
stop() -> 
  erlycairo:stop().        

create_images()->
  rect("rect.png", 100, 100, {1.0, 0.2, 0.7, 1.0}).

rect(File, Width, Height, {Red, Green, Blue, Alpha}) ->
  case erlycairo:new_image_blank(Width, Height) of
      ok ->
          erlycairo:set_source_rgba(Red, Green, Blue, Alpha),
          erlycairo:rectangle(0, 0, Width, Height),
          erlycairo:fill(),
          erlycairo:write_to_png(File),
          erlycairo:close_image(),
          ok;
      {error, Reason} ->
          exit(Reason)
  end.

Amazon EC2 / S3 Network Performance

Thorsten von Eicken (CTO at RightScale) did some bandwidth measurements, here an overview of his analysis:

EC2 <-> EC2 (large instances):

• one connection: totally 75 MB/s
• three connections: totally 96 MB/s

S3 <- EC2 (large instance, HTTPS, download):

• one connection: totally 12.6 MB/s
• eight connections: totally 49.8 MB/s

the results for non-SSL (HTTP) are about the same.

S3 -> EC2 (large instance, HTTPS, upload):

• one connection: totally 6.9 MB/s
• twelve connections: totally 53.8 MB/s

TextMate is the most popular Rails development environment

So I am hopefully on the right track with my attempt to improve Erlang support for TextMate !
Tim Bray asked 1000 Ruby / RubyOnRails about their preferred development environment, below just the most popular ones, for the complete table go to Tim Bray's Ruby survey.

	All	%	Ruby	%	Rails	%
TextMate	478	30.35	216	24.24	262	38.30
Vi family	345	21.90	217	24.35	128	18.71
NetBeans	184	11.68	93	10.44	91	13.30
Eclipse	175	11.11	101	11.34	74	10.82
Emacs family	198	12.57	132	14.81	66	9.65

Disaster recovery nightmare with mozy.com

Shit happens and usually when you don't expect it, in my case I lost some personal data on my mac (iTunes and iPhoto library). I had signed up with Mozy for automatically backing up my data and easily restoring it, so I thought it just needs a few mouse clicks and everything is fine again. Unfortunately that was not the case. The pictures above show how both restore methods they offer (online restore and download-file-and-restore) failed (even after multiple retries).

Mozy support staff seems to responsive and friendly, but that's all. The copy-pasted text below is from an email they sent me 21 hours ago:

I will continue to do more research on this, and tomorrow our mac specialist will be in the building and from there we can work on a more permanent solution.

Lessons learned:

• Don't trust your data to a fast growing startup company. In my case it was Mozy. But I believe this can happen with any other of these over-hyped cheap-service companies.
• Don't test the backup service by just restoring a simple file. Take the time to do a test restore of your real valuable data at its full size.
• Choose at least two totally independent backup solutions.

Faster interaction between Erlang and TextMate

My first attempt of combining TextMate key bindings to Erlang scripts was fully based on escripts. Unfortunatley starting an escript costs some time (because of the startup of an Erlang VM), on my mac it is about 0.15 seconds. Not much, but enough to be perceived as short delay by the user. So I was looking for a faster way of interaction and Erlang core developer Bengt Kleberg pointed me to erlaunch, a kind of erlang scripting environment where the VM runs in the background and dosent need to be restarted for each script. So I guess something along that lines is the way to go for TextMate Erlang integration, probably a TextMate plugin written in C which takes care of the Erlang background VM during TextMate uptime. Need to learn now how to write a TextMate plugin ...

s3fox - a great Amazon S3 explorer and backup tool

I recently had to do some disaster recovery on my mac and made the painful experience that mozy sucks in every way one can imagine. After several failed attempts I managed to restore most of my stuff. Ok, mozy mac version is still in beta, and it is dirty cheap but it is also completly useless for the less experienced user, because it was not straight forward for me to restore large sets of files and it is useless for the more technical user, because it gives you little control of what it is doing and you can't even backup hidden files (e.g. a git repository) and when I asked them about that, I just received a non-answer from somebody who did not understand what I asked. This is my personal experience. I am sure there are plenty of happy mozy users out there.
So I had to reevaluate my options for personal backups. Because I was already a heavy user of Amazon S3 in relation with various web projects, I tried out some of the common S3 desktop tools:

• JungleDisk (commercial, cheap) - sucks because what you store with JungleDisk you can only retrieve with JungleDisk (with additional effort you can do anything of course).
  
• S3 Browser (free) - sucks because there is only a mac version and it has very limited functionality.
• BucketExplorer (commercial, free while in beta) - sucks because it is written in Java and therefore looks ugly and just feels strange on the mac.
  

It's a crowded market and there are a lot of other S3 explorer and backup tools out there. I actually had been using one heavily and with great success when it came out: s3fox. It's a slick Firefox extension and it was the only one which exactly worked the way I actually expected it to work. But there was one problem: after initial release, the author did not maintain it, and Firefox evolved, the S3 protocol itself evolved and when I switched to a mac half a year ago, I couldn't use it anymore (I don't remember for what particular reason). Now I checked again and - nice surprise - there is a new version (0.4), which solved all the problems and even got drag & drop and some functionality for synchronizing local and remote folders. Easy to install, easy to use, free. I didn't have to look any further.

MochiWeb got a HTML parser

Since MochiWeb went open source I have been working with Tait Larson on a Comet server (HTTP push) built on top of MochiWeb. We are not there yet, but it has been a very pleasant experience so far, especially considering an earlier (failed) attempt of mine to build such a thing on top of yaws, where I had to patch yaws and deal with all kind of annoyances (don't get me wrong, yaws is great for 99% of all possible use cases, just HTTP Push belongs to the other 1%).

So today I was just doing some online housekeeping and I noticed that MochiWeb got a HTML parser. Thats great ! So far, it has been asked many times on the Erlang mailing list how to parse HTML. And sooner or later somebody points to the yaws HTML parser, which works reasonably well. One time it was me asking that question and when I got the answer, I started to play with that yaws HTML parser and some simple XHTML (if I remember properly) examples and everything looked fine. But things turned nasty when I tested the yaws parser with real world HTML.

Now I hope people are starting testing and crashing (I just did) the MochiWeb parser with real world HTML and provide feedback to the developers so they can further improve it !

Rounded corners with canvas tag graphics

There exist many different approaches for creating rounded corners on HTML pages, and each of these approach has its problems. Javascript based solutions consume too much CPU cycles (and even more if doing anti aliasing), background-image based solutions are complicated (and even more, if done properly, e.g. sliding-doors based) and involve additional HTTP requests and solutions based on a bunch of DIVs (with border and varying margin to emulate the corner radius) turn any page into a horrible tag soup. The slickest solution, with the cleanest HTML and Javascript code I have seen so far is Greg Houston's mocha. It uses the canvas HTML tag, which is supported by Firefox, Safari and Opera 9. The ugly part of this solution is the heavy work-around required for Internet Explorer.

Amazing browser-based Javascript IDE

A few months back I put together a summary about browser-based Javascript code editors. Then I tried myself to create one, with dojo, but I did not get very far. Then Nicola Rizzo released CodeTextArea, which is a dojo widget and a very promising approach. A demo is available and it currently best works with Firefox on MS Windows. Today I learned about TIDE, a full fledged browser-based Javascript IDE, which loads fast, looks amazingly good, provides a lot of typical IDE functionality and has even a bunch of working Javascript demos to play with. It uses the following open source libraries:

• ExtJS for the GUI
• CodePress for the Javascript Editor
• Narcissus (from Brendan Eich) for Javascript parsing

The webhosting reviews of the different hosting companies give very informative and useful details about their hosting packages. The basic aim of seo services is to provide all essential tools of search engine optimization to enhance the web ranking according to the criteria of main search engines. The domain registration service providers offer very affordable rates for registering domains. The one of most leading hosting service providers, dotster has been offering very reliable hosting packages for the potential clients in reasonable ratings.

Amazon S3 will soon support upload via HTTP POST

One limitation of Amazon S3 is its strict REST behavior: upload only via HTTP PUT, and that is not possible with most of the browsers. Today a proposal was published by Amazon staff, so the final solution is probably not too far away anymore.

Easy Erlang compiling with TextMate

Today I created a custom command for easy Erlang compiling with TextMate. Errors (if there are any) are popping up hyper-linked on a HTML window, and if you click on them, you get straight to the file and line with that error.

• Download: Erlang Build command
• See it in action: Screencast (Quicktime, ca. 4 MB)
  

This is work-in-progress and just compiles the current file. The following features will be added in future versions:

- only compile modified files
- parse Emakefile if available, otherwise:
- crawl directories to figure out src, include and ebin folders.
- code reload if node is running

Which Javascript library ?

Update: Simon Willison's Javascript library slides, which where embedded at this place, are not accessible any more, so I took out the embedded slide show.

If it is just about pepping up a webpage with some AJAX and some flash-imitating eye-candy, any of the popular Javascript will serve similarly well. But when it comes to full fledged web applications, the dojo toolkit is leading the pack. Vance Dubberly shares the same vision and today he has provided a detailed answer on the dojo mailing list, why he is choosing dojo:

Because it's an application development framework. Most of the
javascript libraries out there serve some special purpose, Ext is a
gui toolkit, Prototype is a util library, and everything else is
pretty much the same. Dojo holds a special place in the community in
that it provides a unified framework for developing applications in
javascript. What I mean by this is that it provides everything from
simple utility classes to full implementations of design patterns.
Dojo helps me organize my application layout, debug it's output, and
prepare it for distribution.

Drawbacks? The most glaring one is the horrible documentation. It's
gotten a lot better but it's still neither consistent, clear, nor
complete. The documenters are working mostly API style docs. The dojo
book reads like a brick wall. The project is badly in need of some
"Here is how to accomplish (X) style docs" Despite this the pay off is
high for those who invest the time and suffer through the exponetial
increase in 4 letter words coming from between their lips.

The widgets are a nice bonus but I gotta tell you I used it before .9
and wouldn't touch the widget system because it was heavy, slow, and
well, annoying. ( no longer an issue ) It has much to offer beyond
widgets. Take a look through the dojox directory ...

Large file sizes aren't really an issue anymore, and what's better is
that you can if you choose, optimize your builds to include only what
you want in the dojo.js file.

Oh and one last thing, a thing which I think is often overlooked. The
Dojo foundation isn't simply building a javascript library. They spur
and drive innovation in the community often developing or incubating
break through technology. This kind of behavior deserves to be
rewarded.

First step in adapting Textmate for Erlang

I have been using Textmate for two weeks now, for all my coding needs, my productivity has increased (I know, all the real good programmers use Emacs or Vim, so I am not one of them). Today I spent some time on starting a little side project: improving Textmate Erlang integration. I have created templates with skeletons for a general Library module and also for the the following OTP behaviors:

• application
• gen_server
• gen_event
• gen_fsm
• supervisor
• supervisor_bridge

The module header details such as author, license, etc. can be specified via environment variables.
There is a long way to go, to create a complete Textmate bundle and reaching an Erlang integration as provided by Emacs. If anybody wants to join, you are welcome. An feel free to use the templates and adapt them to your own needs:

Download: Textmate Erlang templates

Note: if you set environment variable TM_MY_LICENSE to "MIT", then the MIT license gets included in the header as comment (more open source licenses to be added). Other custom environment variables I use: TM_MY_EMAIL and TM_MY_WEBSITE. You can set them in the preferences.

Mochiweb Erlang HTTP server toolkit now on googlecode

Mochiweb is a small and extensible web server written in Erlang. I have mentioned it before. Today Bob Ippolito and his team at Mochiads have started the mochiweb project on googlecode:

• mochiweb.googlecode.com
• groups.google.com/group/mochiweb
  

I already have hacked the sources a bit, just to make it better fit my needs (e.g.: adding fd_server) but the real fun begins when people start to extend it. I mean extensions which do not modify the core engine, and which can be separate projects at different repositories. Anybody interested in collaborating on OpenId, oAuth and COMET extensions ?

Open social insecurity

Everyone is praising google's open social initiative. So let me do the opposite and try to put together a few weak points:

• Open social apps already have been hacked.
  
• Open social is not really open yet, only the client API has been published so far. It allows to put iframe apps with Javascript based business logic (or should I say "social logic") into open social containers such as orkut. Currently you have to join the waiting list, to get notified when the server API specification gets published.

Dave Winer goes even a step further and criticizes the motivation behind open social:

Standards devised by one tech company whose main purpose is to undermine another tech company, usually don't work.

In this case it's Google trying to undermine Facebook.

And I don't think it's going to work.

Time will tell ...

[Update:] Just after posting, I found more critique thoughts, by Julien Bond:

As a geek it pisses me off because there's absolutely no accountability or transparency in how those standards are developed. It's every bit as bad as MS trying to force the Word XML standard through the standards bodies. Google is something of a black box. There's no way to influence them. Stuff appears out of the black box fully formed.

And here a detailed article about, exploiting open social XSS vulnerabilities on ning.

Erlang and textmate

Two days ago, I felt the need to reevaluate my options for an Erlang text editor or an IDE. I have been using aquamacs (the mac-ified emacs) for a couple of months now. It has a steep learning curve, but looks reasonably well on the mac (yes, that matters for me, and therefore I try to avoid Java desktop apps) and it provides good Erlang integration. And everything can be customized, in elisp, a LISP dialect. That is what I did not like. I would prefer to customize things in Erlang itself.
So what are the other options ? Beside of of emacs, a vim hack and Java IDEs (erlide, ErlyBird) there are no other open source options I am aware of. So I investigated about proprietary solutions and it turned there is one with decent Erlang support: Textmate. That was a pleasant experience, quickly downloaded and installed and it looks great. Textmate ships without Erlang support, but it's very easy to install the Erlang bundle, I just had to install and run GetBundle, a helper app to browse and install additional language bundles.
The support for Erlang is not that much yet:

• Syntax highlighting
• Code snippets

There is also a language bundle for Makefiles, but it seems to simplify only running make without arguments, therefore it is not very useful. The most interesting part, which I haven't explored yet are custom scripts accessible via user-defined key-binding, which can take current file or text selection as input, process it and overwrite the input with the result. Those scripts can be in any scripting language, so it should work with with Erlang escripts. Will experiment with this ...
There exists also a windows editor called E-TextEditor, which is language bundle compatible
with TextMate.

The Case for the Open Web

Alex Russel, dojo toolkit project leader, presented recently the slides below and I spotted "Web 3.0" on the cover slide ...

Prism - Turn any web page into a desktop app

Today Mozilla Labs launched Prism, a tool for easily turning web pages into desktop apps with their own window, desktop shortcut but no URL address bar (everything customizable, of course). It's not a new project, just the relaunch of what was previously mostly unknown as WebRunner. If the Mozilla guys do it right this time, Prism will compete head-to-head with Adobe AIR.

An early alpha version, currently only for windows, is available for download. I tried it out, for this blog, it is very easy to use, just one dialog for defining Name, URL and optional properties for the web page to be turned into a desktop app.

Google AdSense might mess up your page

Javascript expert Andrea Giammarchi points to some potential flaw in Google Adense. If you put on the same page Google AdSense and your own or other third party Javascripts, AdSense could mess with the others in a destructive way so that nothing works anymore on that page. The origin of the problem is that AdSense deletes properties of global objects, without checking whether these properties actually belong to AdSense.

What can one do to prevent problems ? The usual recommendation for dealing with the insecure aspects of Javascript: don't use global variables and put your stuff in your own namespace. See also my previous posts about Javascript security.

HTML5 movie trailer