Showing posts with label s3. Show all posts
Showing posts with label s3. Show all posts

Wednesday, November 28, 2007

Amazon EC2 / S3 Network Performance

Thorsten von Eicken (CTO at RightScale) did some bandwidth measurements, here an overview of his analysis:

EC2 <-> EC2 (large instances):
  • one connection: totally 75 MB/s
  • three connections: totally 96 MB/s
S3 <- EC2 (large instance, HTTPS, download):
  • one connection: totally 12.6 MB/s
  • eight connections: totally 49.8 MB/s
the results for non-SSL (HTTP) are about the same.

S3 -> EC2 (large instance, HTTPS, upload):
  • one connection: totally 6.9 MB/s
  • twelve connections: totally 53.8 MB/s

Saturday, November 24, 2007

s3fox - a great Amazon S3 explorer and backup tool

I recently had to do some disaster recovery on my mac and made the painful experience that mozy sucks in every way one can imagine. After several failed attempts I managed to restore most of my stuff. Ok, mozy mac version is still in beta, and it is dirty cheap but it is also completly useless for the less experienced user, because it was not straight forward for me to restore large sets of files and it is useless for the more technical user, because it gives you little control of what it is doing and you can't even backup hidden files (e.g. a git repository) and when I asked them about that, I just received a non-answer from somebody who did not understand what I asked. This is my personal experience. I am sure there are plenty of happy mozy users out there.
So I had to reevaluate my options for personal backups. Because I was already a heavy user of Amazon S3 in relation with various web projects, I tried out some of the common S3 desktop tools:
  • JungleDisk (commercial, cheap) - sucks because what you store with JungleDisk you can only retrieve with JungleDisk (with additional effort you can do anything of course).
  • S3 Browser (free) - sucks because there is only a mac version and it has very limited functionality.
  • BucketExplorer (commercial, free while in beta) - sucks because it is written in Java and therefore looks ugly and just feels strange on the mac.
It's a crowded market and there are a lot of other S3 explorer and backup tools out there. I actually had been using one heavily and with great success when it came out: s3fox. It's a slick Firefox extension and it was the only one which exactly worked the way I actually expected it to work. But there was one problem: after initial release, the author did not maintain it, and Firefox evolved, the S3 protocol itself evolved and when I switched to a mac half a year ago, I couldn't use it anymore (I don't remember for what particular reason). Now I checked again and - nice surprise - there is a new version (0.4), which solved all the problems and even got drag & drop and some functionality for synchronizing local and remote folders. Easy to install, easy to use, free. I didn't have to look any further.

Monday, November 19, 2007

Amazon S3 will soon support upload via HTTP POST

One limitation of Amazon S3 is its strict REST behavior: upload only via HTTP PUT, and that is not possible with most of the browsers. Today a proposal was published by Amazon staff, so the final solution is probably not too far away anymore.