Red Leopard

Things don’t change. You change your way of looking, that’s all.
— Carlos Castañeda

Early this spring, I made some big architectural changes in the company’s website. The two most far-reaching changes involved Amazon’s Web Services and Adobe’s Flex product. Sometimes you regret big changes. I only regret having not made the changes earlier.

I admit that I wasn’t always a flex fan. Indeed, I dismissed the flex out of hand in the early days mainly as a response to Macromedia’s steep pricing model. Ouch. Since then, Adobe had bought Macromedia and the pricing model changed several times. I never noticed. Such is the cost of writing something off.

Then I saw Peter Armstrong’s presentation at the Third Annual Silicon Valley Ruby Conference (April 18-19, 2008). Turns out, flex is now free. (uhhhmmm, ok. you will want to buy the developer ide but it’s possible to use the free command line compiler.)

I bought Armstrong’s book and started working through the tutorials. By the time I finished Chapter 6, I knew that I was going to change our webapp from a javascript-driven front-end to an flex-based, actionscript-driven front-end. We deal with a lot of tabulated data. Over a million sets and growing. Some of the datasets are small and some are quite large. Furthermore, each row had numerous event listeners attached. We had plans to add more, add mashups, add slicky bells and wistles. The larger tables were dozens of megabytes large. Which isn’t a problem in and of itself except that…

Javascript sucks.

Well, not really. It’s the browser’s that suck. All of them. We had to back off functionality just to get IE to render. Firefox’s javascript+dom engine (including ff3) would render the tables (if it didn’t crash) but not all the event listeners worked. Safari (web-kit) would render and all events worked but then Apple released the 5-second script warning. We schemed endlessly on how to break the problem up. Mostly we hoped one of the miracle javascript libraries would solve these problems. We knew it was unlikely but we hoped.

I made some side-by-side comparisons on my MacBookPro and actionscript beat javascript hands down. Orders of magnitude faster to load and render. Less code. Consistency between browsers. And finally, actionscript/flex worked. And worked well.

What didn’t work well was our MySQL database. Our million datasets occupied hundreds of millions of database rows in scads of tables and hundreds of gigabytes of disk space. A 3ware RAID10 disk subsystem with master/slave configuration was in complete IO overload all the time. I had Matt from mysql come in to help us unravel our mess. This guy was good. He identified, explained, documented and proposed a prioritized list of solutions.

But I started thinking, why the hell do we even store the datasets in MySQL in the first place? The only time we ever change a dataset is when we replace it. The relational database solution had always been a problem and was getting worse by the day. Thinking like a mathematician, I decided to solve a different problem. Why not just save the datasets in flatfiles and overwrite them when they change? Just store them in flatfiles. Like pictures.

Of course, every solution brings its own problems. And storing a million flatfiles has its problem: you cannot store a million files in a single directory (not on Centos+ext3 you cannot). I recalled reading about mogileFS some time back. It presented what looked like a flat directory in which you could store millions of files in a single directory, distributed the save over multiple machines with redundancy, and so on. Looking at that solution lead me to smugFS. And to Don MacAskill’s presentation on Scalability. Crap! I said. I don’t need to host the flatfile’s at our colocation center. I’ll put them on Amazon S3. And While I’m at it, I’ll build a RESTful API on Amazon EC2 to import the datasets.

And my team has done it. We have completed the move to EC2/S3 and Flex. I couldn’t be happier. Armstrong’s book, Flexible Rails, lit the spark for our migration to Flexible Web Services.

I will write about some of the hurdles we’ve overcome in future posts. Some of the things we’ve solved are (i) building an EC2 image from scratch, (ii) compressing flat files in a format which actionscripts’ ByteArray can uncompress, (iii) using the struts2 REST plugin, (iv) XML parsing using StAX.

It’s confirmed. My Tuesday evenings are booked till December.

I’m building a study group to meet once a week out of class. All students in either the Monday or Tuesday class are welcomed to join. (Which suggests we won’t meet on either Monday or Tuesday)

Introductory Chinese Language and Culture

“With an emphasis on conversation and an introduction to reading and writing Chinese characters, it focuses on the acquisition of basic communication skills for travel, business, and everyday use.”

Stanford Continuing Studies
Tuesdays, Sept 23 – Dec 2, 7-9 pm

“Try to learn to breathe deeply, really to taste food when you eat, and when you sleep, really to sleep. Try as much as possible to be wholly alive, with all your might, and when you laugh, laugh like hell, and when you get angry, get good and angry. Try to be alive. You will be dead soon enough.”

William Saroyan
(1908 – 1981)

This is a busy week. Monday is a Memorial Day in the United States, ostensibly a holiday. These days, holiday is code for catching up on specification writing, of course.

But on Wednesday-Thursday I’m heading up to Google I/O. I’ll be in the social track. I didn’t see a single session in social track I want to skip. But one session stands apart as one I simply must see. ‘OpenSocial – Scaling and Analytics, Nuts & Bolts’ delivered by Nat Brown.

I’ll be at the conference both days. If you’re at the show, look me up. See you there!

Churchill Route
3 miles
34m 12s
11.24 pace

Buddha Belly Begone!

Last summer, I retired my old Titanium PowerBook G4 (800MHz) as the Company bought my a shiny new MacBook Pro (2.4GHz). Sweet!

A year later, the latch on my laptop stopped latching. I sent it in for repair. In the interim, I Carbon Copy Cloned my hard drive and am using that hardrive to boot a 20″ iMac on loan from the office. I love it.

One thing I do miss: I really do use my laptop as a lap top. I can lean back with feet up on the ottoman and work with laptop on my lap. Sometimes, I need a book under the laptop. It gets hot.

I get my laptop back next week. I will miss the iMac but not enough to switch back to a desktop.