You work for me, Computer.

By Brandon Bloom

One Year After Accidentally Falling in Love With Rails

About a year ago, I wrote a post titled How two Pythonistas accidentally fell in love with Rails. Some people predicted that the honeymoon would be over quickly and that I would soon hate it.

So, do I still love Rails?

Yes. Absolutely.

The prototype and first beta version of my startup was built entirely on Rails. The framework enabled us to build our site significantly faster than we would have been able to without it. For that, I still love Rails. It was critical in helping us get to where we are today.

It’s the perfect hammer for the v1, nebulous direction, rapid development, nail.

That said, we’re not using Rails for any new components.

After getting a lot of feedback from our first beta, we realized we had the wrong approach to the UI. Now, in our second attempt, we’ve got a much richer client-side front-end, written in CoffeeScript and bootstrapped by Node.js. Ruby/Rails is simply not the right tool for that job. In the new world of the browser app, even the folks of 37signals have been accused of propping up their successor.

Luckily, we had a pretty good handle on the models in the problem domain. So I ran git rm -r app/views and effectively turned the Rails app into a Json API backend. The models have evolved, but not by much. In fact, there were a few months where the Ruby code remained practically unchanged while we developed the frontend.

As time goes on, we’ve been breaking our API layer apart into several services. It makes development, deployment, and versioning easier. Service Oriented Architecture is working great for us… now that we know what services we need. These services are each too simple to justify the overhead of Rails. Some are Node.js, some are Sinatra, some are cron jobs and shell scripts. We’re polyglots, so we’re careful to choose the right tools for each job.

We’d have gotten it very wrong if we tried to build services from the start. The monolithic Rails app was the correct evolutionary intermediate form.

Given a time machine, would I have done anything differently?

Only one thing: I wouldn’t have used ActiveRecord.

I guess I should have seen that one coming. ORMs are an insidious, broken abstraction. At first, they save you time and make you feel productive. They work out splendidly for a while, but like a virus, they infect every part of your application. One day, you hit that abstraction brick wall and it’s too late to do anything about it without significant effort.

Are there things I hate about Rails?

Ignoring ActiveRecord, only one thing really stands out: Performance.

I’m not even talking about page load times. I’m talking about development time.

Even with our modestly sized application, the impact on productivity is atrocious. Our test suite runs far too slowly to be waited on for every checkin. Rake’s startup time is too great to for its usage to be considered “interactive”. Simple shell scripts have replaced practically all of our custom tasks; they are instantaneous.

Isn’t this basically the same story I told about Django?

Yeah, I guess it is.

The story basically goes:

  1. Use a framework
  2. Be productive
  3. Encounter shortcomings
  4. Stretch it to work
  5. Watch it break
  6. Replace components
  7. Loop steps three to six
  8. Be productive
  9. “Where’s the framework?”
I still stand by my original criticisms of Django (although, I’m sure much as changed in the last year). I’m also re-affirmed in my belief that big frameworks are generally bad. It just so happens that Rails is good enough and matches my mindset well enough to justify the framework pains.

Imported Comments


If you are annoyed by your slow test times you should watch this presentation by Corey Haines at gogaruco


I’m going through using Django (1.2) coming from Rails to maintain and extend a legacy web app. Generally I find Django more limited and painful than the Rails equivalent. Maybe it’s the way we create apps or just personal preference.

I think it really depends what kind of web application you’re building as to what works out well.

Most of the web apps we deploy have pretty limited usage and are closer to the prototype side of things. They are niche, custom, broadly educational-market things that have a lot of customized UI flourishes, but underneath everything are more or less standard data driven forms-type apps.

The indentation of Python that some people fuss about doesn’t bother me at all since I indent correctly anyway. Python’s methods often feel misplaced with the wrong objects to me, though.

The Ruby ‘end’ statements take a bit more space but Ruby optimizes in other ways such as with blocks and things that make the code shorter in other ways. It’s more or less a wash as far as I can tell for code brevity.

A lot of people I know and respect love Python and Django. I can’t tell them they’re wrong, but I’m personally much more productive in Ruby even after getting used to Python for a number of months.

Python performance is better than Ruby’s, but Rails 3.1/Ruby 1.9.2 feels much faster than previous older versions of Ruby and Rails to me.

I don’t think there is a clear winner for all situations in the Ruby vs. Python and Rails vs. Django religious wars, but I know I prefer and work much better with Rails than with Django even after some significant ramp-up time.

Stupid Brain Trick: Tests and Vertical Splits

Recently discovered a stupid brain trick that I thought was worth sharing.

When writing code with unit tests, I find it helpful to view the tests and the implementations side by side. I use vertical splits in Vim to do this.

The stupid brain trick is this: Put your tests in the left-hand split!

Why? I traditionally used the left-hand split for the “main” thing I’m working on and the right-hand split for reference. I’ve watched many other developers do the same thing. The right split is constantly sub-split horizontally and always changing to various different files.

In trying to be better about writing tests first, I created a test code file before the implementation file, so it just happened to be in my left-hand split. After an hour or so of development, I realized that the “main” file placement of the tests meant that I focused more on them. I viewed the actual implementation file as just some ancillary file I had to tweak to get my tests to pass. It changed the focus of what I was doing from “Write a method that does X” to “Describe X”. Describing something involves breaking it up in to smaller and smaller pieces until the implementations are so trivial that they basically write themselves.

Where Rails Rocks (and Where Python Doesn’t)

Here’s a list of things I like about Ruby on Rails. Almost every single one of these are available in at least one Python framework in some form, but the best thing about Rails is how polished the pieces are and how well they all fit together. The whole is truly greater than the sum of its parts.

Fantastic OOBE

Rails has a fantastic out-of-box experience. Simply execute rails new and you’re off to the races with a directory full of goodies that every real web app needs: dependency management, build system, multi-environment configuration, development storage, logging, custom error pages, robots.txt file and more. See section 3.2 of the getting started guide for a complete list. Hell, they even fill out a .gitignore file for you! A place for everything and everything in its place.

Dependency Management: RubyGems

In the Python world, easy_install and its ilk are all kinds of broken. You can’t even uninstall a package! Luckily, there is Pip and Virtualenv, but assuming Pip supports all your packages, you’ve still got to make sense of dependency freezing and manually setup a virtual environment. With Rails, that all just works: rails server runs in a virtualized environment and loads gems as defined by your Gemfile; rails console for a repl. No need to muck with sys.path! Any seasoned developed has spent many an hour in dependency hell, but it is still hard to justify the infrastructure investment before writing a line of HTML. Bonus points for the “vendor” directory, in case you need to patch a third party package (or Rails itself) and want to keep it under version control.

Build System: Rake”>Makefiles. A lot of people really hate them, but they are just so damn useful. Frequently, you need to transform some set of files into some other set of files, but only when they change. Makefiles help you do that reliably. They accomplish their feat explicitly by not being a programming language, but a dependency and build rule declaration language. I’ve seen a lot of attempts at build systems that aim to be idiomatic with respect to their host languages. Java has Ant and Maven. Python has Scons and others. Scala has Simple Build Tool. None of them are as good as plain old Makefiles. Then, I discovered Rake. Rakefiles rock. They are 90% Makefiles, and 90% Ruby. Rake expertly uses the features of Ruby to craft an API/DSL that looks a lot like Make declarations without compromising on general purpose programmability.

Markup and Style Sheets: Haml and Sass

Haml and Sass are so amazing, they ought to be illegal. Just click those links and check out the HTML and CSS comparisons. It is no contest. I never want to write HTML or CSS ever again. As far as I’m concerned, HTML and CSS are object code: to be generated by a compiler. Whenever I have to look at the generated code, it feels like I am reading assembly. Simply add gem ‘haml’ to your Gemfile, then start writing .html.haml files instead of .html.erb and .sass files instead of .css. Use html2haml and sass-convert to upgrade your crusty old markup and style sheets instantly.

Object Relational Mapper: ActiveRecord

I’ve discussed my preference for database schema reflection at great length, so I won’t reiterate here. I will, however, say that ActiveRecord makes simple things simple and complex things possible while embodying successful best practices in its various subsystems, like migrations. It doesn’t try to be everything to everyone, unlike SqlAlchemy. And it seems to be nicely onion layered for pealing back when necessary. For example, ActiveModel validations can be used on plain old Ruby objects. Which brings me to…

Form Handling: FormBuilder, params and mass assignment

I don’t need to learn and manage three different data marshaling techniques to map a database record to a web form. In fact, I don’t need to manage any. Schema reflection spares me in the model layer, FormBuilder/FormHelpers in the views, and the hash & array structured params in the controllers layer. Almost every user interaction boils down to interpreting a form’s worth of JSON-like data structures as an API call, implemented via a (secured) mass assignment to virtual attributes.

Templating: Layouts, Partials, and Helpers

When I’m getting my Haml on, I frequently want to run some non-trivial code. Feel free to shout about your separation of concerns religion, but I’ve got real things to worry about. Furthermore, we as engineers should be writing our production views/templates, not some hypothetical designer who is technical enough to write loops and branching statements, but not to be trusted with a function definition. Views are composed of markup and presentation declarations in the templates, presentation logic goes in helpers (which conveniently share scope with templates and controllers), and are neatly organized into partials. Simple, but effective.

URL Design: Routes

Rail’s routing system makes clever use of blocks to provide a hierarchical, declarative URL map that just makes sense to read. I used to design URLs in an indented text file and then map it to a list of regexps, but the routes.rb file is close enough to a spec and doesn’t require me to even think about regexes, ‘nuff said.

Authentication: Authlogic

Authentication is hard. Really, really hard. And boring. Really, really boring. But important. Really, really important. Everyone knows that the sign-up funnel is critical to get right and security is easy to get wrong. Most quality sites have a little bit of domain-specific special sauce in their authentication model. Trivial sign-up and log-in forms just don’t cut it. One size simply does not fit all for authentication front-ends, but the authentication back-ends tend to be pretty repeatable. Authlogic turns repeatable into re-usable, without the baggage of views you’re going to need to rewrite anyway. Thinking about authentication as CRUD operations on a user-session model blew my mind slightly. I had a full authentication system tuned to my needs up and running within two hours.

Authorization: CanCan

I hate ACLs. Role-based authorization simply has too much impedance mismatch for most problem domains. CanCan is a clever little library which makes zero assumptions about how permissions are represented. Simply declare a list of everything a user can do and how to tell if they may. Query with the can? function, or assert with the authorize! method. Beautiful.

Rope to Hang Yourself: Ruby Itself

One keyword in my previous post triggered much discussion: magic. The concept of magic is a topic for a whole ‘nother blog post (as I’ve already been hung a few times), but as several commenters pointed out: it’s all just Ruby. Between mixins, include, require, blocks, method_missing, symbols, and many other features, Ruby is a very capable internal domain specific language development toolkit. Most of the time, you don’t want a DSL. If you’re not an expert in both the host language and the DSL, there is almost certain confusion and maintenance danger. Most projects are either small or distinct enough to justify writing more verbose code that can more readily be understood by successors or even your future self. The gains of a DSL do not always outweigh the costs. Python excels at not supporting DSLs; it is basically executable pseudocode. However, when it comes to a task as common as web application development, it’s worth the time to develop or learn a DSL or two. Rails is loaded with DSL features and “magic” behavior that increases the learning curve, but also increases peak productivity.


These sites have been invaluable: The Ruby Toolbox, RailsPlugins, Railscasts. The quantity and quality of Ruby libraries on Github continues to impress. The educational infrastructure is extensive. Somehow, the Ruby community seems to consistently be at the forefront of the biggest trends in software development. Much like Rails compared to other frameworks, the community distinctions are subtle, but meaningful. For example, Rubyists championed Git while Python adopted Hg. To the untrained eye, they are virtually identical tools, but extensive experience with both has convinced me that Git is significantly superior. I’m new to the community, but it seems like Rubyists tend to insist on quality and bet on the winning horses.

Other Random Things

Just who the hell do I think I am?

I’m nobody. Just some opinionated computer geek working on a start-up. Not unlike many of you! My co-founder and I are currently participating in TechStars Seattle and hope to launch something killer come demo day, November 11th, 2010. We’ve got a grand vision for next generation enterprise collaboration software, but we’re starting small by focusing on making weekly status reports less painful and more useful. More details soon, but please visit Thanks!

Imported Comments


You’ll probably remove this too (at least you read like a fan-boy) but … you don’t show where Python is lacking at all in this article. I’m not going to start a war but TurboGears has an equivalent to almost everything you listed there (Although I’m not so sure about Rake), and I was looking forward to the “comparisons” your title implies.

Brandon Bloom

I removed the previous comment because the author deleted it himself which left a vestigial block in the comment stream saying that the post was deleted.

He wrote “Why would you compare a framework to a programming language?” to which the answer is: because I’ve tried dozens of libraries and several frameworks in Python. Then I tried Rails. Python as an ecosystem lost out to Rails as a particular framework community for my needs.

To your comment: I don’t really think TurboGears does. Or at least, they are not first class citizens. For example, the documentation on form handling presents several options include FormEncode, ToscaWidgets, etc. It requires that I create an additional schema class where I declare widgets outside of the templates — or awkward @validate decorators.


This is a poorly researched article. You’re comparing the language Python and it’s vast sea of tools to Rails the framework written in Ruby.

Pylons (which is also the foundation for TurboGears 2) provides all you have listed above, if not more. I’ve also noticed a strong tendency for Rails, Django, CakePHP, and many other “kitchen sink” frameworks to do much of the work for the developer and if I wanted to get around it – it becomes a day long hackathon of digging through docs, source code, and IRC logs.

Pylons, has its own quirks and dirty sides too; but I highly prefer having to spend a lot of up-front time programming/configuring my own framework environment “flavor” (and being able to use it across many different projects) then having it just be “Rails”.

Next time you post something like this, compare Rails to Django, or TurboGears, or Pylons; not just “Python”.

Brandon Bloom

@lxmatus: I am quite aware of the distinctions between a language and a framework. Thanks.

In this case, I am genuinely making the case that the Python web-development ecosystem as a whole has a story to tell which is less coherent and less attractive than the single framework: Rails.

I make this argument because I was able to use everything in this article without having to write a lot of glue code. I spent months researching and experimenting with Python libraries. Then I spent a week using Rails. Each of these things worked beautifully together without any heavy setup. The resulting complete stack aligned very closely with my preferences. And I’m a picky bastard: I tried a huuuge list of Python libraries before I found a stack I liked and I’m still not totally satisfied.

You can say “TurboGears has these” or “You should try” but that’s precisely the problem. If I install Ruby, install Rails, and then visit and choose the most popular plugin for each thing I need: I’ve got a highly pleasant, well integrated, stack in hours instead of months.


@Brandon … sorry for the accusation! I’ve watched far too many baseless arguments between the Ruby and Python groups, so maybe I’m a bit sensitive.

In any case, the example you gave is exactly the type of information I was expecting to find in the article. It’s very valuable to get the kind of comparative information you provide in your comment.

As noted by Ixmatus, I find both the frameworks to have their quirks … it’s the learning curve that you’re paying for when using either.

So I generally recommend TurboGears to people that already know Python and Ruby with Ruby on Rails for those that want to learn a 4th generation scripting language and use it with a RAD web framework. RoR definitely has the media hype and resulting community (or it could be the other way around).


Great list of reasons to love rails (may I add: migrations, UJS defaults, and restful controllers). The “python ecosystem” is a appropriate phrase and the whole “comparing framework to language” debate is out of context for this article. It is clear you taking the various frameworks and I-can-glue-this-library-to-this-one solutions into account in your support of the rails framework.

As far as rails doing too much work for the developer, that is the point. That is the whole point.


Horses for courses. Python frameworks have their fanbois too!

Bruno Cezar Rocha

You probably does not know web2py.

web2py has everything you mentioned in that list.

William Brian Smith

I’d recommend devise over authlogic. Devises is build around warden which is kind of an authentication framework of sorts. Allows for pluggable auth strategies and already has a lot of extras(fb, twitter, imap, cas, openid). Also in rails 3.1, you’ll be able to flush the buffer before the page is done rendering, which should greatly increase load times. It will also come with a built in sprite system and css compression

Brandon Bloom

@William: I followed the advice in comment #10 here:

I tend to want to customize pretty much everything heavily :–) Also, I also explicitly did not want any views/templates/etc done for me. However, I didn’t look at Devise closely. If I ever revisit our authentication, I’ll reinvestigate it.

Looking forward to those new features too. Thanks!

How Two Pythonistas Accidentally Fell in Love With Rails

One week ago, my co-founder and I found ourselves staring down a thick stack of paper UI prototypes; almost 50 pages total. One week before that, we threw out a perfectly good, working beta product because customer feedback suggested a significantly better path. The tale of how we ended up at that point is entertaining as well, but it will have to wait for another time. This is the tale of how our paper prototype turned a pair of long time Python developers (and documented framework haters) into Ruby on Rails developers.

Paper prototyping proved to be an extremely worthwhile exercise, but we really wanted something we could click around. We devised a strategy to build a clickable prototype that would evolve into production code. The plan was to stub out all of the views with static HTML and CSS. The CSS could be directly carried over to the product and the HTML could be retrofitted as templates. Having become religious about Sass about a year ago, I fired up the file watcher (which automatically rebuilds CSS) and set to work filling a directory with sass and html files.

After some time, I asked myself “I wonder if Haml has a --watch argument?” (it doesn’t) because Sass makes me hate Html’s verbosity. Armed with practically zero Ruby knowledge, I hobbled together a simple Rakefile and Haml watcher script and got back to work. A few Git pushes later, my co-founder, being the clever bastard that he is, simply installed Rails and the Haml plugin to replace my hacky scripts. “Neat”, I said and thought nothing of it.

Then we designed our URLs in an indented plain text file, which pretty much directly mapped into the routes.rb file and some trivial (empty method) controllers. Whoops! Now we’re writing Ruby.

“At this point, we might as well try Rails proper.”

Generated some models and corresponding migrations: “Wow, explicit schema with reflective model objects. None of this declarative schema bullshit.”

Wired up some controllers. “The structured params infrastructure is pretty cool.”

“Ruby is kinda fun. Blocks are super useful.”

Integrated Authlogic. “Wow, that was easy. I really like how it does not force any front-end decisions on me.”

“Rails is full of evil black magic that I do not understand, but it seems to read my mind, so I don’t really care.”

Started to apply some polish and work on the 10% that takes 90% of the time. “I think I can safely add Ruby to my resume now.”

Late at night, at the very end of the week, exhausted, we stared at each other. “Dude, I think we can show this off to people tomorrow.” Walking out the door of the office, my co-founder said to me “I feel so dumb for having ignored Rails for so long. I just assumed it was Ruby’s Django”.

We accidentally implemented our entire beta product in one week, with zero prior Ruby or Rails experience.

When I have some more time (and experience), I’ll try to write about specific design decisions that set Rails apart.

Imported Comments

Veezus Kreist

Great story! I love the black magic quote – but I think it will feel more apropos when you use rspec as a test framework.

Joe Lallouz

Sweet. A very similar situation happened when my co-founder and i switched to Groovy and Grails.


How did I never notice you on Hacker News before? I read it every day!

Andrew Ingram

Weird, I moved to Django because I was fed up with building sites using Rails :p

I guess everyone has their own preferred style.

Eric Floehr

The best language for you is the one in which you are most productive in, so awesome!

My first public website was in Rails, and I assumed Django was just Python’s Rails.

I have since converted it to Django because Django allows me to be more productive, and ActiveRecord didn’t work well for me.

Best of luck, and congrats again for finding such productivity!


lol, do you always call your co-founder a bastard?


Did you give web2py a try? I believe it provides in Python what you found in Rails.


“lol, do you always call your co-founder a bastard?” I’m guessing he’s in the UK :)

Brandon Bloom

@Dan: I haven’t been hiding :–)

@Andrew: I think that’s why these topics are such hot issues. It is more preference and religion than anything else. Django is a solid piece of code and a stellar community, but it just doesn’t mesh with my brain.

@Rives & JonnieCache: I called him a clever bastard. I consider that a complement. And no, I’m not from the UK. I’m from NY, he’s from Philly, and we are currently located in Seattle.

@Max: I’ve played with web2py, but wasn’t super impressed. But hey, I thought the same thing about Rails. I guess you need to really try to make something to understand. What about it is interesting in particular?


Just out of interest, which version of rails are you using?

Brandon Bloom

Rails 3


And here I am going from Rails (because of black magic :) to Pylons, which took a lot from Rails, but its in Python!

Perhaps you should give Pylons a look over, just some food for thought.


I originally began programming in Python with Django myself, but I saw Rails and immediately went.. WTF!? to Django and Python in general.


I think the real lesson here is to find the right tool for the right job. Both Rails and Django are pretty strong pieces of framework, allowing you to do a lot in a little amount of time.

I had a similar experience, where I quickly prototyped a website in Rails. I showed it off and said, “see…this is pretty much done.” And then developers on my team felt the need to rebuild the site in PHP since that’s what our company ‘officially supports.’ :)

On another project, I was able to quickly use the Django ORM to introspect an existing MySQL database, and build an admin interface for some of our internal users. That was a HUGE win for me to be able to quickly make the admin site.

So, the right tool for the right job…



Given your article I thought your would like the fact that in web2py everything has a default. For example is you just type:


web2py creates the table (if it does not exist, or alters it as necessary), a web based database administrative interface, and crud.create, crud.update,,, forms that you can insert in your app like

It also includes role based access control out of the box with pluggable authentication mechanisms.

Web2py also has a component/plugin architecture that is best described by this video

Since you are coming from Python, I though you may be interested. Anyway, I too really like Rails and web2py was originally designed to be in Python but closer to Rails than other Python frameworks.


Semantic indenting is far and away the best thing about Python, and Haml/Sass prove Ruby should adopt it too.

Brandon Bloom

@brokenladder: Hell yes.


Well in my case I am way more productive in Python, especially when prototyping with Django. It all depends on the mastering level of the tool. One point I will give to rails is haml tho. We need a proper equivalent well integrated in Django/Python.

Rafael Rosa Fu


Next time you need to authentication, try Devise instead of Authlogic, and check Ruby Toolbox for more interesting Ruby tools.



Could you expand on your file watcher script? Running a background process (e.g. re-compile) on change is a common pattern that I haven’t found a tidy solution to…

Brandon Bloom

@kevin: I’ve written a bunch of watchers in the past. Each had their pros and cons.

Steve has a simple Python script here:

However, in this case I used FSSM:

FSSM has the advantage that it uses the platform’s native file notification mechanism, so it tends to respond more quickly (no polling delay).


Awesome post! I’ve been comfortable with Django though I am dabbling into Rails but I’m having trouble with getting to grips with Ruby. Would you recommend anything you’ve read that allowed you build a working prototype? I’m curious about how you managed to learn it so quickly. Thanks!

Brandon Bloom

@TT: I took advantage of the fact that many people switch to or from Python. I searched for phrases like “python’s foo in ruby”. Ruby is so similar to Python, it was pretty easy to just pretend it’s the same and go back for cleanup as you learn the idiomatic approaches.

Also, I generally just have an interest in programming languages. I’m pretty fluent with about a half dozen languages, so each marginal new language is that much easier to pick up.

As an aside: the “end” keyword is just sooo unnecessary!

NoScript Add-on Install Base

The official NoScript site downloads via the Mozilla Add-ons page, so its download count is probably accurate: 67,616,402

NoScript does not publish their individual add-on usage statistics, but the global download/usage ratio can be calculated from the statistics on the Firefox Add-on home page:

About 8% of downloaded add-ons are still in use. Assuming NoScript’s usage ratio is comparable to the average, approximately 5.4 million installations of FireFox are running NoScript. Let’s ignore the fact that NoScript’s usage ratio is probably much lower than the average, due to the fact that it breaks most web pages.

I’d wager that the average NoScript user has at least two machines, so the total number of NoScript users is probably less than 2.7 million.

There are over 230 million internet users in the USA.

Even if 100% of NoScript users were Americans, they form 1% or less of the general population. If you, like me, believe that I have been generous to NoScript here, it is likely that no script users are no more numerous than 1 in 1,000.

Even if the user is using NoScript, they can whitelist your site. You can probably add <noscript>WARNING: THIS SITE IS BUSTED WITHOUT JS</noscript> to the top of your page and call it a day. If you are feeling generous, redirect no-script users to the mobile version of your site and tell them why.

tldr: Assume that human user agents have Javascript.

Imported Comments

Anthony DiSanti

What’s the inspiration behind this post? Did you think you actually needed to support a strong noscript experience? JS is mandatory at this point, screw anyone that’s turning it off.

Brandon Bloom

Facebook’s BigPipe tech announcement prompted some Javascript-required discussions on Hacker News.

ORMs and Declarative Schemas

My prior post was more controversial than I anticipated. In hindsight, I should have realized what a hot button issue web frameworks are. One assertion went all but unnoticed. I expect it may be even more controversial:

the schema-generative ORM paradigm is fundamentally flawed.


I came to this conclusion while working with Django’s ORM, but this post is completely object-relational mapper agnostic. We are now using SqlAlchemy, but we are not using any of the many available declarative layers. Instead, we are using schema reflection and semi-automatically configured mappers. This is not an argument against ORMs. It is an argument against generating database schemas from ORM declarations. By extension, this is an argument against Django’s ORM because Django uses an exclusively schema declarative model. That said, Django’s ORM is far from alone in this camp.

Data Outlives Code

When code is dead and gone — be it through rewrite, obsolescence, or by other means — the data will still be there. Longevity implies slower evolution; data is always more difficult and riskier to change. Data is also more valuable. What if Facebook rebooted their database? They’ve already rebooted their software several times.

Schemas are data. As data, schemas are longer lived, less flexible, and more valuable than code. These factors alone suggest that the database itself should hold the authoritative schema, not a class declaration in the code.

If you have inherited data from another project, you already know this lesson. You can’t generate the schema from code because the schema already exists. You can mimic the authoritative schema in your declarations, but it is easier and more accurate to use reflection.

Ineffective Domain Objects

Object relational mapping is primarily a serialization problem. Every serialization solution has its quirks. The scale and number of quirks seem directly proportional to the absolute difference between the runtime and storage representations. Since a database is a completely distinct type system, rather than an opaque byte array, serialization to a database can have particularly quirky quirks.

Modeling is one of the fundamental challenges of software development. Capable developers prefer highly expressive or unconstrained type systems to aid with modeling. Generally, runtime type systems are more expressive than those found in databases. Declarative relational mappers, however, constrain the programming language type system to its less expressive counterpart. When building domain objects, the developer must think in terms of the database’s type system, not the programming language’s.

While using a declarative object relational mapper, developers are effectively trying to design storage and runtime models simultaneously. On average, superior results are achieved by modeling these two concerns separately and then solving an additional subproblem: serialization mapping. You might wind up with slightly more code, but it will be easier to understand and maintain.

SQL Is Not Going Away

Despite forcing a less expressive type system on to the developer, declarative ORM layers attempt to treat the database as an implementation detail. However much we wish this were true, the database is not a detail which can be ignored. Sooner or later, you are going to have to open your database shell and write a SQL expression. This requires knowledge of your database’s particular SQL dialect and idiosyncrasies. Exacerbating this issue, generated schemas are typically full of name mangling and other ugliness. It is far more pleasant to work with a carefully designed schema than one that compromises for the ORM or the runtime type system.

Schema and Declarations Diverge

Data migrations present the most pressing need to work with SQL directly. As an widely unsolved problem, automation can not be trusted. Unless you are painstakingly simulating the schema generator, the production database schema will slowly diverge. Most deviations are tolerable as they will not affect the runtime behavior, but it is wise to minimze differences between production, staging, and development environments. To faciliate this, store migration scripts and backed-up schemas in version control.

Some deviations will directly affect runtime behavior. For example, consider the case where two versions of an application are running in production. An is_read boolean column was added to a message table in the database. It’s default value is false. When a row’s page is viewed, the is_read column is set to true. The old version of the application doesn’t know about the column, so it can not set the flag. When the new version is rolled out to everyone, affected user will see a bunch of read items marked as unread! The solution is to set the column’s default to true, but initialize it to false in the application. Declarations must either deliberately deviate from the schema or present a misleading default value to be overridden during initialization. This is just a simple example, real world schema migrations can be significantly more complex and suffer from numerous more subtle problems.

Imported Comments

Martin Diers

These are all good points, and largely irrelevant for someone developing to Django. Yes, if you are using Django to develop to a long-term data store, enterprise data, etc., you are dead right, and you should be using SQLAlchemy with Django instead of the Django ORM.

However, consider Django’s original target use: a newspaper, on the cutting edge, putting up quick apps on-the-fly with a very short development cycle. A CMS. A Polling app. A Blog. Think about the type of apps that people actually build with Django, and you will find that the Django ORM makes perfect sense. Short development cycle. Easy deployment. “Good enough” query optimizations.


I agree with Martin, Jacob Kaplan-Moss himself said that, when you get to a point where you’re building something really big (e.g. Twitter) you basically have to throw your framework away.

Django is really good, but, like any technolgy, sometimes is not enough.

Brandon Bloom

@Martin: The “original target use” argument is no longer relevant. Django is being pushed as a general purpose web framework for developing sites or apps, big and small.

@Ezequiel: I’m not building something Twitter big. I’m building something that has to deal with real user data and survive real design evolution. The problems I describe affect any application with any longevity of any kind.


I agree I’d rather go schema first than model first. As a side note though it’s interesting to think about it in the context of the whole NOSQL mouvement that’s been gaining a lot of traction over the last year. Facebook, LinkedIn or Amazon for some subset of its data and others are doing away with the relational stores and going with pure key/value pairs persistence.

Not relevant for every application but very interesting nonetheless.


Have you ever considered something else than django? Like pylons or repoze.bfg? Django is really not so good solution for more complicated tasks if you want to work efficiently.

URL Routing and Views

As promised, I’d like to elaborate on the URL routing system I came up with.

Weighing at less than 200 lines of code (including example), I’ll let it speak for itself: download it.

This approach seems to be working great for us. Love it? Hate it? Feel free to let me know what you think.

Imported Comments

Dead Man

You are hardcoding child views in the children method of each view which means the view can’t decide what (type of) children it has based on the current path segment. For example, I have a use case in which some content types can contain children of more than 1 type, so for example /foobar/child/ can either be a TypeAView or TypeBView depending on whether a TypeA with name “child” can be found or not – if it cannot be found, TypeBView will be returned. If your children() method took a path segment as an argument, it would probably solve this problem.

BTW have you checked out how Restish resolves/dispatches URLs? I think they have it right when it comes to dynamic URL trees.

Brandon Bloom

Child views are not “hard coded”. They are determined by the children method. You could pass segment, but it would be easier for the bind method to just store something on self for use later.

I looked at Restish a bit. It seems like a nice library; definitely the closest to my taste from what I’ve seen.


I’d recommend taking a look at Werkzeug for your routing & other wsgi basic needs.

As a follow-up to the previous post, Jinja 2 makes for an excellent templating engine (think django but much faster and without the stupid limitations).

And as a shameless plug, have a look at WTForms for your forms:

Andriy Drozdyuk

I am sorry I still don’t get how this serves as an improvement over Django’s url system?

Can you provide me with a concrete example of what you cannot do in Django in terms or url-routing?

Andriy Drozdyuk

in terms of* I meant. Spelling is not one of my strongest areas eh?

Brandon Bloom

There is nothing that Django’s routing can’t do. In fact, the first version of this simply generated Django routes. This is about making is faster and easier to define hierarchical permissions and trees of URLs in a way that queries the database on a per-url-segment basis.

Dropping Django

My co-founder and I built a non-trivial web site on Django. When the next version ships, there might not be a single Django module imported.

We’re not trying to drop Django; it is just sort of happening. Piece by piece, it is failing to meet our needs. Despite the marketing copy on the Django site, most components of the framework are tightly coupled enough to make customization frustrating. It is often easier to rewrite core framework components than to implement them on top of the existing extensibility points.

What follows is a loose chronology of our migration away from Django.

URL Routing

A flat list of patterns violates the DRY principal when creating nested URLs. Trees are a superior representation. Having a tree of views also enabled us to optionally associate a “binder” function with each node. These bind functions are executed for each URL component from left to right, filling the template context as they go. Breadcrumbs are automatically generated as each node binds, but only the last node executes its full view logic.


Our site enforces permissions on every resource, but Django’s database ACLs would have been prohibitively numerous. Instead, views or their URL binders may raise an AccessDenied exception. Upon catching such an exception, a middleware layer serves a login form. This ensures users have permission to access the current resource, as well as all ancestor resources bound to the URL.


Both of Django authentication’s key extensibility points are flawed. These two extensibility points are “user profiles” (storing additional per-user data) and custom credentials (such as for logging in via email address instead of username). Django’s documentation and numerous internet sources cover both topics, but all of the guidance lacks important caveats. The admin UI, in particular, is very easy to break with either extensibility mechanism.

Extending the User model with the ORM requires a one-to-one database relationship. This relationship can be implemented with a “user profile” setting, an explicit foreign key, or model inheritance. Each approach has its own strengths and weaknesses in terms of performance, API semantics, subtle behavioral changes, and outright bugs.

Enabling custom credentials requires implementing a trivial authorization “backend” object. Unfortunately, it is non-trivial to replace usernames with email addresses. The admin UI’s login form refuses to accept email addresses without hacking the template. Even if you hacked the template, the User model would still enforce a non-null constraint on the username field and the generated database schema enforces a uniqueness constraint as well. It turns out to be easier to fill the username field with a dummy value and “support” both forms of authentication with your backend, but you won’t come to that conclusion until your head has already bore a hole in your desk.


We do our best to keep view and template logic separate. Django’s templates are targeted at designers, who aren’t implementing any real logic anyway. However, we’re a pair of hackers. Sometimes it is just more convenient to put a little bit of logic in the views. Besides, templates are code; code needs to be reviewed and tested. We wouldn’t ever hire a designer who couldn’t pass a code review for some trivial template logic.

We needed a pragmatic template language to replace Django’s idealistic one. Any template language with greater expressive power would have been welcome, but Jinja2 fit the requirements and provided the easiest migration path. Ultimately, we’d prefer to use something like HAML, but there doesn’t seem to be a Python equivalent besides the inactive GHRML project. We are, however, using SASS. I will never write CSS by hand again.

ORM and Admin UI

One of Django’s most touted features is the Admin UI. For simple “active record” style database models, the Admin UI is a huge time saver. Sadly, it struggles a little bit with nullable fields and is tricky to customize. You’ll definitely need to write custom UI for complex models, but by and large the admin solves the problem at hand: viewing, creating, updating, and deleting database rows.

After using the Admin for a little, I found myself missing Microsoft Access. I never thought I’d say that, but it is true. Django’s admin does not support sorting, filtering, or other impromptu queries. Edit: It turn’s out I was mistaken about sorting and filtering, but I stand by the core message of this section. I found myself writing impromptu queries in the database and Python shells. After a while, I just gave up and installed a desktop client. I haven’t visited the Admin UI since.

Django’s ORM has shortcomings with respect to querying, especially for joins and aggregation. It has been improving over time, but it will likely never reach the capability of projects solely focused on databases, such as SqlAlchemy. With the admin having fallen into disuse, the Django ORM lost all advantage. Beyond Django’s specific weaknesses, I’ve come to believe that the schema-generative ORMs paradigm is fundamentally flawed. That is a topic that deserves an entire (Django-agnostic) post of it’s own. We are now using SqlAlchemy via schema reflection; no declarative layer.

Form Validation and Generation

Here is where our chronology meets present day. We are still using Django form validation, but never used form generation beyond scaffolding. Nearly all of our templates customize labels and display of errors. Additionally, embedding widget information in the Python code is cumbersome during template development. Django forms is a quality validation library, but there are some inconsequential style things that I like better about FormEncode. Preferences aside, the difference isn’t large enough to justify switching.

While I like FormEncode, I’m still not sold on its anti-form-generation companion, htmlfill. I think there is a middle ground with form generation that provides scaffolding during development, smoothly transitions to production use, and cooperates with validation. As we implement more complex client views, I’ll be on the lookout for ways to improve our form development toolbox.

So, ugh… What’s left?

Besides a few isolated helper functions, not much is left of Django.

The last big ticket item is the HTTP framework and WSGI server. We could continue using Django as if it were CherryPy or Paste, but Django has this nasty habbit of insisting on running your code for you. The settings and infrastructure are fiddly for deployment and don’t really add any value over simple scripts using our application like a library. Might as well use a simpler WSGI library, and replace those over-engineered management/commands/ files with vanilla scripts/ files.

Moral of the Story

I’m sure there are numerous lessons to be generalized from this journey. Personally, I’ve developed a moderate fear of the word “framework”, as well as altered the way I think about software abstractions. I think the most important lesson, however, is one I already knew: choose the right tool for the job. Unfortunately, we had no idea what the right tool was when we started. I’m not sure we know any better now.

Imported Comments


Maybe it worked out exactly like it should have. Django bootstrapped your app to a certain point. Got you further faster than you would have if you implemented everything from scratch. Then from there, you identified the things you considered inadequate and replaced them. If it all goes away who cares. You have learned something, shared it with us and moved on.

You seem to have successfully moved away from it to your own custom code. I am impressed that it sounds like your decoupling your app from Django was not as traumatizing as it might have been.

Brandon Bloom

Yeah, everything seemed to work out OK. Django was definitely a big help, especially with my initial limited web experience (previously, I was a game and desktop application developer).

If I were to do it all again, I probably wouldn’t choose Django. That’s not to say Django wouldn’t be the right choice for someone else. I just think that I’d choose a collection of tools that are less likely to get stuck glued together.


I think Pylons would be a great fit for you. It matches the way you want to work.


Your URL routing sounds intreging. I like have been looking at Werkzeug, Routing and Django. I prefer Django to all three, but still I feel like something is missing.

Could you elaborate on your solution?


Take a look at Werkzeug. Despite the weird name is the most modular and yet easy to use wsgi wrapper and dispatcher I know. Combine it with Jinja2 and SQLAlchemy… and voilà.


Good thing there are people like you who would share your improvements back to the community!

Brandon Bloom

Thanks Juan, I’ll take a look at Werkzeug.

Eric: I’ve been looking at Pylons, but I think I want to go with something more like Juan’s suggestion.

Varikin: I was able to implement my URL routing over Django by simple matching (.*) and then writing a view which splits by ‘/’. I then loop over each component and perform a depth-first search of a tree structure. Each node has an optional bind(request, template_context, path_component) function. After I find the node which matches the path, I call the ancestor chain’s bind functions and then finally the current node’s view function. We’ve got some decorators and helper functions to make building this tree easy. I’d share it, but it is too tightly tied to our application. Maybe I’ll write more about this later.

WhatDoYouThink: My improvements are application specific. I’m sharing my experiences to help the community. That is far more than most people do.


This comment has been removed by the author.


The tree stuff and permission scheme you’d like perfectly matches up how repoze.bfg works with traversal and security based on traversal schemes (also known as object dispatch).

I’d highly recommend giving it a shot as it seems to mesh quite well with the way you’ve laid out the app.

Tucanae Services

Think Web2Py…..

Chris Shenton

Ditto what gaspode said. I’ve been fighting Django for an app whose content is basically hierarchical and BFG’s traversal would really be a relief.


I’d like to add my 2 cents. Restish has no threadlocals (compatiable with eventlet/spawning) and focuses on resources/content negotiation. I believe the URL routing is similar to what you want


Enabling custom credentials requires implementing a trivial authorization “backend” object. Unfortunately, it is non-trivial to replace usernames with email addresses. The admin UI’s login form refuses to accept email addresses without hacking the template. Even if you hacked the template, the User model would still enforce a non-null constraint on the username field and the generated database schema enforces a uniqueness constraint as well. It turns out to be easier to fill the username field with a dummy value and “support” both forms of authentication with your backend, but you won’t come to that conclusion until your head has already bore a hole in your desk.

We wanted to let users log into our app using their email address through contrib.auth.view.login. We don’t care about having email addresses log into the Django Admin.

You have to dig around a bit, but it can be done with a few lines:


I think you were looking for Pylons not Django. Each has its own advantages. I learnt that Django should be taken as is, once you start stripping it apart, IMHO you should consider using Pylons.


Sounds like you should look at is a nice microframework which stays out of the way as much as you want it to.

Jonathan Ellis

If you like SQLAlchemy, you will probably like FormAlchemy for form generation from SQLA models.


I’ve written my last few web apps on top of Berry, which just lets you map URLs to methods (and lay them out however you want). And obviously you can import SQLAlchemy, Jinja, and whatever else you want yourself. I’ve found this gives me the most flexibility possible without actually writing a raw WSGI app.

Michael Galpin

You really need to read this It will help you understand why you were doomed from the beginning :–)

Brandon Bloom

Thanks Michael. Despite the distracting laughter that was an interesting read.


After using the Admin for a little, I found myself missing Microsoft Access.

it still happens to me every time i start a new data-entry project on the web. sometimes i think something went wrong in the latest 20 years…

Dead Man

I’m also building a project on Django (a medium sized, multi language web encyclopedia about a Northern-European state Estonia) and I have to admit that I’ve run into mostly the same problems. What surprised me the most is that I’m implementing a nearly identical to yours solution for URL mapping as our content is also in a (generic) tree structure. The difference being that I’m going to use class based views that allow nesting/composition of other views for the purposes of chaining that you described.

I’m also thinking of open sourcing the code as the project matures more as it’s already written in a clean way and decoupled from project specific code.


Beyond Django’s specific weaknesses, I’ve come to believe that the schema-generative ORMs paradigm is fundamentally flawed. That is a topic that deserves an entire (Django-agnostic) post of it’s own. We are now using SqlAlchemy via schema reflection; no declarative layer.

Well you caught my attention with that. Are you planning to write such a post in the near future, say in the next two weeks?

Andrew Shultz

One of the big things I appreciated about django is the unit test supports. What’s your plan to replace that stuff?


Almost all of your technical feasibility issues are easily addressed by Django. URLs, authN/authZ, etc are all as flexible or more than any other python environment out there because it is so simple to leverage python to implement this things in an incredibly simple manner. This blog entry demonstrates it better than I could. It’s really just a matter of perspective and context. Django may not be the context that best fits your perspective but it most certainly has the capability and power to address all the technical concerns you brought up. Good luck finding the one that fits you best. I actually thought I’d be a TurboGears guy but ended up Django. Both are great and others are good as well.


I am working on building a medium size application in Django too. It is indeed a good way to start. But I am hitting issues too. I have multiple “agencies” so need to have users with multiple roles in agencies. These are within single site. That also means multiple “admins” with one “super admin.” So Django Admin interface is out of the question and role/permission infrastructure is built from scratch. Another thing is importing data. I started writing scripts within Django framework (using, etc.) but need some tables/entities created during import and then dropped. Oops. Have to look at how South does it.

I realize that I am moving out of Django domain just because of the nature and complexity of the application I am building. Django cannot and should not be everything for everybody.

If you are looking for a “glue” framework, I recommend checking out Cherrypy: I like its configuration and request/response handling better than Paster + WebOb. It also has several types of routing, pick what you like or mix it (I would not recommend the latter though), one of them is Routes.

My next move will most likely be to Cherrypy + SQLAlchemy + FormAlchemy + Jinja2.

By the way, repoze.bfg is not microframework either, it pulls quite a large chunk of Zope code, primarily for managing configuration but for templating too.

RJ Ryan

Got a few issues with the things you said — I’ll list them by section.

URL routing is not a flat list. You can make them hierarchical. How do you think all of the admin site routes get added? It’s in the template for any starter project, so I really don’t see how you missed this.

Authorization — I don’t think I understand what you are saying is wrong — “too numerous?” But designing a permission system using the Django auth system has usually been pleasant for me. To require certain permissions just use a decorator on your views which checks for it.

Authentication. There’s nothing that says you have to make a one-to-one with User and whatever profile system you have. That’s just how most people do it because most projects want 1-1. Make a ManyToMany from your Profile to User if you want. There’s nothing stopping you. Also, using email address as your login was simple — swap the auth backend. It’s like 10 lines of code.

Templating: Fair enough, I’ve wnted the templates to allow a little bit more in the way of doing logic only for purposes of output, but you realize if they make allowances for that, then it opens the templates up to abuse — and then you get PHP. I’ve actually come to appreciate the template strictness now, and when I come across something I just have to do in the template, I write a templatetag.

ORM/Admin — I can’t really understand what you’re trying to say here. The Admin UI is a very nice piece of software, and it is extensible at every point. Though, the documentation on doing so is skimpy. Take a look at Satchmo — they do amazing things to the Admin UI that really showcase its flexibility. As for the ORM — if it really doesn’t suit your needs, upgrade to django-sqlalchemy or something.

Forms — it’s true forms are sort of a go-between between views and templates. It does seem odd to specify the widget in the form declaration. Shrug, though the form is more of a prototype of a form, so you design the prototypical form that is instantiated in a view. This makes sense to me.

I’m sure you actually did run into problems with Django that caused it to be frustrating for you, and to that I say ymmv — I just couldn’t help reading your post and being like “but that’s not true, because I’ve done X and he said it can’t do X”.

Shantanu Kumar

Did you consider Jython? You could then probably use any framework from the Java-space.


Faced same problems with Django during 2 years of its using. Looked also at Paste,, Werkzeug, Pylons, WebOb to replace over-weight framework with more flexible tools. My current best python tools is Werkzeug, Mako, SQLAlchemy.


Alexei said> By the way, repoze.bfg is not microframework either, it pulls quite a large chunk of Zope code, primarily for managing configuration but for templating too.

I guess we will have to agree to disagree. By alexei’s definition 20 packages isn’t ‘micro’. By mine 4 packages I wire together isn’t a framework…but to clarify some misnomers:

repoze.bfg is a microframework. it consists of 20 packages, 8 of which are zope packages. Although, BFG uses them you never have to.

The default templating package implements TAL, but it is NOT a zope package. You are not bound to it either. You can plug in Jinha2, Mako, Genshi or whatever you want.

There is no default database, form library or anything else.

FWIW zope is about 170 packages. 8 is not a ‘large chunk’ of that by any definition.


If you’re looking for a HAML Python equivalent, there is PAML (Pamela) at — also, you can check CleverCSS as a SASS replacement.


I totally agree with RJ Ryan. No framework will suit every project, but Django covers a lot of the basics in a way that’s good enough and doesn’t get in the way when I need to extend it. Still, thanks for telling us why it wasn’t the tool for you.

Tony Landis

I use pylons + mako + sqlalchemy and love it

Brandon Bloom

@Dead Man: I’ll probably publish something about our approach to URL routing soon. It seems that several people have shown interest in it.

@ars: Yes, I will probably be writing about declarative ORM layers in the next week or so.

@Andrew Shultz: We’re not doing a whole lot of automated testing yet. Small helper functions typically have doctests, we’ve got a couple unit tests, and we’re currently playing with Selenium for full tests. I’m much more confident in tests which drive the UI than tests which only poke at the views.

@proteusguy: I just read that blog entry and will respond to it separately. Again, I’m not bashing Django so much as I am saying we’ve been slowly choosing tools that better fit each need as we identify those needs. Technical feasibility aside, we’ve been looking for a path of least resistance to accomplishing our goals and this post was just to describe them.

@Alexei: That’s the impression I got too; Django would have had greater longevity on my tool shelf if I needed a highly customizable CMS. I don’t need more flexibility, I need less constraints.

@RJ Ryan: Like I said about, I’ll try to follow up with more info about our URL routing system. What needs we had and why I felt I had to subvert Django to get them.

@Shantanu Kumar: There are already too many choices of Python frameworks! :–)

@Sebastien: Thanks for the PAML link! I’ll take a look at it. As for CleverCSS, I don’t see a need to switch from SASS. We only need Ruby and SASS installed on dev machines, the server only ever works with the compiled CSS.


“Beyond Django’s specific weaknesses, I’ve come to believe that the schema-generative ORMs paradigm is fundamentally flawed.”

I too would like to see some more detail on this. I maintain a Ruby ORM called DataMapper, and while the API is completely different, it is still declarative. I prefer the declarative style to ActiveRecord, since I can specify rich types and constraints than I could reflect from a database. The way I see it, at some point the schema needs to be defined, and I’d rather do it in a richer DSL and have the schema, validation and other constraints generated from that.

Even so, I’d love to hear a different view point.


A somewhat dated (and biased) comparison of Python/Django with Ruby/Rails.


This is a subset of the reason for why we made web2py. Thanks for explaining them so clearly.

Ian Lewis

pylons is great being a framework that allows mixing and matching libraries.

My ideal would be

I would probably build on Django’s idea of applications and middleware as that’s a good paradigm.

Brandon Bloom

For anyone interested and tracking this thread, I’ve posted a follow-up:


You can have non-flat URL routing. After your first point demonstrated a failure to grasp more than the basics before whining, I tuned out.

Nested URL routing confs and includes (new in 1.1) are what you should have used instead of bitching on the internet. It wouldn’t have made you seem as cool, but you would have gotten the job done instead.

Brandon Bloom

Without the changes in Django 1.1, modules were the only recursive mechanism; too heavyweight for my taste. The 1.1 beta was released in March. I had developed my own solution to this problem several months before then.

Even with the 1.1 improvements, Django doesn’t meet my other requirements: namely binding URL segments to the database and creating breadcrumbs while filling a template context dictionary. Since that work involves parsing request.path myself, I might as well just bind (.*)+ and do my own routing. It was relatively trivial.


Shawn’s AppWeek post inspired me to write one too. AppWeek is our chance to be creators for a little while and it was a lot of fun. I didn’t set out to build something nearly as ambitious as “Super Avatar Sample Smashup EXTREME! – ‘Capture the Cat’ edition”, but I did get to take a swing at a game I’ve wanted to build for a while: Rock’em Sock’em Avatars Avatar Boxing. Avatars, being a new feature in this release, were an unwritten requirement for all of the AppWeek games. Between SASSECTCE, my game, and the many others, Avatars were chasing cats, beating each other up, play futuristic sports, falling off buildings, dancing in a cloud of gems, being launched from canons to save the world, and much more. All this excitement was almost too much for a bunch of exhausted engineers, but that’s what the beer was for during the game unveilings.

Here’s what the game looked like with the basic animations wired up. You’ll notice that the avatars have been hitting the gym. That’s because their arms were too short to reach each other! I added a little extra bulk because I was laughing too hard not to. I directly bound the game pad triggers to the shoulder and elbow joints and rigged up the chase camera sample to inspect my work. There wasn’t much game play yet, but it was already fun. That’s always a good sign.


Even with just one week, I decided to invest some time into debugging visualizations. That turned out to be a really great idea.


Then, I added some collision spheres for the heads, hands, and upper bodies. This was a hacky, trial and error process. Thankfully, C# compiles quickly.


At this point, I spent an entire day working on the physics. I wanted the avatars to bounce/wobble when they got hit, so I rigged up some complex spring systems. Things were starting to work, but I’m generally pretty bad at this sort of thing and my simulation routinely exploded. The avatars arms went shooting off into space and I was getting pretty frustrated. No screen shots of that chaos because I am embarrassed.

With half a day to go, I added the obligatory damage bars and some rudimentary hand-to-head collision detection.


I was feeling pretty good about the game, despite my physics failures, it was pretty fun anyway. I wondered down the hall to chat with Jace, who had just added sound effects to his game. His game was hilarious before, but the sound effects were priceless. I ejected the sound effect CD out of his machine, yoinked it, and took off running. An hour later (and 10 minutes after the deadline), my game had some sweet punch and miss sounds. I also made the avatars’ heads pop up when their damage bar was full, accompanied by an awesome zip-tie sound.


At our team happy hour, I’d like to think Avatar Boxing was a fan favorite. I certainly had fun making it! I hope everyone enjoys Avatar support in the new XNA Game Studio.

Imported Comments

Danny Tuppeny

I really wish you guys were releasing these games – they sound so funny and it would be great to see how you did some of the stuff with Avatars.

Brandon Bloom

As Shawn said: “We did an AppWeek shortly before shipping Game Studio 1.0, which produced minigames by Dean and Minjie, plus a bunch of stuff that was too unfinished or too much of an IP violation to release.”

Believe me, the code for my game isn’t remotely useful :–) Hacks upon kludges upon hacks galore. It would be a significant chunk of work to turn it into something educational. Even if it was interesting, it is technically not mine to release. It would be a conflict of interest to publish it on Xbox LIVE Indie Games, since I created it during the course of my job.

Since our games are only being seen by the team, many developers use copyrighted assets in the interest of time. Fair use allows us to show them at our happy hour, but someone would certainly come knocking if we released them. It is very expensive to clean that up because now you need to involve artists and rework the code for the new art (AppWeek games aren’t robust, art changes are breaking).

Everyone would love to share, but it isn’t universally practical. Luckily, several cool Avatar-centric samples were inspired by app week, so we will have something to share. I’ll also see about recording some videos of the IP-safe games…

Danny Tuppeny

No worries, it makes sense. I’m sure we’ll see more Avatar samples soon that aren’t hacked together we can learn from :-D


This looks like it could be turned into a real fun game. It’s too bad that we can’t buff our Avatars up like this =(

Brandon Bloom

Yeah, sadly it appears that scaling individual body parts is against the rules (which were unwritten when I did this project). I believe the motivation is for accommodating future avatar accessories which might look awful with unknown joint transforms. Don’t quote me on that :–)

PowerShell: Condemned to Reinvent

I tried PowerShell when it was first released, but never used it for real work. I recently attended a “brown bag” presentation about PowerShell. This presentation spurred me to augment our team’s environment with PowerShell and I have been using it every day since.

In the past weeks using and abusing PowerShell, I have drawn two conclusions:

First things first: if you spend any time working with Windows, get PowerShell. Now. Stop reading my blog and go download it immediately. It mops the floor with cmd.

The key premise behind PowerShell is that it operates on live .NET objects. This is beneficial because it eliminates a lot of the text cutting and manipulation common in shell scripts. Additionally, it puts the full .NET Base Class Library into your scripting toolbox. PowerShell tools, known as commandlets, typically only render the most common fields for their objects, but the less common fields are easily available in memory. By convention, Commandlets are named with a verb-noun pattern and support a common command line parsing behavior. The repository of commandlets and the command line options of each are easily queried and highly consistent. All this meta-data makes PowerShell a breeze to learn.

I fell in love with the the discoverablity and ease of use when I tried to kill a collection of runaway processes:

PS> get-command -noun process

CommandType   Name           Definition
-----------   ----           ----------
Cmdlet        Get-Process    Get-Process [[-Name] <String[]>] [-Verbo...
Cmdlet        Stop-Process   Stop-Process [-Id] <Int32[]> [-PassThru]...

PS> get-process notepad | stop-process
PS> get-alias | where { $_.definition.contains("Process") }

CommandType   Name   Definition
-----------   ----   ----------
Alias         kill   Stop-Process
Alias         ps     Get-Process

PS> ps someotherapp | kill

OK, that’s pretty cool and oh-so-very Unixy — right? Wrong. Notice the “CommandType” column in the results of get-command. There are many other types of commands besides commandlets: functions, filters, scripts, applications, etc. Each of these has slightly different semantics for pipes and parameters. Applications, for example, have no way of accepting .NET object pipes. You must develop a separate commandlet. Yikes!

Compare to Unix: all commands are applications which accept a command line and pipe byte streams in and out. Much simpler, but byte streams aren’t as friendly, discoverable, and maintainable as object streams. However, the brilliantly simple thing about Unix is that, when you get right down to it, object streams are just byte streams! There is absolutely nothing stopping you from implementing get-process and stop-process as Unix programs which pipe object references, JSON, pickled Python objects, XML, S-expressions, or any other data format you fancy. Doug Mcllroy, the inventor of Unix pipes, was right: text streams are the universal interface.

Actually, this is no different on Windows. All of the PowerShell commandlets could have been implemented as applications which import a library. This library would replace main in much the same way as winmain, provide a metadata enriched implementation of getopt, man, etc. There is no need to invent a new shell in order to acquire the power of piping objects. Sure, cmd is old and needed to be retired for many other reasons, but it is a real shame that the PowerShell toolset is not available to those of us stuck in batch scripts.

Personally, I would really like to see such a library developed. Microsoft has certainly proved one thing with PowerShell: steep learning curves are not intrinsic to command line interfaces. Unfortunately, commandlets are two steps forward and one step backwards. I have no doubt that we can retake that forward step.

Imported Comments


I find the 2nd point interesting given that most of the powershell team were unix only guys before this project.

Brandon Bloom

I don’t know the history of PowerShell or any of the team members. I’m sure they are all very smart and have a strong grasp of Unix in general. Maybe they understand exactly what I’m getting at, but explicitly rejected it for some non-obvious reasons (such as scheduling, implementation complexity, target users, etc). They probably know better than I do. My apologies for exaggerating to make a point :–)

Howard T. Snidbiscuits

I agree, PowerShell is way better than the primitive, DOS style Windows CMD. However, it’s nothing like a normal Unix shell for me… and I’d rather just use cygwin.


Sorry but no: Pipelining objects is much smarter. You can still pipe strings or binary data if you want, but you don’t have all the drawbacks.

And then there’s the security implications. While theoratically possible, are all your scripts safe against whitespace|“‘`` injection? Piping objects is sooo much easier AND it does not kill your system if the bytestream containsrm -rf /`.

Just my two cents…

Powershell Jedi

Unix is optimized around flat files windows is optimized around objects. It’s not the they don’t understand Unix. The Powershell Team has a strong unix Background. It’s just just that M$ has 87% market share and a trillion dollars in the bank share that says F@ck unix.

Brandon Bloom

Piping objects has draw backs too. Here’s a case that came up on the internal PowerShell mailing list…


cat foo | wc -l


Get-Content foo | Measure-Object -line

Ignore the line length, as I didn’t use any aliases for the PowerShell version. The issue here is that Get-Content returns a String, not a StreamReader. The result is that for large files, the entire String must be read into memory and for really large files, swapped back out to disk!

You could easily work around this with objects; this example is just to show that objects have drawbacks just as byte streams do. Everything is always a balance. I like the Unix approach because it is a simple system with drawbacks that are well understood. I dislike the object approach because it is a complex system with less obvious drawbacks that don’t bite you until much later.


Are you kidding? MS couldn’t even get the simplest thing right. Their “compare-object” is supposed to be their “equivalent” to diff. Not so. Not even close. Try comparing these two files:

file 1:

file 2:

Unix diff correctly shows how they are different. MS compare-objects says they are the same. Why? Because the latter doesn’t care about order.

Microsoft is not “very smart”. They are demented.