Fun with Threads

During the release of rails 2.2 we made a lot of changes to ensure that we could dispatch requests in parallel by using multiple threads. Some like to describe this as ‘threadsafe’, but as I’ve argued before that’s a little too vague for me.

One of the key lessons we had in the very very early days of rails is that it is impossible to require code when you have multiple threads on the go. It takes a while for people to get their heads around this, typically they ask things like “Why not just put a mutex around require?”. I figured I should write this up for future reference as I haven’t seen it described anywhere else, and people are still learning this the hard way.

This isn’t an issue with autoload. Autoload makes the problem much worse by deferring the require statements to some ‘random’ moment in the future. If you call require from threaded code, you’re going to get bitten by this feature. Let’s start with a simple example:

We’ve all written code like this before, it’s the very basis of how ruby works! Now let’s imagine we want to run this code in multiple threads. Easy right? We’ll require ‘customer’ if it’s not already defined, then do the same thing.

If you run this enough times you’ll get an error like this:

NoMethodError: undefined method `scale!' for #<Customer:0x5f00dc>

WTF is that thing on about? The method is RIGHT THERE! This issue lead some less experienced people to say silly things like “When I used rails in a multi threaded environment methods kept appearing in the wrong thread”. But given how common and underdocumented this issue is, it’s hardly their fault.

The problem comes about because of the way constant definition works in ruby. It isn’t atomic. The second the interpreter has read the “class Customer” line of your file, it has defined the constant Customer, but the constant doesn’t have any methods, or a constructor. It’s like a baby-constant, and if you try to do anything with it, you’ll get all kinds of crazy errors. If you’re playing along at home and want to see this happen, the easiest way is to do this:

Now at first glance this is stupid behaviour, why on earth doesn’t ruby just avoid defining the constant until the very last end statement? Remember back to when you picked up the pick axe and it told you how to define class methods? There are a bunch of different types of syntax available. What about this one?

If Customer wasn’t defined immediately, that syntax couldn’t work! Class methods in ruby aren’t something magical, they’re just methods defined on that particular instance of a Class. Also it’d be impossible to have bi-directional references in your class declarations. ask the delphi guys how much fun that is

So the problem with require and autoload isn’t something about the implementation of how require works, or how autoload works. It’s an inherent limitation of the way ruby works. It’s completely impossible to define constants in one thread, without those constants being accidentally used before they’re ready.

Thankfully we knew about this problem going into the threadsafety project, and the vast majority of the work josh did was to catch all the places where we defined methods and classes, and do that work prior to the dispatcher starting up. This means you pay a penalty at start up, but your application will work correctly irrespective of which constants you reference and when you reference them. We intend to use autoload in non-threaded deployments for 2.3, and to support that we have to preload everything in a similar fashion.

One of the more challenging things with this issue is catching the problem. If all you ever test is a single method, then the first warm-up call will require everything and nothing will break. Hello world is an awful test case here.

So if you’re thinking of trying out jruby and using the nice native threads, be sure to audit every line of code you have for calls to require at runtime, including autoload declarations. If you’re using a framework or library which uses autoload, be sure to work around this problem by requiring everything explicitly ahead of time. If you don’t your code is only ‘thread safe’, and you’re just waiting for trouble.

If you’re a library author and would like a hand figuring out a solution for your case, drop us a line on the rails core list. We’d be happy to help.

Note: I’ve used gist to handle the snippets here, you noscript users will have to trust me :)