Monkeypatching is Destroying Ruby

(The title of this post is intended to be deliberately provocative, as well as being a nod to Steven Colbert’s “The People Destroying America” segments. It’s provocative because I want to get people talking about this issue.  I don’t actually think that monkey patching is “destroying” Ruby, but I do think the proliferation of the technique has real and troubling implications for Ruby’s future.)

“Monkey patching”, for anyone who doesn’t know, refers to the practice of extending or modifying existing code by changing classes at run-time. It is a powerful technique that has become popular in the Ruby community at least in part because the Ruby language makes it so easy. Any class can be re-opened at any time and amended in any way.

I believe the term first arose in the Python community, as a derogatory term for a practice which that community tended to frown on. The Ruby community, on the other hand, has embraced the term and the practice with enthusiasm. I’m starting to think that the Pythonistas’ attitude may have been justified.

Here’s what crystalized it for me. The other day I wrote a small Rails plugin (“NullDB”:nulldb). It was inspired largely by another plugin, “UnitRecord”:unitrecord. UnitRecord is by “Dan Manges”:[http://www.dcmanges.com/], a talented Rails developer whom I have a lot of respect for.

UnitRecord is implemented almost entirely as a set of monkey patches. When invoked, it dynamically modifies several standard Ruby and Rails classes, including @ActiveRecord::Base@, @Test::Unit::TestCase@.  As a result of this implementation, it is tightly coupled to the inner workings of ActiveRecord.  A small change to Rails and it could cease to work, and such a failure would be difficult to debug. Indeed, one of the reasons I decided to write “NullDB”:nulldb was because of just such a failure.

In writing “NullDB”:nulldb, I discovered that I could achieve the same functionality without resorting to monkey patching.  Instead of modifying existing classes, it implements the Rails “Database Adapter API”:[http://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/AbstractAdapter.html].  The finished library is shorter than “UnitRecord”:unitrecord, is composed entirely of implemetations of public APIs, and contains zero monkey patches.  The code is also easier to understand, in my opinion, because of the lack of metaprogramming.

Now, I did not write this post to gloat about how my library is better than Dan’s.  As I said before, I have a lot of respect for Dan.  He’s a smart guy; I’ve learned a great deal from “his blog”:[http://www.dcmanges.com/]; and it’s safe to say I would be a poorer Rails programmer if it weren’t for hm.

And this is really the point.  Monkey patching is the new black.  it’s what all the hip kids are doing.  To the point that smart, experienced hackers reach for a monkey patch as their tool of first resort, even when a simpler, more traditional solution is possible.

I don’t believe this situation to be sustainable.  Where I work, we are already seeing subtle, difficult-to-debug problems crop up as the result of monkey patching in plugins.  Patches interact in unpredictable, combinatoric ways.  And by their nature, bugs caused by monkey patches  are more difficult to track down than those introduced by more traditional classes and methods.  As just one example: on one project, it was a known caveat that we could not rely on class inheritable attributes as provided by ActiveSupport.  No one knew why.  Every Model we wrote had to use awkward workarounds.  Eventually we tracked it down in a plugin that generated admin consoles.  It was overwriting @Class.inherited()@.  It took us months to find this out.

This is just going to get worse if we don’t do something about it.  And the “something” is going to have to be a cultural shift, not a technical fix.  I believe it is time for experienced Ruby programmers to wean ourselves off of monkey patching, and start demonstrating more robust techniques.

I see the problem as one of convenience: sure, Ruby provides us with an immense toolbox, bigger than that of most other languages. But we’re like lazy carpenters: sure, we have a whole garage full of tools, but there’s a hammer laying right there on the floor next to us, and it’s easier to reach for the hammer instead of getting up and fetching the right tool for the job.  But those tools do exist.  Indeed, most of the patterns in the “software pattern literature”:[http://c2.com/ppr/index.html] are inspired at least partly by the need to manage extension in robust and maintainable ways.

I look at the Emacs community as an inspiration.  Emacs, as many of you probably know, is essentially just a lisp machine.  Emacs Lisp, the language it is written in, is every bit as dynamic as Ruby.  Functions can be replaced at any time, by code in any package.  And because all Emacs Lisp code is intended to extend the Emacs editor, it is not uncommon for hundreds of different Emacs Lisp packages to be running in the same process, all coexisting and interacting. And yet, for the most part this melting pot of extensions all function together smoothly and without breakage.

Why?  I think the biggest reason is community conventions.  I’ve read a lot of ELisp code, and I have very rarely seen the equivalent of Ruby-style monkey patching.  Even the @advice@ mechanism, an AOP(Aspect Oriented Programming)-like feature which enables functions to be dynamically wrapped and chained, somewhat like Rails’ @alias_method_chain@, is used sparingly.  Instead, every mature Emacs extension exposes a plethora of “hooks”, extension points that other packages can attach their own handlers to.  Other packages add their handlers to these hooks, and to hooks that the core Emacs code provides, and thus they cooperate largely without collisions.

Hooks are just one of the techniques available to us for robustly handling extension in a language as dynamic and powerful as Ruby.   Another is using the dependency injection style[1] to decouple classes from each other and allow third parties to substitute their own classes without monkey patching.  Another un-sexy but important practice is simply writing clear and comprehensive documentation of our classes’ APIs and extension points.

But the most important thing we can do is set an example.  This is my call to action, my line in the sand: as experienced Ruby programmers, let us demonstrate solid, sustainable practices in our own code.  Let us use monkey patching sparingly, as a tool for exploration and experiment, and only in production code as a last resort.  Let’s produce APIs that are amenable to extension without having to be patched.  And let us begin to develop some community-wide conventions and tools for writing extensible classes.

fn1. UPDATE A commenter on ruby-talk felt that my use of “Dependency Injection” revealed an “Enterprisey” bias.  Let me be clear that when I say “dependency injection style”, I’m not talking about using elaborate DI frameworks.  I’m just talking about writing classes that allow their collaborators to be passed in, either via constructors or setters, rather than having hard-coded collaborators.

[nulldb]http://avdi.org/projects/nulldb/

[unitrecord]http://unit-test-ar.rubyforge.org/

47 comments

  1. Many good points in this article. Teenagers never listen to their parents’ advice. The Ruby community by rejecting some well-known practices (like dependency injection and proper encapsulation) will have to reinvent it in some form in the future.

  2. sorry but ruby!=rails

    your problem is that all the monkeys are using rails at the moment, and consequently 95% of rails developers are monkeys

    give it a few more months and i am sure they will all jump on the bandwagen and head off to the Next Big Thing(tm)

  3. sorry but ruby!=rails

    your problem is that all the monkeys are using rails at the moment, and consequently 95% of rails developers are monkeys

    give it a few more months and i am sure they will all jump on the bandwagen and head off to the Next Big Thing(tm)

  4. In general I agree with you that monkey patching is a dangerous technique, and it’s use should always be weighed against the risks. However, none of the monkey-patching related bugs that I’ve come across have been hard to track down.

    I generally use small commits, and have reasonably solid tests, so I can tell if the introduction of a plugin or library breaks something.

    Once I’ve identified a problem, I pull out the debugger. It’s pretty easy to spot monkey patches when you’re stepping through code.

    I’m curious as to what it was about your Class.inherited() issue that required months to find. Care to elaborate, in the interests of illustrating how monkey patching shouldn’t be done?

  5. That you had to look for this “Class.inherited()” scares me to death. I must admit that I don’t consider myself a RoR developer, but I haven’t had to look for anything like that in any other language I have used. Lame.

  6. Some other precedents for why doing stuff like this is a bad idea.

    The Perl community has a long-standing love/hate-affair with making changes that impose “spooky action at a distance”.

    They call it “black magic” and it is generally considered it a last resort. Black Magic that makes GLOBAL changes to things like inheritance is often characterised as being “Octarine” (see disk world novels), because it tends to work ok when there’s only one person doing it, but start to mix a few together and KABOOM!

    Second example is the JavaScript prototype.js library, which modifies the core Object class, breaking a few things in the process. One of the main reasons people quit using prototype is that it doesn’t “play well with others” (libraries) as a result. Half the other libraries out there seem to have been spawned in part because of this problem.

  7. @Adam, RE: prototype, you have outdate info – that was in the bad old days when Prototype extended not just Object but “Object.prototype” which caused problems. It no longer does this.

  8. @Adam, RE: prototype, you have outdate info – that was in the bad old days when Prototype extended not just Object but “Object.prototype” which caused problems. It no longer does this.

  9. A few things…

    You do monkey-patch in your NullDB. You reopen AR::Base to add a nulldb_connection method. Maybe it isn’t as magic as eval, or defnine_method but it is still altering an existing class without subclassing or mix-in.

    Subclassing isn’t always possible. With AR::Base subclasses have special semantic meaning, ie. Single Table Inheritance. That is why everything with ActiveRecord is done through mix-ins +/or eval. (You can subclass AR::Base not for STI but it’s a pain)

    People write bad code in any language. Ruby is interpreted. Classes are objects and can be modified at anytime. Hell, even attr_accessor is a monkey-patch. You are correct that we need to be more careful about how we use Ruby. But the really important thing is to understand all of the code you use. Whether it is the Rails framework or a plugin, understand it enough to debug it. Then you can eval as much as you like.

  10. @andrew:

    You do monkey-patch in your NullDB. You reopen AR::Base to add a nulldb_connection method.

    True. This, unfortunately, is part of the contract of the
    ActiveRecord connection adapter API – you are expected to do this if
    you want connection_establish :adapter => :foo to work. Whether this
    is a true “monkey patch” is debatable; a lot of people only consider
    such a thing a monkey patch if it actually redefines existing
    methods. Of course, the Rails guys could easily have avoided the
    whole question by giving us a less-silly API that allowed us to simply
    register a new database connection class.

    Subclassing isn’t always possible. With AR::Base subclasses have special
    semantic meaning, ie. Single Table Inheritance. That is why everything with
    ActiveRecord is done through mix-ins +/or eval. (You can subclass AR::Base
    not for STI but it’s a pain)

    It’s not a pain, you just set abstract_class = true.

    Whether it is the Rails framework or a plugin, understand it enough to debug
    it. Then you can eval as much as you like.

    This is easy to say, but in real world large projects it simply isn’t
    feasible to understand all of the code in the project, let alone all
    of the third-party code it depends on. That is why community
    conventions are essential for maintainability.

    Thanks for the comment!

  11. “monkey patch” is debatable
    Ah, there is no such thing as a ruby “monkey patch” or “meta-programming” anyway.

    http://dablog.rubypal.com/2007/1/7/meta-shmeta-learning-ruby-horizontally

    less-silly API
    true

    self.abstract_class = true
    good in theory, not in practice ( if your inheritance is more than one level deep, easier [and more maintainable] to whip out some eval than set this in each subclass )

    community conventions are essential for maintainability
    also good in theory, but deadlines kill the virtue and bring out the kludge, i’m still sticking to knowing your code 🙂

  12. @andrew: so you’re saying I can’t take Rails as a black box and simply use it as my web application framework? I have to know the internals as well, just so I can avoid being clobbered by a bad monkeypatch? This is wrong. A framework should support my application, not undermine it. Don’t get me wrong, I like Ruby and I like Rails, but I do not like the monkeypatch-as-programming-model approach that many Rails developers seem to have taken.

  13. Nathan: You can use Rails as a black box framework. You can read some tutorials and start programming, use the API docs as needed. But you cannot throw a bunch gems, plugins, generators and engines into your app and expect to be able to debug without an understanding of the code you have written and used. I have worked with .net developers in the past. I present them with a bug and they shrug their shoulders, “that’s part of the framework, i don’t know”. Hoorah for open-source, huzzah for eval, I can change anything because I can read and understand the source ( all of it ), and can modify everything.

  14. I totally agreed with you on Monkeypatching as a style to code libraries. Patches are just patches. They are not organized codes. However, Monkeypatching does have is values in patching buggy codes and it should stay that way. Combined with the open source nature of Ruby, it is a powerful tool to patch things up the correct way without modifying the 3rd party codes.

    We used it in our ShellShadow merb app and it’s working pretty well.See a blog post here.

    http://odwks.com/2008/04/monkeypatching-merb/
    And yes, it should not be the means for developing libraries(which should uses public api’s).

  15. There are “programming language communities”? People actually hold affiliation to programming languages?

    This is all so very odd. Are there tattoos involved? I bet there are.

    You guys should like, have a Ruby sports team of some kind, which would play against the Python team, and all the others of course.

    Man, it sounds like people need to take off their Buddy Holly glasses and stop drinking so much damn coffee.

  16. There are “programming language communities”? People actually hold affiliation to programming languages?

    This is all so very odd. Are there tattoos involved? I bet there are.

    You guys should like, have a Ruby sports team of some kind, which would play against the Python team, and all the others of course.

    Man, it sounds like people need to take off their Buddy Holly glasses and stop drinking so much damn coffee.

  17. Classes are objects and can be modified at anytime. Hell, even attr_accessor is a monkey-patch. You are correct that we need to be more careful about how we use Ruby. But the really important thing is to understand all of the code you use. Whether it is the Rails framework or a plugin, understand it enough to debug it.

  18. Nice post, but it doesn't seem like much changed. I've just spent a day to find this in Amazon S3 gem's sources.

    class XmlSimple # :nodoc:
    def self.xml_in(*args)
    FasterXmlSimple.xml_in *args
    end
    end

    Well, maybe it's better to head back to Java.

Comments are closed.