a little baby ruby poem

A little Ruby poem, engendered during a discussion of how to run a block at fixed time intervals (like cron).

gem install activepoem

class Fixnum
  def minutely_do(times = 0)
    begin
      sleep(self*60)
    end while yield and (times -= 1) != 0
  end
end

some literature

Now we can write:

1.minutely_do { puts "a minute!" or true }

And that totally rhymes.

We would need to put the call in its own thread if we wanted to use this in real life, in most cases.

smart plaintext wrapping

Very often you need to break plaintext at a specific width while retaining readability. However, it is easy to get caught up in fencepost errors and similar irritating issues. I wrote the particular implementation here to help display the cheat files on Cheat. I have no doubt that this has been done many times before, but my method does do a few neat things.

features

Indents are preserved.

Linebreaks never occur within words unless the word is longer than the maximum width.

You can set an additional hanging indent so that wrapped lines clearly belong to their parent.

Also optionally, you can have it hang wrapped lines in a list underneath their parent text instead of underneath the list item delimiter.

usage examples

input text
class Employeed < Status
  belongs_to :employee, :class_name => "Person", :foreign_key => 'employee_id'
end

1. To find out what class an instance of the superclass use @instance.class or you can use the value of the type column from other systems that do not go through AR.
2. Fixtures for a STI model must go in the fixture file for the superclass.

Straightforward wrap (notice how the belongs_to line preserves its indent):

chloe:~/Projects/misc/misc eweaver$ ./wrap.rb text 50 0 false
class Employeed < Status
  belongs_to :employee, :class_name => "Person",
  :foreign_key => 'employee_id'
end

1. To find out what class an instance of the
superclass use @instance.class or you can use the
value of the type column from other systems that
do not go through AR.
2. Fixtures for a STI model must go in the fixture
file for the superclass.

How about with some hanging indents on the list, too?

chloe:~/Projects/misc/misc eweaver$ ./wrap.rb text 50 0 true
class Employeed < Status
  belongs_to :employee, :class_name => "Person",
  :foreign_key => 'employee_id'
end

1. To find out what class an instance of the
   superclass use @instance.class or you can use
   the value of the type column from other systems
   that do not go through AR.
2. Fixtures for a STI model must go in the fixture
   file for the superclass.

Etc.

code

wrap.rb
#!/opt/local/bin/ruby

class String

  def wrap(width, hanging_indent = 0, magic_lists = false)
    lines = self.split(/\n/)

    lines.collect! do |line|

      if magic_lists
        line =~ /^([\s\-\d\.\:]*\s)/
      else
        line =~ /^([\s]*\s)/
      end

      indent = $1.length + hanging_indent rescue hanging_indent

      buffer = ""
      first = true

      while line.length > 0
        first ? (i, first = 0, false) : i = indent
        pos = width - i

        if line.length > pos and line[0..pos] =~ /^(.+)\s/
          subline = $1
        else
          subline = line[0..pos]
        end
        buffer += " " * i + subline + "\n"
        line.tail!(subline.length)
      end
      buffer[0..-2]
    end

    lines.join("\n")

  end

  def tail!(pos)
    self[0..pos] = ""
    strip!
  end

end

if __FILE__ == $0
  File.open(ARGV[0]) do |f|
    puts f.read.wrap(ARGV[1].to_i, ARGV[2].to_i, ARGV[3] == "true")
  end
end

It’s really just a few regular expressions and some string munging; that’s all.

coming soon, double polymorphism with activerecord

I am going to add doubly polymorphic associations to the has_many_polymorphs plugin. They will continue to support self-reference and directionality.

possible API
class ChaseRelationship < ActiveRecord::Base
  has_many_polymorphs :join_class => self, :reflective => false,
                      :parent_key => 'chaser', :child_key => 'chasee',
                      :parents => [:dogs, :cats, :birds],
                      :children => :parents

end

Stay tuned. I need to decide what the API will be, because it won’t make sense to specify the relationship in a master model class anymore, since no class will have priority (except, possibly, the join class).

Is ‘directionality’ a real word, even? ‘Bi-direction’, maybe.

Also, if you missed it, I changed the license for has_many_polymorphs to the Academic License 3.0 a while ago, and the minor version was bumped. Functionality did not change.

anatomy of an attack against 1.1.4

Some people are still unclear about where exactly the flaws in the 1.1.x release series were, how they could be exploited, and what the patches did (or didn’t do). So step by step, I am going to lead you through the two main avenues of attack against 1.1.4, and explain the problems with the withdrawn 1.1.5 patch.

testing environment

I am testing this on OS X 10.4.7, Webrick, and MySQL 5. Rails tags are checked out into /vendor/, and the application is a slightly modified version of the well-known Cookbook example.

denial-of-service

This is what people have been referring to as the routing bug.

I added some logging code into rails/action_pack/lib/action_controller/routing.rb. First let’s see what happens when we request a correct controller and action.

in "traverse_to_controller" method
  Recognized route segment: recipe
  Looking for controller RecipeController in module Object
  in "attempt_load" method
  Loading RecipeController in module Object in path recipe_controller
    found script/../config/../app/controllers/recipe_controller.rb as a possible match
    loaded RecipeController
    found script/../config/../app/controllers/recipe_controller.rb as a possible match
    loaded RecipeController
  No controller found, so looking for module: Recipe
  in "attempt_load" method
  Loading Recipe in module Object in path recipe
    found script/../config/../app/models/recipe.rb as a possible match
    loaded Recipe

in "traverse_to_controller" method
  Recognized route segment: recipe
  Looking for controller RecipeController in module Object
  Found it

The reasons why it executes in the order that it does are not entirely obvious. Singleton classes get tossed around in the source and it can be difficult to follow.

Still, it is easy to see the look-up functions stepping through the module hierarchy and checking for already loaded constants for the relevant controller and model name. If they are not found, it sees if there are any files it can load that might contain them. If they are still not found, it tries to decend into whatever modules are available (or directories, which it dynamically creates as modules for namespacing purposes), and looks again.

Let’s try a malicious URL:

in "traverse_to_controller" method
  Recognized route segment: breakpoint_client
  Looking for controller BreakpointClientController in module Object
  in "attempt_load" method
  Loading BreakpointClientController in module Object in path breakpoint_client_controller
  No controller found, so looking for module: BreakpointClient
  in "attempt_load" method
  Loading BreakpointClient in module Object in path breakpoint_client
    found script/../config/../vendor/rails/railties/lib/breakpoint_client.rb as a possible match

Whoops. Controllers and models aren’t buried deep in /vendor/rails/railties/lib/. Still, it managed to find breakpoint_client.rb, and, being very optimistic about where you might like to store your controllers, tried to load it up to see if it was the right thing. Only after that load does the framework check if it found the class it was actually looking for.

Class definitions in Ruby are executed just like any other method call, so the entire file has to get evaluated during the load in order to really guarantee what is inside it. Of course, now the server is hanging around waiting for a breakpoint from itself, and the app no longer responds.

database corruption

What if we continue with the routing bug, but instead try something really nasty? First, how many recipes are in the database?

roaming-205-133:~/Projects/cookbook eweaver$ script/console
Loading development environment.
>> Recipe.count
=> 1
>> 

Looks like we won’t be eating very well.

in "traverse_to_controller" method
  Recognized route segment: db
  Looking for controller DbController in module Object
  in "attempt_load" method
  Loading DbController in module Object in path db_controller
  No controller found, so looking for module: Db
  in "attempt_load" method
  Loading Db in module Object in path db

in "traverse_to_controller" method
  Recognized route segment: schema
  Looking for controller SchemaController in module Db
  in "attempt_load" method
  Loading SchemaController in module Db in path db/schema_controller
  No controller found, so looking for module: Schema
  in "attempt_load" method
  Loading Schema in module Db in path db/schema
    found ./db/schema.rb as a possible match

...

The application eventually returns a routing error, and continues to function. Except:

in development.log
[2006-08-12 02:57:05] INFO  WEBrick::HTTPServer#start: pid=14539 port=3000
-- create_table("categories", {:force=>true})
   -> 0.0459s
-- create_table("recipes", {:force=>true})
   -> 0.0059s
-- initialize_schema_information()
   -> 0.0007s
-- columns("schema_info")
   -> 0.0029s

Cue stunned silence.

roaming-205-133:~/Projects/cookbook eweaver$ script/console
Loading development environment.
>> Recipe.count
=> 0
>>

Start the database restore, because otherwise we’re going to starve. You could probably also force the server to execute migrations if you could guess their names, and damage the database that way, too.

remote code execution

This is what people have been referring to as the $LOAD_PATH bug, but it’s really an extension of the same attack as above. What if we could drop an arbitrary .rb file into one of the directories being examined, then load it?

We’ll add some pictures to our recipes using the file_column plugin, so we add a filename field to our recipes table, and put the file_column :filename macro in the Recipe class definition. We also upload a picture to Recipe 1. Let’s look on the filesystem to see where file_column put it.

~/Projects/cookbook/public/recipe/filename/1/yum.jpg

Fine, whatever. But maybe the application author didn’t check file extensions properly. Can we upload an .rb file?

Turns out that we can. Now we have ~/Projects/cookbook/public/recipe/filename/1/hax.rb, too. Can we find a route that will load it?

Routing Error
Recognition failed for "/public/recipe/filename/1/hax/"

No, we can’t, because “1” is not a valid module name. However, if you have text-based primary keys, or use a custom upload process that puts files in a subpath of /public/ such that each subdirectory name begins with a letter, you are vulnerable.

For example, Typo 4, last time I checked, uploads files into /public/files/. On an unpatched system, anyone with a login to the Typo instance could effectively run any Ruby code they wanted on the server—a classic privileges escalation.

(Actually, the file doesn’t always have to be in a subdirectory of public, because ”.” is included in the load path. On some webservers such as Lighttpd, ”.” will usually point to /public/, but on others, it points to RAILS_ROOT, leaving any file within the entire Rails tree potentionally loadable on such servers.)

From where exactly is the framework coming up with all of these “alternative” directory choices? Let’s add a little bit more logging code and request a route that doesn’t exist.

in "traverse_to_controller" method
  Recognized route segment: whatever
  Looking for controller WhateverController in module Object
  in "attempt_load" method
  Loading WhateverController in module Object in path whatever_controller
    entered directory ./script/../config/../vendor/rails/actionwebservice/lib/action_web_service/v
endor/
    entered directory ./script/../config/../vendor/rails/actionmailer/lib/action_mailer/vendor/
    entered directory ./script/../config/../vendor/rails/actionpack/lib/action_view/helpers/../../
action_controller/vendor/html-scanner
    entered directory ./script/../config/../vendor/rails/actionpack/lib/action_view/vendor
    entered directory ./script/../config/../vendor/rails/actionpack/lib
    entered directory ./script/../config/../vendor/rails/activerecord/lib
    entered directory script/../config/../test/mocks/development
    entered directory script/../config/../app/controllers/
    entered directory script/../config/../app
    entered directory script/../config/../app/models
    entered directory script/../config/../app/controllers
    entered directory script/../config/../app/helpers
    entered directory script/../config/../components
    entered directory script/../config/../config
    entered directory script/../config/../lib
    entered directory script/../config/../vendor
    entered directory script/../config/../vendor/rails/railties
    entered directory script/../config/../vendor/rails/railties/lib
    entered directory script/../config/../vendor/rails/actionpack/lib
    entered directory script/../config/../vendor/rails/activesupport/lib
    entered directory script/../config/../vendor/rails/activerecord/lib
    entered directory script/../config/../vendor/rails/actionmailer/lib
Routing Error
Recognition failed for "/whatever"

Crazyness! That’s a lot of places to look for whatever_controller.rb. Remember, too, that any subdirectories under any of these places are vulnerable if they can be properly namespaced in the route.

It turns out that Rails constructs the safe_load_paths array from several different places. Most of them are set in the application’s environment.rb or in the framework itself, and are immutable. But it also adds the contents of the $LOAD_PATH environment variable.

Previously I said that on some webservers, sending a Load-Path: header in the request might add the contents to the Rails $LOAD_PATH. I have tested, and I was wrong. There was no vulnerability related to the header. I don’t know if there are server-side ways to manipulate the $LOAD_PATH, though.

what did the 1.1.5 patch do?

The important change occurred in routing.rb, although 1.1.5 also added an integration testing method relating to the $XHTTP_LOAD_PATH request parameter. This turned out to be not relevant to production environments. Also, a test was added for an SQL injection vulnerability, but there is no known exploit related to that test. It seems to be merely a preventative measure.

You can get a diff of the changes between 1.1.4 and 1.1.5 in routing.rb here. There were some other changes, too, but this is the most relevant. If you want to look yourself, be sure to delete /doc/, /test/, and other irrelevant things like that before you run diff -r or you will get a huge mess of changes.

routing.rb
# removed
base[0, extended_root.length] == extended_root ||
  base =~ %r{rails-[\d.]+/builtin}

# added
base.match(/\A#{Regexp.escape(extended_root)}\/*#{file_kinds(:lib) * '|'}/) ||
  base =~ %r{rails-[\d.]+/builtin}

Let’s see what that does.

logger output for 1.1.5
    entered directory ./script/../config/../vendor/rails/actionwebservice/lib/action_web_service/v
endor/
    entered directory ./script/../config/../vendor/rails/actionmailer/lib/action_mailer/vendor/
    entered directory ./script/../config/../vendor/rails/actionpack/lib/action_view/helpers/../../
action_controller/vendor/html-scanner
    entered directory /opt/local/lib/ruby/gems/1.8/gems/RedCloth-3.0.4/bin
    entered directory /opt/local/lib/ruby/gems/1.8/gems/RedCloth-3.0.4/lib
    entered directory ./script/../config/../vendor/rails/actionpack/lib/action_view/vendor
    entered directory ./script/../config/../vendor/rails/actionpack/lib
    entered directory ./script/../config/../vendor/rails/activerecord/lib
    entered directory script/../config/../app/controllers/
    entered directory script/../config/../app
    entered directory script/../config/../app/models
    entered directory script/../config/../app/controllers
    entered directory script/../config/../app/helpers
    entered directory script/../config/../lib
    entered directory script/../config/../vendor/rails/railties/lib
    entered directory script/../config/../vendor/rails/actionpack/lib
    entered directory script/../config/../vendor/rails/activesupport/lib
    entered directory script/../config/../vendor/rails/activerecord/lib
    entered directory script/../config/../vendor/rails/actionmailer/lib
    entered directory script/../config/../vendor/rails/actionwebservice/lib
    entered directory ./script/../config/../vendor/rails/activesupport/lib/active_support/vendor
    entered directory ./script/../config/../vendor/rails/activesupport/lib
    entered directory /opt/local/lib/ruby/site_ruby/1.8
    entered directory /opt/local/lib/ruby/site_ruby/1.8/i686-darwin8.6.1
    entered directory /opt/local/lib/ruby/site_ruby
    entered directory /opt/local/lib/ruby/vendor_ruby/1.8
    entered directory /opt/local/lib/ruby/vendor_ruby/1.8/i686-darwin8.6.1
    entered directory /opt/local/lib/ruby/vendor_ruby
    entered directory /opt/local/lib/ruby/1.8
    entered directory /opt/local/lib/ruby/1.8/i686-darwin8.6.1
  No controller found, so looking for module: Whatever
  in "attempt_load" method
  Loading Whatever in module Object in path whatever

@file_kinds:
[:app, :lib]

file_kinds(:lib):
[:app, :lib]

It turns out that the regular expression change merely limits the safe_load_paths to directories prefixed with /lib/ or /app/ relative to some fixed starting points (give or take). This broke Rails engines, which keep their libraries somewhere else. Also, there were still some unsafe files living in some of the directories listed above that could cause denials-of-service if requested via a route.

A stop-gap measure, mainly.

what does the 1.1.6 patch do?

Again, a diff of the changes between 1.1.5 and 1.1.6 in routing.rb is here.

The new method file_kinds, added by 1.1.5, is gone again. The regular expression is now:

base.match(/\A#{Regexp.escape(extended_root)}\/*(app|lib|components)\/[a-z]/) ||
  base =~ %r{rails-[\d.]+/builtin}

Another logger output. I’m getting tired of these.

logger output for 1.1.6

in "traverse_to_controller" method
  Recognized route segment: whatever
  Looking for controller WhateverController in module Object
  in "attempt_load" method
  Loading WhateverController in module Object in path whatever_controller
    entered directory script/../config/../app/controllers/
    entered directory script/../config/../app/models
    entered directory script/../config/../app/controllers
    entered directory script/../config/../app/helpers
  No controller found, so looking for module: Whatever
  in "attempt_load" method
  Loading Whatever in module Object in path whatever
    entered directory script/../config/../app/controllers/
    entered directory script/../config/../app/models
    entered directory script/../config/../app/controllers
    entered directory script/../config/../app/helpers

Look how short it is! So much nicer. The fix itself, though, still seems kind of shady. The new routing and constantizing code in Rails edge is a big improvement. Personally, I recommend that you update to edge if it won’t break your application, in case there are still vulnerable URLs or routes in 1.1.6.

in the wild?

I have heard very few reports of actual public-facing Rails installations being exploited. Please let me know if you know of (or own) any such sites, and any details about the attack that you have.

conclusion

That’s all, really. I completely understand that after such a marathon patch session, the team was tired and that their patience was wearing thin. They have been good (I think) about letting critical comments stay on the weblog. But still, Hansson, core, and everyone, please don’t bully us poor users, especially right after a previous patch rollback and keeping in mind this ticket from June 16.

Anyway, lessons learned by all, etc., etc. Next time will be better… or it will be Django >:) .

I do know that Camping’s routes are immune to this particular problem. There has also been a little bit of speculation on IRC about back-forking Rails into a security-audited version every six months or so. IronRails, anyone?

Finally, I would like to thank a few people on IRC who gave me clues or encouragement during this ordeal: defunkt, isotopp, bitsweat, icblenke, and joelmichael.

postscript

I will be presenting on Camping at the September Philly-on-Rails meeting.

explanation of the rails security vulnerability in 1.1.4, others

introduction

second update

Please also see my Anatomy of an Attack Against 1.1.4, which is more detailed and more accurate regarding the mechanics of the flaw.

first update

The problem is not limited to remote execution; there are denial-of-service issues too. See this thread on Ruby-Forum, and the official explanation on the Rails weblog. However, as Chris Carter points out on Peter Krantz’s blog:

“The scary part is that it can run your schema.rb file, which will drop all your tables and recreate them.”

I always use a SQL schema instead of a Ruby schema because I use a visual database modeler, so I was immune to this particular use of the bug. Your first line of defense should be to rename schema.rb to schema.something_else.

A vulnerability tester (not an exploit) has been posted to Pastie. Usage instructions and some tests of different combinations of webservers and Rails versions are here, on the Caboose wiki. The testing script was written by Zed Shaw. Be very careful to check that it won’t accidentally do something dangerous to your app. Warning: don’t run it against production, because it will reload your database schema if your application is vulnerable.

details

The patch location is easily discovered with the elite hacking tool diff -r:

routing.rb 1.1.4
  base[0, extended_root.length] == extended_root || base =~ %r{rails-[\d.]+/builtin}
routing.rb 1.1.5
  def file_kinds(kind)
    ((@file_kinds ||= []) << kind).uniq! || @file_kinds
  end

  file_kinds :app

  base.match(/\A#{Regexp.escape(extended_root)}\/*#{file_kinds(:lib) * '|'}/) || base =~ %r{rails-[\d.]+/builtin}

It looks like, for example, that if your Rails installation is in /www/rails/, passing a string such as /www/rails/../../tmp/ would pass the old validation, and if you had managed to upload a file such as hax_controller.rb to /tmp/, a route request to /hax/ would force Rails to run your arbitrary code. It seemed like the browser-submittable header field $HTTP_LOAD_PATH would get rewritten to $LOAD_PATH during certain requests. This may not be true in production mode, or it may depend on your webserver. Even so, file_column and other plugins upload things to places within the Rails tree (e.g. #{RAILS_ROOT}/public/files/model_name/id/filename), so it wouldn’t have been necessary to backtrack on the path, because /public/files/model_name/id/filename would have been a valid route even though there was no such controller. (I think. I am not an expert on routes.) If you could predict the landing-place of an uploaded .rb file, you could compromise the system.

However, as is mentioned in the update above, the default Rails install tree contains some files which are already dangerous and could be executed without having to manipulate the $LOAD_PATH. This means there is a built-in potential for denials-of-service as well as database corruption if the schema is reset or if migrations are forced to run in the wrong order.

criticism

The security-through-obscurity response of the Rails team has been widely criticized. As a commenter notes on the Rails blog, this is the kind of thing that gets software permanently rejected by organizations where risk management is a high priority. Kristian Köhntopp reminded me that Theo de Raadt tried the same approach with the famous OpenSSH vulnerability. It didn’t work then either. Open-source and limited disclosure policies tend not to go well together.

In other news, I posted release 12 of the has_many_polymorphs plugin.

has_many_polymorphs release 11

Release 11 of the has_many_polymorphs ActiveRecord plugin is available; go here for the update. This will be important to you if you use callbacks in your models (after_find, after_initialize), or if you want to verify that the plugin passes comprehensive tests before using it in a production environment.

has_many_polymorphs gets official

The has_many_polymorphs plugin is now hosted on Rubyforge. Read about it on the brand-new official page. The new version has a bunch of performance enhancements behind the scenes.

In other news, there seems to be a proliferation of view-like (jackchristensen.com) plugins (habtm.com). I understand using views in the database itself. But I am not sure that the convenience of putting them in the ActiveRecord model outweighs the muddling of the class structure that you get. Views aren’t very DRY. Thoughts, anyone?

ajax and camping, together at last

I had to mock up a demo for Intel for my work and went with _why’s Camping. However nobody on the IRC channel the other day knew how to use ajax with it. It turned out to be easy.

serve static files

You need your Camping app to be able to serve the javascript libraries to the client. Follow these instructions from the wiki. We have updated them recently to be more robust.

Once you actually deploy your app, it will be faster and more stable to serve the static directory either directly from Apache, or with a mongrel dirhandler. Both ways bypass your static camping controller. There is an example of the mongrel dirhandler in rv version 2.

require the javascript

Copy the Prototype and Scriptaculous javascript libraries out of a Rails installation, and require them in the head element of your layout.

in your layout method, in App::Views
head do
  ['prototype', 'effects', 'controls'].each do |s|
    script :src => "/static/#{s}.js", :type => 'text/javascript'
  end
end

make a link

Make a link in one of your views that will call the ajax’ed action and specify the element to update with the response.

in your view method, in App::Views
id = "some_element"
onclick = "new Ajax.Updater('#{id}', '#{URL(MyAction, some_parameter)}',
           {asynchronous:true, evalScripts:true}); return false;"
a "Link text",
  :id => id,
  :href => R(NoJavascriptFallbackAction, some_parameter),
  :onClick => onclick                

return something

Then have your action just return a raw string instead of calling render:

in App::Controllers
class MyAction < R '/myaction/(.*)'
  def post param
    # do something with param
    "Mepw!"
  end
end

the Ajax.Updater will replace the element you specified with the action’s response.

Or a feral cat. Mepw.

sqlite3, camping, and sessions

If you use Camping::Session, this could be useful to you.

hair

I was tearing my hair out because all of a sudden after I deleted my ~/.camping.db, my camping app completely failed despite the .create method that had worked perfectly well before. The error

ActiveRecord::StatementInvalid ActiveRecord::StatementInvalid:

is identical to the one you get if you try to use the sqlite3-ruby gem without swig bindings installed.

So, I tried every possible combination of installs, uninstalls, library paths, and even looked in the sqlite3-ruby sources, where it seemed that some table was empty when it wasn’t supposed to be.

solution

It wasn’t until I switched to MySQL that I really figured out the problem.

No such table 'sessions'.

If you are using Camping::Session, you need the line Camping::Models::Session.create_schema in your .create method. I had run the camping examples before, and this had created the sessions table in my ~/.camping.db. But when I reset the database that table got destroyed, never to return.

:(

sqlite3-ruby’s error reporting is less than awesome.

self-referential, polymorphic has_many through helper now a plugin

Thanks to Chris Wanstrath, my polymorphic, self-referential has_many :through module is now a handy plugin.

Read more.

In the process of working with it, I discovered this ActiveRecord oddity:

Dog.find(:all, :conditions => {:name => "Sparky", :likes_treats => 1}) works fine.

But Dog.find(:all, :conditions => {:name => "Sparky", :likes_treats => 1, :favorite_tree => nil}) doesn’t work at all. Apparently ActiveRecord is turning the “ => nil” into “ = NULL” when it needs to be “ IS NULL”, at least for MySQL.

I would look to see if there was a ticket for this already, but the Rails trac is still down. Anybody else run into this?