snax

scaling rails

bleak_house 3 tells you your leaks

BleakHouse 3:

  ...
  58%: core rails (783 births, 1241 deaths, ratio -0.23, impact 1.66)
  66%: recipe/new/GET (15182 births, 14593 deaths, ratio 0.02, impact 1.77)
  75%: core rails (766 births, 1168 deaths, ratio -0.21, impact 1.60)
  83%: recipe/list/GET (16423 births, 15991 deaths, ratio 0.01, impact 1.64)

65992 births, 66458 deaths.

Tags sorted by immortal leaks:
  recipe/show/GET leaks, averaged over 4 requests:
    5599 String
    80 Array
    2 Regexp
    2 MatchData
    2 Hash
    1 Symbol
    ...
  core rails leaks, averaged over 4 requests:
    238 String
    10 Array

Tags sorted by impact * ratio:
   0.0739: recipe/show/GET
   0.0350: recipe/new/GET
   0.0218: recipe/list/GET
  -0.6686: core rails

That's a Symbol up there; the new BleakHouse walks the sym_tbl as well as the regular heap. We now track the history of every individual object instead of just class counts. This means we can accurately (fingers crossed) identify where lingering objects were spawned.

On the flipside, analyzing the log file is slow (a decent-sized logfile will have hundreds of millions of rows). I wrote a pure-C CSV parser, which helps, and there's always the "better hardware" answer. I've been mainly running it on my Mac Mini; if I use the Opteron 2210 it goes much faster, since the analyzer is CPU-bound.

It doesn't make pretty graphs anymore but I'm not sure exactly how they would help. It would be easy enough to add them back.

go go go

A gem, not a plugin, because it needs to compile a C extension. First, uninstall the old versions to prevent version problems:

sudo gem uninstall bleak_house -a -i -x 

Now:

sudo gem install bleak_house

Also, you need to rebuild your ruby-bleak-house binary, even if you already have one. Just run:

bleak --build

The RDoc has updated usage instructions.

September 25, 2007

21 comments

Zach Dennis says (September 25, 2007):

I think the RubyGem mirrors may be behind. It shows up when searching


$ gem search –r bleak_house
* REMOTE GEMS *
Bulk updating Gem source index for:
http://gems.rubyforge.org

bleak_house (7.1, 7, 6, 5.3, 5.2, 5.1, 5, 3.1, 3.0.2, 3)

But fails to install…


$ sudo gem install –r bleak_house –v 3.0.2
ERROR: While executing gem … (OpenURI::HTTPError)
404 Not Found

Come on Rubyforge! I only have so long on lunch!

evan says (September 26, 2007):

Get it from here if you’re still having trouble. You also will need Ccsv.

anonymous girlfriend says (September 26, 2007):

I just want all you programmers out there to know just how hard this guy works, and how little time I get to actually see him. For the good of the cause I guess…. ;-)

karl says (September 26, 2007):

Is it 3.2 or 3.0.2?

Still cant find on gem mirrors…

evan says (September 26, 2007):

Karl: Use the direct links to the two gems, above. 3.2 is the latest. I’ve filed a Rubyforge ticket about the mirrors being messed up.

Martha: Soon ;)

Dr Nic says (September 26, 2007):

evan: Make sure you remove that “anonymous girlfriend” spam… :P

evan says (September 26, 2007):

Oh, snap!

By the way, the gem mirrors are fixed.

Stephen Sykes says (September 30, 2007):

Evan: Line 42 in build.rb is a tiny bit mangled:


exec(“rm #{binary}; cp ./ruby #{binary};
chmod–755 #{binary}; echoe \” Done\"")

chmod doesn’t need ‘–’ and echoe is the olde fashioned command, nowe deprecated :)

One other thing, wget doesn’t exist on my mac–no prob, I modded it to use curl–but you might want to use curl by default.

evan says (September 30, 2007):

Oops! Fixed in trunk. I’ll switch to curl, too.

There’s another bug in the queue, so once I fix that, I’ll update the gem.

Justin Williams says (October 01, 2007):

ld is throwing up errors when it tries to make bleak_house. Here’s the error: http://pastie.caboo.se/102696

Ideas?

evan says (October 01, 2007):

Ah; crap; ld is trying to link against the generic ruby binary rather than the ruby-bleak-house one.

I’ll figure something out.

evan says (October 03, 2007):

Actually, it looks like you’re using Leopard? I don’t know how to fix compilation against the bundled package (also, I don’t have Leopard). If you install Ruby from MacPorts you should be ok.

Martin says (October 09, 2007):

Just thought I’d tell you that on my system (Ubuntu) I had to remove the ‘&’ characters from the system calls in “/usr/lib/ruby/gems/1.8/gems/bleakhouse-3.2/lib/bleakhouse/support/build.rb” to be able to use “bleak—build”. The ‘&’ forks the command so that the script continues without waiting for the command to complete, which causes the following commands to fail.

evan says (October 09, 2007):

Strange that &> would background rather than redirect STDERR. Are you not using Bash, maybe?

Justin Williams says (October 13, 2007):

I was able to successfully install bleak_house on my main dev machine running Tiger.

You’ve got 2 weeks to release a version that’s Leopard compatible. :-P

evan says (October 14, 2007):

I think you mean, Apple has two weeks to make sure the Leopard Ruby build isn’t confused like the Tiger one was. :)

Martin says (October 15, 2007):

Yes, I am using bash, so my conclusion would be that the ruby interpreter uses another shell.

evan says (October 15, 2007):

Hmm, well, I definitely can’t reproduce it. What does ruby –e ‘puts `env | grep SHELL`’ say? And perhaps bash —version?

I changed the &> in BleakHouse 3.3 to > and 2>&1; maybe that will work better for you.

Aphelion says (December 03, 2007):

I think that it might have to do with leopard building things universal by default... I'm not sure. But even with ARCHFLAGS="-i386" it's not happy.

suzaku:~ travis$ ruby-bleak-house
require 'rbconfig'
-:1:in `require': no such file to load -- rbconfig (LoadError)
	from -:1

chendo says (December 03, 2007):

I'm getting this error when trying to use it with Rails Edge:

/var/www/apps/XXXX/releases/20071130025107/vendor/rails/activesupport/lib/
  active_support/core_ext/module/aliasing.rb:31:in 
`alias_method': undefined method `process' for class `Class' (NameError)
	from /var/www/apps/XXXX/releases/20071130025107/vendor/rails/          
activesupport/lib/active_support/core_ext/module/aliasing.rb:31:in 
`alias_method_chain'
	from /usr/lib64/ruby/gems/1.8/gems/bleak_house-3.5.1/lib/bleak_house/
          rails/action_controller.rb:9

Any ideas?

evan says (December 03, 2007):

Please report these bugs on the forum; it's easier to handle them there.

Comments are closed.