memcached gem release

August 4, 2009 By in Uncategorized 15 Comments

One of the hardest gems to install is no more. It’s now easy to install!

Memcached 0.15 features:

  • Update to libmemcached 0.31.1
  • Bundle libmemcached itself with the gem (antifuchs)
  • UDP connection support
  • Unix domain socket support (hellvinz)
  • AUTO_EJECT_HOSTS bugfixes (mattknox)

Install with gem install memcached. Since libmemcached is bundled in, there are no longer any dependencies.

on coordination

Andreas Fuchs suggested several months ago that I include libmemcached itself in the gem, but at the time I resisted. I was wrong.

My opposition was based on the idea that libmemcached itself would be an integration point, so running multiple versions on a system would be bad.

In real life, the hash algorithm became the integration point, not the library itself. And since the library’s ABI kept changing, the gem always required a very specific custom build. This annoyed the public and caused extra work for my operations team, who had to make sure to upgrade both the library and the gem at the same time.

Updates can come thick and fast now because I don’t have to worry about publishing custom builds or waiting for the libmemcached developers to merge my patches.

In retrospect it seems obvious—it’s always a win to remove coordination from a system.

linker woes

Unfortunately, it was easier to make that decision than it was to implement it. Linux and OS X link libraries differently, and I had a lot of trouble making sure that no system-installed version of libmemcached would get linked, instead of the custom one built during gem install.

When you link a shared object, OS X seems to maintain a reference to the original .dylib. Linux does not, and depends on ldconfig and LD_LIBRARY_PRELOAD to find the object at runtime. Since you can’t modify the shell environment from within a running process, there’s no way to override LD_LIBRARY_PRELOAD, so I needed to statically link libmemcached into the gem’s own .so or .bundle.

The only way I could do this on both systems was to configure libmemcached with CFLAGS=-fPIC --disable-shared, rename the libemcached.* static object files to libemcached_gem.*, and pass -lmemcached_gem to the linker rather than -lmemcached. Otherwise the linker would prefer the system-installed dynamic objects, even with the correct paths and -static option set.

Note that you can check what objects a binary has linked to via otool -F on OS X, and ldd on Linux.

Feel free to look at the extconf.rb source and let me know if there’s a better way to do this.

15 responses

  1. What about instead of bundling the binaries, you bundle the source and it gets compiled during installation? This is how ruby-ffi has been doing it, and it works well for installing on OS X, Linux and even on Windows with the new MinGW builds and DevKit.

    That would make the code more portable.

  2. I do bundle the source. It gets compiled locally to the gem during installation, and linked.

    I’ll take a look at the ruby-ffi extconf.rb file. It looks like it’s using libffi sources directly during build, with no intermediate object, which is something.

  3. We do something similar in moxi (the memcached proxy) for somewhat similar reasons and had challenges with the different linking between Linux and OS X too.

    On the linking side of things though, you should be able to use -rpath (Mac OS) --rpath (Linux) and -R (Solaris) to build in a path to search for a library. That combined with versioning may get you what you need though there certainly seems to be more magic on OS X and it’s done things I’ve not expected before.

    Personally, I think longer term we’re planning to just use the installed libmemcached, so we’re going the opposite direction. :) It’ll still be a while before we can expect a reasonably modern libmemcached on any given platform though.

  5. If you set linker flags like that they really should be in the LDFLAGS environment variable, not in CFLAGS. gcc is order dependent on some things and that may make the difference on Linux.

    Added some other comments in the code.

  6. Is changing get_multi in the Memcached::Rails compatibility wrapper the right approach to fixing the libmemcached branch of cache-money for this release (0.15.2)?

    Also note that add_method = options[:use_udp] is never used in Memcached#set_servers

  7. No, the underlying API was changed for a reason; it’s safe to consider it pretty stable now though. So I would think that Cache-money is the thing that should be patched.

    Thanks for the bug report about UDP; I committed a fix. Please report any more issues via the GitHub issue tracker.

  8. I got pretty excited because I thought :use_udp was a drop-in speed boost. Unfortunately, as I learned from here, get is not supported in UDP mode.

    So, if I understand correctly, you can’t just start using UDP everywhere you use the memcached gem. For example, if you replace the Rails cache with Memcached::Rails using UDP, it will die with a Memcached::ActionNotSupported every time you try to read/get.

  9. Yeah, UDP only works for operations that don’t require a response. It’s usually used for invalidations that have to be spammed to many servers.

    Incidentally, UDP support looks totally broken in my gem. Fixing.

  11. I’m running Snow Leopard and I installed the memcached gem, but when I require ‘memcached’ it complains that “LoadError: no such file to load — memcached”. There were no errors printed when I did the install. libmemcache is installed and works (all of the tests in “make test” pass). I have confirmed that the directory in gems was created and is full of stuff, including memcached.rb . Any suggestions?

  12. Well after much playing around I have it working. I am using rvm so I can develop with multiple Ruby versions. I had to make sure I clean out the old attempts at building memcached.

    gem uninstall memcached
env ARCHFLAGS="-arch x86_64" gem install memcached

    And it worked!

  14. I had trouble installing it on my Solaris 10 with 32bit / gcc compiled ruby but managed it with a few modifications to extconf.rb:

    1. added “–disable-64bit” to the libmemcached configure arguments

    2. added “-std=gnu99” to CFLAGS (the rlibmemcached_wrap.c compilation was failing without that)

    3. added an extra -R path for ext/lib – not sure if this was needed actually

    4. recreated the rlibmemcached_wrap.c with swig (it removed a bunch of methods, not sure if this will bite me later)

    5. added three extra libraries “-lnsl -lsocket -lposix4” to resolve a “symbol getaddrinfo: referenced symbol not found” relocation error with rlibmemcached.so (might only need libsocket)

  15. # gem install memcached
Building native extensions.  This could take a while...
ERROR:  Error installing memcached:
	ERROR: Failed to build gem native extension.

/usr/local/bin/ruby extconf.rb
Building libmemcached.
tar xzf libmemcached-0.32.tar.gz 2>&1
Patching libmemcached source.
patch -p1 -Z < libmemcached.patch
sh: patch: command not found
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers.  Check the mkmf.log file for more
details.  You may need configuration options.

Provided configuration options:
	--with-opt-dir
	--without-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=/usr/local/bin/ruby
extconf.rb:38: 'patch -p1 -Z < libmemcached.patch' failed (RuntimeError)
	from extconf.rb:28:in `chdir'
	from extconf.rb:28