snax

memcached gem release

One of the hardest gems to install is no more. It’s now easy to install!

Memcached 0.15 features:

Install with gem install memcached. Since libmemcached is bundled in, there are no longer any dependencies.

on coordination

Andreas Fuchs suggested several months ago that I include libmemcached itself in the gem, but at the time I resisted. I was wrong.

My opposition was based on the idea that libmemcached itself would be an integration point, so running multiple versions on a system would be bad.

In real life, the hash algorithm became the integration point, not the library itself. And since the library’s ABI kept changing, the gem always required a very specific custom build. This annoyed the public and caused extra work for my operations team, who had to make sure to upgrade both the library and the gem at the same time.

Updates can come thick and fast now because I don’t have to worry about publishing custom builds or waiting for the libmemcached developers to merge my patches.

In retrospect it seems obvious—it’s always a win to remove coordination from a system.

linker woes

Unfortunately, it was easier to make that decision than it was to implement it. Linux and OS X link libraries differently, and I had a lot of trouble making sure that no system-installed version of libmemcached would get linked, instead of the custom one built during gem install.

When you link a shared object, OS X seems to maintain a reference to the original .dylib. Linux does not, and depends on ldconfig and LD_LIBRARY_PRELOAD to find the object at runtime. Since you can’t modify the shell environment from within a running process, there’s no way to override LD_LIBRARY_PRELOAD, so I needed to statically link libmemcached into the gem’s own .so or .bundle.

The only way I could do this on both systems was to configure libmemcached with CFLAGS=-fPIC --disable-shared, rename the libemcached.* static object files to libemcached_gem.*, and pass -lmemcached_gem to the linker rather than -lmemcached. Otherwise the linker would prefer the system-installed dynamic objects, even with the correct paths and -static option set.

Note that you can check what objects a binary has linked to via otool -F on OS X, and ldd on Linux.

Feel free to look at the extconf.rb source and let me know if there’s a better way to do this.