snax

ruby performance

sweeper

Automatically tag your music collection with metadata from Last.fm.

what it is

A while back Last.fm released a command line tool to retrieve metadata for an arbitrary mp3 from their new fingerprint database. I tried it yesterday and it seemed way better than MusicBrainz. So, as a person with a lot of random mp3s, I cooked up a script for retagging entire folders of songs.

Some neat things used in the script:

  • id3lib-ruby for handling mp3 tags
  • Text for calculating Levenshtein distance to the nearest correct genre name—amatch is a compiled version of the same thing, but not Windows-compatible
  • the incredibly comprehensive Last.fm API
  • XSD::Mapping for parsing the XML responses (better than Hpricot for small, well-formed documents)

A handy feature in the script is the ability to add the top 10 tagged genres to the comment field, so you can use iTunes or Foobar smart playlists for fancier multi-genre sorting. This is similar to lastfmtagger, but not Mac-specific.

demo

Before running sweeper --genre:

$ id3info 1_001.mp3
*** Tag information for 1_001.mp3
*** mp3 info
MPEG1/layer III
Bitrate: 128KBps
Frequency: 44KHz

After:

$ id3info 1_001.mp3
*** Tag information for 1_001.mp3
=== TPE1 (Lead performer(s)/Soloist(s)): Photon Band
=== TIT2 (Title/songname/content description): To Sing For You
=== WORS (Official internet radio station homepage): http://www.last.fm/music/Ph
oton+Band/_/To+Sing+For+You
=== TCON (Content type): Psychadelic
=== COMM (Comments): ()[]: rock, psychedelic, mod, Philly
*** mp3 info
MPEG1/layer III
Bitrate: 128KBps
Frequency: 44KHz

quickstart

Documentation is here, but for OS X:

sudo port install id3lib
sudo gem install sweeper
sweeper --help

Linux is similar to the above, depending on your distribution.

On Windows, you can just download a zipfile from the Rubyforge page and extract sweeper.exe to somewhere in your path.

I expect this to be eventually replaced by an official Last.fm tool, but for now, patches are welcome. It would be especially nice if someone could write a tutorial to help non-Ruby people install the script.

If you are going to contribute some code, grab the SVN checkout from Fauna, since the gem doesn't ship with the test mp3s.

SVN, I know—how embarrassing!

April 13, 2008

22 comments

machinehuman says (April 14, 2008):

Nifty. I'll give it a try. Thanks for sharing.

doktorjung says (April 15, 2008):

I suppose I'm one of the non-Ruby people. I got Ruby installed well enough on my WinXP machine, but the sweeper install fails on installing amatch due to a mismatch between MS Visual C versions used to make Ruby (version 6) and the version 7 I have installed. I guess I'll post a solution if I figure it out, short of finding a version 6 to use or re-compiling Ruby under 7.

Sounds like a neat idea, though. Thanks for making it!

evan says (April 15, 2008):

My fault. I didn't realize the amatch gem used a compiled extension...it's not Windows compatible. I switched to the Text gem instead which should be cross-platform.

Try again with version 0.3.

Daniel Fischer says (April 16, 2008):

I'm trying to install the sweeper gem on osx and I get this error on building native extensions:

checking for id3.h... no
You must have id3lib installed in order to use id3lib-ruby.
...
--ruby=/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby

The thing is, if I run sudo port install id3lib it says I already have it.

evan says (April 16, 2008):

Since you're using the OS X Ruby, and not MacPorts Ruby, you probably have to specify the library location. Something like:

sudo gem install id3lib-ruby -- --with-id3=/opt/local/

Daniel Fischer says (April 17, 2008):

Sorry, I know this sounds so noobish but I'm having a hard time getting this installed and it looks really useful.

I tried the above, and I got no luck.

I also tried compiling it manually and I can see it in /usr/local/lib/libid3

I'm trying sudo gem install id3lib-ruby -- --with-id3=/usr/local/

And no luck with anything I try. Arg :(

evan says (April 17, 2008):

I guess try putting the libid3.dylib or whatever it is in /usr/lib, maybe it can find it there. There is also a chance it might work if you copy it to the /ext/ folder of the gem and then manually run ruby extconf.rb and make.

I don't use Leopard or the built-in OS X Ruby so I can't really help you much more; sorry.

doktorjung says (April 17, 2008):

Thanks for the change, Evan. However, now I get a buffer-error somewhere. This is kinda spammy, but I printed a backtrace:

C:\>gem install sweeper --backtrace
ERROR:  While executing gem ... (Zlib::BufError)
    buffer error
C:/ruby/lib/ruby/site_ruby/1.8/rubygems/package.rb:621:in `initialize'
...
C:/ruby/lib/ruby/site_ruby/1.8/rubygems/command.rb:70:in `invoke'
C:/ruby/lib/ruby/site_ruby/1.8/rubygems/command_manager.rb:121:in `process_args'
C:/ruby/lib/ruby/site_ruby/1.8/rubygems/command_manager.rb:92:in `run'
C:/ruby/lib/ruby/site_ruby/1.8/rubygems/gem_runner.rb:30:in `run'
C:/ruby/bin/gem:23

It installed a number of dependencies successfully, and now just gets the error immediately upon running the install command. Dunno if the trace helps, but thanks for at least trying to support multiple OSs.

Mike says (April 17, 2008):

I had some trouble getting libid3-ruby installed with the instructions here. I finally had success by downloading libid3 directly and manually installing it, then using the install instructions at libid3-ruby: http://id3lib-ruby.rubyforge.org/doc/files/INSTALL.html

Thanks for the tool! It's running now and is looking great!

evan says (April 17, 2008):

doktorjung: Try this. Sorry it's so difficult... it installs fine for me in Windows on Rubygems version 1.0.1.

Mike: Thanks for the tips!

Seth Thomas Rasmussen says (April 17, 2008):

You forgot to mention Choice in your list of neat stuff! I was looking around at ways to do that kind of thing, and I dare say it looks like the nicest choice (I didn't plan that) I've seen so far.

evan says (April 18, 2008):

doktorjung: I bit the bullet and made a Windows pre-packaged binary. Download a zipfile from here and the included sweeper.exe should work on any system, even without Ruby installed.

Thanks for your patience.

Daniel Fischer says (April 19, 2008):

Thanks to Mike I figured it out. I had to run this:

sudo env ARCHFLAGS="-arch i386" gem install id3lib-ruby

Then it compiled properly on Leopard. :)

evan says (April 19, 2008):

The script is currently returning errors because of the recent Last.fm outage...

chris says (April 20, 2008):

evan, this might be a stupid question, but I am getting "Skipped (Fingerprint failed)" error message. Is this due to last.fm outage too? I'm asking because I am not sure if the error is in generating fingerprint or matching the fingerprint.

Daniel Fischer says (April 20, 2008):

Ya I get what Chris is getting as well as this: Unknown error: #<nomethoderror:>

Jack Danger Canty says (April 22, 2008):

This is what finally worked for me on Leopard with macports:

sudo port install id3lib
sudo env ARCHFLAGS="-arch i386" gem install id3lib-ruby -- \ 
  --with-opt-dir=/opt/local/

evan says (April 22, 2008):

Man, so much trouble on Leopard.

Anyway, the Last.fm servers seem to be back up. Sometimes you might still see a skipped error, but there's not much I can do about that. It only happens for me about 5% of the time. I'm not sure why you'd get nomethoderror; I've never seen that.

doktorjung says (April 23, 2008):

Hey, Evan. I actually tried the initial suggestion first (with the updated method in Ruby), and it seems to have built fine now. I get a few of the skipped issues you mention, and I can't see anything in the existing tags that's much different... Oh well.

Neat project. Now I just have to work up the courage to run it on everything. A backup's probably a good idea, heh. Any plans to support WMA, or is that not even possible using Ruby?

doktorjung says (April 23, 2008):

Just to add: my skipped error messages are a little different from the one mentioned above:

Skipped (XMLParserError - syntax error)

Maybe that's somehow enlightening. Thanks again.

W. Andrew Loe III says (May 12, 2008):

Is there a reason why a lot of my comments are random strings?

Like for AC/DC - Hells Bells I have:

00001BE4 00001A51 000058FF 00004AD4 0002E3E0 00021E9F 00008000 00008000 0000EA53 0000B931

But Last.fm lists some great tags: http://www.last.fm/music/AC%252FDC/_/Hells+Bells

Suggestions?

W. Andrew Loe III says (May 12, 2008):

Bah turns out that I did not pass -f to overwrite my old comments.

Add a comment

Various HTML tags allowed. Use <pre> for code blocks and <code> for inline references.