Service Discovery and The Big Library

Last week I attended the DLSR workshop [1] put on by the NSDL, DLF, and JISC/UKOLN. It was a great opportunity to meet numerous people whose work I'd admired from a distance for some time, and to also catch up with friends.

It was also a great time to do a little hacking.

During a portion of the day that filled somewhat (as these things do) with jargon handshaking, I took some time to fiddle with what DNS-based service discovery might look like. As a few of us (edsu [2], who first suggested it, and jaf [3], whose staying power that night was legendary) discussed at 3am PST one night during code4lib 2006 [4], Apple's Bonjour [5] pretty much does a huge piece of what we need to make library service registries fly, and there are already useful summaries [6] and an animal book [7] and both free software [8] and official implementations for both OSX and Win* [9].

So what's a repeat Access Hackfester to do, but to try to hack it up in realtime.

Here's what we did:

• I keep a regularly-refreshed copy of unalog.com on my laptop, and simply turned it on
• ifconfig told me which IP the local wireless had assigned evora, my ibook
• I wrote a little zeroconf-registration script for the unalog instance (with a name, description, the IP, and port number) on evora using pyzeroconf
• The service, with name, description, IP and port number, appeared in the Bonjour Browser [10] on my machine instantly, and
• It also showed up on my next-chair-neighbor's laptop instantly
• My next-chair-neighbor was able to drag that record and drop it into his browser, literally, and see evora's unalog instance

That was pretty cool. But, that's what bonjour/zeroconf does. Ever used iTunes's music sharing across a subnet? That's bonjour. Ever used the delicious library library sharing thingy across a subnet? Same thing. The best part of it all is that it's just DNS.

Why am I so excited about this?

Well, as some of us have written in the past [11], service discovery is something of a holy grail of One Big Libraryness. If you and your computer and your personal library can automaticaly, cheaply, and usably find and interact with your colleagues' personal libraries, and all of those can, in turn, talk to your institutions' big libraries, and they can talk to two libraries, and they can talk to two libraries, and they can talk to two libraries... you see where I'm heading with this.

Here's the technical tradeoff that makes me think this might be a magical solution: DNS administration is heavyweight. Few people do it; it's important to get correct; supporting it isn't cheap; it's mission-critical. Registering dozens-to-hundreds of constantly-changing big library services in DNS won't ever fly.

But adding one little SRV to DNS that simply says "the library is over $HERE" and rarely changes might.

If, at $HERE, you're running something like the IESR [12] or OCKHAM [13] registries, then anybody on your network can be automatically routed to local services when the application context is appropriate. And -- perhaps more importantly -- people *not* on your local network can ask your local network where to route you.

Since I've been obsessed with journal article linking for more than eight years now, here's an example of how that might work for getting to full text of an article from a citing reference (so obsessed I am, I was ranting about it just the other day [14]):

1. You visit some remote service.
2. The remote service asks your host (bonjour) and your network (normal DNS) where your registry is.
3. Finding it one way or another, the remote service queries your registry for and then caches a resulting pointer record to your local resolver.
4. You read something that cites an article you care about.
5. The remote service knows where your resolver is, so...
6. ...it simply sticks the resolver links where they need to be.
7. You follow the links to the article through your own library.

Note what didn't happen in this little scenario:

• The remote service didn't preconfigure a big list of IP addresses for your institution
• Your local institution didn't have to talk to the remote service to give them that big list of IP addresses
• You, the user, didn't have to install some wacky browser extension [15] to do the connecting

None of that happened. It just simply worked.

Or, well, at least it worked in my head.

I'm excited about this because it's a zero-configuration (there's that word again) answer for users. I'm excited about this because it speaks directly to existing widespread infrastruture. I'm excited about this because it might offer a reasonable handoff between the heavyweight maintenance piece (DNS) and a lighter-weight yet more ongoing maintenance piece (the registry). I'm excited about this because I'm a librarian and librarians like nothing more than to make catalogs of stuff and a service registry is really just a catalog of service-level records (as opposed to item-level records or collection-level records). I'm excited about this because *every* necessary piece is already in place and specified and accepted and adopted and scaled and inexpensive. Well, except for the local library service registry piece, but from what I could tell after having had the good fortune to spend about 20 hours with the world's leading experts in solving that particular problem, it's not too tough of a nut to crack, and we've already got a very good head start. Which is exciting.

So, yeah, I'm excited about this.