One Big Library.

(Last in a series.)

Last week I gave the talk I'd been leading up to. Here it is in a nutshell (photo by Nicole Engard):

The premise of the talk is: people are already starting to ask us "why doesn't the library work like iTunes?" By this they really mean (I think) "how come when I walk into the library everything in the library doesn't just show up on my machine like when I open iTunes and my friends' music just shows up on my machine?"

I think we need to take this question very seriously. Especially because there's no good reason we can't do just what that not-so-hypothetical user wants. A great candidate networking protocol (ZeroConf) is already out there, flowing in the network. We could layer OpenSearch over our metasearch interfaces, and intermingle search and resolver services fairly easily, and along the way *every* one of those steps would have side benefits: merging OpenURL resolution and metasearch UIs into a single view simplifies things for users; adding OpenSearch means you can put your library's catalog right into your users' browsers; using ZeroConf to announce your service makes it easy for everybody to find your interface from anywhere.

During the talk I demonstrated just how much ZeroConf is already flowing all around us - we're swimming in it!. Apparently the video didn't video, so to see how it works for yourself just try the follow the next time you're on a network with a lot of other users (like, a public wifi spot would be ideal, but it should also work at your workplace).

• If you haven't seen it before, turn on iTunes music sharing and look for people sharing their music. It's fun and incredibly easy.
• Install iStumbler and look at the bonjour services tab (note that this is also a useful tool for browsing wireless nets.
• Install Bonjour Browser to get a closer look at the network traffic underlying the more user-friendly screens in iStumbler.

The conference wireless AP was unreliable, so I bought my own and turned it on as an open wireless net (named "librarywebclique", in honor of librarywebchic). In the middle of the talk I invited participants to join the net so I could show the ZeroConf traffic in iStumbler and Bonjour Browser. In a matter of seconds, about 20 people had done just that and all of us could see who was sharing iTunes, who had visible web servers, SMB shares, or ssh ports open, among other things. From where I was standing, it was a pretty convincing demonstration, and all it entailed was showing everybody what their machines were *already* doing.

I can't stress enough how important it is to keep a close eye on what's already happening out there on the net at a global scale. The microformats.org project is frustrating in many ways but they're dead right in their insistence on recognizing what's already out there as primary (though I'd quibble over their self-consistency in enforcing that, but that's another story). This traffic is flowing on the network - lots of widely-used apps already use it - there's no reason we can't just hop aboard.

I'm attaching slides and audio as best I can, but imho it was the kind of thing you had to see - it's very easy to make a point about what's possible by showing that it's actually already out there. Still, try those apps above (they're OSX only, but the traffic isn't, so you can probably find similar apps on other platforms) and you'll get the point.

Near the end I took a few questions, but didn't think to repeat them. One was "what about remote access?", to which I responded that this model doesn't really improve much on that set of issues or make it much worse, though in theory by adding desktop-level discoverability it's possible it could improve things slightly. Another was "what do you think the interface to this would look like", which is a bit tougher to answer. There are examples like iTunes and Delicious Library to follow for what it should look like when a library just "shows up" on your machine. But I think my immediate answer was the right one - I don't know, but it could show up in a lot of places, like the browser, or a desktop app, or on server apps, because this is a platform/infrastructure level service opportunity. In one of those answers I stated that "if we don't provide this level of ease of use, then we're failing", and because we don't, I think we are. Failing, that is.

There is a lot of talk out there in biblioblogoville about "getting into the flow", but I can't imagine a better way to do that than to radically integrate our most flow-oriented services (metasearch and openurl service resolution) and advertise its availability on our networks. And it can be accomplished quickly, cheaply, uniformly, and with a variety of side benefits.

Which, of course, probably means that it'll never happen, because our community isn't really capable of this kind of realization, let alone acting quickly upon it at scale.

That doesn't mean that I don't think you should bother with it - of course not, I think this is all very doable, and I hope that I met my objective of trying to frame all of this in a positive light: see the last few slides, where I suggest how to respond to "can't we make the library work like iTunes?": we should respond by saying "yes, let's."

The slides are attached as pdf, and here's the audio to lead you through 'em. You can hear my clicky-taps to advance the slide most of the time.

The jake site at Yale is now closed. The project is officially over.

On a brighter note, LibraryFind is now available. The project is officially open.

I worked up and have been giving a new talk recently about the directions I think library services should go. As far as I could tell, they were quite well received by both technical and non-technical audiences alike, and I've been exchanging messages with several people offline about it in one way or another.

The more I think about it, the more I'm convinced that this direction, stumbled into with gentle nudges and nudzhes alike from people like Ed and Jeremy and Terry, is the right next set of steps to take. I'm not sure how the outcome of it all will turn out, but I don't think we can afford not to try, and soon. The best thing about it all is that it's all inexpensively accomplished, or at least prototyped, and we should be able to find out quickly whether it's Good (tm) or not.

The thing is, it would be better to engage a *lot* of people on these topics, and this kind of project, because we'll only see if it works if it can be taken up quickly and broadly. So this is the first of a series on ZeroConfMetaOpenSearch, introducing the plan one piece at a time, spinning out what's more compactly worded in those slides.

Where to start?

Imho the best place to start is with a challenge. Have you ever used iTunes? More specifically, have you ever used its music sharing function? It's awesome. You can see what other people near you are listening to, and you can listen to their stuff, and they can find and listen to yours. The best part about it is that once you enable the feature in your iTunes preferences, it Just Works (tm). There's no configuration needed - the protocol works on top of a local-net service discovery protocol that is the most magical protocol this side of HTTP itself.

So the challenge is: what are we going to say when our library patrons start asking us: "why can't we share our personal libraries with your big library as easily as we can share music with iTunes?"

Got an answer?

There is no good answer, because there's no reason we can't apply a glossy sheen to our sloppy mess-o-OPACs and metasearches to play very nicely with others, just like iTunes does. (Actually, afaik, delicious library already does use the same networking protocol for this exact purpose, but, we don't all have that, do we.)

When I say "there's no reason why we can't", what I really mean is that everything we need to do it is already sitting around waiting for us to plug in. Just like on the front page of today's WSJ, featuring how the Zimbra folks assembled their Exchange-competitor largely out of FLOSS components, nearly all the components we need are already available. But the key thing is that they have to plug in easily, cheaply, and *universally*, with no "first we need a new standard to do X" step. And they cannot require end-users to have to plug anything in or install or configure anything. To accomplish this, what we really should be doing is using the best existing standards, even if they're (gasp!) not from the library community.

In the talk I use these three tests to judge whether a new approach adequately meets these objectives:

1. Is it *simple*? i.e., can any competent coder grok it in a quick sitting, and churn out compliant apps in a few hours?
2. Does it require a new specification? We don't have time form more committees, I fear... we have to move forward with what's already here.
3. Does it Just Work? Do normal people have to actually follow any kind of elaborate instructions to make it happen, or stand on their head, or sign something, or offer sacrifices and pay for a warranty with a mail-in rebate? 'Cuz those are right out.

iTunes meets all the needs above (we could quibble about DAAP and #2, but, it's zeroconf and HTTP that make DAAP possible as much as DAAP itself). We have to, too.

Where is this all leading? Essentially, to the three points listed here. In the next few posts, I'll spin out toward each one with a little more detail, and will hopefully draw some of you into the discussion along the way.

Next time, I'll evaluate how COinS and unAPI stack up against the three tests listed above. (Then we'll get into the collapsing of metasearch and resolver interfaces into a single interface, and after that how OpenSearch could layer usefully over all of that, and then I'll close with the ZeroConf piece and why it ties everything together so neatly.)

But first things first: shouldn't our whole libraries be as easy to connect and share as iTunes?