One Big Library.

Here are the slides (all one hundred and thirty-freakin'-five of them) from the keynote/"vision" talk I gave yesterday morning in Louisville, KY at NASIG 2007. I was very sorry to miss most of the rest of the conference, but was quite glad to have the chance to present this talk to a very engaged crowd (especially considering it was 8am on a Sunday) and catch up with several friends I hadn't seen in ages after the talk.

The name of this talk is -

A New Approach to Service Discovery and Resource Delivery

...and that's what it's about.

In it I reprised some slides and concepts from previous talks I've given at a NISO meeting last November and code4libcon this past March and a few more slides from an earlier Access talk. But rather than just rehash slides explaining why COinS and unAPI are useful, I tried to place their potential benefits in a critical light, and from that perspective, I tried to state a much higher standard we need to try to reach for, and not settle for less, and one obvious (to me) way to get started.

That path is through the dynamic service links we now see everywhere. In this talk you'll see a series of slides about 2/3 through that start to explain what I think we should work on next - a way to unify interfaces to those peskily incompatible service link boxes that would open up a ton of doors that would remain closed even if COinS and unAPI both really exploded.

So have a look, and, if they don't make sense on their own, rest assured that a healthy number of serials librarians (at least a good 1/3 to 1/2 out of several hundreds at one point or another) seemed to be nodding vigorously in agreement with the proposal early yesterday morning.

I'll do my best to give it a more direct blog writeup in the next week or so, but I'll be offline a lot this week for some old-fashioned Stuff To Do and Places To Go. In the meantime, enjoy the 135 slides. :P

Looks like the good folks employing me over at Computers in Libraries chose to publish my April 2007 "Libraries in Computers" column online. This might be a fun one for anyone reading this who isn't a Computers in Libraries subscriber.

I'm happy about the way this one turned out - it expresses something I've been thinking about a lot lately, that it's important to remain objective about the tools and standards available to choose from. This is especially true for those of us (me included) who've been involved in standards development and hold particular standards dear due to the blood, sweat, and carpal-tunnel pain we've earned during the development of those standards.

If you're really into standards, get your hands on a print copy of the April 2007 Computers in Libraries (27:4) because it's also packed with brief primers on all of "Atom, COinS, MADS, MARC 21 / MARCXML, MIX, MXG, OpenSearch, PREMIS, RESTful HTTP, unAPI, XMPP (aka Jabber), and ZeeRex", among which I wrote the ones on Atom, COinS, OpenSearch, RESTful HTTP, and unAPI (but you should read 'em all, natch :).

I've added an xisbn response to isbn queries against OPA. The updated opa.py is attached; see earlier posts for the rest of the whole package.

Sample links:

• http://opa.onebiglibrary.net/?id=urn:isbn:1565923928
• http://opa.onebiglibrary.net/?id=urn:isbn:1565923928&format=xisbn

Note that neither this nor the earlier update is redirecting properly, rather they are republishing the remote data. This is a bug needing fixing, but that will have to wait for a full refactoring. (Hmm, that would be a good project during the upcoming code4lib conference...)

Holy cool, Mike Giarlo just refactored the WordPress unAPI plugin so that it fits cleanly into the WordPress architecture. And now it's under svn at wp-plugins.org.

I've heard more stirrings from down CHNM way that unAPI support for Zotero remains a priority... how cool would it be to be able to pull arbitrary blog entries as records straight into a citation database?

Must... update... wordpress... instances...

Hooray, mjgiarlo (and pbinkley)! :)

[Update: one down, two to go.]

(This is part two of a series that began here.)

If you've absorbed the weight of the question "why can't our libraries connect up to each other as easily as iTunes?", here's another question for you: is the landscape of November 2007 is going to look anything like that of November 2006?

Think about this. Three years ago, did you know about del.icio.us? Two years ago, did you know about flickr? Last year, did you know about youtube? Face it, we are living in an era of enormous *churn*. Six months ago myspace was headline news; now it's headline news that it's not headline news anymore. How many feedreaders have you tried? How many computers have you owned, and how different in capacity and design were they?

The great thing about churn is that innovation and quality typically wins out, but even yesterday's great ideas become passe if they aren't constantly improved and made continuously relevant. Ever hear the maxim that four out of five restaurants fail within five years? Well, most software products fail too. Proprietary vs. FLOSS, windows vs. mac, desktop vs. webapp, it doesn't really matter - for every dozen or so apps you know and love and have used for a while there are probably hundreds that you probably don't know about, and either way, nearly every one of these things goes away over time. How many restaurants do you know in your area that have been open your whole lifetime? And how many software applications have you used -- in their current form -- for more than a year? Probably a few of each, right? What fraction of *all* the restaurants and applications that you've ever tried is that?

Things are particularly churny just now because it's gotten much easier to build apps and there's a lot of money in the system incentivizing eager hackers. And this is just as true of Google as those two guys you know hanging out in their basement - Google, like 3M and other great innovative companies before it, is laser-focused on making it easier for their own people to build new, innovative applications, and throws a lot of money at the people who build the new ones that stick.

I think it's safe to say that the churn isn't quite ready to slow down.

But here's the thing: we are.

C'mon, you know you are. For all the hype of web 2.0 and (horrors!) library 2.0 and whatever it might be about these hype memes that gets people excited (though there's something deeply important about the architecture of participation that matters, but I'm writing about that elsewhere, and it's orthogonal here), you are getting tired of all the new apps showing up in your alphablogger feeds every damned day. There's not enough time!

More important to us here, though, is: what do you do with all the data you've strewn about in yesterday's hot sexy webapps? In part, you make a judgement about longevity and trust and maybe even pay a few (I pay for flickr and imho it's still $25/yr well-spent but next year I might change my mind, yknow?) to help them sitck around for a while. But there are generally two classes of new apps that have your data, and I'm splitting them on what you'll want to do about all your data when those apps are no longer quite so new to you. You'll either want to:

1. keep it around for a while - so you'll pay, or back it up somewhere, or
2. forget all about it

I must have 50 accounts in 50 different online apps from newspapers to feedreaders to media sharers to networkers. I don't care about nearly all of them, but I really *do* care about the two or three that I still use, and which I've invested some time and energy into. Actually, that ratio starts to feel close to the what succeeds/what fails ratio, doesn't it?

And when you think about that some more, there's another important factor to consider: it might be likely that the apps to which I grow loyal might be broadly popular, but there's *nothing* guaranteeing that those apps will be among the few that stick around. Or that even if they stick around, that they won't get bought up and have their terms of service changed and blah blah blah not be interesting anymore.

Basically, it's a crapshoot. It's your data, but although it's most interesting in its current application (and social, perhaps) context, there's no guarantee that that dynamic won't change. Rather, you should assume that it will.

If you work in a library, there's no possibly way to profile your users like you could even five years ago. Does "uses Word, EndNote, IE, and Outlook, and that's pretty much it" sound familiar to any of you? Well, that guy left town and there's a random scattering of people and apps and combinations of both in his place and the combinations are always changing and we have to serve them all.

So you're left with your data strewn among apps that tend to fail, and your users are in the same situation, but with a spectrum of apps. I wonder what the distribution of those apps would be... but I'm guessing the chance of most of those people and their problems with their data and their apps being similar from one case to the next is *way* lower than it's ever been since the PC era began, and falling steadily.

The problem becomes: how can we move our data in and out of these diverse apps reliably? Back in 2004 several colleagues and I wrote an oddly named paperthat anticipated this:

"...it seems clear that we are in the middle of a wave of innovation and integration of these new services. The common thread running through these innovations is that each new service helps individuals move and connect more kinds of information from more diverse resources through the various information communities in which they participate. We are still at a stage where each innovation adds value within a well-defined community or information context, even while we are learning that we will have to meet the needs of users who regularly move between formal and informal communities, and between public and private contexts. Before long, our ability to meet these users' needs will be limited by our inability to allow users to create and connect information sources and services as they see fit."

I can't help but want to repeat that last line:

"...our ability to meet these users' needs will be limited by our inability to allow users to create and connect information sources and services as they see fit."

To allow people to create and connect sources and services as they see fit, we have to have some sort of common interfaces that apply to all sources and all services. There's no chance otherwise -
the complexity is too great. The flickr API is cool, but, well, so is the amazon API, and the basecamp API, and the delicious API, and the google API, and the yahoo API, and the xISBN API, and well you get the point. Do you want to have to build and maintain software to match the Churn API?

I don't. I want one or a few APIs that do one or a few common things and I want them to work as part or all of *every* API.

The cool thing is that this is happening now, largely with OpenSearch and the Atom Publishing Protocol. Which is way cool. If you don't believe that, check out the new Open Media Profile for OpenSearch from Six Apart, the people who thought up trackback. But more on OpenSearch later.

What does all of this have to do with COinS and unAPI?

I don't run a sexy web2.0 site. Indeed I run the least sexy social bookmarking service ever (but I still love it). And I work in libraries, and I have friends who work in libraries, and among us are people who write interfaces and have control of how our library sources and services appear on our users' desktops. There aren't many great library APIs yet, but one of them is OpenURL. Sadly version 1.0 of the spec turned out way too complicated, but it's widely implemented nonetheless across the STM journal publishing industry and there are well over 1000 OpenURL resolvers in university libraries worldwide. OpenURL is an attempt to push data around consistently between publishers and library services, but it doesn't typically get presented consistently in our browsers. This gives us the too many APIs problem all over again.

COinS is an attempt to make OpenURL work the *same* way in our browsers. If you publish OpenURLs, please read the COinS spec and tweak your site templates to publish your OpenURLs in COinS. We're starting to see the benefits of this approach: the biggest non-profit book database in the world (worldcat) supports COinS. The biggest user-generated encyclopedia in the world (wikipedia) supports COinS. Two awesome firefox extensions (zotero and libx) support COinS. This means that with COinS, and with things set up properly in your browser, you can get big benefits from the common OpenURL interface rendering today: browse worldcat or wikipedia, save good references for later citation in zotero: two great tastes that taste great together.

unAPI is a slightly different fit. Like COinS, it specifies how to publish certain HTML fragments, this time for object identifiers. Unlike COinS, though, it also specifies a simple set of methods to call on a server to get those objects from the server in whatever format the server makes available. That's all it does: the spec is so short it prints out onto 1.5 pages (with another page of helpful notes). We've been able to add unAPI support to weblogs, OPACs, citation databases, and wiki frameworks, and fake it as a proxy layer over otherwise sexy-yet-disparate APIs like those from flickr, amazon, pubmed, and OAI-PMH-based resources. That's a pretty wide spread.

This part has already gone on too long - our network connection went out this morning when I was about four paragraphs up from here, so I know the tone changes a bit. But here's the last point I want to leave you with for now: churn is temporary. In a few years, or a decade, or two, things will settle down, and there will be common interfaces that work on all resources and that will make creating and connecting sources and services much easier for everybody.

Until then, though, we have a *lot* of work to do. I don't know if COinS and unAPI will become those common specs - honestly, it's safer to assume that they won't - but they're helping us to figure out what those specs are going to have to accomplish. And they're cheap and easy and fun and actually useful today. So check 'em out, you might find something you like.

But they're not without other issues. I'll dive into those in part three.