Here are the slides (all one hundred and thirty-freakin'-five of them) from the keynote/"vision" talk I gave yesterday morning in Louisville, KY at NASIG 2007. I was very sorry to miss most of the rest of the conference, but was quite glad to have the chance to present this talk to a very engaged crowd (especially considering it was 8am on a Sunday) and catch up with several friends I hadn't seen in ages after the talk.

The name of this talk is -

A New Approach to Service Discovery and Resource Delivery

...and that's what it's about.

In it I reprised some slides and concepts from previous talks I've given at a NISO meeting last November and code4libcon this past March and a few more slides from an earlier Access talk. But rather than just rehash slides explaining why COinS and unAPI are useful, I tried to place their potential benefits in a critical light, and from that perspective, I tried to state a much higher standard we need to try to reach for, and not settle for less, and one obvious (to me) way to get started.

That path is through the dynamic service links we now see everywhere. In this talk you'll see a series of slides about 2/3 through that start to explain what I think we should work on next - a way to unify interfaces to those peskily incompatible service link boxes that would open up a ton of doors that would remain closed even if COinS and unAPI both really exploded.

So have a look, and, if they don't make sense on their own, rest assured that a healthy number of serials librarians (at least a good 1/3 to 1/2 out of several hundreds at one point or another) seemed to be nodding vigorously in agreement with the proposal early yesterday morning.

I'll do my best to give it a more direct blog writeup in the next week or so, but I'll be offline a lot this week for some old-fashioned Stuff To Do and Places To Go. In the meantime, enjoy the 135 slides. :P

Looks like the good folks employing me over at Computers in Libraries chose to publish my April 2007 "Libraries in Computers" column online. This might be a fun one for anyone reading this who isn't a Computers in Libraries subscriber.

I'm happy about the way this one turned out - it expresses something I've been thinking about a lot lately, that it's important to remain objective about the tools and standards available to choose from. This is especially true for those of us (me included) who've been involved in standards development and hold particular standards dear due to the blood, sweat, and carpal-tunnel pain we've earned during the development of those standards.

If you're really into standards, get your hands on a print copy of the April 2007 Computers in Libraries (27:4) because it's also packed with brief primers on all of "Atom, COinS, MADS, MARC 21 / MARCXML, MIX, MXG, OpenSearch, PREMIS, RESTful HTTP, unAPI, XMPP (aka Jabber), and ZeeRex", among which I wrote the ones on Atom, COinS, OpenSearch, RESTful HTTP, and unAPI (but you should read 'em all, natch :).

In case you didn't catch the updated text at the top of the last post here, Jon Udell kindly followed up on a comment he left here recently and interviewed me for his Friday podcast about OpenURL and other topics persistent and linkish.

...in which I:

• try to explain OpenURL linking
• fail to recognize or usefully distinguish the differences between archive.org and arXiv.org (I just heard "archive.org" and went with that even though Jon meant "arxiv.org", at least early on, I think)
• reference the 50+-year research and publishing history of Eugene Garfield
• dredge up the old DSpace identifier wars
• discuss some issues with the Handle System and the centralized/distributed governance spectrum
• describe COinS and how to use COinS browser extensions at your library
• highlight the COinS Generator, the OCLC OpenURL Resolver Registry, and LibraryFind projects
• crib Jon's notion of a publishing surface area and reference FRBR as a framework for considering the surface area of a distinct intellectual or artistic creation
• say "Mess is Lore" and reference the Book Arts Press valentines the *week* of valentine's day without saying they're valentines
• mention the Computer History Museum
• try to convince Jon that he should find an archives to work with to start collecting his papers (online and otherwise) and correspondence without actually saying that so succinctly or usefully
• give Jon the scoop about my upcoming new job

It was a real pleasure to get to talk with someone whose writing I've read for many years and to appear on a podcast I've been listening to since its inception. Jon's long-time crossover interest in libraries (c.f. Library Lookup) is one of the strongest bridges we library hackers have to the broader software and systems communities. It's also a privilege to follow great folks like Tony Hammond, both John Blyberg and Ed Vielmetti, John Wilkin, and Lou Rosenfeld (both of whom I was lucky enough to interact with while at grad school at umich, check the podcast feed for these older, pre-Jon's new blog links) as a library-interested interviewee.

Unfortunately I've been too busy to put together more Library Geeks interviews, but I plan to start that back up again once I settle into the new gig, I promise. It's easier to be the one asking the questions!

(This is part two of a series that began here.)

If you've absorbed the weight of the question "why can't our libraries connect up to each other as easily as iTunes?", here's another question for you: is the landscape of November 2007 is going to look anything like that of November 2006?

Think about this. Three years ago, did you know about del.icio.us? Two years ago, did you know about flickr? Last year, did you know about youtube? Face it, we are living in an era of enormous *churn*. Six months ago myspace was headline news; now it's headline news that it's not headline news anymore. How many feedreaders have you tried? How many computers have you owned, and how different in capacity and design were they?

The great thing about churn is that innovation and quality typically wins out, but even yesterday's great ideas become passe if they aren't constantly improved and made continuously relevant. Ever hear the maxim that four out of five restaurants fail within five years? Well, most software products fail too. Proprietary vs. FLOSS, windows vs. mac, desktop vs. webapp, it doesn't really matter - for every dozen or so apps you know and love and have used for a while there are probably hundreds that you probably don't know about, and either way, nearly every one of these things goes away over time. How many restaurants do you know in your area that have been open your whole lifetime? And how many software applications have you used -- in their current form -- for more than a year? Probably a few of each, right? What fraction of *all* the restaurants and applications that you've ever tried is that?

Things are particularly churny just now because it's gotten much easier to build apps and there's a lot of money in the system incentivizing eager hackers. And this is just as true of Google as those two guys you know hanging out in their basement - Google, like 3M and other great innovative companies before it, is laser-focused on making it easier for their own people to build new, innovative applications, and throws a lot of money at the people who build the new ones that stick.

I think it's safe to say that the churn isn't quite ready to slow down.

But here's the thing: we are.

C'mon, you know you are. For all the hype of web 2.0 and (horrors!) library 2.0 and whatever it might be about these hype memes that gets people excited (though there's something deeply important about the architecture of participation that matters, but I'm writing about that elsewhere, and it's orthogonal here), you are getting tired of all the new apps showing up in your alphablogger feeds every damned day. There's not enough time!

More important to us here, though, is: what do you do with all the data you've strewn about in yesterday's hot sexy webapps? In part, you make a judgement about longevity and trust and maybe even pay a few (I pay for flickr and imho it's still $25/yr well-spent but next year I might change my mind, yknow?) to help them sitck around for a while. But there are generally two classes of new apps that have your data, and I'm splitting them on what you'll want to do about all your data when those apps are no longer quite so new to you. You'll either want to:

1. keep it around for a while - so you'll pay, or back it up somewhere, or
2. forget all about it

I must have 50 accounts in 50 different online apps from newspapers to feedreaders to media sharers to networkers. I don't care about nearly all of them, but I really *do* care about the two or three that I still use, and which I've invested some time and energy into. Actually, that ratio starts to feel close to the what succeeds/what fails ratio, doesn't it?

And when you think about that some more, there's another important factor to consider: it might be likely that the apps to which I grow loyal might be broadly popular, but there's *nothing* guaranteeing that those apps will be among the few that stick around. Or that even if they stick around, that they won't get bought up and have their terms of service changed and blah blah blah not be interesting anymore.

Basically, it's a crapshoot. It's your data, but although it's most interesting in its current application (and social, perhaps) context, there's no guarantee that that dynamic won't change. Rather, you should assume that it will.

If you work in a library, there's no possibly way to profile your users like you could even five years ago. Does "uses Word, EndNote, IE, and Outlook, and that's pretty much it" sound familiar to any of you? Well, that guy left town and there's a random scattering of people and apps and combinations of both in his place and the combinations are always changing and we have to serve them all.

So you're left with your data strewn among apps that tend to fail, and your users are in the same situation, but with a spectrum of apps. I wonder what the distribution of those apps would be... but I'm guessing the chance of most of those people and their problems with their data and their apps being similar from one case to the next is *way* lower than it's ever been since the PC era began, and falling steadily.

The problem becomes: how can we move our data in and out of these diverse apps reliably? Back in 2004 several colleagues and I wrote an oddly named paperthat anticipated this:

"...it seems clear that we are in the middle of a wave of innovation and integration of these new services. The common thread running through these innovations is that each new service helps individuals move and connect more kinds of information from more diverse resources through the various information communities in which they participate. We are still at a stage where each innovation adds value within a well-defined community or information context, even while we are learning that we will have to meet the needs of users who regularly move between formal and informal communities, and between public and private contexts. Before long, our ability to meet these users' needs will be limited by our inability to allow users to create and connect information sources and services as they see fit."

I can't help but want to repeat that last line:

"...our ability to meet these users' needs will be limited by our inability to allow users to create and connect information sources and services as they see fit."

To allow people to create and connect sources and services as they see fit, we have to have some sort of common interfaces that apply to all sources and all services. There's no chance otherwise -
the complexity is too great. The flickr API is cool, but, well, so is the amazon API, and the basecamp API, and the delicious API, and the google API, and the yahoo API, and the xISBN API, and well you get the point. Do you want to have to build and maintain software to match the Churn API?

I don't. I want one or a few APIs that do one or a few common things and I want them to work as part or all of *every* API.

The cool thing is that this is happening now, largely with OpenSearch and the Atom Publishing Protocol. Which is way cool. If you don't believe that, check out the new Open Media Profile for OpenSearch from Six Apart, the people who thought up trackback. But more on OpenSearch later.

What does all of this have to do with COinS and unAPI?

I don't run a sexy web2.0 site. Indeed I run the least sexy social bookmarking service ever (but I still love it). And I work in libraries, and I have friends who work in libraries, and among us are people who write interfaces and have control of how our library sources and services appear on our users' desktops. There aren't many great library APIs yet, but one of them is OpenURL. Sadly version 1.0 of the spec turned out way too complicated, but it's widely implemented nonetheless across the STM journal publishing industry and there are well over 1000 OpenURL resolvers in university libraries worldwide. OpenURL is an attempt to push data around consistently between publishers and library services, but it doesn't typically get presented consistently in our browsers. This gives us the too many APIs problem all over again.

COinS is an attempt to make OpenURL work the *same* way in our browsers. If you publish OpenURLs, please read the COinS spec and tweak your site templates to publish your OpenURLs in COinS. We're starting to see the benefits of this approach: the biggest non-profit book database in the world (worldcat) supports COinS. The biggest user-generated encyclopedia in the world (wikipedia) supports COinS. Two awesome firefox extensions (zotero and libx) support COinS. This means that with COinS, and with things set up properly in your browser, you can get big benefits from the common OpenURL interface rendering today: browse worldcat or wikipedia, save good references for later citation in zotero: two great tastes that taste great together.

unAPI is a slightly different fit. Like COinS, it specifies how to publish certain HTML fragments, this time for object identifiers. Unlike COinS, though, it also specifies a simple set of methods to call on a server to get those objects from the server in whatever format the server makes available. That's all it does: the spec is so short it prints out onto 1.5 pages (with another page of helpful notes). We've been able to add unAPI support to weblogs, OPACs, citation databases, and wiki frameworks, and fake it as a proxy layer over otherwise sexy-yet-disparate APIs like those from flickr, amazon, pubmed, and OAI-PMH-based resources. That's a pretty wide spread.

This part has already gone on too long - our network connection went out this morning when I was about four paragraphs up from here, so I know the tone changes a bit. But here's the last point I want to leave you with for now: churn is temporary. In a few years, or a decade, or two, things will settle down, and there will be common interfaces that work on all resources and that will make creating and connecting sources and services much easier for everybody.

Until then, though, we have a *lot* of work to do. I don't know if COinS and unAPI will become those common specs - honestly, it's safer to assume that they won't - but they're helping us to figure out what those specs are going to have to accomplish. And they're cheap and easy and fun and actually useful today. So check 'em out, you might find something you like.

But they're not without other issues. I'll dive into those in part three.

I worked up and have been giving a new talk recently about the directions I think library services should go. As far as I could tell, they were quite well received by both technical and non-technical audiences alike, and I've been exchanging messages with several people offline about it in one way or another.

The more I think about it, the more I'm convinced that this direction, stumbled into with gentle nudges and nudzhes alike from people like Ed and Jeremy and Terry, is the right next set of steps to take. I'm not sure how the outcome of it all will turn out, but I don't think we can afford not to try, and soon. The best thing about it all is that it's all inexpensively accomplished, or at least prototyped, and we should be able to find out quickly whether it's Good (tm) or not.

The thing is, it would be better to engage a *lot* of people on these topics, and this kind of project, because we'll only see if it works if it can be taken up quickly and broadly. So this is the first of a series on ZeroConfMetaOpenSearch, introducing the plan one piece at a time, spinning out what's more compactly worded in those slides.

Where to start?

Imho the best place to start is with a challenge. Have you ever used iTunes? More specifically, have you ever used its music sharing function? It's awesome. You can see what other people near you are listening to, and you can listen to their stuff, and they can find and listen to yours. The best part about it is that once you enable the feature in your iTunes preferences, it Just Works (tm). There's no configuration needed - the protocol works on top of a local-net service discovery protocol that is the most magical protocol this side of HTTP itself.

So the challenge is: what are we going to say when our library patrons start asking us: "why can't we share our personal libraries with your big library as easily as we can share music with iTunes?"

Got an answer?

There is no good answer, because there's no reason we can't apply a glossy sheen to our sloppy mess-o-OPACs and metasearches to play very nicely with others, just like iTunes does. (Actually, afaik, delicious library already does use the same networking protocol for this exact purpose, but, we don't all have that, do we.)

When I say "there's no reason why we can't", what I really mean is that everything we need to do it is already sitting around waiting for us to plug in. Just like on the front page of today's WSJ, featuring how the Zimbra folks assembled their Exchange-competitor largely out of FLOSS components, nearly all the components we need are already available. But the key thing is that they have to plug in easily, cheaply, and *universally*, with no "first we need a new standard to do X" step. And they cannot require end-users to have to plug anything in or install or configure anything. To accomplish this, what we really should be doing is using the best existing standards, even if they're (gasp!) not from the library community.

In the talk I use these three tests to judge whether a new approach adequately meets these objectives:

1. Is it *simple*? i.e., can any competent coder grok it in a quick sitting, and churn out compliant apps in a few hours?
2. Does it require a new specification? We don't have time form more committees, I fear... we have to move forward with what's already here.
3. Does it Just Work? Do normal people have to actually follow any kind of elaborate instructions to make it happen, or stand on their head, or sign something, or offer sacrifices and pay for a warranty with a mail-in rebate? 'Cuz those are right out.

iTunes meets all the needs above (we could quibble about DAAP and #2, but, it's zeroconf and HTTP that make DAAP possible as much as DAAP itself). We have to, too.

Where is this all leading? Essentially, to the three points listed here. In the next few posts, I'll spin out toward each one with a little more detail, and will hopefully draw some of you into the discussion along the way.

Next time, I'll evaluate how COinS and unAPI stack up against the three tests listed above. (Then we'll get into the collapsing of metasearch and resolver interfaces into a single interface, and after that how OpenSearch could layer usefully over all of that, and then I'll close with the ZeroConf piece and why it ties everything together so neatly.)

But first things first: shouldn't our whole libraries be as easy to connect and share as iTunes?