One Big Library.

Attached at a link at the bottom of this entry is a dashboard widget that lets you keep a list of items from unalog right on your OS X desktop. You can see 5, 10, or 20 recent items from the feed as a whole, or for any particular user or tag. Here's an example, for the tag "xkcd":

It was pretty easy to do using Dashcode and its RSS widget template. If you want to try writing your own widget, you'll find it easier if you already know some javascript+html and have some minimal event-driven UI development background to figure out which events need to happen and when. Or you could just grab some existing widgets you like and do what they do. I'm attaching the dashboard project source for this, too, in case you want to take this one apart.

If you use unalog and OS X, give it a try!

Dan Cohen, Josh Greenberg, and Dan Stillman of the Center for History and New Media at GMU joined me to discuss their work on Zotero, their about-to-be-released Firefox 2 based citation manager-cum-bibliographic everything keeper. "Z-Day", as they call the upcoming public release day, is scheduled for this Thursday, October 5, 2006.

There's a lot to be excited about in Zotero, as I wrote recently. Listen in and learn all about it straight from the source through the local podcast feed, iTunes, or straight to the mp3 file.

Show notes:

• To use Zotero, you'll need Firefox 2.
• Zotero uses the Citation Style Language, or CSL, by Bruce D'Arcus.
• Zotero can recognize, parse, and collect bibliographic information found in COinS.
• Support is planned in Zotero for unAPI.

Luckily, I was invited to have a look at the Zotero private beta several weeks ago. Today I finally had time to take a look.

Very first impression: Wow.

Second impression: Ooh, sqlite!

Third impression: This is why we wrote unAPI, and COinS.

I love the idea and am all for it and want to get hacking at it. A few thoughts on what I want to add follow below.

But first, here's what I did on my first look around:

Bring in a reference from our local library catalog, presumably via screen scraping a la LibraryLookup:

Bring in a book reference via worldcat.org, presumably through the COinS on the page:

Imported a swath of records from unalog via its MODS export function. Worked just fine, considering how the unalog MODS generation is a bit haphazard:

Pulled a reference out of CiteULike, presumably via a COinS:

Visited that reference through our local OpenURL resolver, which I'd configured by hand in Zotero's preferences:

I have a lot of little interface nits to mention but I won't presume that they're not all related to how awkward my brain remains at windows-navigation-shortcuts while using windows in OSX w/parallels. I wish items with a URL would allow linking back out through the url in the notes section at right.

Let's see how it does importing a 28Mb file of 10,000 MODS records derived from MARC... yknow, because I can. (Several minutes later...) Didn't work, but it still passes the test because (a) it's running in a lowish-memory virtual machine, and (b) nothing crashed. Failing silently is a good outcome. Especially because (c) it's their first beta release, and it's not even public yet.

I'd love to see Zotero become a unAPI consumer... I think this is in the works but I haven't checked in with them about it lately. In the meantime the first thing I'll do when I get time to poke around under the hood is look for this or where it should go, and see what it'll take to get going.

I want to be able to absorb my friends' Zotero databases and get updates. They've said they're working on this through some sort of central server-based mechanism for this, which makes sense.

My one concern about using sqlite is the concurrent-file-access issue. I'd like to write some code outside of Zotero that would use the sqlite file to expose OAI-PMH and OpenSearch interfaces. These should both be doable (last I checked multiple processes could read sqlite dbs safely, but only one can write safely). But, if I'm a remote user looking at my pal's Zotero through OAI-PMH or OpenSearch or Atom and I want to load some or all of their data into my Zotero, can I do it safely from outside of Firefox/Zotero? Probably not, today, though all of that presumably could happen on the consumer side strictly through web access, which might be enough, and even preferable.

I want to add zeroconf discovery so if I walk into a room with 28 other people using Zotero, I can bind in and metasearch/harvest all their data. And chat, too, but that's a whole other story.

In short - it's projects like these that take numerous ideas floating around and ground them in something practical that I can use right now that really wind my crank. The CHNM folks know their stuff -- they've worked on various flavors of this kind of app for years, and they're already most of the way to something lots of people could use today. I'll be keeping a close eye on it (especially because they asked me to!) and maybe you should too.

Finally had more time to work on the unAPI copy / Atom paste implementation in unalog. At this point, unalog will examine a "robust" jsondidl object saved with a unalog entry for alternate unAPI-formats, will provide links back out to them (with proper attributes rel="alternate" and type="[mimetype as per unapi]" on the anchors), and will display a small image for every saved entry if the entry contains at least one image of size 100x100 or smaller. Here's a screenshot:

I've tweaked the jsondidl model and the output from OPA (and updated the version running at opa.onebiglibrary.net) to now also include attributes "url" and "unapi_source" on resources. "unapi_source" is a direct link back to the unAPI URL from which the resource (in its specific format) came. "url" is a direct link to the original URL to which OPA redirects for unAPI format redirects. This allows unalog to both (a) provide links back to unalog users to the unAPI service for every format, and (b) render an item directly from its source link if that source link is public, rather than having to bounce back through the unAPI service.

Perhaps unalog should publish that image to the world right from its own stored copy. But, I'm not sure if that's a good idea... I don't want people using unalog.com to republish arbitrary data objects -- especially images -- for which they might not have the rights to do so. Users *themselves* should be able to see their own saved data, and maybe small groups of their friends or colleagues (unalog's had public and private groups since early 2004, so that shouldn't be hard to implement). But not arbitrary users looking at other users' images. But then again, there's nothing wrong with one user being given unAPI links back to an image; if the user can obtain rights to see the source data, they can navigate to it for themselves.

Working on this raises tons of issues. Here's a working list.

• Should users who've saved robust objects get sub-object bitstream access via a public url? How would that even work?
• Need to reconcile the distinction between the URI for entries to show arbitrary entries and the URI for Atom-GET. I'm very confused about whether these are the same things or not, and that unalog entries are going to become jsondidl objects just confuses me further w/r/to this issue.
• Which file formats other than images should receive "extra rendering treatment" in the public unalog UI? Like, if there's dublin core metadata, should it just render that inline?
• I think I will likely break down and agree with the sentiment that unAPI needs to be able to handle UNAPI?uri=FOO&uri=SPAM&uri=BAKEDBEANS calls with multiple format lists in the response. Separate hits for each of multiple URIs per page is just evil waiting to happen.
• Metadata pass-through: when additional metadata is available on a unAPI source object like a flickr image (description, "owner", created/updated dates, tags, etc.), what is the proper way for OPA to pass it through to the unAPI client? For flickr images, for instance, I want the option to auto-populate the unalog tags for an entry with that image source's flickr tags. Or the same for a blog entry marked up in hAtom, etc. What's the best approach... should the flickr metadata blob become another jsondidl resource, or a parallel item in its own right? For the hAtom blog entry, should OPA translate to AtomAtom? If not, should unalog translate to AtomAtom and render metadata from that? It gets tricky that way.

Not to mention that I still need to drop everything for a few days to completely rewrite the unalog "save this page/link" screen to incorporate all these features for pages with unAPI URIs and microformats, to newly bundle all the data up into a jsondidl via javascript and paste via Atom, among other things.

But first, let us retell the story of Exodus...

Here's a rambling, 18 minute screencast of me firing data through a social clipboarding service in tracer bullet mode. A quick summary of web clipboards leads into a demo of using unAPI to copy image data from flickr and paste it through unalog using the POST and GET functions of the atom publishing protocol and a json-wrapped data model based loosely on mpeg21 did.

I've never done a screencast before, so go easy.

Oh!, and I forgot one important thing. I left off showing that the pasted-in flickr data is available in unalog listed just like any other pasted-in link, with its tags and so forth. Here's a screenshot just to prove it: