infoSpace - Save Everything

Neil Beagrie on Personal Digital Libraries and Collections

tkiehne — Mon, 16 Oct 2006 07:37:50 +0000

I finally got around to reading Neil Beagrie's D-Lib article, "Plenty of Room at the Bottom? Personal Digital Libraries and Collections" (June 2005), and I regret not having done so sooner (alas, I have a great deal left in my "to read" folder). This article touches on several major themes in my academic pursuits of the last few years, which I will briefly describe here.

What drew me to the archival field was the overarching concern I have about the potential loss to our external memory in the sense of our information bearing objects. Being firmly seated in the digital generation, my concern is mostly over digital materials, and having completed my information science degree I find that, though I still worry about our institutions' digital preservation efforts, it is the enormous amount of personal digital information that people the world over possess that really worries me. Beagrie's article attacks this issue head-on, naming this body "personal digital collections" and enumerating not only the threat of loss, but the challenge these non-traditional collections pose to our "memory institutions."

Personal digital collections are subject to the same threats to persistence that the large institutional and academic projects are â€“ obsolete formats and media, access regimes such as passwords and DRM, and so on. Beagrie also enumerates missing data as a threat, with the parenthetical "email, webpages, etc." It seems that he means links to web pages, references to emails that have been deleted and so on, but I also wonder if mere information mismanagement is also intended? A recent episode in my own personal digital information management should elucidate.

As part of my ongoing audio encoding project, I have been preserving some of my own audio works from the last decade. I have also been checking my music collection, including these personal works, against an online discography database, Discogs.com. Every release in the Discogs database represents a physical object (CD, LP, etc.) released by a specific entity (record label), and lists not only the track information, but catalog information, liner notes, and cover art. As you can probably guess, the music that I created and released was not widely known or distributed (I still have a day job), so naturally there were no previous entries in Discogs.com. In the process of updating the database with my defunct label's releases, I found to my horror that I had lost some of the original digital files containing artwork and layout for some of my releases! Granted, I have not always been preservation-minded, but I had always assumed that these files were migrated from computer to computer over the past decade. Certainly lapses of this sort pose a significant hazard to personal digital collections, and I'm sure that it qualifies as "missing data."

Interestingly enough, my Discogs example also touches on Beagrie's discussion of "information banks." Although Discogs does not store the actual information represented in its indexes (the music), it is easy to visualize how it could were it not for the copyright regime so voraciously defended by the music industry. This worn argument aside, Discogs does implement a social networking component of the likes proffered in Beagrie's discussion of information sharing services such as blogs and sites like Flikr. By adding a social networking component, all of these sites, whether they publish unique user content or merely aggregate collected information (like Discogs), add a layer of informational value in the form of contributed information (e.g.: blog comments) or linked information (e.g.: relationships between artists in Discogs). But perhaps more importantly, the creation of these information banks, whatever their form, supports my assertion that digital preservation efforts must be aggregated at some level beyond a single (physical) entity's capabilities -- that only distributed efforts will ensure that digital assets are adequately preserved and accessed, let alone described and identified. This is as true for the National Archives as it is for Joe Q. Public's personal works.

As an aside, I could not help but notice that all of the talk about social networks and personal collections seemed to echo writings on digitally mediated identity by Danah Boyd. Beagrie's Venn diagram showing the definition of "public persona" begs comparison to Boyd's thesis work in faceted identity. I imagine that there is much to explore about the intersection of faceted identities or, for that matter, multiple personal public persona's, and the consequences to the "Lifetime Personal Web-spaces" concept mentioned at the end of the article.

In closing, one quote in particular caught my attention as it factors into my explorations into the "save everything" debate. Beagrie says (which he credits to Michael Lesk): "The combination of cheap digital storage and very sophisticated retrieval tools is shifting the balance of costs: digitally it is becoming cheaper to collect and more expensive to select, and cheaper to search than to organize." In other words, the scarcity argument is shifting from "we don't have enough space" to "we don't have the time to organize what we have," but as Beagrie seems to say, it no longer matters so long as you do not expect traditional access mechanisms. Or, more succinctly (with a nod to Catherine Stollar for originally expressing it): "what we do... will change, but why we do it does not."

Reflections on the SAA 2006 Annual Conference - Part II

tkiehne — Thu, 24 Aug 2006 03:25:50 +0000

This entry is a continuation of my observations on this year's SAA annual conference. For more, see Part I.

Plenary Session II: "Technology"

Each of the three plenary sessions was hosted once each by the three joint conference organization, the second one headlined by SAA. This year, SAA president Richard Pearce-Moses opened the session with a talk summarizing his work over the last year in exploring the "new skills" needed by archivists for the digital era. Between his writings in Archival Outlook (here and here) and the New Skills Colloquium in June I have already heard much of what he had to say, but it was nice to see it presented so succinctly to a room full of archivists â€“ a name drop didn't hurt, either!

There were a couple of key points that he made which validate opinions I have had and expressed in the past. One is that traditional archivists have a tendency to avoid the challenges presented by digital records â€“ paraphrasing Pearce-Moses: to hope that someone else will deal with it instead. Second, he essentially stated that if archivists do not rise to the challenge, other professions will. I have previously expressed my concern over how terminology and practices that technology vendors use come into direct conflict with those that archivists use so it was encouraging to hear it put to the audience.

Following Pearce-Moses was a talk by Brewster Kahle of Internet Archive fame, which was a pleasant surprise, mainly because I was curious to see how he would present the "save everything" argument in this venue. Kahle's presentation was decidedly oriented towards a lay audience, being rather shallow in scope and simple in terms of technical detail, but I can understand his trepidation over being inaccessible to a decidedly non-technical audience â€“ In fact, I have seen this happen on numerous occasions when tech industry professionals or computer science academics are asked to speak to librarians or archivists.

Aside from this lapse, however, Kahle definitely had a couple of key points to make and drove them home. One main point can be paraphrased as: we can save everything digitally because in the grand scheme of things it's not really that expensive. He threw out some general figures based on estimated amounts of data found in print, film, etc. and showed how these figures are inexpensive in an institutional or government context. Kahle didn't address appraisal and selection, which I am certain many in the audience would have loved to bring up, but I believe that addressing such concerns would have made for a significantly longer presentation. Second, he mentioned very little about preservation and preservation strategies and how they might impact the costs and requirements for storage and management. The main point he made about preservation was to reiterate the LOCKSS principle, saying essentially that the only proven way to keep information safe is to make lots of copies. But, I can understand why he would not delve too deeply into this topic as it brings into play discussion of formats, technological obsolescence, and of course, increased storage and costs. In summary, I appreciated his presentation as a general position statement, but I can easily imagine that few skeptics in the audience were turned.

The plenary session was wrapped up with a star appearance by "Cokie" Roberts, writer and ABC News correspondent. Her speech was quite entertaining, the content of which was mostly focused on her experiences in researching for her various books and how her experiences in advocating for breast cancer research could apply to helping fund archives and archival research. The most interesting part of her presentation was most likely an unintended argument for "save it all."

During her speech, Roberts discussed how difficult it was for her to find source documents regarding or by the wives and women related to the "founding fathers" for her book Founding Mothers. Some of the difficulty was due to the usual mis-management of documents, including deliberate destruction by the creators, but more of a problem was the fact that the perspectives of the women of the subject period were considered to be inferior to those of the men â€“ in other words, there was a conscious selection judgment made on the part of archivists not to keep such records. These decisions could be waved through as sexist or as some related conspiratorial power struggle, and no doubt some of it is, but the issue I keyed in on is that no one can know with certainty what will be of interest to future researchers. This is perhaps the strongest argument for "save it all," not only because of the value to users, but because it is not a technological reason. It is this one thought that weaved Robert's speech seamlessly into the previous two, a feat that is tempting to attribute to her renowned brilliance, but then again may just as likely be due to the latent inertia behind the notion to "save it all."

Exhibit Hall and Student Poster Sessions

Having presented a poster at last year's exhibit, I felt a responsibility to take a look this year's presentations. Two posters caught my attention this year. The first was "Search and Preserve: Collecting the Punk and Hardcore Communities" by Debi Griffith of the University of Wisconsin at Madison. I found this poster to be of personal interest for many reasons: First, it embodies a core argument behind my desire to save everything, that being that relying on conventional institutions and selection and appraisal can become biased against "fringe" or unpopular communities, thus ensuring a bias in or an incomplete cultural record. Another reason is that I have been a participant in some of these communities, from punk and alternative music, to industrial, techno, and experimental music. I am a semi-avid collector of DIY-style zines and publications put forth by these communities, a habit that started well before I had any idea about archives and such. My participation in these communities has taught me how the "mainstream" can easily, if not deliberately, misrepresent such movements and how important it is to ensure that the record includes the perspectives and views of the communities in question.

The second poster that I caught my interest was "Digital Object Identifiers and Resource Identifiers in Archival Description" by Krista Ferrante of Simmons College. This poster was fairly simple, presenting DOI and handle servers as a means of providing persistent identification of electronic records, but it did remind me that I need to finally get something together on the XRI/XDI specification in the archival context.

Session #508: "Future Shock: Saving the Signals of Audio-visual Records"

I attended this session for much the same reasons I attended session #208, that is, to validate the decisions that were made in formulating the CHAT preservation plan and see what new work had been done in digital video preservation and access since early last year. The difference between this session and #208 is that this session covered projects specifically dealing with audio, video, and audiovisual research rather than TV.

The first presentation was by Steve Weiss of the University of North Carolina at Chapel Hill, who presented his work with restoring and preserving African American cultural audio works. His presentation was heavy on demonstrations of the various music and voice recordings, but fairly light on process and lessons learned. One idea that I took from his presentation had to do with software for testing CD recordable media prior to use. All during the CHAT research I had not come up with such software, but it struck me as not only plausible but desirable to confirm recordable media before attempting to write data in order to avoid having to troubleshoot bad recordings after the fact. No specific software was mentioned, but knowing that such software exists should make it easy to find â€“ more research is necessary here.

The second presentation was given by Joanne Rudof of the Fortunoff Video Archive for WW-II Holocaust Testimonies. Rudof described in detail the process used to migrate and preserve a large number of Beta-SP cassettes of oral histories and testimonies. Much of the initial process she described sounded similar in concept to the CHAT plan: surveying and inventorying existing media, developing a "triage" plan to prioritize preservation efforts, etc. The major portion of the effort centered on the implementation of an experimental robotic system called SAMMA which comprised a semi-automated system for copying the existing cassettes to newer media and creating MPEG-2 digital surrogates. It was difficult to tell from the information presented how much material (in hours) was actually migrated â€“ one figure I heard was about 250 hours or 10 TB of MPEG-2 â€“ but the final number of cassettes migrated came out to over 2000. The mini-DV cassettes used by CHAT are newer and at less risk than those of the Fortunoff archive, but if the number of hours was correct, then we managed to develop a plan that took more time and individual work effort, but only a fraction of the cost of this project â€“ several hundred thousand versus a few thousand. I've been meaning to revisit the CHAT project in terms of results and I think the low budget aspect may be the tack to take.

Some research findings were presented, one set by Virginia Danielson of Harvard University, who gave an overview of her work with the "Sound Directions" project, and the other a short update by Jim Reilly, who was brought in by the session chair to discuss some of his work. Danielson's presentation focused on some of the best practices determinations made by her project, or as she put it, "not bad practices." One thing I noted to research from her presentation is the IASA TC-04 preservation manual. The main takeaway from Reilly's presentation was, paraphrased, that there is no single or simple cause of physical degradation of magnetic media. This reinforces my doubt, stated in the CHAT plan, over the long-term efficacy of tape media as an archival solution. As optical and disc-based magnetic media overtake magnetic tape in storage capacity, the days of tape media certainly seem numbered.

Notes: Diaries, On-line Diaries, and the Future Loss to Archives by Catherine O'Sullivan

tkiehne — Mon, 14 Nov 2005 06:16:54 +0000

Published in American Archivist, Vol. 68, No. 1, Spring/Summer 2005, pp. 53-73.

One thing that has become clear in the short time since I was brought into the archival fold, is that online (not merely electronic, mind you) information really puts old-guard archivists into a bind. Having been to one annual conference of the Society of American Archivists, I feel that this apprehension is palpable â€“ almost like the tension amongst people who do not wish to discuss the rather large and smelly elephant in the room. And with good reason. Hypertextual resources (Web pages, databases, and the like) dissolve the boundaries between groups of information by removing the physical constraints that have for so long defined information access. For someone who has defined their career's work in terms of discreet information objects of a physical nature, this lack of physical structure is likely to induce a sort of intellectual nausea.

Not that all archivists are cranky old dead tree advocates, but in my opinion, the momentum towards the issues inherent in the preservation of online information has yet to build up. The crux of the problem, and the solutions that are to be devised, lie in a common area -- technology. Having come from the Web design and applications development field, these challenges are still formidable, but I am not intimidated by the technology itself. A greater understanding of the technology is needed in order to make progress in preserving the information products contained therein. (much more is required than merely understanding the technology -- intellectual property is one of the big ones -- but I digress)

It is to this end that O'Sullivan seems to approach. Much has been made about the influence of blogs in media and other social and political venues. From the perspective of a long time online information users, this hardly seems worthy of note; After all, BBS systems, USENET, and Web message boards have been around "forever." So what makes blogs so special?

O'Sullivan takes a literary approach to analyzing blogs, likening them to written diaries, which provides a clues as to why blogs are different. Her research into the styles and types of diaries over the centuries indicates a form of information that has changed from that of a highly formal, often religious imperative, to that of casual social observation of a very personal nature. Bringing blogs back into the picture, this is part of the hype that the ephemeral nature of email lists and BBS's were unable to capture â€“ the highly connected, personal observations so prevalent in blogs and online journals. But that is not all. The main observation here is that blogs are the literary descendant of diaries, an assertion which is verified by the availability of non-technical tools for publishing them (like Drupal -- touchÃ©). Just as paper diaries only require knowledge of written language and simple technology (i.e.: paper and pen or some analog thereof), blogs also require language knowledge (in some cases merely spoken, as is allowed by some experimental blog transcription services) and simple technology, which these days includes internet access and computer with browser â€“ all freely available at your public library or on the cheap in numerous other ways. The diary metaphor is certain to provide much comfort to heretofore squeamish archivists -- it is a boundary, after all, that is still quite elusive, but can be grasped without too much knowledge of the underlying technology.

Having made the link between old media and new, O'Sulivan moves on to describing the magnitude of information wrapped up in blogs and some discussion of similarities and differences between the two. Using a brief explanation of the ephemeral nature of blogs, and thus the threat of a great loss of cultural information, she segues into the question of preservation. As can be predicted, the Internet Archive is brought up, to which she quickly acknowledges the WayBack Machine's inherent (ironically) archival limitations. From this, she retreats to the comfortable territory of acquisition and appraisal, that is, leaving the problem up to individual, existing collections to decide which resources are worth preserving in order to keep a more complete record, including much of the context that surrounds, and lends authenticity to, the resource. Such tactics continue, however, to support the retention of only what is deemed at the time to be "important" -- which inevitably leads to preferential treatment for high-profile resources. I might as well state at this point my undying devotion to the idea of a balanced, complete cultural record, not merely of the rich and famous. Besides, as a wise woman once challenged me, since storage is inevitably going to be cheap as air, why not keep it all? Indeed, why not?

It is here that I diverge from the reductionist view, that is, narrowing our focus to one type of online information resource, and throw out this proposition: in order to gather a complete record of the global internet, we are going to have to throw open the gates of archival process and make it available to the masses. Do I expect archival theory to become familiar to the common person? Absolutely not. What I propose is that the systems we use, and the technology of archival storage, be made easier to grasp to the common person, and as with blogs just as easy to use. These are no small tasks (the information retrieval and integrity aspects are immense in and of themselves), but they are what I have defined as a life's work for myself.

Perhaps this will be distasteful to the archival establishment, but this proposition might imply a break with formal diplomatics since we aren't talking about property records or a succession of command, but the reconstruction of a place and time with implicitly less dire consequences for imperfection serving considerably less skeptical needs. The lives of common people and organizations can contribute greatly to the understanding of the context in which greater political and social shifts occur. As our cultural record moves online, the imperative to preserve a wide swath of it increases as well. Technology alone will not provide all the answers, and O'Sullivan rightly brings together the new and the old in order to more fully engage the archival mindset in the problems at hand.