infoSpace - Archives

Bringing Records to the Users

tkiehne — Tue, 17 Apr 2007 07:06:37 +0000

I've volunteered at the local National Archives branch for over a year now. Over this time I have gained the acute impression of lament over the dwindling number of researchers and members of the public who make the trip out to the archives to do research. Indeed, it is not uncommon for me to enter a virtually empty research room during my Thursday afternoon visits.

What is seldom spoken of, but is vitally central to the issue, is the instant information gratification that the general populace receives from their increasingly ubiquitous internet connections. The reams of information available through Google, or the convenience of accessing Ancestry.com from home (instead of for free at the archives), keeps them at home and only adds insult to the injury of fewer patrons.

NARA is not alone in this lament. Günter Waibel of OCLC recently spoke of it in a Hanging Together blog post covering the IMLS WebWise Conference, where he reposts a quote by Deanna Marcum of the Library of Congress that rather eloquently sums up the issue:

“What I think our challenge is, it is not enough for us to create the perfect finding system, we know from all the user studies that individuals, who are looking for information, go directly to the open web, and our marvelous catalogues are not getting used. We have to find ways to take our content and the metadata and move that content to the open web.”

NARA relies on an online system called the Archival Research Catalog (ARC) to catalog and describe its considerable physical holdings. This system was derived from an earlier prototype called NARA's Archival Information Locator (NAIL), which primarily contained images and other non-textual surrogates. Each of these systems are of late-1990's vintage – a Web lifetime ago – and, quite frankly, the user interface shows it. One of my contacts at NARA once stated that the government seems to be about a decade behind when it comes to information management, and in this case it seems true enough.

Granted, the scope of these projects is monumental and one cannot expect them to be updated at Web speed to, say, incorporate some of the more useful “Web 2.0” principles. However, given the exodus of potential patrons to full-text search engines, there is a particularly devastating truth hidden behind the aging ARC code. To paraphrase Marcum, we need to take our content and metadata to where the users are. As it turns out, some informal research provides a striking illustration of this need.

One evening I found myself looking through the Web site of the Northwest Digital Archives, a group of education, government, and private archives across the Pacific Northwest and Alaska. I was curious about a number of things, but not the least of which was the potential for sharing finding aids for holdings at NARA's Seattle branch with the NWDA. This led me to search and browse some of the NWDA finding aids available via their search engine.

After browsing some of their EAD-encoded and HTML transformed finding aids, it occurred to me that this method of presentation very likely exposed the finding aids to Web agents acting on behalf of search engines like Google. I immediately wondered if the same was true of ARC and set out to verify.

Google exposes a number of advanced search parameters that can, among other things, limit the results to one site or domain. Using this feature, I performed a search for the number of records returned by Google for the NWDA and ARC respectively (click these links to see for yourself or reference the images below). The results are shocking: on the order of 3000 results from the NWDA, most of which are finding aids that have been full-text indexed; but only one result for ARC – its main search page!

Figure 1: A Google search result for the Northwest Digital Archives

Figure 2: A Google search for ARC records

I should not need to explain this to you, but the point is clear: the catalog information about holdings of the members of the NWDA are being seen by the masses searching Google, while not one of the multitudinous ARC records is being seen outside NARA's domain. Although seasoned researchers will know to go directly to the NARA site to search their holdings, the well-meaning masses will not. But which is easier: changing millions of people's Web searching habits, or altering an application to get the data to where those millions of people already are?

Fortunately, this example was very clearly understood by NARA staff at the Seattle branch, and apparently also with the staff in D.C. Within weeks of my demonstration I heard reports of changes being made to ARC to allow direct links to the catalog records, and a plan to phase in search engines by transferring indexes to these links directly to the major search engines. Regardless of where the impetus came from or what plans were already in motion, I am happy to know that progress is being made.

I've been working with the marketing sector as a programmer for over nine years. The companies I have worked for have specialized in Web and online marketing, and, although I have no direct role in marketing planning, I have of necessity become intimately familiar with such concepts as ROI and performance tracking, sales cycles, and lead/customer database development. Because of this exposure, the notion of how to reach people online is somewhat intuitive to me, but that knowledge is not yet widely dispersed among the archival profession. From this example, the intersection of Web marketing and archives seems to be an interesting space for me to explore.

Feeling Out Finding Aids

tkiehne — Fri, 24 Mar 2006 02:42:23 +0000

I've now spent five afternoons over the last five weeks volunteering my time at the local NARA branch. The bulk of my work has focused on developing user-friendly finding aids for patron use. For the benefit of my memory, here's a recap of some of the things I have done in creating these items.

When I first began the project, I was given a copy of a finding aid that had already been developed that was considered to be the ideal final product. From this, I created a spreadsheet that displayed an information "crosswalk" between the two major primary sources of data, ARC records and accessions records, and the desired end product. The crosswalk enabled me to understand all of the different ways that the same information is represented among the different sources and to communicate these relationships quickly and effectively, should anyone else need to create finding aids. Next, I created a Microsoft Word Template to ensure that formatting and layout would be consistent for all of the products.

Each finding aid contains more information than any one of the component sources of information from which they are created. The National Archives divides records among hundreds of "Record Groups" (RG) that are based upon government agencies, commissions, or other formal bodies. Looking through the list sequentially gives the impression of a semi-chronological subject listing about US Government operations and history. I create a finding aid for each RG that the regional branch carries. The first page lists the RG number and title. Given the government agencies often undergo reassignment, name changes, dissolution, and transfer, each RG may contain many sub-agencies, often with different regional or functional offices underneath them. Each agency or sub-agency that has records at the region has a separate descriptive page within the finding aid that describes the function and history of the agency, followed by sections for each series for that agency. Series is as it sounds -- a grouping of records as they were originally maintained by the agency in question. each series has a descriptive sheet with scope and content note, inclusive dates, geographic and subject references, extent and location, and arrangement notes. The series sheet is a cover sheet to a box or folder inventory that describes in more detail what is included in the fonds.

As I mentioned in my previous post, there are a number of information sources from which I assemble the finding aids:

ARC Database Records: I have found that these records, if they exist, comprise the most accurate and up-to-date description of each series. For each of the RGs that I have processed so far, there has been an ARC record that I have used to find most of the data for the agency and series sheets.

Printed and Online Indexes: For the agency and sub-agency history, especially originating and successor agency descriptions, I have used indexes published by NARA, as they are the most authoritative references available for such information.

Internal Databases: I have a copy of a database table that lists a great deal of internal inventory information. Anything that I have not found in the ARC record can usually be found here, especially the shelf locations and extent information.

Existing Inventories: Most of the RGs have already been inventoried and have Word documents containing these inventories. I have used these to create the box/Folder listings.

Accession Records: In some cases, however, no inventories exist. For these, I have had to use the accession records and any annotations made by the region to generate a box listing. In one case, I was brought into the stacks to verify the extent and arrangement represented by a very old accession record, and I expect that I will have to do this again many times as I progress through the project.

Through all of this, I am getting a full grasp on how information is generated and flows through the regional branch which I think will serve me well as I move toward description and records processing.

Chronicles of NARA

tkiehne — Sun, 05 Mar 2006 07:07:03 +0000

As it turns out, my relatively recent move has placed me within easy reach of our region's National Archives and Records Administration branch in Seattle. This branch serves government agencies and the public in Idaho, Oregon, Washington state, and to a lesser degree, Alaska. Curiosity got the best of me, and before long I discovered that they offer volunteer positions in a number of areas within the archives. Given that my current line of work is not as near to the archives profession as I might desire, I decided that volunteering was a good way to keep my head in the archives field while simultaneously getting to know how an archives works from the inside.

Having an MSIS degree helped the application process quite a bit. The average, untrained volunteer usually performs more of a customer service role: helping researchers with their needs and questions, watching over the research room, and so forth. Volunteers with a little more training, such as library students from nearby UW, may take on more intellectually stimulating tasks such as describing holdings or creating products such as finding aids. The latter is where I have fallen in.

I am currently committed to about four hours per week. In the grand scheme of things, four hours a week is not much, but I have certainly gained from what I've done so far. My initial sessions were focused on grasping the current state of information generation relating to the branch's finding aids and beginning the process of generating such materials for public use. The ultimate goal of my current work is to produce easy to understand finding aids that represent the various record groups held by the Seattle branch down to the folder level. This work puts me more in the realm of education projects rather than in the core of the archive's management operations, but the overall experience I have had thus far is rather illuminating.

I took the opportunity to perform a quick, informal information audit. The information management situation at the Seattle branch is not unlike many such environments I have encountered in similarly sized organizations, especially my experience in the military, and I imagine that the scene in other branches is not too much different. With respect to finding aids and inventories of the holdings, I have already encountered obsolete or inadequate paper inventories, disparate electronic systems for maintaining inventory, and a parent organization that is implementing a completely different electronic system that does not completely fulfill the branch's needs.

From what I have seen so far (and I expect that there is plenty that I am as yet unaware), resources for finding records within the local holdings are thus: Washington DC's current initiative, ARC, Microsoft Access databases on the local area network, hand-generated Finding aids, and accession forms with the originating agency's inventory records but little other descriptive or location information. As you might guess, there is quite an ecosystem here, replete with version control, data synchronization, and controlled vocabulary problems.

The ARC database is probably the best organized information asset of those available. As the archivists enter the requisite information into ARC, quite a bit of quality control goes into the process, from rechecking the actual holdings, to enforcement of controlled vocabularies, to detailed review of insertions. Unfortunately, ARCs emphasis is at the series level and above, which does nothing to help the agency keep its holding inventories organized or make them available to public search. For that, the archivists use an Access database; well, they use Access as a glorified spreadsheet: one monolithic table containing a plethora of columns, many of which are rather cryptic in appearance; no relational structures for data integrity; no attempt at enforcing controlled vocabulary; and no explicit workflow procedures. Additionally, though the database does address units of inventory below series level, it does not contain a box/folder level item list. To further complexify the situation, a copy of this database (not a view, mind you, a copy) is used to keep additional information for education-related projects. This indicates a potential rift between the archivist's operations and the public-centered education and research activities, but I have not had the time to feel out the details here. Regardless, the overall feeling is one of great information disunity â€“ an old metaphor something along the lines of one hand not knowing what the other is doing.

My first instinct (from a Web applications developer's perspective) is that the whole database scheme must evolve to one of a centralized network application. Instead of different copies of the database being manipulated separately, the same data should be presented with different views, as appropriate to the work that is being done. Once the existing data is being maintained properly, the schema could be extended to incorporate box/folder level lists which could then be used to generate finding aids with little effort. Even better still, this data could be used to populate a Web application that could be linked through ARC to provide the detailed inventories to researchers, thus taking advantage of our increasingly network-centric research culture.

Unfortunately, there are a number of factors working against this plan: four hours a week is not much time to implement such a scheme, funds are short, and the organizational inertia against such change is apparently formidable. For the time being, I have decided that I still have a great deal to learn about the nature of the branch's holdings, how to describe and index them, and the overall information and political structure within the organization. These, coupled with the urgency imparted to me for improving and creating useful finding aids has convinced me to take the path of least resistance for the time being. The materials that I generate will have an immediate impact not only on my own education, but in helping the branch's customers in their work. Normally I would feel that greater internal information organization is more of an imperative in that it prevents duplicate work, and thus lost time, but in the near term this situation is an exception. From what I've been told, the Internet has taken quite a bite out of the number of customers that the research room sees every day. In order to restore awareness in the public of what the archives has to offer, it is important that the products that describe the holdings measure up to the more stringent expectations of information seekers that have become accustomed to full text indexing and search. Although a folder index is not the same as searchable text, it is a number of shades better than a standard inventory form without annotations or understandable descriptions.

Walk first, then run...

Blogs: Electronic diaries or Commonplace Books?

tkiehne — Fri, 18 Nov 2005 21:43:40 +0000

No sooner than I post a reading review that compares blogs to diaries, a Metafilter post proffers the idea that blogs most represent "commonplace" books.

Certainly the myriad content found in a commonplace book, including diagrams and illustrations, reflects much of the current "blogosphere" where topics drift from one post to the next and taken as a whole resembles an ad hoc compendium. Ultimately, however, the comparisons between blogs and diaries or commonplace books must necessarily be made in general terms -- perhaps as a way to describe archetypes of blogging rather than all blogging. Certainly we have seen both diary style and commonplace book style blogs; why try to narrow the description of such a widespread networked information type to a single physical container type?

Fortunately, the extension of the metaphor does little to change my argument -- that the use of such metaphors is useful for helping traditional archivists think more clearly about preserving networked information.

Notes: Diaries, On-line Diaries, and the Future Loss to Archives by Catherine O'Sullivan

tkiehne — Mon, 14 Nov 2005 06:16:54 +0000

Published in American Archivist, Vol. 68, No. 1, Spring/Summer 2005, pp. 53-73.

One thing that has become clear in the short time since I was brought into the archival fold, is that online (not merely electronic, mind you) information really puts old-guard archivists into a bind. Having been to one annual conference of the Society of American Archivists, I feel that this apprehension is palpable â€“ almost like the tension amongst people who do not wish to discuss the rather large and smelly elephant in the room. And with good reason. Hypertextual resources (Web pages, databases, and the like) dissolve the boundaries between groups of information by removing the physical constraints that have for so long defined information access. For someone who has defined their career's work in terms of discreet information objects of a physical nature, this lack of physical structure is likely to induce a sort of intellectual nausea.

Not that all archivists are cranky old dead tree advocates, but in my opinion, the momentum towards the issues inherent in the preservation of online information has yet to build up. The crux of the problem, and the solutions that are to be devised, lie in a common area -- technology. Having come from the Web design and applications development field, these challenges are still formidable, but I am not intimidated by the technology itself. A greater understanding of the technology is needed in order to make progress in preserving the information products contained therein. (much more is required than merely understanding the technology -- intellectual property is one of the big ones -- but I digress)

It is to this end that O'Sullivan seems to approach. Much has been made about the influence of blogs in media and other social and political venues. From the perspective of a long time online information users, this hardly seems worthy of note; After all, BBS systems, USENET, and Web message boards have been around "forever." So what makes blogs so special?

O'Sullivan takes a literary approach to analyzing blogs, likening them to written diaries, which provides a clues as to why blogs are different. Her research into the styles and types of diaries over the centuries indicates a form of information that has changed from that of a highly formal, often religious imperative, to that of casual social observation of a very personal nature. Bringing blogs back into the picture, this is part of the hype that the ephemeral nature of email lists and BBS's were unable to capture â€“ the highly connected, personal observations so prevalent in blogs and online journals. But that is not all. The main observation here is that blogs are the literary descendant of diaries, an assertion which is verified by the availability of non-technical tools for publishing them (like Drupal -- touchÃ©). Just as paper diaries only require knowledge of written language and simple technology (i.e.: paper and pen or some analog thereof), blogs also require language knowledge (in some cases merely spoken, as is allowed by some experimental blog transcription services) and simple technology, which these days includes internet access and computer with browser â€“ all freely available at your public library or on the cheap in numerous other ways. The diary metaphor is certain to provide much comfort to heretofore squeamish archivists -- it is a boundary, after all, that is still quite elusive, but can be grasped without too much knowledge of the underlying technology.

Having made the link between old media and new, O'Sulivan moves on to describing the magnitude of information wrapped up in blogs and some discussion of similarities and differences between the two. Using a brief explanation of the ephemeral nature of blogs, and thus the threat of a great loss of cultural information, she segues into the question of preservation. As can be predicted, the Internet Archive is brought up, to which she quickly acknowledges the WayBack Machine's inherent (ironically) archival limitations. From this, she retreats to the comfortable territory of acquisition and appraisal, that is, leaving the problem up to individual, existing collections to decide which resources are worth preserving in order to keep a more complete record, including much of the context that surrounds, and lends authenticity to, the resource. Such tactics continue, however, to support the retention of only what is deemed at the time to be "important" -- which inevitably leads to preferential treatment for high-profile resources. I might as well state at this point my undying devotion to the idea of a balanced, complete cultural record, not merely of the rich and famous. Besides, as a wise woman once challenged me, since storage is inevitably going to be cheap as air, why not keep it all? Indeed, why not?

It is here that I diverge from the reductionist view, that is, narrowing our focus to one type of online information resource, and throw out this proposition: in order to gather a complete record of the global internet, we are going to have to throw open the gates of archival process and make it available to the masses. Do I expect archival theory to become familiar to the common person? Absolutely not. What I propose is that the systems we use, and the technology of archival storage, be made easier to grasp to the common person, and as with blogs just as easy to use. These are no small tasks (the information retrieval and integrity aspects are immense in and of themselves), but they are what I have defined as a life's work for myself.

Perhaps this will be distasteful to the archival establishment, but this proposition might imply a break with formal diplomatics since we aren't talking about property records or a succession of command, but the reconstruction of a place and time with implicitly less dire consequences for imperfection serving considerably less skeptical needs. The lives of common people and organizations can contribute greatly to the understanding of the context in which greater political and social shifts occur. As our cultural record moves online, the imperative to preserve a wide swath of it increases as well. Technology alone will not provide all the answers, and O'Sullivan rightly brings together the new and the old in order to more fully engage the archival mindset in the problems at hand.