infoSpace - NARA

Bringing Records to the Users

tkiehne — Tue, 17 Apr 2007 07:06:37 +0000

I've volunteered at the local National Archives branch for over a year now. Over this time I have gained the acute impression of lament over the dwindling number of researchers and members of the public who make the trip out to the archives to do research. Indeed, it is not uncommon for me to enter a virtually empty research room during my Thursday afternoon visits.

What is seldom spoken of, but is vitally central to the issue, is the instant information gratification that the general populace receives from their increasingly ubiquitous internet connections. The reams of information available through Google, or the convenience of accessing Ancestry.com from home (instead of for free at the archives), keeps them at home and only adds insult to the injury of fewer patrons.

NARA is not alone in this lament. Günter Waibel of OCLC recently spoke of it in a Hanging Together blog post covering the IMLS WebWise Conference, where he reposts a quote by Deanna Marcum of the Library of Congress that rather eloquently sums up the issue:

“What I think our challenge is, it is not enough for us to create the perfect finding system, we know from all the user studies that individuals, who are looking for information, go directly to the open web, and our marvelous catalogues are not getting used. We have to find ways to take our content and the metadata and move that content to the open web.”

NARA relies on an online system called the Archival Research Catalog (ARC) to catalog and describe its considerable physical holdings. This system was derived from an earlier prototype called NARA's Archival Information Locator (NAIL), which primarily contained images and other non-textual surrogates. Each of these systems are of late-1990's vintage – a Web lifetime ago – and, quite frankly, the user interface shows it. One of my contacts at NARA once stated that the government seems to be about a decade behind when it comes to information management, and in this case it seems true enough.

Granted, the scope of these projects is monumental and one cannot expect them to be updated at Web speed to, say, incorporate some of the more useful “Web 2.0” principles. However, given the exodus of potential patrons to full-text search engines, there is a particularly devastating truth hidden behind the aging ARC code. To paraphrase Marcum, we need to take our content and metadata to where the users are. As it turns out, some informal research provides a striking illustration of this need.

One evening I found myself looking through the Web site of the Northwest Digital Archives, a group of education, government, and private archives across the Pacific Northwest and Alaska. I was curious about a number of things, but not the least of which was the potential for sharing finding aids for holdings at NARA's Seattle branch with the NWDA. This led me to search and browse some of the NWDA finding aids available via their search engine.

After browsing some of their EAD-encoded and HTML transformed finding aids, it occurred to me that this method of presentation very likely exposed the finding aids to Web agents acting on behalf of search engines like Google. I immediately wondered if the same was true of ARC and set out to verify.

Google exposes a number of advanced search parameters that can, among other things, limit the results to one site or domain. Using this feature, I performed a search for the number of records returned by Google for the NWDA and ARC respectively (click these links to see for yourself or reference the images below). The results are shocking: on the order of 3000 results from the NWDA, most of which are finding aids that have been full-text indexed; but only one result for ARC – its main search page!

Figure 1: A Google search result for the Northwest Digital Archives

Figure 2: A Google search for ARC records

I should not need to explain this to you, but the point is clear: the catalog information about holdings of the members of the NWDA are being seen by the masses searching Google, while not one of the multitudinous ARC records is being seen outside NARA's domain. Although seasoned researchers will know to go directly to the NARA site to search their holdings, the well-meaning masses will not. But which is easier: changing millions of people's Web searching habits, or altering an application to get the data to where those millions of people already are?

Fortunately, this example was very clearly understood by NARA staff at the Seattle branch, and apparently also with the staff in D.C. Within weeks of my demonstration I heard reports of changes being made to ARC to allow direct links to the catalog records, and a plan to phase in search engines by transferring indexes to these links directly to the major search engines. Regardless of where the impetus came from or what plans were already in motion, I am happy to know that progress is being made.

I've been working with the marketing sector as a programmer for over nine years. The companies I have worked for have specialized in Web and online marketing, and, although I have no direct role in marketing planning, I have of necessity become intimately familiar with such concepts as ROI and performance tracking, sales cycles, and lead/customer database development. Because of this exposure, the notion of how to reach people online is somewhat intuitive to me, but that knowledge is not yet widely dispersed among the archival profession. From this example, the intersection of Web marketing and archives seems to be an interesting space for me to explore.

Feeling Out Finding Aids

tkiehne — Fri, 24 Mar 2006 02:42:23 +0000

I've now spent five afternoons over the last five weeks volunteering my time at the local NARA branch. The bulk of my work has focused on developing user-friendly finding aids for patron use. For the benefit of my memory, here's a recap of some of the things I have done in creating these items.

When I first began the project, I was given a copy of a finding aid that had already been developed that was considered to be the ideal final product. From this, I created a spreadsheet that displayed an information "crosswalk" between the two major primary sources of data, ARC records and accessions records, and the desired end product. The crosswalk enabled me to understand all of the different ways that the same information is represented among the different sources and to communicate these relationships quickly and effectively, should anyone else need to create finding aids. Next, I created a Microsoft Word Template to ensure that formatting and layout would be consistent for all of the products.

Each finding aid contains more information than any one of the component sources of information from which they are created. The National Archives divides records among hundreds of "Record Groups" (RG) that are based upon government agencies, commissions, or other formal bodies. Looking through the list sequentially gives the impression of a semi-chronological subject listing about US Government operations and history. I create a finding aid for each RG that the regional branch carries. The first page lists the RG number and title. Given the government agencies often undergo reassignment, name changes, dissolution, and transfer, each RG may contain many sub-agencies, often with different regional or functional offices underneath them. Each agency or sub-agency that has records at the region has a separate descriptive page within the finding aid that describes the function and history of the agency, followed by sections for each series for that agency. Series is as it sounds -- a grouping of records as they were originally maintained by the agency in question. each series has a descriptive sheet with scope and content note, inclusive dates, geographic and subject references, extent and location, and arrangement notes. The series sheet is a cover sheet to a box or folder inventory that describes in more detail what is included in the fonds.

As I mentioned in my previous post, there are a number of information sources from which I assemble the finding aids:

ARC Database Records: I have found that these records, if they exist, comprise the most accurate and up-to-date description of each series. For each of the RGs that I have processed so far, there has been an ARC record that I have used to find most of the data for the agency and series sheets.

Printed and Online Indexes: For the agency and sub-agency history, especially originating and successor agency descriptions, I have used indexes published by NARA, as they are the most authoritative references available for such information.

Internal Databases: I have a copy of a database table that lists a great deal of internal inventory information. Anything that I have not found in the ARC record can usually be found here, especially the shelf locations and extent information.

Existing Inventories: Most of the RGs have already been inventoried and have Word documents containing these inventories. I have used these to create the box/Folder listings.

Accession Records: In some cases, however, no inventories exist. For these, I have had to use the accession records and any annotations made by the region to generate a box listing. In one case, I was brought into the stacks to verify the extent and arrangement represented by a very old accession record, and I expect that I will have to do this again many times as I progress through the project.

Through all of this, I am getting a full grasp on how information is generated and flows through the regional branch which I think will serve me well as I move toward description and records processing.

Chronicles of NARA

tkiehne — Sun, 05 Mar 2006 07:07:03 +0000

As it turns out, my relatively recent move has placed me within easy reach of our region's National Archives and Records Administration branch in Seattle. This branch serves government agencies and the public in Idaho, Oregon, Washington state, and to a lesser degree, Alaska. Curiosity got the best of me, and before long I discovered that they offer volunteer positions in a number of areas within the archives. Given that my current line of work is not as near to the archives profession as I might desire, I decided that volunteering was a good way to keep my head in the archives field while simultaneously getting to know how an archives works from the inside.

Having an MSIS degree helped the application process quite a bit. The average, untrained volunteer usually performs more of a customer service role: helping researchers with their needs and questions, watching over the research room, and so forth. Volunteers with a little more training, such as library students from nearby UW, may take on more intellectually stimulating tasks such as describing holdings or creating products such as finding aids. The latter is where I have fallen in.

I am currently committed to about four hours per week. In the grand scheme of things, four hours a week is not much, but I have certainly gained from what I've done so far. My initial sessions were focused on grasping the current state of information generation relating to the branch's finding aids and beginning the process of generating such materials for public use. The ultimate goal of my current work is to produce easy to understand finding aids that represent the various record groups held by the Seattle branch down to the folder level. This work puts me more in the realm of education projects rather than in the core of the archive's management operations, but the overall experience I have had thus far is rather illuminating.

I took the opportunity to perform a quick, informal information audit. The information management situation at the Seattle branch is not unlike many such environments I have encountered in similarly sized organizations, especially my experience in the military, and I imagine that the scene in other branches is not too much different. With respect to finding aids and inventories of the holdings, I have already encountered obsolete or inadequate paper inventories, disparate electronic systems for maintaining inventory, and a parent organization that is implementing a completely different electronic system that does not completely fulfill the branch's needs.

From what I have seen so far (and I expect that there is plenty that I am as yet unaware), resources for finding records within the local holdings are thus: Washington DC's current initiative, ARC, Microsoft Access databases on the local area network, hand-generated Finding aids, and accession forms with the originating agency's inventory records but little other descriptive or location information. As you might guess, there is quite an ecosystem here, replete with version control, data synchronization, and controlled vocabulary problems.

The ARC database is probably the best organized information asset of those available. As the archivists enter the requisite information into ARC, quite a bit of quality control goes into the process, from rechecking the actual holdings, to enforcement of controlled vocabularies, to detailed review of insertions. Unfortunately, ARCs emphasis is at the series level and above, which does nothing to help the agency keep its holding inventories organized or make them available to public search. For that, the archivists use an Access database; well, they use Access as a glorified spreadsheet: one monolithic table containing a plethora of columns, many of which are rather cryptic in appearance; no relational structures for data integrity; no attempt at enforcing controlled vocabulary; and no explicit workflow procedures. Additionally, though the database does address units of inventory below series level, it does not contain a box/folder level item list. To further complexify the situation, a copy of this database (not a view, mind you, a copy) is used to keep additional information for education-related projects. This indicates a potential rift between the archivist's operations and the public-centered education and research activities, but I have not had the time to feel out the details here. Regardless, the overall feeling is one of great information disunity â€“ an old metaphor something along the lines of one hand not knowing what the other is doing.

My first instinct (from a Web applications developer's perspective) is that the whole database scheme must evolve to one of a centralized network application. Instead of different copies of the database being manipulated separately, the same data should be presented with different views, as appropriate to the work that is being done. Once the existing data is being maintained properly, the schema could be extended to incorporate box/folder level lists which could then be used to generate finding aids with little effort. Even better still, this data could be used to populate a Web application that could be linked through ARC to provide the detailed inventories to researchers, thus taking advantage of our increasingly network-centric research culture.

Unfortunately, there are a number of factors working against this plan: four hours a week is not much time to implement such a scheme, funds are short, and the organizational inertia against such change is apparently formidable. For the time being, I have decided that I still have a great deal to learn about the nature of the branch's holdings, how to describe and index them, and the overall information and political structure within the organization. These, coupled with the urgency imparted to me for improving and creating useful finding aids has convinced me to take the path of least resistance for the time being. The materials that I generate will have an immediate impact not only on my own education, but in helping the branch's customers in their work. Normally I would feel that greater internal information organization is more of an imperative in that it prevents duplicate work, and thus lost time, but in the near term this situation is an exception. From what I've been told, the Internet has taken quite a bite out of the number of customers that the research room sees every day. In order to restore awareness in the public of what the archives has to offer, it is important that the products that describe the holdings measure up to the more stringent expectations of information seekers that have become accustomed to full text indexing and search. Although a folder index is not the same as searchable text, it is a number of shades better than a standard inventory form without annotations or understandable descriptions.

Walk first, then run...