Audio Encoding Project: Initial Progress Report

I made some progress this week on my audio encoding project. In a total of three sessions I have ripped the contents of about 50 CDs to a lossless digital format with basic embedded metadata. This entry recounts the requirements, procedures, and observations on the project thus far.

The overall requirements for the project involve encoding all of my physical music media to a compact yet lossless digital format for long-term preservation. Additionally, the encoded objects will be accompanied by as much descriptive metadata as possible and provide a testbed to experiment with developing structural and technical metadata (see Gilliland-Swetland "Setting the Stage"), namely, describing relationships between individual objects and the technical data regarding their encoding and creation.

I have defined a set of software requirements to help me to meet these high-level requirements. There are a plethora of ripping programs available for free, as shareware or commercial products, or bundled with hardware. Some of these are specifically designed to support only certain codecs, formats, or hardware devices, but many are format agnostic. The software I require for this project should ideally allow format agnostic file extraction at a cost that is commiserate with the quality of its interface, and its stability and reliability.

In addition to the basic features of the encoding software listed above, it should also support appropriate audio codecs. Popular digital audio file formats, such as MP3, Ogg Vorbis, and Windows Media do well at compressing an audio file to a very small size, but do so at the expense of fidelity to the original recording (for more on audio codecs see the NDIIPP formats page). The very principle behind long-term preservation and archiving of any object demands high-fidelity with the original. For binary formats such as images and audio, this means using a lossless codec – one that allows all of the original recording's features to be reconstructed upon each subsequent access, without loss of data, whether perceivable by humans or not. In addition to being lossless, the selected codec must also be open (as in Open Source) and well documented. The reasoning here is that should a closed, scantily supported, or otherwise obscure codec be used, the risk of obsolescence, and thus loss of access in the future, increases (see my description of transparency concerns in my digital video preservation plan).

Whatever software is selected, it should support metadata creation. Most current digital audio formats will support at least some amount of embedded metadata – that is, descriptive data encoded with the object itself, usually artist, title, date, genre, etc. The ideal solution will support metadata retrieval using either CDDB or FreeDB. These online databases have already encoded much of the descriptive metadata required, thus saving untold hours of labor and error in manual reconstruction. Some programs will also support the creation of other types of metadata such as playlists, which, though not necessarily standards-based may help create more sophisticated structural metadata at a later time.

Finally, an additional requirement on my part is support for metadata configurable filenaming, including path references. In addition to the explicit metadata described above, I, like most people, like to use filesystem metadata to assist in search and retrieval. This is simply a way of saying that the location of an object within the filesystem indicates some basic metadata that is used for retrieval and description, such as artist name, album name, and song title. The ideal program will be able to use metadata from CDDB/FreeDB to create filenames and directory paths that are descriptive of the object's contents.

Among the software that I evaluated, Nullsoft's Winamp offers an extensible platform with a wide variety of third-party plug-ins that seems to best match the requirements stated above. Winamp supports CDDB queries as well as CD ripping, metadata configurable filenaming, and a variety of codecs. In addition to meeting these requirements, the program is not beholden to supporting specific formats or encumbered with DRM, and is inexpensive (approximately $20 US for the full version).

Winamp natively supports encoding to WAVE format, which is a widely used audio codec. Despite being openly documented, WAVE is nonetheless a proprietary format developed by IBM and Microsoft. Although it is unlikely that the format will endure catastrophic technical changes in the near future, the legal landscape could conceivably change in deference to future lossless audio formats that these vendors might wish to promote. For this reason alone, a pure open format is desired. Fortunately, a codec named FLAC (Free Lossless Audio Codec). In addition to comparable sound quality, FLAC appears to support a more flexible metadata structure. Unfortunately, Winamp does not natively support encoding or decoding to/from FLAC. To enable this, I found third-party plug-ins developed by Michael Facquet that enable Winamp to support FLAC.

It is with this combination of software that I began encoding my collection. Within the space of time it takes to encode 50 CDs, however, some problems are already evident. Some mixture of circumstances has resulted in not infrequent instability including program crashes and encoding failures. Though not frequent enough to warrant aborting the effort, it has taken some undue attention to manage something that should be nearly effortless. Most of the problems occur upon loading a new CD into the drive and during Winamp's response to the change. The worst crashes have occurred just after the CDDB query has completed, within a few seconds of the CD being inserted. This could be the result of program control failure (failing to pass from one state to the next in the correct sequence), hardware communication issues (the CD player not communicating with Winamp correctly), or potentially some other system or program instability as a result of the current configuration.

The second type of failure occurs during encoding. Apparently some sort of error condition encountered while encoding will cause Winamp to attempt to play one of the CD tracks before the encoding process is complete. The resulting conflict causes the encoding process to fail, usually to the effect of an incompletely ripped track. A more minor variation of this problem results when a dirty CD (usually foggy due to age, not so much scratched or damaged) causes the encoder to quit the track in question, also resulting in an incomplete track. Neither of these situations is caught or reported by the FLAC encoder; the former being caught by Winamp ("CD currently in use"), the latter not reported at all (I accidentally discovered the incomplete track while listening to it). The lack of error recovery is a disturbing development. Otherwise, however, the encoding and metadata gathering process has worked as expected.

Before continuing, it is obvious that I must determine the cause of the crashes. I have already varied the CD ripping speed (configurable within Winamp) to no avail. Next, I will begin to disable unused plug-ins within Winamp, especially those that were installed prior to my upgrading to the full version. I may also need to upgrade or re-install Winamp and/or the FLAC plug-in. Finally, I need to do some research to determine if this problem has occurred with other users. At the least I should notify the FLAC plug-in creator about the problems with error correction and checking in the hopes that he can improve future releases of his product.

If these efforts prove unsuccessful, I will need to search for a more stable FLAC plug-in or an entirely different software platform. The combination of features in Winamp will be hard to beat, though, which keeps me hopeful that I may find the solution to the problems that I am encountering.