All posts by Matthew S.

About Matthew S.

Web developer, errant philosopher, idiosyncrat.

A Change of Focus

I thought I’d write about where we are with the microfilm digitization part of our project. In particular, we had a minor setback with our scanning that proved (to me, at least) to be quite educational.

We enlisted the help of The Crowley Company, a commercial digitization lab that offered state-of-the art equipment and a clean room, and that, along with experienced operators doing the work, convinced us that we would produce digital copies of these collections of the highest possible quality. But what of the microfilm itself? Were the reels in good enough condition, and was the original photography done well? We were eager to view the results, and it fell to me to analyze the scans and assess their quality.

A page selected at random from our first scanning of the a microfilm edition of the Papers of Rutherford B. Hayes
A page selected at random from our first scanning of the a microfilm edition of the Papers of Rutherford B. Hayes

How’s it look to you?

I’ll admit to having been a little unsure of myself. I explored various reels, looking for poor exposure, blurry images, anything indicating an error or gap in the digitization process. Eventually, I got a sense of what the overall resolution of the images was. But something didn’t feel right. When I looked at the top of the page (and why wouldn’t you start there?), things looked fine, but occasionally I saw things that looked blurry, and this gave me pause. Was I looking at the maximal sharpness of the image, or could things be improved? Without the microfilm on hand, I couldn’t even determine whether it was the digital photography that was to blame or the original microphotography. If the latter, there was really nothing to be done, short of re-shooting the papers, and that wasn’t an option!

Fortunately, when we had an on-site visit from Crowley, I had a chance to raise the issue with their representative. He very quickly identified a curious trend: the top of each page was crisp and clear, while the bottom was less clear. A few minutes later, he was on the phone to the scanning lab, as they still had the microfilm in their possession. At his request, the film was examined under a microscope, and we confirmed that original microphotography was in focus. This can be done ad hoc, by looking at a frame selected at random. But, fortunately, there is a more systematic method for answering questions such as these.

Resolution Calibration Test Targets Are Your Friends

When microfilm is professionally manufactured, there is a calibration chart included in the material photographed. The photographs of the chart document the focal quality of the entire batch of images. Moreover, they are designed to exhibit the limitations of any optical lens used in photography: aberration. The target looks like this:

Resolution Target Microphotograph from Reel 1 of the Rutherford B. Hayes Papers
Resolution Target Microphotograph from Reel 1 of the Rutherford B. Hayes Papers

 

The four seemingly redundant charts at the corners tell you something very important: how the photography resolves an image at the extreme edges of the camera lens. It’s not enough to have good focus at the center, a lens must be designed and calibrated to produce sharp images at the periphery, and these charts document how the photographic lens performed for the microfilming session. Like an eye chart, finding the smallest lines that can be resolved (and are not just a grey blur) indicates the limit of your ability to focus in on a detail in an image.

As this is a standard, highly accurate printed chart, rather than pen ink on a hundred-year old manuscript, you can get a much more reliable sense of what the photography (and subsequent scanning) captured. And here we can see the variation between a top-corner and bottom-corner chart on the very same exposure:

Upper Left Resolution Target from first scanning run (good focal quality)
Upper Left Resolution Target from first scanning run (good focal quality)
Lower Left Resolution Test Target (poor focal quality)
Lower Left Resolution Test Target from first scanning run (poor focal quality)

 

I find that if you examine the bars labeled “2.0”, you can see a distinct difference.  (Click on the image to expand.)

Once we’d established that at least some of our scanned reels resulted in partially out-of-focus digital images, more detective work was required. Our Crowley representative quickly established that there was a calibration error with the scanner that processed the samples we examined, and their records indicated the entire run was done with the same machine. They very quickly sent that scanner to be repaired and re-calibrated, and since they still had the microfilm on hand, they offered to rescan the material as soon as the machine was deemed fit to return to service.

We were more than happy with this arrangement. I’m hoping to use these images in an effort to extract transcripts and/or metadata from the documents photographed, and clarity is very important. Consider this side-by-side example:

The first scanning attempt:

Close-up view of text scanned in October, 2014
Close-up view of text scanned in October, 2014

And the results of recalibration:

Close-up view of text re-scanned in December, 2014
Close-up view of text re-scanned in December, 2014

I think the increase in clarity is pretty obvious (your mileage may vary!)

I’m glad we investigated this, and the turnaround time for the rescanning was excellent. So thanks again to Crowley for their diligence and expert eyes! I have a much higher degree of confidence in our digital images, and look forward to the next phase of our work.

UPDATE: The nice folks at Crowley have blogged about this project as well.

Advertisements

“President Warren Harding may not have been the most popular president, but he sure knew how to write a love letter.”

That was one editor’s headline on one media outlet’s report on the Library of Congress’ publication of Harding’s private correspondence with Carrie Fulton Phillips, evidence of an oft-rumored extramarital affair. Political satirists including John Oliver have poked fun at Harding for his unrestrained prose, and the archival community is generally abuzz as these interesting materials resurface after a fifty-year embargo, coming into public view for the first time.

Warren G. Harding

Warren G. Harding, half-length portrait, seated, facing left, at Elks’ National Home, Bedford, Va.
Copyright by F.H. Richardson (expired). [Public domain], via Wikimedia Commons

The letters, their survival until the present day, and the story of how they were sealed until July 29th, 2014, are all of interest to historians and curators of our cultural heritage. The New York Times has a good account of how the letters came to be in the public domain.

It is particularly interesting to read how Harding’s correspondence with Mrs. Phillips included major political topics of the day, such as U.S. entry in World War One (Harding was in the Senate in 1917, and voted in favor of declaring war.) Mrs. Phillips was pro-German, and Harding warned her to avoid public statements in favor of the German Empire, lest she attract the attention of federal investigators.

Harding’s presidency was, of course, cut short by his untimely death in 1923, and his reputation is largely colored by the scandals surrounding his time in the Oval Office. Collections such as these letters can help us understand Harding’s life and work, by adding perspective on, not to mention detail into, his thoughts and actions at a given time. Hopefully, the emphasis on the ‘steamy’ nature of the letters will pass, and students and researchers can use these materials to learn more about the milieu that directly proceeded Harding’s rise to the Presidency.

News of the impending release of the letters prompted us here at the Miller Center to turn to one of our on-line exhibits, made a few years ago. Thanks to some diligent research, and preservation work, from Harding’s nephew, Dr. George Harding, III, we have copies of President Harding’s recorded speeches, something of a rarity for the time period. A newspaper editor by trade, Warren G. Harding showed a keen interest in the then-new technology of analog recording, and he recorded excerpts of his popular speeches for pressing and distribution as phonographs.

In 1921, President Warren Harding spoke into a recording
apparatus to create a phonographic copy of one of his speeches.

You can read about these recordings at the exhibit’s home page. Transcripts and MP3 audio of the individual speeches can also be found here.

The Harding family deserves praise for the efforts to preserve these recordings and make them of greater use today. Dr. Harding supervised, in the 1970s, the re-recording of the original discs as they were converted from their original 78-rpm format to the more common 33 & 1/3 rpm. Then, in 2004, Dr. Harding’s son, Warren G. Harding, III, saw that the recordings were digitized and issued on Compact Disc. Just as the letters made their way through various hands and venues to make it to 2014, so the recordings of Harding’s orations have had a circuitous journey to the Web.

Given what seems to be an unusually-high amount in interest in Harding, we took the opportunity to update our catalog records for these items, and have decided to add the collection to CPC. This marks the first batch of material pertaining to Harding, and adds one more presidency to our list (with more to come soon!)

Here’s a quick link to our site, presidentialcollections.org, with the Harding speeches highlighted.

New Look and Feel!

As promised, I’ve updated the CPC website, which now features updated versions of Blacklight, Ruby on Rails, and Solr.

More importantly, it’s got its own logo (nicely done, Amber!)  I think that’s reason enough to go have a look.  We’ve also made some changes to how you search and discover documents, so by all means visit the site and let us know in the comments what you think.

Joining the Team

It’s introduction time! This has been a while in the making, but I’m delighted to say I’ve joined the CPC project, and the Miller Center, as the new web developer.

New Computer by N1NJ4, on Flickr

Creative Commons Attribution 2.0 Generic License  by  N1NJ4  (source: Flickr)

My name is Matthew Stephens, and most recently I was a technologist at Alderman Library, here at the University of Virginia. This is where Blacklight was born, and I’ve been a fan of it and Apache Solr for many years. Prior to my coming to UVA, I worked at Intelex, a digital publisher here in Charlottesville, and that’s where I cut my teeth on metadata aggregation, something very much on the minds of everyone here. Needless to say, I’m very excited to work on a project where I can build upon past experience and learn new ways to do things I care about.

The first item on my (growing) to-do list is updating the CPC site. Open source software is a moving target, and many of the components of the site are due for an update. The Blacklight team has released version 5.3.0 in the past month, and upgrading to this, along with many dependencies, will keep the CPC application current with the broader community of Ruby-on-Rails and Solr enthusiasts. Along with the upgrades, I will be streamlining a few things under the hood, including the indexing of the metadata provided by our partners. A Solr index is many a splendored thing, and part of my role will be to ensure that the information we receive from partners can be effectively searched and discovered by our users.

I’m delighted to join a team that has already impressed me with their creativity, expertise, and drive. I’ve also learned of their passion for the serial comma, so you may view that last sentence as a peace offering, given my expressed agnosticism. (I’m sure we’ll work that out in the months to come. I’m also from Canada, so my pronunciation of the letter ‘Z’ may be an issue.)

Modern web development is a fascinating endeavor, but so much more so when the endless possibilities are shaped by serving a community. I invite any and all interested readers to have a look at the site as it changes over the coming months. We’re all eager to make something interesting and useful, and we welcome any suggestions you’d care to make.

I tweet, occasionally and somewhat whimsically, at this address.