Paleospectrophony

NOTE: This initial account of methods for transducing images into sound—first posted in 2009—is maintained here for its historical interest. You can also find it archived in Archive.org's Wayback Machine. For current information, please visit griffonage.com.

[<< Part 2]

3. Automatic Musical Instrument Programs

3.1—In 2002, the National Recording Preservation Board chose the first fifty items for its National Recording Registry. Included in the list was "Scott Joplin ragtime compositions on piano rolls." When I first saw this, I thought: "Wait a minute. Those aren't actually sound recordings! What's going on here?"

The Board defines "sound recordings" as "works that result from the fixation of a series of musical, spoken, or other sounds," specifying only that audio synchronized with motion pictures isn't part of its scope (the National Film Preservation Board already had dibs on that). A piano roll may document a musical performance, but it does so by recording actions that trigger the production of sound, not the sounds themselves—the timbre of the original piano and so forth. Although Joplin's piano rolls are important cultural artifacts, they're not phonograms ("sound recordings") in the sense we ordinarily understand, but things of a different order.

Still, there is some overlap. Thanks to paleospectrophony, automatic musical instrument programs are often directly interchangeable with phonograms. The one can be converted into the other by setting just two basic parameters: frequency range and time.

3.2—Take the following illustration from Edward Wheeler Scripture's Researches in Experimental Phonetics (1906):

On one hand, the bottom inscription is in the same format as those we've been considering, plotting frequency along one axis against time along another axis. We can play it through paleospectrophony, like this (the duration is arbitrary):   . On the other hand, this inscription is also in the same format as many automatic musical instrument media. The notes look just like they would if cut into a typical piano player roll.

3.3—Now consider the illustration accompanying U. S. Patent 231,323, G. H. Ireland's "Mechanical Musical Instrument," granted August 17, 1880:

For paleospectrophony, this inscription requires degridding followed by a vertical flip to orient it like a modern sound spectrogram. I've also cropped the image so that it spans one octave, from a B to another B:

Paleospectrophony doesn't treat these marks as C, D, E, F, G, and A, but as pieces of acoustic data at specific distances between the bottom (set to B=246.94 Hz) and top (set an octave up, at 493.88 Hz). Here's what it sounds like (the duration is again arbitrary):  

Note that the lines corresponding to the notes in this inscription are evenly spaced, whereas some notes should be a full step apart and others only a half step. As a result, the intervals sound somewhat "off." But the melody is still quite recognizable.

3.3—Having established this principle, we can now apply it to more complicated inscriptions and extend it back further in time.

Athanasius Kircher's Musurgia Universalis (1650) contains five plates illustrating how to pin an organ barrel (you can see a facsimile of the complete work here, courtesy of ECHO, European Cultural Heritage Online). The first plate looks like this:

For paleospectrophony, this plate needs to undergo degridding and a 90° counterclockwise rotation:

The octave is divided here into twelve evenly-spaced steps, so paleospectrophony gives us something close to twelve-tone equal temperament. This is true of plates 1, 2, and 4, but plate 3 divides the octave into just eleven intervals, and plate 5 (representing a piece of microtonal music) divides it into twenty-eight. Listen to all five plates played in order:  

3.4—Another important source for plates of this sort is Dom François Bedos de Celles, L'art du Facteur d'Orgues (1778), which includes examples prepared by Marie Dominique Joseph Engramelle, a leading expert in barrel-pinning. These require an extra stage of processing for paleospectrophony: Engramelle showed the sounded and unsounded parts of each note with different shadings, so the unsounded parts need to be erased.

The second plate of the "Barcelonnette"

Romance by Claude Balbastre, pinned as played by the composer, duration specified to the second: Barcelonnette: Fragments du Cylindre (examples of ornamentation):

3.5—This procedure differs from the digital conversion of piano rolls and similar formats into MIDI—for example as described here—in that (1) there's no need to assign an arbitrary "voice" to the results and (2) the original markings are read as acoustic data, not parsed into notes with individually assigned pitches. I don't claim that mine is a better method—but it's a different method, and one that demonstrates that these formats can be "played" as inscriptions comparable in nature to "sound recordings," even if they weren't originally intended that way.

[Part 4 >>]