A new virtual reality project reconstructs the city’s historic soundscape.
You’re standing on the High Line. The walkways are damp, and fellow strollers have zipped on windbreakers to ward off the drizzle. Look up, and you’ll find a sky the color of poured mercury. The sound of a revving motorcycle ricochets in the canyons between the buildings. Bend your gaze to the street and you’ll stop for a moment on windows glowing amber before settling on cabs, startlingly yellow against all the gray.
A few seconds later, 400 years melt away. A recent 360-degree video dissolves into a threadbare black-and-white sketch; traffic sounds give way to waves lapping against a shoreline and distant bird calls. The present disappears, and in its place is an aural recreation of a distant past in the same footprint.
The High Line’s slice of Manhattan’s western edge is one of four sites that comprise “Calling Thunder: The Unsung History of Manhattan,” a new virtual reality project that pulls New York City’s contemporary soundtrack into conversation with its predecessors. Bill McQuay, an audio producer at the Cornell Lab of Ornithology, collaborated with David Al-Ibrahim, a graduate student in the interaction design program at the School of Visual Arts, on the immersive audio landscape to capture what the island probably sounded like in 1609, just before the Dutch washed ashore on a coast inhabited by the Lenape people.
It seems logical to invite listeners into an immersive audio world—along with smell, hearing is a sense that we already use in 360 degrees. But making the worlds of Inwood Hill Park, Collect Pond Park, and the modern-day American Museum of Natural History historically accurate required a lot of legwork.
Before the team stitched together the component sounds—drawn from the archives of the Macaulay Library—they had to locate topographic features that would have affected them. The presence of hills or water, for instance, would alter how sound carried. To reconstruct the lay of the land, Al-Ibrahim and McQuay consulted Eric Sanderson, a landscape ecologist at the Wildlife Conservation Society and author of Manahatta: A Natural History of New York City, which compiled block-by-block annotations of the island’s historical ecology. Sanderson inferred species distribution by running contemporary models and consulting historical engravings and maps plotted by European arrivals. (Few archaeological traces of Lenape settlements persist, Sanderson says, and there are very limited extant written records or maps of these features that predate Dutch arrival.)
Hundreds of years ago, present-day Manhattan played host to black bears, wolves, and mountain lions; whales and porpoises schooled thick in coastal waters. When deciding which sounds to place in the landscape, Al-Ibrahim says, the team omitted extinct species such as passenger pigeons out of fear that they wouldn’t be able to replicate their sounds or placement with a high degree of fidelity. They also skipped any species that were infrequent visitors, since the aim was to illustrate “a typical day, not a rare moment,” Sanderson says.
Drawing from Sanderson’s research about species density and distribution, McQuay slotted the sounds where he thought they belonged: frogs by the pond, for instance, and birds vocalizing from trees either closer to water or deeper in the glens of trees. He then adjusted the echoes, frequencies, and reverberations to telegraph distances, and tapped an ornithologist to fact-check his work.
The creators describe the project’s visuals as a “coloring book for sound.”
At first, the team imagined that the 15th-century installments would play in darkness, to completely immerse the listener in the acoustic elements of the landscape. But, in focus groups, “people kept thinking it was broken,” explains Al-Ibrahim. Now, the calls of the ring-billed gull, osprey, and common raven are coupled with visuals, but they’re bare bones. The scene consists of white features drawn against a black background: ghostly trees; the outlines of hills, like graceful brushstrokes; stippled waves; smudges of drifting clouds.
The moving clouds help create a sense of dynamism and motion as the viewer rotates the VR glasses without seeing the shape of the creatures whose sounds they’re hearing. It’s a surprise to hear a bird call, seeming to arc right over your head, and see no flash of wing or buoyant shadow. The creators describe the visuals as a “coloring book for sound,” adding perimeters and suggestions without filling in all the blanks.
The project’s larger purpose is more than romanticizing or lamenting bygone centuries. Accompanying narration celebrates local efforts to shore up pollinator populations and incubate native plants—strategies that cities around the globe are pursuing as grassroots efforts and official policy position green infrastructure as a crucial arm of environmental remediation.
The project’s co-creators anticipate that the project’s scope will expand as they earmark more funding and the landscape of VR technology continues to shift. Already, they’ve enlisted technology “that wouldn’t have been possible eight months ago,” Al-Ibrahim says. Next, they’ve set their sights on adapting the material to different user experiences, viewable both with and without VR goggles, and on both mobile and desktop, as well as through an app.
McQuay also hopes the project will help modern New Yorkers appreciate the contemporary urban soundscape, instead of blotting out the roar of the city with ever-more-sophisticated headphones. We’re missing out on something important, he says:
“Our environment is as much about what we hear as what we see. We’re always listening.”
This feature originally appeared in Citylab.