[dropcap style=”font-size:100px; color:#992211;”]D[/dropcap]aredevil made it look easy – using the sounds of the immediate environment to create a mental picture as accurate as that we perceive with our eyes.
Can we do it in reality though? Exactly how bat-like can humans get? Is our sound perception actually better than we give it credit, might it be part of the reason we always seem to know when someone else is in the room, even if we can’t see them.
Who knows? But the Notre Dame cathedral in Paris seems as good a place as any to put it to the test.
Paris’ Cathedral of Notre Dame has a ghost orchestra that is always performing, thanks to a sophisticated, multidisciplinary acoustics research project that will be presented during Acoustics ’17 Boston, the third joint meeting of the Acoustical Society of America and the European Acoustics Association in Boston, Massachusetts.
In the project, computer models use recordings from a live concert held at the cathedral and detailed room acoustic simulations to produce a novel type of audience experience: a virtual recreation of the live performance using spatial audio and virtual reality.
Researchers reproduced the recordings using computerized acoustical data and enhanced it with computer-generated virtual navigation — 3-D visualizations made with immersive architectural rendering that float the viewer through the complex acoustics of the acclaimed medieval gothic cathedral.
Combined, the multimodal sound and image footage of the ghost orchestra produce a spectral tour to the sounds of the 19th century opera “La Vierge” — The Virgin — performed live during the 2012-2013 concert season to celebrate the Cathedral’s 850th anniversary.
Multimodal virtual reality integration is central to the project’s significance, said Brian F.G. Katz, lead investigator and CNRS Research Director at the Institute Jean Le Rond d’Alembert, Pierre and Marie Curie University, in Paris.
“3D-audio is the hot topic today in virtual reality (VR) that is currently a very active subject in both academic and industrial research,” Katz said. “With the commercialization of affordable VR systems — the cheapest allowing for VR on smartphones — spatial audio is rapidly immerging from the laboratory.”
The next stage in spatial audio is personalized audio rendering that involves being able to adjust the rendering to match one’s individual head and ear details.
“The importance of multimodal interactions, how visual and auditory cues balance in spatial perception, is key to VR and the sense of immersion, of being ‘in’ the VR world,” Katz explained.
He envisions many applications emerging from the investigation.
“For me, spatial audio is a domain on the boundary of physical acoustics, psycho-acoustics, perception and cognition, and digital signal processing.”
His work focuses on using each of the fields to learn more about the others — from virtual audio scenes exploring how visually impaired people understand and remember architectural spaces, to improving understanding of spatial audition, to refining virtual reality rendering capabilities.
Created in the context of the French funded research projects FUI-BiLi, (Binaural Listening and ANR-ECHO), the acoustical foundation of the project is a 45 channel close-mic recording of the live concert made by the Conservatoire de Paris combined with a detailed geometrical room acoustics computational model that was created and calibrated based on in situ measurements of reverberation and clarity parameters.
Next, the team will apply the methodology to other complex multimodal environments such as theatre simulations.
“Aside from the acoustics in this project, I think we definitely learned a lot about computer graphics and VR content production, which has opened up a lot more dialogue with those communities that we intend to pursue,” Katz said.
Further reading
Academic Articles (Google Scholar)
Feature Selection for Multimodal Acoustic Event Detection (PDF)
Source: Eurekalert/Acoustical Society of America
Image: Pixabay/Tpsdave
Some of the news that we find inspiring, diverting, wrong or so very right.
