Hearing is one of our most powerful senses, but one we don’t always use in a conscious way. Without realizing it, we can filter out all the noise in a busy room except the conversation we’re having. Think of the way you can hear within a word or two down the phone how your friend is feeling!
But all too often, what we mostly do is block sounds, either with our thoughts or with music on headphones. Hearing is also arguably the most important skill anybody needs to understand and appreciate music. All the technique, knowledge or musicality in the world won’t help you if you aren’t really listening, and really hearing.
Hearing is under-developed in games in general, and AR in particular. We have incredible directional hearing ability; we can pinpoint where a sound is coming from, and detect subtle changes in volume or tone. So this gathering was created to hack, and then develop, an AR music game that will both help users to learn to listen, and to understand a bit more about classical music, whilst also developing the potential of audio in AR.
Together with students from the Media Lab, the Berklee School of Music, and the Parsons School of Design, Director's Fellows Sheila Hayman and Colleen Macklin spent a weekend leading the group through the process of game design, interaction, and coding.
We sang with straws to warm up our voices and remind ourselves to play
We sang ‘Row Row Row your Boat’ to learn about canons in music, and how you can shift things along and they still ‘work’.
We introduced our partner orchestra, the Orchestra of the Age of Enlightenment in London, which is a very adventurous band with a commitment to ‘historically informed performance’. This means, in practical terms, using sheep-gut strings and goatskin timpani, but in conceptual terms also the importance of finding out the context of the music’s original composition, performance and place in the culture - all relevant to the purpose of ‘adding meaning’ to their performances in different ways.
Then we introduced the idea: AR applications are generally focussed on picture, rather than sound. Classical music needs new ways to reach younger audiences and explain itself. Geolocation is a technology that is being widely exploited for utilitarian purposes (maps, targeted messaging in shopping malls, etc) but is underutilized in the playful space.
Can we find a way to bring these three thoughts together, using counterpoint and the different ‘voices’ of a piece of music as an example? The answer is yes!
In two days, much was accomplished: four unique game prototypes were developed, from eating creatures to using your body as a brush and painting in space, all in search for interactive ways to bring people closer to the sounds that surround them. Although we still have lots of work to do on the final design of these prototypes (and we need to protect its content), we are happy to report it was an incredible experience shared with a very creative and collaborative group of students and guests.
We'll report back after our second session later this year.