A Sound Way To Make Data Tangible

Using sound to give form to data offers new possibilities for understanding the human body, but so far has not taken the prominent place that seems possible.

Sound has so far not taken the prominent role that seems possible in giving form to data.

An introduction…

Aqua Vita, our previous collaboration with the Netherlands Metabolomics Centre, was our first journey into the world of ‘body data’, triggered by a fascination with the supposed paradigm shift in healthcare towards a future of daily self monitoring and diagnosis. You can probably refer to Aqua Vita as being a first step towards making sense of these changes on a personal level, an attempt to comprehend the ever growing quantity of abstract information from the body. I say abstract, because without the notions of time, context and establishing comparisons, this data remains what is essentially is – a sea of numbers. Decoding the human genome involves analysing 3 billion base pairs. This took ten years the first time it was achieved in 2003, however, this can now be done within a week. As technology evolves and expands to contain, control, transfer and translate this information, so will its sheer quantity. As Alex Szalay, astrophysicist at Johns Hopkins University, notes, the proliferation of data is making it increasingly inaccessible. How to make sense of all this data? Exploring new methods of data collection, and more importantly its translation into new interactive experiences, we hope to investigate the potential for the senses, in this case sound, to generate a new understanding of the body by asking the question, what if we were able to listen in on the electro-chemical messages sent by our bodies?

In The Rhythm of Life project the data itself stops being seen as a by-product of bodily processes but something much more potent. Discussing the potential of big data in healthcare, Craig Mundie, Chief Research and Strategy Officer at Microsoft, explains “you would not just think of data as the ‘exhaust’ of providing health services, but rather they become a central asset in trying to figure out how you would improve every aspect of health care. It’s a bit of an inversion.” This inversion highlights then the importance of new tools to collate, translate, and structure this information. I hesitate to say ‘visualise’ as herein lies a contentious argument.

Can’t ‘hear’ the wood for the trees

On Monday 9th August 2010, BBC Two aired a panel discussion between Neville Brody and David McCandless on Newsnight. In it we hear two opposing takes on the promise and boom in data visualisations.

Advocates of data visualisation maintain that we’re already swamped by data and that new tools and methods are desperately needed to make sense of it all. Data visualisation, it is declared, will actually aid our understanding of an ever more complex world.” The sceptics, as represented by Brody, argue that what in fact data visualisation does is create “a lot of pretty pictures” doing nothing more than pleasing the eye. At this stage of the project, I’m still on the fence, however, I will say that I tend to agree that when data visualisation does not work, it often leads to a kind of visual noise, or banality of information that fails to live up to the preaching of its advocates.

Paul Burgoyne, blogging on the discussion for Creative Review at the time, put if nicely when he states, “In the way that it is employed, data visualisation is no more inherently neutral than any other form of statistical analysis… Graphics that purport to make a statement of fact should be approached with as much caution as a government press release.”

It is true that, as inherently visual creatures, we are bombarded by visual culture, however, the warnings reported above are no doubt the same with sound. One of the biggest challenges of the project will be to manage the translation of the pulses of the photon emissions in a manner that stays true to the raw data itself while stimulating the emotional connection to the information via its transformation into rhythmic melodies. Our challenge then with sound, is much the same as Burgoyne suggests in the same article, “… a certain amount of visual flair  is necessary in order not just to explain complex data but also to encourage engagement with it. The question is at what point does the pursuit of the latter undermine the former?”

Clarity in Sound

In the book “Designing Interactions”, Bill Gaver is interviewed regarding his research into building what he calls ‘a new psychology of sound’:

“It’s important to realise that sound conveys very different information than vision does… Sound works very differently. Sound is created when objects start to vibrate, and the way they vibrate depends not just on their surfaces, but also on their internal configuration, as well as certain details about the interactions that cause them to make sound. Once a sound is created, it bounces around your ear, but it seems to convey less information about the environment than it does about the source…”

As Dave mentioned in his previous post, our aim is to discover people’s unique rhythmic signatures, and what this might in turn say about their physiological or mental condition. The ‘clarity of sound’, to which Bill Gaver alludes, is then, an extremely important aspect of our research. The ability to hear and feel the punctuations in rhythm offers a new means by which to interpret body data, giving physical form to digital information, making bits directly manipulable and perceptible. This perception relates to not only the audible translation of bio-photons, but the hands-on interaction with the Photon Multiplier Tube and indeed the experience as a whole, including audience participation, supplying additional sources of data and the process of ‘waiting’. In my next blog article I will discuss the psychology of the waiting line and the potential effect upon the individual [using the machine] and surrounding audience experience when challenged to provide personal data for public consumption, in exchange for a new interaction with the body.

Moggridge, B (2007). Designing Interactions. Cambridge, Massachusetts: The MIT Press. p577.