The sound of silence: Using technology to recover sound from inanimate objects.

If a tree falls in the woods, and no-one is around to hear it, does it make a sound? An age old conundrum which at its heart really asks, is a sound the vibrations that travel through the air, or is it only a sound when the vibrations are interpreted by humans or animals? The remarkable work by Abe Davis and his colleagues, have created a technology which may confuse the matter even more, but may prove useful in the fields of forensic science, surveillance, history, to name but a few.

Going back to the tree problem, when the tree fall it creates vibrations which travel in the air to our ears. The vibrations are channeled through the inner ear to the eardrum, which communicates the information to the brain as sounds. The vibrations create sound by making the eardrum physically move. Even if we are not in the wood, those vibrations are still present, and when they come into contact with other objects, cause them to vibrate and move on a micro scale, So small that the human eye could not identify the movement.

Davis et al, using a high-speed camera has managed to -at thousands of frames per second, pick up movement at the scale of a micrometer ( a 100,000th of a centimeter) from objects which have been exposed to sound. Even though the movements are much smaller than that of which can be represented by a single pixel, Davis' intuitive software looks for minute changes which occur in the whole image, which then translates these movements back into sound. In his TED talk, he demonstrates this by playing Mary had a little lamb through a loudspeaker to a plant and then plays the recovered sound from the imagery. It is far from the crystal clarity that we expect from modern gadgetry, however it is clear enough to hear the original tune, without straining.

Although in its infant stage, Davis has demonstrated its effectiveness using cameras at range and through obstacles such as glass and has even adapted the technology to run on a shop bought camera. Albeit the shop bought camera produces a less refined sound, it is still possible to identify the tune being played. However, through refinement of this method and perhaps enhancement of clarity of old silent film, we may be able to hear the voices of those who were once lost, or perhaps recover sound from a crime scene using CCTV footage or to be used as surveillance devices.

So in answer to the original riddle, a tree falling in the woods does make a sound when no-one is there to hear it (as we knew it would do). Maybe the question should be changed to "If a tree falls in the woods and no-one is there to hear it, or to have software analyse the video footage of the event, does it make a sound?"

To view the full TED talk, visit: http://cdn.instructables.com/FWH/W1CQ/G7DYJJSW/FWHW1CQG7DYJJSW.LARGE.jpg

Piercing perception, part 2: The plug and play brain.

Where is that damn installation disc?

I remember a time, in the not too distant past, where every peripheral that you bought came with an installation disc, containing the vital drivers required to allow the computer to make use of the device. Without them your gadget was no more useful than a rock, tethered to computer via string (or if the device was wireless...just a rock). Each new computer required the user to scrabble about in old boxes, search disc spindles or CD wallets to find the right disk. Failing that, a trawl through the manufacturers support pages were required to track down the specific software that would recognise how the computer was supposed to interpret and make use of  the electronic signals being sent from the input device. Nowadays, thankfully, you plug a mouse into the USB port and the computer is already installing it, ready to use in seconds. Marvelous.

If we were to consider our eyes, nose, ears, tongue and nerves as input devices, and our brain as the computer, then we already know that we have the right drivers to make sense of the signals we receive from them. However, if the brain can -as the computer does- find its own drivers, can we make use of prosthetic peripherals to enable us to sense the world in ways which have never been achieved before?

Sense and Sensing ability.

In the last post Piercing perception, part 1: A mole new world, I discussed how we perceive only a fraction of what the universe has to offer, due to the restrictions in our ability to process sensory information, as well as our inability to interact with the happenings of the universe on a macro or micro scale (in a meaningful way), without the backing of a well stocked science lab. Furthermore, we discussed the super senses of animals and how technology could potentially be used to harness their abilities, hypothetically allowing us to experience the world in new and exciting ways: expanding our umwelt (the world as it is experienced by a particular organism).

Artificial Inference

In the TED talk Can we create new senses in humans?, Eagleman (2015) explains that we already know that our brains are capable of adapting to receiving information from electronic devices, stating that many thousands of people can hear due to having cochlea implants or can see from having retinal implants. In a cochlea implant, a microphone picks up the sound, turning it into a digital signal, which is relayed to the inner ear. Likewise, with a retinal implant, a camera device captures the images, turns them into digital signals which are directed to the optic nerve. The brain can adapt to these new devices and find its own drivers to enable it to make sense of new signals.

But how does the brain speak digital? How does it translate these signals and convert it into something more familiar? The answer is that when the brain, for example, sees an object, it does not really see anything at all. All that happens is it receives electrochemical signals from the eyes, likewise when you hear something, the brain actually hears nothing but receives,  electrochemical signals. The brain sorts out these signals and makes meaning of them. It does not discriminate between what kind of data it is receiving, it just takes in everything and then figures out what to do with it, which Eagleman explains provides an evolutionary advantage, allowing for "...Mother Nature to tinker around with different types of input channels". So perhaps if we had the heat pits of a snake, the electro sensors of the Ghost knifefish or Magnetite which is in some birds to help with navigation our brain would be able to adapt to what they could pick up, allowing us to perceive our world in new ways.

In summary.

Our brains do not see, hear, taste, smell or touch, no more than a computer can see a digital feed from a camera. The computer just takes the patterns of electronic signals and sorts, uses and stores them in such a way to derive meaning. Our brains do the same. They are not fussy about what type of information they receive, they just find a way of using it. The versatility of our brains allows those who have sensory impairments to experience the world as though they would without their limitation; imitating the peripherals which time and evolution have bestowed upon us. The question now is not  whether our brains are capable of receiving digital input, but "How adaptable are our brains?", "Can our brains cope with new sensors, which evolution has denied us?" and "How will this alter our perception of the world?". 

Images

MRC (2012),  Cochlear Impant, viewed 9th July 2015, <http://www.mrc-cbu.cam.ac.uk/improving-health-and-wellbeing/cochlear-implant/>

Shutterstock (2012), altered by Novis, J. (2015) Plug and play brain, viewed 9th July 2015, <http://images.gizmag.com/hero/brainpolymer-1.jpg>

Piercing perception, part 1: A mole new world.

Introducing the umwelt.

You could be highly poised, caffeinated and keen; the type of person who remembers even the smallest detail, which others overlook and still, by the very nature of being human, only experience a slither of what the universe has to offer. Our capacity to understand our surrounding is limited to our 5 senses (arguably, there are more), our size and how we manipulate science and technology. This is known as the umwelt: the world as it is experienced by a particular organism.

Our World.

Our senses, touch, taste, smell, hearing and sight are extremely efficient at providing us with the relevant information necessary to exist, however we know that surrounding us are microwaves, x-rays, radio waves (but to name a few) which bounce off or pass through our bodies constantly, and we have no natural way of being attuned to their presence. Likewise, our size restricts us to what we can perceive. As we attend to our daily duties we are no more aware of the plight of a bacterium as we are to a star dying in a distant universe. We are creatures who live in a world of a certain size, who can operate successfully without  consideration for happenings on a micro or macro scale.

Despite our size and senses, we have evolved to deftly wield the tools of science and technology to expand our understanding of our universe. We can detect and utilise x-rays for medical reasons, microwaves for the convenience of cooking and look upon the stars and microbial matter with ease. However, even though we can artificially expand our umwelt, this leads to an intriguing question: What exists beyond it?

The world of the star-nosed mole.

Consider the star nosed mole, found in the wetlands of Canada and North East America...

...it is roughly the size of rat, lives underground and in water, and is blind. Initially we may think that this creature has a very limited umwelt. It lives in the dark, in the cold and cannot see: a pitiful existence, especially if you can imagine how you would live, as a human  in these conditions. However, the star-nosed mole is an incredible creature and that has the most sensitive touch organ of all mammals; its star shaped nose. Each digit-like appendage (ray) has approximately 25,000 sensory neurons, in contrast, the human hand has only 17,000 (Smithsonian.com). The mole can, by puffing out air underwater, sense with its star where prey is, it can move each ray with speed, it can touch up to 12 different objects per second and can detect electrical fields (Epic Wildlife).

A mole new world.

Great. Good for the mole. But what about us? Imagine, if through the use of technology we could utilise the sensitivity of the mole's nose. Imagine that we could sense electrical fields or could experience touch 32,000 times more than we can already. This has an obvious benefit for those that are blind. For example, the technology could allow the individual to know their surroundings by using their own breathing (much as the mole uses its puffing) and the feedback from it to sense objects and people. However, beyond this initial application, what if we could all sense our surroundings this way, layering the information we receive from the technology over that which we receive with our eyes. What if we could overlay, on top of that, being able to perceive electrical fields? Our perception of our world would be transformed completely, and so too would our interaction with it.

The star-nosed mole is just one exception of the animal kingdom, with fantastic abilities. Think about how our world would alter if we had the infrared capabilities of the pit viper or the powerful nose of the bloodhound? From these super powers we would be able to sense things that were there all along, but which had previously been inaccessible to us, thus expanding our umwelt. These new insights could potentially yield scientific breakthroughs in natural sciences, medicine and technology. 

Ahem! Excuse me... we already have the technology. What is your point?

We already have night vision goggles, Geiger counters, thermometers, powerful telescopes and electron microscopes, etc, to view our world. I agree. Although we do not use them, about our person, all of the time. The technology is limited to use in certain scenarios, allowing us momentarily to expand our perception. They are (and I do not mean to trivialise their worth) accessories to who we are and what we are capable of. The technology that I speak of would be worn most of the time, as one would wear, for example: regular glasses, transforming our ability instantly from our command. This could be wearing something akin to Google Glass and saying "Infrared vision" allowing the user to see infrared, or to say "microscopic vision" and instantly being able to zoom in and focus on minuscule objects. Or perhaps allowing one eye to see the changed view, whilst the other stays normal, permitting the user to create an augmented view. Having the power to experience the world at will, without the constraints of a science lab, would truly be as close as a humankind could get to having extra senses, without invasive alterations or through the natural process of evolution.

Final thought.

In summary, there is enormous potential in using technology to reveal the various aspects of our world which are currently hidden to us, That we may be able to toggle 'on' or 'off' powers which have, thus far, been confined to select species or  the realms of fiction is not only exciting, but will enable us to make rapid advances and discoveries in science and technology.  Although an equally thrilling prospect is, what else exists in the universe that exceeds what we can tap into with technology, and what lies beyond our collective knowledge. The nature of this mystery means that there are potentially limitless discoveries to be made from revealing our unseen world.

In my next post :

Piercing perception, part 2: The plug and play brain (due 4th-6th July), I shall discuss whether our brains are capable of adapting to receiving sensory information from new technology.

Images

Epic Wildlife (2014) Star-nosed mole, viewed 2nd July 2015, <https://www.youtube.com/watch?v=Egz2f5_Ip3U>

Novis, J. (2015) perception infographic, viewed 2nd July 2015,

USSLC (2014) Bubble burst, viewed 2nd July 2015, <http://usstudentloancenter.org/student-loan-bubble-not-going-to-burst/>