FIGHT! Virtual Reality Binocular Rivalry

FIGHT! at Ars Electronica 2017
Carter, O. L. et al. Meditation alters perceptual rivalry in Tibetan Buddhist monks. 2005. Paper, Presentation

Context

The way our brain constructs a conscious visual percept is quite complicated, and not necessarily a direct representation of what is out there. i.e. what we see in our minds, is a reconstruction. And this reconstruction is based on who we are. Even though everybody is presented with the exact same images, what you see in this experience is different to what I see. I cannot see what you see, and you cannot see what I see. Furthermore, what we both see, is different to what is actually presented. We are both unable to see the ‘entirety’ of the ‘ground truth’, so to speak.

Introduction

There’s a few themes that I’d like to reflect in this work: i) what we perceive to be real, what we see, is a reconstruction in our minds, a simplified model of the world, limited by our biology and physiology, ii) perception, including vision, is an active process, it requires action and integration; iii) the actions that we take, affects the reality and the meaning that we construct in our mind; iv) perhaps most importantly: even when presented with the same information, the same images, everybody will experience — will see — something unique and personal, which nobody else can see or maybe even understand.

i) What we perceive to be real, what we see, is a reconstruction in our minds, a simplified model of the world, limited by our biology and physiology

Excerpts from Jörg-Peter Ewert & IWF: Visually guided prey-catching and threat-avoidance behaviors in toads and the underlying neurophysiological processes. http://www.joerg-peter-ewert.de/13.html
Kanizsa G., Kanizsa triangle. 1955. Can you see a white triangle, that isn’t there? Example of https://en.wikipedia.org/wiki/Illusory_contours
Kitaoka A., Rotating snakes. 2003. Example of https://en.wikipedia.org/wiki/Peripheral_drift_illusion

ii) Perception, including vision, is an active process, it requires action and integration

from http://publishing.cdlib.org/ucpressebooks/view?docId=ft438nb2fr&chunk.id=d0e340&toc.id=d0e340&brand=ucpress
Al Hasan Ibn Al Haytham, Book of Optics. c1000 CE.
Alfred L. Yarbus, Eye Movements and Vision, 1967

iii) The actions that we take, affects the reality and the meaning that we construct in our mind

Yarbus Eye Tracker https://en.wikipedia.org/wiki/File:Yarbus_eye_tracker.jpg
Alfred L. Yarbus, Eye Movements and Vision, 1967, on Ilja Repin’s (1844–1930) The unexpected visitor
Alfred L. Yarbus, Eye Movements and Vision, 1967. Superimposed colour images from http://www.cabinetmagazine.org/issues/30/archibald.php
Pelphrey, K. A. et al. Visual scanning of faces in autism. J. Autism Dev. Disord. 32, 249–261 (2002).
Anagram. Door into The Dark. 2015.

iv) Everybody’s experience is unique and personal, nobody else can see or maybe even understand

Binocular Rivalry (BR)

(See here for a great introduction to BR by Olivia Carter, and here for a review of BR used in an artistic context by Michael Scroggins).

Carter, O. Binocular Rivalry Tutorial. (2006). Available here.

FIGHT!

Fight at Ars Electronica 2017

Narrative journey

I designed the narrative journey (i.e. how the experience develops over time) with a few specifics in mind.

(slightly sped up for online documentation)

Outside vs Inside the head

It’s also worth mentioning, that usually when we see, the images that we see appear to be taking place outside the head. I.e. when looking at the world, it appears that the world is external to us, and we are looking at it through a window — a hole in our head — which is our eyes. Even when looking at a flat 2D image, like a painting, a photograph, or an image on a screen, it still appears to be external, it appears as a flat image on an external medium — such as a canvas or screen.

Transitioning from ‘inside the head’ to ‘outside’.
‘The Penetrating Gaze’ (slightly sped up for online documentation)

Psychedelics & Tripping

Quite a few people have likened this experience to taking (psychedelic) drugs and ‘tripping’. In a sense, this is understandable. Hundreds of millions of years of evolution went into developing our sensorimotor systems, including the development of vision and the integration of vision with movement and other senses to produce this coherent singular conscious experience of an external world that somehow ‘makes sense’ to us. Hallucinogens, such as LSD or Psilocybin ‘magic’ mushrooms etc., create perceptual distortions. They ‘hack’ these mechanisms that give rise to our conscious perceptual experiences.

‘Look Painting’

Technical

For anybody wondering about the technical implementation details: the piece was developed in the Unity game engine (C#), the inbuilt Oculus Rift integration and a simple hack to allow different objects to be visible to different eyes (i.e. two cameras, and different layers for each eye). The audio uses HRTF binaural spatialisation with an 8-track score distributed across 8 static virtual sound sources positioned around the centre, and 3 additional moving virtual sound sources depending on the developments in the scene.

Acknowledgements

Commissioned by STRP
Score by Rutger Zuydervelt (Machinefabriek)
Producer: Juliette Bibasse
Assistant: Rob Homewood

References

1. Anderson, E., Siegel, E. H., Bliss-Moreau, E. & Barrett, L. F. The Visual Impact of Gossip. Science (80-. ). 332, 1446–1448 (2011).

--

--

computational ar̹͒ti͙̕s̼͒t engineer curious philomath; nature ∩ science ∩ tech ∩ ritual; spirituality ∩ arithmetic; PhD AI×expressive human-machine interaction;

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Memo Akten

Memo Akten

computational ar̹͒ti͙̕s̼͒t engineer curious philomath; nature ∩ science ∩ tech ∩ ritual; spirituality ∩ arithmetic; PhD AI×expressive human-machine interaction;