Synesthesia Symposion in Berlin

Romana Schuler (2011)

FACETS OF PERCEPTION

On early research into the compound eye, autokinetic sensations, and some remarkable experiments with prism glasses

"Change the eye of man and you will change his world view,"[1] the well-known experimental physicist Ernst Mach pointed out. The actual import of this realisation, as so often is the case, only became clear a lot later, in historic retrospect. Thus, for instance, the cyberneticist Norbert Wiener, in the twentieth century, was convinced that the rapid networking on the basis of technical mobility was the only major reason why the so-called United States of America were able to unite both politically and culturally in the nineteenth century.[2] At the beginning of the twentieth century, the emerging technical picture media triggered a radical change that led to the explosion of images, the so-called pictorial turn, initiated mainly by photography and film. To be followed by television, video, computer, as well as the Internet and finally mobile devices such as iPhones etc. As the technical picture, sound and communication media suddenly became available also to non-experts, non-scientists and non-artists, the pictorial turn, significantly, descended on the epistemic domains and the visual arts. The animation of hitherto static images, for the first time realised with the help of the stroboscopic disc (the birth of film) − which later, through information technology, has lead to interactive media − was no more than a further logical step in the direction of these media developments.

With the emergence of faster technical modes of transport the new facilities of mobility (railway, telegraphy, photography) too became a subject for the disciplines of sensual physiology and cognitive psychology. The cognition of movement was thus given a greater importance than ever before. That human vision could be altered, controlled or manipulated by means of special visual instruments or cognitive apparatuses had already been proven in earlier psychological and physiological experiments.

Since the beginning of the nineteenth century, the perception of movement from a thoroughly empirical perspective was being pushed in scientific experiments. Decisive impulses came, among other things, from two inventions: the stroboscope (Plateau, 1832; Stampfer, 1833) and the stereoscope (Wheatstone, 1838). These significant innovations in the production of moving resp. three-dimensional pictures not only led to intensified research into the human perception of movement, but also a veritable flood of other inventions, i.e. all kinds of cognitive apparatuses meant to raise detailed questions with the help of a new scientific method, namely experimental empiricism, questions such as: How do we perceive movement, and how do phenomena like apparent movement or apparent spatiality come about? Which stimuli cause a reaction on the retina when we perceive actual movements or also apparent movements? Which area of the retina registers the movement most clearly?

The scientific investigations undertaken by the disciplines of psychological and physiological research, that had dealt with the perception thresholds between actual and apparent movement, formed the basis of the worldwide, all-encompassing tendency towards mobility. In the course of the nineteen-thirties, new scientific disciplines such as cybernetics, and later cognitive sciences, emerged which went in search of a universal computer, the functions of which were to enable not just the automatic recognition of movement but also the direct interaction between humans and an intelligent machine. Through the modern media of our day and age, such as virtual reality in general or their continuation into augmented reality, actual and virtual movements or apparent spatialities have become a natural, integral part of all areas of life.

Sigmund Exner's early experiments into vision and the compound eye

In order to observe the perception of (apparent) movements, the physiologists of the early nineteenth century preferably chose, for primary research material, special pictorial constructions of visual tricks, like the Müller-Lyer or the Zöllner illusion. For naturalists, natural phenomena in the field of apparent movement, like the moon or shore illusion, as well as many observations from the animal world too played a central role.

One of the most celebrated studies of the time, that dealt with the subject of the viewing of movement and aimed to explain the latter by means of the compound eye, was the one done by the Viennese physiologist Sigmund Exner (1846−1926). Exner had studied with Ernst Wilhelm Brücke in Vienna and Hermann Helmholtz in Heidelberg. Both, Brücke and Helmholtz, had been students together with Emil Heinrich Du Bois-Reymond of the famous "vitalist" Johannes Müller. The three scientists, from 1847 onwards, described themselves as "organic physicists," thus distancing themselves from the vitalist approach of their teacher. Especially Helmholtz dealt extensively with optical phenomena. He is regarded as one of the founders of the empirical theory of perception, that bore reference to crucial elements of associational psychology[3] and later was developed further by Helmholtz' student Wilhelm Wundt. Exner, on the other hand, was the first person, in 1896, to formulate a concept of the neural network, starting from the assumption that the brain was a store of associations that by means of a sort of matrix used the neural nodes for its elements.

Throughout his life, Exner gave expression to his frustration with the lack of conceptual clarity that he encountered in the field of the theory of perception. Which is why, first of all, he attempted to define more clearly the concepts of "perceiving" and "feeling" by means of reflecting on the second and the minute hand on a clock. For this purpose, he put in rotation a black disc the diameter of which was marked in white. The speed of rotation corresponded to the speed at which a minute hand moves around a clock. The movement by and by became apparent by the degree in which the white marker line slowly changed position. When the speed was increased, though, there was a moment when a movement became apparent to the observer, while with the slow speed previous the movement could only be assumed. It follows, so Exner's definition, that the impression of the slow movement counts among the perceptions, while with the faster movement what happened was a pure feeling of movement. Among such feelings of movement Exner, subsequently, also subsumed all other possibilities of apparent movement. His theory thus states that all experiments with stroboscopic discs were not only instances of apparent movement, but in each case also actual feelings of movement. Exner had arrived at the opinion that the "most primitive qualities of our eye"[4] were reserved exclusively for the perception of movement. An inkling of this significance of movement for our perception we already encounter in Aristotle (384−322 BC), who also regarded light as a sort of movement emerging from a glowing body. The actual perception through the eye, therefore, did take place solely thanks to the movement of the glowing body.[5]

Based on observations of animal behaviour, of animals able to recognise their prey only thanks to manifest motions, and thus to react and successfully chase it, Exner arrived at the assumption that the perception through the eyes as found in humans and higher animals was probably necessary only for making out movement. This conclusion led him to examine the functions of the compound eye, in order to answer the question which purpose movement vision actually served.

If light hits one unit of the compound eye, Exner remarked, and then the next and the next after that, then the light ray (for instance from a candle) must be experienced as a very intensive kinaesthesia. The intensity depended on the number of nerve endings stimulated by light.[6] Moreover, the compound eye could orient itself in two directions: on the one hand, it localised the movement, and at the same time registered the intensity of the feeling. The more units there were, the better the localising. A highly refractive so-called crystalline cone, situated behind the cornea, prevented the sensation's intensity from decreasing. Exner compared the compound eye with a sort of light condensator, an instrument similar to the one he found inside his microscope serving to illuminate the examination object. (Abb.)

One question in particular seems to have occupied Exner, namely whether looking through the compound eye resulted in a little picture on the retina, as was already known from the eye of vertebrates. The repetition of an earlier experiment with a fly (Musca vomitans), originally undertaken by Johann Christoph Gottsche, remained unsuccessful. Only with the water beetle (Hydrophilus piceus) did Exner think he could detect so-called compound images.

In an experiment on the compound eye, Exner placed a convex lens, with a focal length of two inches, inside a room in such a manner that through the lens he could view the image of the mullion and transom crossing. Eight inches before the lens he placed a pencil in a vertical position. When the observer, consequently, positioned his eye about one foot behind the lens, he could make out the window and the pencil fairly clearly. By means of a translucent screen Exner then took the image of the window, only to "find totally missing the image of the pencil," as he wrote. "When I move the screen two to three inches away from the lens, I get the image of the pencil and no more than a hazy bright blotch for an image of the window."[7] Exner assumed that the image of the compound eye, supposing it really existed, had to form on various different levels. He ultimately became convinced that the compound eye, due to its anatomy und its optical qualities, could not produce a miniature image comporable to the retinal image. Thanks to the large number of perceptory units, however, the sensitivity was greatly increased, which had to be due to the "light condensers." The light of a candle alone, Exner stated, would stimulate more nerve endings than was possible in the human eye. Therefore, the compound eye to him was the "key" to the "way the eye works as an organ recognising movement."[8] Although not ideally suited for a two-dimensional or spatial capturing of the outside world, the compound eye, consisting of many individual eyes, in his opinion was perfect for recognising movements. The construction of compound eyes in the shape of spheres creates a field of vision of nearly 360 degress. The object to be observed is being identified by a whole group of eyes, which is why each change in position of the visual object too is being registered only through a process of alliesthesia affecting the entire group of perceptory units. Exner compared the workings of the compound eye to the reaction of the peripheral areas of the human retina to processes of alliesthesia.[9]

On July 15, 1875, he submitted his study at the Vienna Academy of Sciences for the so-called Vienna Sessions. The research findings were published in 1876 under the title Über das Sehen von Bewegungen und die Theorie des zusammengesetzten Auges (lit. On the Perception of Movement and the Theory of the Compound Eye).

Autokinetic sensations

In 1896, Sigmund Exner published his treatise Über autokinetische Empfindungen (lit. On Autokinetic Sensations)[10], in which he described instances of subjective processes of alliesthesia in darkened rooms. His inspiration he had taken from the early research of his friend Hermann Aubert (1826−1892), who had dealt with this sort of illusions before him and had also published an essay on the processes of alliesthesia around 1887. Besides the description of his own experiments, Exner, in his text, also compiled an historical review on the phenomenon. Thus, for instance, Alexander von Humboldt, in 1799, had written about stars low in the sky performing oscillating movements. Humboldt's astronomical observation was declared a subjective illusion by Kaspar Gottfried Schweizer, director of the Moscow Observatory, in his 1858 study entitled Über das Sternschwanken (lit. On the Oscillating of the Stars). Schweizer had experimented with a so-called “artificial star,” an opaque lantern with a narrow light slit, which he had observed inside a dark room, thus being able to explain the phenomenon as an apparent movement. Schweizer's experiments were of crucial importance for Exner, as the former had observed the largest elongations so far in the movements of objects. In this context, Schweizer described the following experiment, for instance: "If you paint a black spot or also a larger blotch on a white wall and stand at a distance so that the spot or blotch is still visible fairly clearly, you will, if you fix them with your eyes over a period of time, notice the strange phenomenon that the spot or blotch will seemingly begin to move, namely in different directions, but each time to return to its original location. […] While the separate objects appeared to change position slightly, they also seemed to change their appearance round the edges, so that especially the black spot made the impression as if an insect sat on the white wall trying to crawl here and then there, but each time returning to its starting point. The illusion of observing a living thing on the wall is so complete that the person not clearly aware of the opposite has to move closer in order to make sure."[11]

In 1886, the French physiologist and neurologist Pierre-Marie-Augustin Charpentier (1852−1916) carried out quite similar tests, probably without prior knowledge of Schweizer's experiments. Charpentier described how a point of light, that one stared at for minutes on end inside a darkened room, soon appeared to move. According to his notes, this effect was taking place even within a few seconds, and that one gained the impression of a gentle shift. Charpentier called such phenomena "subjective visual sensations." Hermann Aubert, in his studies on the perception of movement, arrived at an almost identical conclusion and may have been the first to sum up these phenomena with the term "autokinetic sensation." According to Exner, what happens here are associations of mental images. The mental image of direction gives us the impression of the point of light stared at moving in the same direction. Exner was convinced that the mental image alone sufficed for a balloon hovering in the distance, for example, to be associated with a bird. One question, however, remained unanswered for him: "Why does movement force itself on our consciousness? For this happens long before a mental image, that could associate itself with the perception of the point of light, appears."[12] These observations probably make Exner the first person to hint at the significance of neuronal links that are responsible for thinking and consciousness being connected to vision.

The Innsbruck visual experiments with prism glasses

After the experimental psychologist Franz Hillebrand had been given a professorship in philosophy at the University of Innsbruck in 1896, he established a laboratory for psychological experiments there at the beginning of the twentieth century. Hillebrand had studied philosophy with Franz Bretano, and after his graduation in 1881 had worked with Ewald Hering[13] and with Ernst Mach in Prague. In his later research he focused primarily on visual perception and the geometry of space, as well as the theory of cognition. Starting out from the findings of the brain researcher Ewald Hering and the physicist Ernst Mach, he concentrated his own research on questions of visual perception. Franz Hillebrand dealt extensively with stroboscopic movement and, in 1922, published his findings on the subject of apparent movements, a study written with the intention of "basing stroboscopic movements on a theory as free of hypotheses as possible."[14] "We comprehend stroboscopic movement as an expression of a gradual reevaluation [of the retinal position] […] We do not take movement as the completion of a gap, but as a reevaluation that the second object's retinal position undergoes. We can, as Wertheimer has observed, create stroboscopic movement between a plant, a bird cage and a grape. And indeed we make out such effects between percepts that do not have the least inner similarity, in the context of which therefore there can be no question of identification. If the letters of an electrically illuminated billboard, spelling the word CINEMA, successively are lit up, everyone has the impression of movement, but we cannot say the C moves on into the I, the latter into the N etc. The C, much rather, disappears and the I comes in from the left until it reaches its final position."[15] Hillebrand, in other words, assumed that under certain circumstances the relocation of the field of view does not take place synchronously with an extension of the place of attention (retinal position), but that it is delayed.

Hillebrand died unexpectedly in 1926. His successor at the University of Innsbruck was the Swiss psychologist Theodor Erismann. From 1928, Erismann began to deal with the phenemenon of the upside-down image on the retina by submitting the healthy eye to "artificial" defects of vision, experimenting to that purpose with spectacles. In particular, the scientist looked for means of regenerating visual impairments. The increased occurrence of eye injuries was one of the consequences of World War I.[16] Erismann drew on the work of the American experimental psychologist George M. Stratton[17] who, from 1895 to 1896, had undertaken a number of experiments at the University of California on the retina and upright vision. His questions were generally aimed at the necessity of image reversal for our upright vision.[18] Towards the end of the nineteenth century, two theories basically dominated the scene explaining the problem of why the retinal image was the "wrong way" up, namely the projection theory and the eye movement theory. Stratton constructed a visual instrument that guided the right eye's gaze through two convex lenses of the same focal length that are placed closely together along a visual axis. (Abb. 3 Skizze von Strattons Umkehrbrille) The tube in front of the left eye was covered with black paper. On the first occasion, he wore this apparatus for a period of three days. The double-lens glasses presented Stratton with a completely new field of vision that showed everything in reverse. The image had to be mentally reinterpreted so that the researcher could get his bearings. Physical movements, though, were not automatically executed in the new direction, which led to a contradiction between the visual appearances and the tactile sense. The direction Stratton reached towards initially did not correspond with his visual perception. Yet after a certain length of time his field of vision lost this reversed, strange character and he returned to an acceptable visual normality. For Stratton, this was proof that the collaboration between tactile sense and vision had to be important for spatial orientation.[19] "The experiment indicates that if we were to see a thing long enough in any given place, we should, sooner or later, also feel it there."[20]

For similar experiments at the Innsbruck Institute of Psychology, Theodor Erismann at first used a specula placed horizontally below the eyes. The larger the specula, the larger the field of vision. However, this construction cut off the view of one's own feet, for which reason the experiments could be carried out only within the institute. Ivo Kohler, still a student of Erismann's at the time,[21] assisted in the experiments from 1939 onwards. Besides the replication of Stratton's experiments, Erismann and Kohler also looked at the question of how we perceive forms and shapes, size and movements, as well as colours. The results of their research led to new insights into the construction and the genesis of physiological perceptions in the context of an everyday milieu of stimuli.

Experiments with binocular prism glasses Erismann had carried out as early as 1933. After two days, during which these glasses were worn permanently, the apparent movements perceived at first had disappeared and the initial curves in straight lines and warping of shapes had markedly improved. (Abb. 11 Bildverzerrungen bei Prismenbrillen) According to the statements of the test persons, the visual disturbances had disappeared completely after ten days. After the glasses had been removed, again there were marked instances of apparent movement, curving and distorted shapes. It was like being inebriated, one test person described the state that was gone by the fourth day the glasses were no longer worn. Kohler, himself acting as a test person, used the binocular prism glasses for a duration of up to 124 days.

From 1947 onwards, the specula could be positioned above the eyes with the help of a newly developed contraption resembling a visor cap. In this way the researcher could use speculae that were twice as large, and the redesigned prism glasses became street-compliant, as it were. (Abb. Umkehrbrille nach Erismann und Kundratitz 9, 10 ) Within four days a cycling tour was undertaken and on the sixth day even a skiing outing. After the glasses were taken off, according to the test person, objects presented in a vertical position were perceived upside down during the first few minutes. Kohler drew a comparison with the viewing of inverted images, e.g. the Schröder staircase.[22]

The Innsbruck-based scientists received considerable international applause. Among others, the German-American art historian Rudolf Arnheim, curator of the exhibition "The Responsive Eye" (1965) at the Museum of Modern Art in New York, that was dedicated to kinetic art and OpArt, found the Innsbruck experiments impressive for the very reason as with these new approaches to perception, as he remarked, a "rubber world" could be created.[23]

For Richard Gregory, well-known neuroscientist and author of "Eye and Brain" (1966), Stratton's and Erismann's resp. Kohler's studies are among the best visual experiments there are, discussed to the present day and inspiring scientists around the world, from Japan to the United States, to undertake numerous replications and follow-on experiments. Most recently, for instance, the interesting long-term experiments with prism glasses were replicated, in 1999, at the Max Planck Institute in Frankfurt, under the direction of neurologist Wolf Singer in the context of a doctoral thesis. The author of this thesis, David E. J. Linden, now professor at Bangor University (Wales), ten years after completing the project gave this comment: "I continue to regard the question of adaption (perceptual vs. cognitive/motoric), as exemplified in prism glasses and other transformations in sensory input, as a very fascinating one."[24]

The brief outline of visual experiments on the perception of movement alone will suffice to show how very receptive the human visual sense is to deliberately misleading (pseudo) realities. Which fact again raises questions as to our intellectual and practical capacities in dealing with apparent movements and spatial illusions. Relatively simple visual apparatuses with mirrors and prisms are perfectly sufficient for our sensory organs to take in, beyond their physiological cognitive ability, also artifically produced, distorted mental images and pieces of information, and to practically process these after a learning phase. The "problem of truth," of the "reality" that we perceive or mean to perceive, has basically always determined the structure of our world view. This quite possibly is the reason why our designations of new media tend to include the term reality, as in "virtual" or "augmented reality," in order thus to legitimise "illusion" and "reality" in equal measure. As if it were a basic need to produce a qualitative equivalence between illusion and reality. Certain facts about the technical possibilities of picture media show marked similarities to experiments with drugs, that have been used in manifold ways throughout the history of mankind.[25] The idea that computer-based virtual worlds, among other things, were a substitute for psychedelic drugs therefore is not so far-fetched. However, these technical, "artificial" surrogate worlds also show significant differences from the "consciousness-expanding" drugs mentioned. The appearances working on our sensory perceptions have in fact been specifically programmed and thus have something highly artificial about them, that moreover promotes the illusion that it can be uniformly perceived by any "user" whatsoever.

We live in an age, in the twenty-first century, that has been characterized as having brought forth the "post-media generation."[26] Technical media, in a well-nigh universal manner, and basically accessible to everybody, explain the world to us, tell us what there is to discover and how to communicate, and also what and when we have to consume, etc. The novel media quality, consolidated through psychological and physiological research, and supported by recently founded disciplines such as neuroaesthetics or molecular aesthetics, has been defined as postmedial and pretends to open up new possibilities of a democratic life. The developments currently taking place in science and art are a clear indication that cognitive research, as for example the specific question whether what we witness is a real or only an apparent movement, will remain highly pertinent for the post-media generation, despite the highly advanced, post-medial technical support at our disposal.