Art and Algorithm

Digital artist and researcher Daniel Jones explores the world of self-generating imagery

Text by Daniel Jones

“We have come to the edge of a world of which we have no experience, where all our preconceptions must be recast.” –D’Arcy Wentworth Thompson, On Growth and Form

In 1882, his eyesight failing and fingers becoming increasingly feeble, the ageing Nietzsche could no longer read his own scrawl. He was forced to make a radical break; rather than abandon writing altogether, he would adopt the new technology of the typewriter. He contacted the inventor of an ornate device known as the Writing Ball, who mailed him one direct from Copenhagen. Now, through touch alone, the hemi-spherical keyboard could convey Nietzsche’s thoughts onto the page, as clear and legible as they still were in his mind.

Yet the substitution of keyboard for pen was not as straightforward as it seems. A composer friend of Nietzsche’s spotted a shift in his writing style, which was becoming “even tighter, more telegraphic”. Nietzsche himself noticed the impact of the new medium: “Our writing tools are also working on our thoughts,” he wrote in a letter.

Today, the ubiquity of digital media has only amplified fears that technology is altering our minds. Popular science writers take great pains to emphasise the dangers: “Technology doesn’t just do things for us,” Sherry Turkle writes in a recent New York Times op-ed. “It does things to us, changing not just what we do but who we are.” This is indisputably true. Any long-term exposure to new ways of interacting with the world – alphabets, instruments, interfaces – will cause structural changes in the brain. And any changes to the brain will affect the way we imagine and make things. But is the impact of tools on our creative process really something to fear

The Western Romantic idea of art involves a lone figure toiling away in a studio. His paintbrushes are inert instruments, transmitting a fully formed internal vision onto the tabula rasa of the canvas: a perfect reproduction of the mind’s eye. The artistic medium itself is meant to be transparent; any visible impact it may have is effectively an impurity, an obstacle to the artist’s self-expression.

Yet our relationship with our tools is itself a living, productive process. Any medium profoundly affects what it transmits, leaving a distinctive signature on the artist’s output and potentially opening up new creative avenues. In a sense, entire artistic movements can be seen as extended dialogues between artists and the particular materials available to them at the time. Paul Klee wrote of “taking a line for a walk”, as if the line itself were straining at the leash. He describes the creative process as a kind of loop: “Already at the very beginning of the productive act, shortly after the initial motion to create, occurs the first counter motion, the initial movement of receptivity. This means: the creator controls whether what he has produced so far is good.”

This interaction is the crux of creativity: each new move or mark you make is a direct response to what you have produced so far. You are engaged in a continuous feedback loop with the work itself, and the materials with which you are creating it. If creativity aspires to originality, why try to simplify this feedback loop? In treating our tools as mere devices to realise a static vision, we miss an opportunity to make the process ever more baroque, tangled and productive.

The move towards artificially complicating the creative process is primarily a modern phenomenon. Influenced by the egoless practices of eastern philosophy, John Cage and his Black Mountain colleagues included chance processes in their compositions, letting external forces such as the I Ching determine harmony, rhythm and instrumentation. The artists of the Fluxus movement likewise left elements of their works open to chance, or delegated the decision-making to performers or audiences. Clearly, art did not require an individual artist to play a privileged role; it could incorporate agencies from the world around it.

At the dawn of the 1960s, the Huddersfield-born philosophy lecturer Desmond Paul Henry came across a Sperry mechanical bombsight computer – used in WWII aircraft to calculate missile trajectories – in an army surplus store. Henry, himself a veteran of the Normandy landings, was intrigued by the movements of the machine’s motors and pneumatic parts. He began to explore using the device as part of his artistic practice. Within weeks, he had successfully converted it into a high-tech drawing machine.

The results were breathtaking. Rigged with a drawing apparatus and ball-point pen, the mechanism swept across the page to create swooping geometric vistas. At first, Henry would gently intervene, disturbing the device’s operations to find new aesthetic effects. Over time, though, he made fewer and fewer interventions, until the machine was creating its artworks virtually independently. During these periods of productivity, Henry was also modifying the machine itself. By tinkering with the mechanisms – tightening screws, replacing parts – he could affect the scope of its behaviour. Elements of randomness could be amplified or minimised, altering the range of results it could produce. In effect, Henry was changing the machine’s artistic style.

The resulting drawings offer one of the first computer-aided examples of “generative” artwork: art made predominantly through the use of an external system. Henry was aware that his work existed in a lineage, with predecessors including the harmonograph, a scientific instrument that, through the cyclical motions of coupled pendulums, creates undulating networks of lines on the page – known in mathematics as “non-linearity”. But there is a crucial difference. With Henry’s machine, the imprecise mechanical elements underlying the system – gears, belts, differentials and motors – could interact to produce chaotic, unpredictable movement. A small change in the initial conditions could make for vastly different results, with varying levels of detail at different spatial scales. This is what gives the images their richness, significantly closer to those produced by the imperfect hand of man than by the geometrically pristine harmonograph.

Since the ’60s, scientific interest in non-linearity has increased exponentially, in line with the technological advances that have allowed us to examine and model its effects. After the meteorologist Edward Lorenz discovered the butterfly effect, in which a small change in weather conditions can have unpredictable, far-reaching consequences, the study of chaotic behaviour spread through countless fields, from physics and biology to economics, sociology and linguistics. Like the coupled parts of D.P. Henry’s bombsight, we see a biological organism or a stock market as a network of interdependent elements, each of whose effects can ripple through and alter the entire system in unpredictable ways.

Alongside the theoretical study of non-linear processes, many researchers have begun engaging with them as a way to produce something new. Most ambitious are the artists and scientists who work with artifical life, creating life-like structures in software or hardware. At the inaugural artificial life conference in 1986, Christopher Langton described its objective as not merely to understand existing, organic life, but to go beyond it – to create life “as it could be”.

Attracted by the results of this work, a new generation of cross-disciplinary artists has begun to draw on it. In Canberra, Jonathan McCabe constructs software models of biological phenomena to make artworks of kaleidoscopic complexity. Nuanced and painterly, his work begins from a simple, abstract simulation of a particular natural process, assigns its behavioural properties to visual elements, then gradually develops and iterates upon the system as it becomes aesthetically richer.

McCabe, as per Langton’s mantra, does not just faithfully reproduce patterns from nature, instead fusing approaches from diverse areas of biology, chemistry and biophysics. Where the existing models fall short, he simply invents his own. It is as if he has a boundless natural laboratory on his computer screen.

One series begins from a growth model devised by Alan Turing, which imitates the chemical reactions that produce the distinctive patterns found in the coats of mammals and the scales of fish. By introducing multiple interacting processes at different scales, McCabe creates a software ecosystem, in which different elements compete for visual space: aggregate flows of coloured forms collide and coalesce, creating a continually shifting equilibrium. McCabe describes his own role as tantamount to that of an explorer “hunting around”. He can alter the underlying code, and bend and relax the system’s natural laws, forging a vast imaginary space of possibilities.

McCabe’s work is fascinating in its resonance with the living forms of cells and organisms. It seems to embody what the Chilean biologists Maturana and Varela describe as “autopoiesis”: the ability of a system to dynamically regenerate its own elements, continually creating itself from within. Equally fascinating is the relationship between the work and McCabe himself. These software systems are developed through an iterative process of growth and reflection between artist and tool. Serendipitous encounters in this “parameter space” can lead to the discovery of radically new aesthetic forms. Accident becomes a central part of the creative process, just as genetic mutations are vital to evolution in the natural world.

In contrast to the stereotypical western mode of detached, theoretical creative practice, McCabe’s process is a thoroughly embodied one – an experimental dialogue between him and the constellation of forces and algorithms that constitute his models. He says: “I imagine that the creative system is composed of myself and the computer running the algorithm, each part having a complementary role as part of a ‘cyborg artist’. There is a flow of information both ways, between the human part and the machine part, rather than a unidirectional flow from the artist to the tool to the work.”

This could not be further from the conception of tools as a transparent conduit for the artist’s will. The combined “cyborg” identity of artist plus tool is far more than the sum of its parts; the result reflects the exploratory interactions between the two. This is an extreme extension of the idea of creativity as a feedback loop. Each change McCabe makes to the code is like a new mark on paper. But whereas the outcome of a single pencil stroke is relatively predictable, the complexity of McCabe’s system means any small change can create an endless cascade of effects.

Of course, McCabe’s relationship with the algorithmic systems is still asymmetrical, with the computer doing the grunt work and the human artist supplying most of the artistic judgement: “A lot of the work of the human part of the system is filtering out the gems from the dross.” Yet you could describe McCabe as entering into a network of agency, in which his internal creative impulses are supplemented by numerous other forces: the constitutive elements of the mechanism; the parameters that govern its behaviour; the visual mappings that render it onscreen. Perhaps there is no single source of agency here at all, but something closer to what the French philosophers Latour and Callon describe in their Actor-Network Theory. Neither human nor technological agency plays a primary role. Instead, they are entwined in a mesh of causality: our tools shape us while we, cyclically and collectively, shape them.

The philosophies behind these autonomous creative systems vary wildly. Some artists try to transmit their own style and judgement directly into their machines. The French artist Patrick Tresset, after training in computing, spent many years drawing and painting portraits. At some point, though, he felt his ability to express feeling in his drawings begin to fade. “After that,” he says, “I started working on systems that could draw autonomously. Initially I was trying to develop a kind of prosthetic.”

Tresset devised and engineered a robotic system that could produce its own drawings, using a mechanical arm and ball-point pen to painstakingly plot an image on paper. Unlike McCabe’s and Henry’s abstract forms, these images are entirely figurative, representing human faces. Tresset’s robot – christened Paul, in its current incarnation – uses a camera to observe its subject, then roughly reproduces what it “sees”. It gradually draws shapes and shading line by line; like a human artist, Paul’s observations and marks are both imperfect, with what Tresset describes as a “clumsy” tendency. Critically, Paul continues to observe its own output, reflecting on each section of the drawing as it begins to take shape. You can watch the drawing develop over a period of roughly 30 minutes, seeing your face appear, filtered through Paul’s perceptions.

Tresset has produced several generations of robots, each with new qualities; one version, nicknamed Peter, would doodle when left alone. Having built a robot, however, Tresset intervenes very little in its creative process. Each drawing is performed autonomously. Tresset’s own artistic input takes place in advance, when he designs specific ways for the robot to perceive and draw, giving it its own stylistic signature. Much of the attraction for Tresset is in the huge range of forms made possible by an open system. Though Paul is modelled on Tresset’s own hand, it is able to produce the unexpected continually: “More than 3,000 sketches later I am still surprised; I can never predict what the drawing is going to look like.”

Who is really the author of these drawings? Paul’s physicality helps confound matters. “Due to embodiment in a robot,” Tresset says, “the system is perceived as having its own agency. In a certain way, I feel this, even if I know it is not true. I could say that I am the author of the system, but the system is the drawing’s author.”

And there is another part to the picture. The artistry of Paul lies not only in the end result, but in the interaction between subject and machine. As Tresset says: “Although getting Paul to draw better or be more autonomous is an important part of my practice, what I exhibit are installations, with robots which are interesting not only for the drawings they produce, but also for what happens between the robot and the sitter, and how this is perceived by the audience.”

It is hard not to ascribe artistic intent as you watch Paul’s robotic hand develop a human likeness on paper. Yet the point is that, regardless of what goes on inside it, Paul itself is very much part of a larger creative process. Tresset’s work is precisely in the interactive theatre of the event itself: the clumsy, hesitant robot adjusting its handiwork; the people whose portraits are being drawn, who are gradually forced to surrender their scepticism as they watch a machine carry out some of the behaviours that supposedly define us as quintessentially human.

It is evident that new technology, from Henry’s mechanisms to McCabe’s digital ecosystems, has blurred the lines of artistic practice. Today, we can produce tools capable of highly autonomous creative acts, with all the appearance of independent thought. Yet the closer we look, the more we realise that the distinction between a tool and an autonomous agent has always been simply one of degree. Even art made by a lone individual is essentially collaborative, with every tool leaving its own stamp on the creative process. Just like so many others, Paul Klee surrounded himself with a constellation of home-made brushes and drawing implements, whose unique properties were fundamental to his development, challenging him and pushing his work in new directions he could not have foreseen. §

Daniel Jones’ latest work, with James Bulley, is Living Symphonies, a site-specific sound installation that grows and adapts in the same way as a forest ecosystem. It will tour four UK forests this summer.

Patrick Tresset will exhibit his new generation of robots as part of Wanted, a show organised by Art Kapsule at Koleksiyon in Clerkenwell, until 25 April.

  • Art and Algorithm
  • Art and Algorithm #1
  • Art and Algorithm #2
  • Art and Algorithm