3. The Art of Breeding Art

This book is about questioning the value of objectives. But how do you begin to question something as basic and common as having an objective? Before the recent ideas that led to writing this book, it’s not as if your authors had spent our lives in the thriving anti-objective protest movement (there is no such movement, thriving or otherwise). We just went about our business setting objectives and following them as happily as everyone else. In the meantime, we were researchers in artificial intelligence, looking for ways to make smarter machines. How we went from there to writing a book against objectives is a strange story that helps to explain the origin of the equally strange idea that objectives might often cause more harm than good.

The story of this book really began when our research group decided to build a website called Picbreeder that was also a unique kind of scientific experiment. At first, the idea behind Picbreeder had little clear connection to objectives. In fact, it was originally conceived as a place where visitors could literally breed pictures. While that may not make sense when you first hear it, it’s actually a simple idea. The plan was to make it work roughly like animal breeding: Pictures on the site would be able to have “children” that are slightly different from their parents (just as animals have children that are unique but still clearly related to their parents). The hope was that by allowing visitors to breed the pictures they find most interesting, over time they would end up breeding works of art that please them, even if the visitors themselves were not artists.

Of course, at first glance, a website for breeding art sounds bizarre. How can art be bred? A Picasso painting can hardly seduce a Van Gogh. But actually there is a way it can make sense. The key to understanding Picbreeder is that when real animals are bred together, their genes are combined to form their offspring. It turns out that scientists in artificial intelligence have found a way to create an artificial “DNA” for pictures stored inside computers. The result is that you can breed the genes of these pictures together just like those of animals. This technology, first introduced by Richard Dawkins in his book The Blind Watchmaker [35], is sometimes called genetic art. Since Dawkins’ original demonstration of the idea scientists have significantly enhanced its capabilities, which is part of what inspired us to make a website where people all around the world can play with it.

To understand genetic art it helps to think of animal breeding. Imagine that you have a stable of horses. If you’re the breeder, then you can decide which stallions and mares will mate, and eleven months later there will be a new generation of fresh-maned newborns. The important thing is how you choose the parents. For example, if you wanted fast horses, your strategy might be to choose two fast parents to mate. Of course, you don’t have to choose parents only for practical reasons. Maybe you just want the prettiest horses to mate, or the silliest. Whatever the reason, by choosing the parents you influence the children’s genes. They naturally end up a mix of the parents’ genes. In the next generation, when all the children you bred grow up, the process can be repeated. And some of the children might end up even faster or sillier than their parents. Over many generations, the animals evolve in a way that reflects the choices of their breeder.

Genetic art programs work in much the same way as breeding horses, except that instead of choosing animals to breed, you choose pictures. What happens is that you see a set of pictures on the screen (perhaps there could be 10 or 20 pictures displayed together at the same time). Then you click on the ones you like, which become the parents of the next generation of pictures. For example, if most of the pictures look circular but you click on the one picture that looks more square, then the next generation will likely contain many square-like images (Fig. 3.1). In other words, square parents make square babies, just as your children might have eyes that look like yours. But like in nature, the offspring don’t look exactly like their parents. There are slight mutations hidden in their genes, though of course you can still see the resemblance.

If you continue to play this kind of game over and over again, clicking on images you like so that they reproduce to form new images, over many generations the images will evolve in a way the reflects your choices, just as with the horses. Playing with genetic art can be fun because it allows you to explore a lot of possibilities you might never have imagined on your own.

*                *                *

In 2006, we had developed a new kind of artificial picture DNA that produced more rich, meaningful images (which you will soon see) than were possible before. But more importantly, this new project, which would become Picbreeder, included another ingredient that made it particularly interesting: Any internet user could continue breeding images that were bred by previous users. This feature was critical to Picbreeder because a big obstacle for systems of this type is that their users can only stand to play for a little while at a time before becoming mentally tired [36]. After all, how long could you tolerate staring at a screen full of pictures in one sitting? It turns out that after 20 generations or so (i.e. after choosing parents 20 times in a row), most people simply can’t continue concentrating. But evolution works best over many generations, and 20 isn’t enough to produce really interesting pictures.

Jimmy Secretan, then a Ph.D. student attending our research group meetings, suggested a clever solution to this problem: Make Picbreeder into an online service. That way, users could share images they had previously evolved with other users, who could then continue breeding them. In other words, if you evolved a triangle on Picbreeder, you could publish it to the website, and then someone else could continue breeding it and perhaps discover an airplane. In Picbreeder, this kind of handoff from one user to another is called branching. The great thing about branching is that it allows breeding to continue far beyond the 20-generation limit. Tired users can continually hand their current product to a fresh new user to add yet another 20 generations to its lineage. Eventually, these linked chains of handoffs can build up to hundreds of generations of evolution.

But for those who don’t want to branch from any previous image, the alternative is to start breeding from scratch, which is how every discovery on the site ultimately began. Starting from scratch randomly constructs artificial DNA (from scratch), which ends up producing a bunch of simple random blobs on your screen. From among those blobs, you can pick a parent blob for the next generation of pictures. Figure 3.2 shows what it looks like for a user to start from scratch. You can see that the user evolved a curvy blob into a more circular form with a mouth-like shape inside it. Perhaps interesting, but not an earth-shattering discovery.

But how impressive do you think these pictures could become if you kept breeding them in this way, picking parents generation after generation? It turns out that they become more remarkable than you might expect. Believe it or not, every image in Fig. 3.3 was bred in this way on Picbreeder, all originally going back to random blobs. What’s more, the breeders of these images weren’t trained artists. They were just curious people who had found the Picbreeder website. In fact, likely few (if any) of them could draw the pictures that they had bred.

Seeing visitors breeding these kinds of images piqued our curiosity. Even with the new artificial DNA, we hadn’t realized how lifelike and meaningful the images would become. In a sense, each one of the images in Fig. 3.3 is a unique discovery. It’s important also to recall that Picbreeder users often continue breeding from each other’s discoveries. For example, the skull-like image in the middle of the second row in Fig. 3.3 was evolved in this way. It’s actually the result of two users branching off of each other’s published images five times in total (for a total of 74 combined generations since the initial random-blob ancestor). So even if the final user who evolved an impressive image didn’t start from scratch, one of her predecessors must have, which means that in the end everything traces back to random blobs. That gives you a sense of how unlikely these discoveries are.

Now here’s where the story gets interesting. Say, for example, that you’re in the mood to evolve a picture of the Eiffel Tower. One thing you might think is that if you visit Picbreeder and just keep choosing images to breed that look increasingly like your objective (the Eiffel Tower), eventually you’d find it. But interestingly, it doesn’t actually work that way. It turns out that it’s a bad idea to set out with the goal of evolving a specific image. In fact, once you find an image on Picbreeder, it’s often not even possible to evolve the same image again from scratch—even though we know it can be discovered!

We confirmed this paradoxical aspect of the system by running a powerful computer program for thousands of generations. First, we chose a target image from those that users have discovered and published on the site. Then, at every generation the program automatically chose parent pictures that look increasingly similar to the target image [37]. The result for the most interesting images—total failure. It’s impossible to breed an image if it’s set as an objective. The only time these images are being discovered is when they are not the objective. The users who find these images are invariably those who were not looking for them.

We can give you a specific example because one particularly surprising image—the Car on the left side of the third row in Fig. 3.3—was discovered by Ken. So we know the precise story of how it was found. The most important fact: Ken was not trying to breed a car. Instead, he had actually chosen to branch from an Alien Face image (Fig. 3.4) that had been bred by a previous user. Instead of thinking about cars, Ken originally intended to breed more alien faces. But what happened next—what always happens before a major discovery on Picbreeder—was serendipity: Random mutations caused the eyes of the Alien Face to descend gradually over several generations of breeding and suddenly it was apparent that the eyes could be wheels (Fig. 3.5)! Who would have thought that an alien face can lead to a car? But it turned out to be just the right stepping stone.

That story would be merely an interesting anecdote if not for the fact that almost every attractive image on the site follows the same story of serendipity. There is always a surprise stepping stone leading to an unexpected discovery. For example, look at all the bizarre stepping stones in Fig. 3.6. As we began to notice this trend, it was hard to avoid its strange and surprising moral: If you want to find a meaningful image on Picbreeder, you’re better off if it is not your objective.

In fact, the only reason these discoveries are being made at all is because users are unwittingly laying stepping stones for each other every time they publish a new picture on the site. The user who evolved the Alien Face never imagined it might one day lead to a car, and neither did the user (Ken) who eventually evolved the Car. No one saw the Car coming. So it’s a good thing that someone did evolve the Alien Face and publish it, because otherwise there would never have been a car (and perhaps no book!). The system works as a whole because it has no unified objective—everyone is following their own instincts. And the most successful users of all are those with open minds, who avoid looking for only one thing in particular.

In other words, the most successful users have no objective. When this fact emerged from studying the discoveries made on the site, it was completely unexpected. You would think that the best breeders would be the ones who conceived an objective image (i.e. something they want to evolve) and then bred towards it, but it turns out to be the opposite—the best discoveries on Picbreeder are always the ones that are unplanned. And this initial observation turned out to apply to much more than just pictures.

After all, why should Picbreeder be different than anything else in life? There’s something you want to create or to achieve, so you start working on finding the stepping stones that lead you there. But how can you be sure that the stepping stones actually look anything like your ultimate objective? What if they are instead like the Alien Face, full of potential, but entirely unlike where they ultimately could lead (like the Car)? In that case, if you concentrate too much on achieving your objective, you’ll end up ignoring the most critical steps to reaching it. Could it really be, we wondered, that this principle, first observed on an obscure picture-breeding website, actually impacts every aspect of life concerned with achieving objectives? If it does, then it must be important, because objectives are everywhere. And as you saw from all the examples in the last chapter, the same story seems to emerge in many facets of life.

But no matter how many stories we show you with this kind of plot, it still doesn’t fully answer why the world works this way. That’s what the next chapter is about. Yes, abandoning objectives is often the best decision, but there’s a reason for this pattern, which is that the stepping stones almost never resemble the final destination, whether planned or not. In other words, no matter how tempting it is to believe in it, the distant objective cannot guide you to itself—it is the ultimate false compass.