Rating: 7.7/10.
Book by AI researchers that questions the need for objectives in scientific discovery and inventions – having objectives is reasonable for many things in life but objectives are harmful for highly ambitious goals where the path of getting there is unknown. Many examples of things that were originally trying to be something else, like YouTube trying to be a video dating site or John Grisham working in law before becoming an author, but these past attempts were useful for the eventually successful pivot.
Much of the book refers to one of the author’s experimental site called PicBreeder, where users would create versions of a picture to kind of evolve randomly, producing impressive results. But it only works if they’re not trying to produce a specific image; users create and share results that they find interesting, which makes it possible for other users to take these results and use them as a starting point for their creations. This is applied to inventions and discoveries in the real world through the concept of “stepping stones” – interesting intermediate results. Many inventions combine the ingenuity of several stepping stones, where each stepping stone is not clearly related to the final product. Eg, vacuum tubes were a prerequisite for computers, or engines were needed for airplanes.
Someone in Ancient Egypt trying to build a computer would have no idea vacuum tubes would be useful, and when nobody knows which direction is even closer to the objective, the best way is to just explore randomly in lots of different directions and try to find interesting things. This is called novelty search – doing something that hasn’t been done before, and tends to produce behaviors that get increasingly complex and avoids falling into local minima, because in order to do something that you haven’t seen before, you have to avoid trivial failure modes. In a maze simulation, the novelty search approach was better than objective algorithms that would repeatedly run into the same dead ends.
Science often produce work that looks kind of stupid or useless, but this is actually ideal because the goal should be to find interesting stepping stones that other experts can use, it would be counterproductive to require scientists justify to a public audience, since it would be narrowly constraining to approaches that are obviously related to the goal. Great inventions don’t come from pursuing a vague and grand vision – they start from what’s already possible, and then they take a leap to the next step. We should be judging stepping stones by how interesting they are and whether they lead to more stepping stones, not by how close they get to any objective.
The book ends on two further case studies of this idea. First, evolution: we usually think of evolution as a process that produces the “fittest” organism in some objective sense, but it’s unclear what this means because bacteria can reproduce more efficiently than humans. Instead, they argue that we should think of evolution as random novelty search with the constraint of being able to survive and reproduce as a filter that limits the diversity of possible organisms.
Second case study on AI research – this can also be thought of as researchers searching for the best search algorithm. AI research uses two main heuristics: (1) new algorithms should beat the previous ones on benchmarks, and (2) it should have theoretical guarantees that it works well. The peer review process acts as a gatekeeper that ensures that these two heuristics are met. The issue is that a method might be a novel stepping stone but be slightly worse than the previously best method by a little bit and fail to pass these heuristics, while less interesting but incremental work might pass.
Overall this book presents an interesting idea: valuing novelty over any objective function. However, it is quite one-dimensional in that it continuously repeats this one idea, with multiple references to the Picbreeder paper, which was an interesting experiment but arguably a stretch to explain all human invention. In practice, it ignores realistic constraints, like when there are limits on resources, researchers, and compute – everyone doing whatever they want is absurd and chaos, but the authors make no attempt to address this, or even mention any of the many examples of science that were done in pursuit of a clear goal like AlphaGo or the Apollo space program just to name a few. A better takeaway of the book’s premise should probably be that solely optimizing for an objective is harmful, and an organization should reward novelty in addition to objective improvements.
