What is the maximum rate of human progress?

Imagine you travelled back in time to the stone age with nearly all of modern science and technology memorized. Let's also stipulate that you can communicate perfectly with the people living back then, and they won't kill you on sight. How quickly could you push human civilization forward to reach the same level of material progress as the world you came from?

It seems obvious the answer is "pretty quickly," assuming you aren't immediately killed by a wild animal or utterly fail to impart any useful knowledge.

But how quickly?

Let's think of it another way. A time traveler from the year 4,000 comes back in time to visit you, right now, today, and brings all the knowledge of the future back with them. Armed with this cheat-sheet, how quickly could you manifest the next 2,000 years of material progress?

Again, "pretty quickly" seems like the answer, but exactly how quickly continues to elude us.

If you consider the first scenario, it's not clear that you could, for instance, jump start something like the industrial revolution merely by giving the right facts to a bronze-age or iron-age emperor. And this is even if they believe your technology works – there's good arguments that the real reason the industrial revolution didn't happen sooner is down to contingent civilizational constrains:

to your dismay, the emperor responds: “Your mechanism is no gift to us. It is tremendously complicated; it would take my best master craftsmen years to assemble. It is made of iron, which could be better used for weapons and armor. And even if we built these engines, they would consume enormous amounts of fuel, which we need for smelting, cooking, and heating. All for what? Merely to save labor. Our empire has plenty of labor; I personally own many slaves. Why waste precious iron and fuel in order to lighten the load of a slave? You are a fool!”

Consider also that progress doesn't move in a straight line. Humanity last visited the moon in 1972, and after the space shuttle program was retired in 2011, the United States no longer had the capacity to send Astronauts into space.

Fast forward to today, and SpaceX continues to wow the world with the latest successful flight of Starship. Studying the history of that company, however, makes us realize that this hard-won progress was also highly contingent–what if SpaceX had never gotten off the ground? How would we access even the aging space station today if we were still entirely dependent on a now-hostile Russia for our rocket launches?

On the other hand, the history of science is also littered with wild coincidences leading to sudden discoveries resulting in durable progress. Two of the most famous stories are penicillin being discovered on moldy bread and the secret of cooking food with microwaves stemming from a melted candy bar in an researcher's pocket; but did you know that pretty much every major synthetic sweetener was discovered by researchers accidentally tasting chemicals?

It is therefore a fact of the world that virtually all the popular synthetic sweeteners were discovered accidentally by chemists randomly eating their research topic.^[1]

I think this is a suspiciously high amount of serendipity. I see two options:

Super-sweet molecules like aspartame are commonplace – there are plenty of molecules hundreds of times sweeter than sucrose, but we only know the few that were ingested by accident,

Super-sweet molecules are very rare, it’s just that chemists accidentally taste a lot of chemicals. Entire chemistry departments routinely taste the entire space of possible molecules, but they don’t notice unless the molecule has a strong taste.

Finally, consider gunpowder. Supposedly Taoist alchemists accidentally discovered it while they were trying to develop an elixir of life, and we all know how profoundly their discovery changed the course of history.

We have two opposing narratives here:

1. Material progress is hard won, highly contingent, and fragile.

and:

2. There's trillion dollar bills just lying on the ground.

Let's think of this another another way – a lot of AI researchers are all agog about the idea that we're about to hit a technological singularity; once we automate AI research itself, AIs will be able to self-improve themselves and we'll get a gigantic burst of technological progress almost instantly; a century's worth of progress in weeks, days, or maybe even hours.

I'm too dumb to make sense of most of the arguments these people put forth, but here's something my smooth little brain can handle: what do different kinds of material and scientific progress require?

On one end of the spectrum, there's math – proving a new theorem or something. In theory all this takes is pure thought – it's not dependent on a bunch of highly specific coincidences all lining up just right to unlock the magic. You just need a smart enough person to work out the puzzle. In this world all it takes to advance human progress is figure out how to make more Terrence Taos.

In the middle end of the spectrum, you have something like chemistry. Discovering a new useful molecule or law of chemistry requires you to be smart but also requires you to directly interact with the physical world running experiments and doing tests and procedures. Each of these procedures has a cost, both in terms of time, as well as physical materials. So it's not enough to be smart, because your progress is limited by how quickly you can do experiments. For a particularly salient example, consider the story of the man who invented the blue LED:

On the far end of the spectrum you have stuff like biology and medicine. Here you not only have to constantly interact with the real world to learn new things, but also this is where the real world starts getting particularly uncooperative. The systems you're working with are incredibly complex and literally alive, and it's difficult to contrive perfectly controlled experiments that let you isolate cause and effect. Furthermore, many of the questions you'd like to answer like, "does this drug increase the risk of cancer in humans" come with ethical barriers and time barriers – what if it takes twenty years to get a sufficiently clear answer? What if the only experiment that could get a definitive answer basically amounts to torture, and therefore you can never run it?

The biggest limiting factor to human material progress is how many other things the next big step forward depends on besides a lot of very smart people thinking really hard about the problem.

Therefore, if you want lots of progress, and you want it sooner rather than later, our marginal focus should be on reducing the time and cost of making the next guess.

There's no better example of this than the contrast between SpaceX and Boeing. Besides being gradually hollowed out following a merger with former rival McDonnell Douglas, Boeing typifies the conventional "cost plus" government contractor. The government agrees to cover all their development expenses plus a fixed percentage of profit up top, which gives the contractor no incentive to reduce costs – the opposite, if anything. SpaceX on the other hand focuses relentlessly on decreasing costs, and subjects themselves to real world experimentation as often as possible. SpaceX makes a lot more guesses much more often than Boeing, and the gains rapidly compound.

This has personal applications too, reflecting on my ~14 year career as a game developer. After having a modest hit early on, I got bogged down in the typical indie death spiral of a 10-year mega project (that should thankfully be shipping very soon, my former colleagues tell me). I had many other accomplishments behind the scenes I'll write about one day, but I'll always regret how bogged down I let myself get on our "main game" project. This was an extremely expensive single guess both in time and money, and I would have benefited from learning its lessons faster and more cheaply.

I contrast this with my experience in my new career doing real estate mass appraisal for local governments. We decided early on that my colleague Will Jarvis should be CEO and I would focus on research and product development. I imagined that we'd spend the first six months quietly building a killer product, and only after we had something to show would we approach clients with it. Will had other ideas–like throwing me into the deep end. We immediately started doing small pilot studies with real clients on small dollar contracts, in order to learn what the problem even was. After a few false starts it became very clear what we actually needed to build, and we soon ramped up to full-scale projects. If we'd followed my approach instead, we would have gone down a complete dead end because the final product customers showed us they wanted was completely different from the one I originally envisioned.

Another thing that surprised me was how quickly I learned the new trade. I owe this to two things – first, expert mentorship from veteran appraiser Paul Bidanset (our first hire), and second, the short durations of our shipping cycle. Get a client, process the data, build the model, get feedback. Then do it again, and again, and again. Before I knew it, I had modelled dozens of jurisdictions. There's still a lot to learn and endless horizons to conquer, but I've acquired more skills and training in the last twelve months than I have in the last five years.

It all comes down to just putting in the reps, day in and day out. And then one day you look at yourself in the mirror and suddenly you have biceps. And you're like, where did those come from? The same place progress does.