Penicillin: a case study in meta-rationality
You were told a heart-warming fairy tale. The truth is much more interesting.
Hello! This post is a new draft section of my meta-rationality book. Readers of previous chunks found them quite abstract, and wanted concrete examples. You were right! Thank you! This section is one long example, illustrating many aspects of meta-rationality.
If you run into any unfamiliar terms, consult the Glossary. I am adding to it as I write draft sections, and it should cover all those used in this post.
This is a partly-paid post. Roughly the first half is free, and there’s a paywall in the middle.
The post is long enough that some email programs cut it off part way through. You may need to click on “View entire message,” “Read in app,” or on the headline to read the whole thing.
The greatest scientific discovery in the history of medicine
There is a wonderful story I would like to use as an illustration of meta-rationality. It is the story of the discovery of penicillin by Alexander Fleming in 1928. I heard it in science class when I was a kid, and have read it again many times since. The version you know may be a little different, but the one I learned went about like this:
Once upon a time, the scientist Alexander Fleming was about to wash out a petri dish—and suddenly had a flash of insight! He had been growing bacteria on the dish, but a spore of a mold called Penicillium had blown in through the window and fallen into it. He noticed that there was an empty ring around where the mold had grown up. All the bacteria near the mold had died. That wasn’t interesting; it happens all the time when you grow bacteria to use in an experiment. Every other scientist before him just threw away the contaminated lot, washed out the dish, and moved on.
But it occurred to Fleming that bacteria cause disease. It’s a hassle when molds kill your bacteria and mess up your experiments, but we want to kill bacteria. That seems obvious now, but no one had considered mold as a potential antibacterial therapy before! Countless previous scientists had missed the point. He stopped himself just before throwing away the mold, and discovered that it produces the drug penicillin.
This was the first antibiotic, which revolutionized medicine and saved hundreds of millions of lives; so he got the Nobel Prize, and is considered one of the greatest scientists in history.
I said I would like to use this story as an illustration of meta-rationality. There are three reasons I can’t.
Almost every part of the story is factually false.
It is the opposite of an illustration of meta-rationality. Instead, it is an illustration of anti-rational Romanticism. It’s a heart-warming fairy tale, endlessly retold, for exactly that reason.
Therefore, it provides an attractive but misleading idea of how science works.
This section begins by explaining why the story is misleading.
The rest tells the story of how penicillin really was discovered, mostly not by Fleming. It’s much more interesting, and it does illustrate meta-rationality! It’s long because meta-rationality characteristically bridges between masses of technical specifics and big-picture considerations of purpose, context, and the coordination of multiple rational systems. The true story unavoidably combines military history, the politics of scientific institutions, the peculiar behaviors of nebulous glop, and automated chemical process optimization.
An anti-rational Romantic fairy tale
Actually-existing science has many unattractive aspects:
Science is a ton of boring facts about mold you are supposed to memorize.
Scientists think they are special and better than you because they are so smart.
Science usually involves math, which makes your head hurt.
Actually doing science, running experiments, is an immense amount of tiresome work, usually with little reward.
Scientists think they can understand everything rationally. That’s crazy and stupid.
The real way you discover new science is by vibing with the cosmos, not by doing a bunch of dead calculations and meaningless experiments.
Each of these points is somewhat true (and somewhat false). We want the benefits of science, but most of us would like to skip the boring and annoying bits. So, what if we decided we just can?
The way Fleming discovered penicillin is proof that great science, as opposed to the boring stuff scientists do, only takes common sense. Obviously we want to kill bacteria because they cause disease. Fleming could just see that the mold kills bacteria. So obviously it could be a drug.
Anyone could see this who wasn’t blinded by boring stuck-up scientific orthodoxy. Learning all that junk in school makes you stupid, actually. What you need to be a great scientist is not any of that, it’s vibratory awareness. If you have the right spiritual headspace, you are connected to the Entire Universe. That powers the creative intuition that reveals amazing paradigm-breaking theories of holistic Oneness, like quantum healing. That’s real scientific discovery! It’s Cosmic Consciousness that makes you Special, not tedious rationalism.
This is what makes the Fleming story so appealing, I think. It flatters your sense that you are Special, just for keeping an open mind and rejecting boring conventional rationality. It’s democratic in a way science isn’t: anyone could have done what Fleming did, and saved hundreds of millions of lives.
This Romantic misunderstanding is not entirely false. It’s false enough, when believed by a large fraction of the population, to severely obstruct science, and human progress more broadly.
What Fleming actually did (and didn’t do)
Learning this is valuable to understand the state of the biological knowledge when Fleming started his penicillin research, and after he gave up on it, so other scientists picked up the ball.
Penicillin was not the first antibiotic. The first was Salvarsan, a synthetic chemical discovered in 1907 and commercialized as a drug in 1911—two decades before Fleming’s work on penicillin. It was the first reasonably effective treatment for syphilis, which is horrific in its late stages and often fatal.
Fleming was a doctor in private practice before he was an academic researcher. He specialized in Salvarsan treatment for syphilis. Unfortunately, Salvarsan was extremely toxic; sometimes immediately lethal to the patient.
This is the fundamental problem in developing antibiotics. It’s easy to find chemicals that kill bacteria; the problem is that nearly all of them also kill people.
During the First World War, Fleming served as a British army doctor in field hospitals in France. Trench warfare conditions ensured that battle wounds frequently were infected with bacteria. The ensuing diseases were often horrific and fatal. More soldiers died of infections than from direct injury from bullets and shrapnel.
Standard procedure in WWI was to apply the chemical phenol to battle wounds. Phenol kills bacteria. It also destroys human tissue. Fleming found that not only did phenol do great damage, it didn’t prevent infection, because it didn’t reach all the bacteria in the wound. It was worse than useless. In 1917, he published a paper explaining this. Unfortunately, it was ignored.
In 1922, Fleming discovered the antibiotic enzyme lysozyme. Unlike Salvarsan, it is a naturally-occurring biological substance, not toxic, and kills a wide variety of bacteria. Unfortunately, none of those are species that cause human diseases, so it was scientifically interesting but a dead end medically. Fleming continued working on lysozyme anyway, and was still working on it when he did his penicillin experiments. It seems likely he initially thought that penicillin was a lysozyme variant.
In the 1800s, decades before Fleming’s “discovery,” many scientists observed that Penicillium mold inhibits the growth of bacteria, including several species that cause human diseases; and some confirmed this experimentally.1 Several suggested that whatever the mold was doing might lead to an effective anti-bacterial therapy, and in one case apparently also showed this experimentally in guinea pigs. So, Fleming’s discovery, if he made one, was certainly not that Penicillium kills bacteria, nor the insight that it might make an antibiotic effective for human diseases.
We don’t know what Fleming actually did during the period in which the supposed moment of discovery occurred. His lab notebook pages from that period are missing. One historical analysis speculates that, when he was becoming famous, he removed and destroyed them because the mundane facts contradicted the engaging story he told.2 However, the story as he told it is factually impossible, both for technical biological reasons and because his timeline is inconsistent with records of where he was around the time he said he made the discovery.3
It seems likely that Fleming was actively looking for better antibiotics (given his long-standing interest in them). If it’s true that he noticed the mold’s activity by accident, there wasn’t a blinding flash of insight “figuring out” that it could be an antibiotic; it was “oh good, yes, this looks like it’s another one.” But perhaps he had read about earlier scientists’ recognition of Penicillium’s antibacterial activity. Or perhaps he heard about it from the mold expert Charles John Patrick La Touche, who was growing Penicillium one floor below in the same building, and who said in 1968 that he’d given Fleming some.
What did Fleming actually do that was new? Two things.
Most importantly—for his reputation—he invented the word “penicillin.” However, he meant something quite different by it than what it came to mean. He grew Penicillium mold on the surface of a liquid “broth” of nutrients, then strained the mold out, and found that the remaining glop had the antibiotic property without the mold. He explicitly used the word “penicillin” only to refer to the glop.
He termed whatever thing in the glop did the work “an antibacterial substance,” which he didn’t name and couldn’t identify. We now know it was what we now call “penicillin.”
Second, Fleming did the first toxicity testing. That’s a critical step toward a drug, and one that would have been top-of-mind given his experiences with Salvarsan and phenol. He injected one mouse and one rabbit with the glop, and neither had significant ill effects. He also bathed human wounds and eyes with Penicillium glop, again with no bad effects. This was enormously less toxicity testing than is required to have confidence in a drug, but he was first to get that started.
Fleming mostly gave up on penicillin after three years’ work trying and failing to figure out what the “antibacterial substance” was. Another thing he didn’t do was test whether it could treat an infection in an animal or human.
Penicillin, in any case, wasn’t a promising drug candidate. It was very difficult to produce more than tiny quantities, so getting enough to do any experiment took a ton of work. It was also chemically unstable, and quickly lost its antibacterial activity.
Development of a new category of antibiotics, sulfa drugs, began before Fleming’s 1928 “discovery.” The first successful one, Prontosil, was demonstrated to be effective in mice in 1931, and became available for human use in 1935. Sulfa drugs are relatively non-toxic, and are effective against many different human diseases. They were “wonder drugs,” and saved hundreds of thousands of lives in the decade before penicillin mostly replaced it. (So, not only was penicillin not the first antibiotic, it was not the first safe and broadly effective one.)
Fleming switched from penicillin to working on sulfa drugs as soon as he learned of them.
Did Fleming discover penicillin?
What counts as a discovery? There’s no clear-cut definition for the word, nor a worked-out explanation of the category. I suggest that what he did should not count. Some historians are indignant that he got credit, and I share that feeling. What matters more, though, is understanding this as case study for clarifying what “discovering” consists of.
I will suggest that the “discovery” of penicillin was not an instant of inexplicable intuition in an isolated individual’s intellect. It was a complex process of physical experimentation and gradually growing understanding, the collaborative work of many scientists, engineers, technicians, civil servants, managers, and administrators, across decades, disciplines, institutions, and continents. This is almost always how difficult technical work, including scientific discovery, gets done.
Fleming’s part in the process was much smaller than that of a research team at Oxford University, a decade later. If there was a discrete “discovery,” they should get the credit for it. I’ll tell that story—featuring meta-rationality—next.
The meta-rational development of penicillin
The aim here is to use concrete examples of meta-rationality in action to give a preliminary sense of what the topic even is.
The Oxford team's development of penicillin is a good case study, not because the work was exceptionally meta-rational, but because it was ordinary science—although done uncommonly well. Only the practical consequences were extraordinary.
I chose the penicillin story almost at random. I could probably have picked any well-documented, well-executed research project and done a similar analysis. That's because well-done ordinary science always has aspects of meta-rationality in it. Meta-rationality isn’t a special, esoteric thing; everyone who uses technical rationality also must do some meta-rational work. The Oxford group did unusually much of it, unusually well.
Case studies in rationality are often quite simple and compact. Case studies in meta-rationality can’t be, because of its panoramic concern for both a project’s big picture and its fine details. Here I omit many fascinating details, yet the story remains quite long and intricate.
I will also analyze it only in terms of the abstract, overview conceptual framework for meta-rationality I’ve presented so far. This Part introduces much more machinery for understanding meta-rationality as it proceeds. Later case studies, of quite different activities, will apply finer-grained analysis to illustrate aspects of meta-rationality this one does not.
In this analysis, I’ll give concrete examples of some of the eleven abstract meta-rational practices discussed in the previous section. I’ll also refer to some entries from the right-most, meta-rationality column of the summary “aspects table” introduced in Part Two and used again in Part Three. Reviewing it now may give you insight as you read further:
In the introduction to Part Four, I wrote:
Meta-rationality answers “what does this situation need?” by engaging with the play of the situation, while orienting to the broad context and to the purposes of participants and potential participants.
Zooming way out, the broad context was England during the Second World War. The Oxford group submitted their first grant proposal for Penicillium research three days after Britain entered the war in 1939; the work was completed, after a colossal scale-up effort, just in time to produce enough of the drug to treat the tens of thousands of casualties on and shortly after the 1944 D-Day invasion of occupied France.
The context of war had two extremely significant, but contradictory, consequences for the research project. For Britain, WWII was a total war, meaning that nearly all economic activity was diverted from civilian to military purposes. Funding for scientific research was cut to nearly zero unless it had direct military applications. In the first years of the project, money was desperately tight, which—as we’ll see—had significant meta-rational implications. In the project’s final years, the United States government recognized penicillin’s military significance, which also radically affected the development project.
The institutional culture of British academic science was another part of the context. Scientists worked mainly alone, supported by tiny individual grants. Fleming claimed, after the Oxford group’s success, that if he had been able to find a competent chemist to collaborate with, they would have been able to purify and characterize the penicillin molecule from the glop, and then it would have been available as a drug ten years earlier. This was probably not true, because the task proved anomalously difficult, but his failed attempts to collaborate illustrate the culture of the time.
The Oxford project, by contrast, operated as a team. I’ll concentrate on the work of two members: the most senior, Howard Florey, and the most junior, Norman Heatley—because they illustrate different aspects of meta-rationality.
In 1935, Howard Florey, a physiologist, was appointed head of Oxford’s School of Pathology. That was the result of complex political maneuvering among the senior university faculty. Florey was considered a brilliant scientist, but a risky choice as an administrator.
He had a vision of a multi-disciplinary team working together on projects too large for any single scientist. This became standard practice in later decades, but it was extremely unusual and perhaps unique at the time. He had full authority to hire and fire, which he used to assemble a group of outstanding scientists with backgrounds and skills in many different fields.
Initially, the scientists did mainly work on personal projects, supported by individual grants that mostly just covered their salaries. Too much of Florey’s time was taken up with fundraising for equipment and operating expenses. The grants he could collect for the School were tiny, often less than £100.4
The initial purpose of the Penicillium project was to get a much larger, longer-term grant. The idea was to open an entirely new field within which there might be many opportunities for discovery, but which would take several years to establish the basic scientific facts. That could justify a much higher price tag.
Another aspect of the context was the state of scientific knowledge at the time. Ernst Chain, a chemist Florey had hired, suggested the mechanisms of biochemical warfare among microbes as a potentially large, important, almost entirely unexplored topic. (Problem selection is a key application of meta-rationality, explored in detail later in this Part.) After scouring the literature, they narrowed the possibilities to three model organisms, and finally settled on Penicillium, for reasons they didn’t document. It was an interesting topic for basic research, not a drug development candidate. Florey and Chain had medical applications vaguely in mind, but those seemed distant and merely theoretical.
British science funders didn’t have the resources for this proto-Big Science project. Fortunately, the American Rockefeller Foundation did, and at the end of 1939 approved a grant for £1,250 per year for five years. That seemed like an enormous amount of money, and a great relief for Florey. He couldn’t have imagined how big a Big Science project his somewhat offhand funding ploy would soon become.
In directing the activities of the School, Florey had to orient to these and other contexts, reasoning across them simultaneously at multiple scales, from global war to individual personality conflicts among his team members.
Similarly, Florey had to make difficult decisions about priorities, on the basis of conflicting purposes of multiple sorts. The School continued several other, unrelated projects concurrently with Penicillium. That project came to dominate, but not all their eggs were in one basket. Florey also had to continually re-prioritize different approaches to penicillin research and development, as incremental progress opened up new possible ways further forward.
Another trade-off, in the face of conflicting purposes, was between basic research and development of penicillin toward application. There are contradictory statements and evidence about how much Florey and the team wanted to make a therapeutic versus discovering fundamental mechanisms in the early years. I suspect they were torn. You get more prestige for basic science than applied, but they did intensely want to reduce suffering and death from bacterial diseases.
You can’t make decisions like these rationally or quantitatively. They must be well-reasoned, but as matters of meta-rational judgement: how best to apply rationality given purposes and circumstances.
How is this going for you so far? I am posting previews of book sections to get your feedback! What’s working? What’s not?
Florey had been a brilliant scientist; now he became a brilliant project manager. In the overall introduction to the book, I wrote:
Meta-rationality becomes increasingly important as you move from being an individual contributor into leadership, management, or entrepreneurial roles. Solving well-defined problems using standard techniques no longer cuts it. Your job is to make sense of messes in which even the problem—never mind the solution—is obscure. Done well, technical leadership is an inherently meta-rational activity: it is about selecting, modifying, and creating systems.
Florey abandoned his own training and research, which were irrelevant to the expanding Penicillium project. His job was now to coordinate the work of scientists from different fields tackling different aspects of the overall effort. This is an instance of meta-systematic activity.
Florey practiced “management by wandering around,” which is characteristic of meta-rational leaders. As I wrote in the introduction to this Part, meta-rationality implies caring more about the concrete situation than representations of it. I said that one failure mode is relying on other people’s reports, rather than going and looking for yourself.
Florey would drop by each scientist’s lab most days for an informal chat, and take a look himself at the physical experiments, to find out what was working and what wasn’t. Sometimes he’d offer encouragement, suggestions, and practical advice in an understated way, while giving the researchers great freedom to work out problems in their own domains according to their own expertise. This is an example of one of the last section’s meta-rational practices:
Bridge the big picture and details. Observe and respond both to the whole system with its broad context, and to the fine-grained details of its operation—specifically with an eye to opportunities for improvement.
Initially, the scope and goals of the project were extremely nebulous. They were walking into an unmapped marshy domain. They didn’t have a Problem. Turning Penicillium glop into a drug was a non-goal at the beginning; that would be an enormously difficult project which they didn’t have the inclination or resources for.
“Make this dish of mold into a drug” is not a Problem, anyway; it’s a hope. So they were, at minimum, in the realm of adventure rationality, meaning without a clear Problem. But there was also no clear way forward once a sketch of a Problem emerged; or (equivalently) many vague and doubtful ways forward. Trying to find Solvable Problems in a marsh made this a matter of meta-rationality, rather than merely adventure rationality.
After a few months:
There were, Florey reasoned, four interlocking puzzles to solve: How to grow the most and the most potent mold; which bacteria were affected by penicillin and to what degree; what, if any, were the effects on human cells and tissues; and what were penicillin’s chemical structure and the means of its action?5
This wasn’t quite right. For example, the main purpose in figuring out the chemical structure was practical: they assumed that once they knew what the active ingredient in the glop was, it would be possible to synthesize it using chemical methods, which would be enormously more efficient than growing it using biological ones. But this turned out to be wrong. The natural molecule was impossible to duplicate artificially. Synthesis still isn’t feasible in practice, and penicillin is still grown in vats full of Penicillium mold.
Each of Florey’s four proto-Problems turned into a major facet of the research project, each consisting of many specific technical Problems. The team encountered endless roadblocks that they successively overcame through intense group effort. Much of this work had to be meta-rational, when existing rational methods failed. As I wrote in the introduction:
There may be no apparent answer—and you remain engaged anyway. If rationality is your default mode, when it isn’t working it’s tempting to disengage and play a video game instead. Meta-rationality builds emotional trust that situations are workable even when there is no obvious way forward.
Let’s shift attention temporarily now from Florey to Normal Heatley: from the most senior member of the team, working primarily at the biggest-picture contextual level, to the most junior, working primarily at the most hands-on nuts-and-bolts level.
Heatley was twenty-five when Florey hired him in 1936, and had just finished his PhD. He was a biochemist, which was relevant, but that expertise was not what made him probably the most important scientist on the project—and arguably the actual discoverer of penicillin.
Heatley actually cared for the project. In the introduction, I contrasted that with:
Caring more about your identity, role, and incentives than for the situation. A key example is wanting the comforting feeling of expert competence and control that you get from doing being rational, more than you want to engage with the situation in all its uncontrolled mess. Relatedly: wanting the institutional rewards that come from being seen to follow rational rules, more than caring whether or not doing so helps.
Heatley displayed extraordinary fluid competence, a key meta-rational attribute we’ll discuss later in this chapter. It’s the opposite of prioritizing the feeling of being an expert doing rationality, and of prioritizing institutional rewards over being useful. It means doing whatever the situations needs, setting aside limitations from your role and identity.
The first difficulty the team faced was getting enough Penicillium glop to do experiments with. In fact, this remained the central practical difficulty throughout the project. The mold grew very slowly and, after many days, a dish produced about a microgram of penicillin. It takes about two grams to treat one patient.
This was no one’s problem, so Heatley made it his problem. He constantly improved the methods. Over the first couple years of the project, he scaled up production at Oxford by five orders of magnitude. Then he contributed to industrial efforts that scaled up an additional five orders of magnitude, enough to treat the D-Day wounded.
Reading accounts of breakthrough science, you often find that the critical factor was not some feat of rational inference, nor of intuitive genius, but new experimental apparatus, often extremely crude. Fortunately, Heatley:
was a most versatile, ingenious, and skilled laboratory engineer on any scale, large or minute. To his training in biology and biochemistry he could add the technical skills of optics, glass- and metal-working, plumbing, carpentry, and as much electrical work as was needed in those pre-electronic days. Above all, he could improvise—making use of the most unlikely bits of laboratory or household equipment to do a job with the least possible waste of time.6
I wrote that meta-rationality may reveal that what a situation needs “may involve an angle grinder, a proclamation, or a line dance.” What this one needed was a bathtub, several metal trash cans, and some discarded dairy equipment.
In 1940, Heatley designed and built a high-throughput laboratory automation system for glop purification. It used those parts, and also featured “an old doorbell to signal when a bottle was about to become empty or full, colored warning lights, nozzles, copper cooling coils that he fashioned, and more junctions between the various bottles and tubes than on the track of a complex electric train set.” The construction budget was approximately £5—research funding was still very tight—so he scrounged most of the bits from the local garbage dump.
Part of the automated glop purifier. The large cylindrical metal containers are milk cans.