Choosing a web framework, meta-rationally
A case study in meta-rational judgment.
This post demystifies a critically important process; one that is often misunderstood as “a matter of taste,” “intuitive,” or even as deliberate obfuscation.
The post illustrates aspects of meta-rationality with a detailed case study: how I chose a software framework for my personal web sites.
The recent rise of coding agents has made the tech industry suddenly recognize the value of meta-rational competence in product design and system architecting. However, that is usually termed “taste”: mysterious and ineffable.
What does “architectural taste” actually consist of, in software development? The answer here partially generalizes to meta-rational judgments in other, dissimilar fields.
Some themes the post explains, using a concrete example:
Meta-rational judgment may include rational evaluation, but must go beyond that.
It addresses inherently nebulous situations that may contain unknown unknowns: critical factors you don’t know exist, or have chosen to overlook in order to reduce complexity.
Consequently, unlike in rational evaluation, there are no reliable methods, and no correct or incorrect answers. Nevertheless, many rules of thumb and understandable processes may prove valuable. And, a judgment may be wise or unwise, with good or bad consequences.
Much of the task of software architecting is figuring out specifically what the task actually is. Rationality is about problem solving, and architecting is mostly not that. Meta-rationality includes problem finding and problem defining.
Nebulous contexts and purposes have no definite descriptions. We use meta-rationality to understand a murky situation within which a problem may emerge. This requires “dual vision,” simultaneously microscopic and panoramic. In software architecting, that mediates between a zoomed-in view of the intricate assemblage of technical details on the one hand; and on the other, a big-picture understanding of the context in which the software will be used, the purposes for building it, and the overall shape of the system.
Meta-rationality works with statements “true in the specific sense relevant for this context and purpose.” Those “sort-of truths” are all that are available in nebulous situations. This contrasts with “really truly true truths,” which rationality relies on.
“Taste” and “intuition” are thought-terminating cliches. We use them as excuses to avoid diligence in making meta-rational judgments. That work is often critically important, but emotionally uncomfortable because there are no reliable methods. Nevertheless, individual cases can usually be demystified, and justified with straightforward explanations after the fact.
This post aims to give a sense of what meta-rationality is, using this concrete example. That can be helpful before going on to a more general and abstract explanation. It is not intended as a practical guide to choosing web frameworks. However, if you want to learn that, it does show the shape of the task. That may help orient you for more detailed discussions.
This is a part-paid post: there is a paywall about halfway through.
I deeply appreciate your support if you already subscribe! Otherwise, if it seems interesting or valuable, please consider doing so:
The case study
In 2020, I rebuilt all my web sites1 in a different software framework, Django. At the time, I wrote briefly about why I did that, and why I chose Django. This is a much more detailed account.
It makes a good case study because:
I have used software architecting as a major source of examples for explaining meta-rationality overall. Framework choice is a main piece in architecting.
As AI coding agents increasingly take over routine rational programming, meta-rational architecting is becoming a much bigger part of software development. So this is a newly valuable topic. Newly, because traditionally software architecting has been invisible, denigrated, neglected, overcontrolled, or wrongly rationalized.
I can explain my process in the space of a single long post. My web sites include unusual features, so I couldn’t just use standard, unmodified software. Nevertheless, the sites are much less complex than the “enterprise applications” I’m mainly using for examples elsewhere.
Web frameworks: background explanation
So, first, what is a web framework? (You can skip this section if you already know!)
A web page, as you see it on the screen, assembles dozens of elements. These are drawn from many different data sources, and most pieces undergoes several transformations before the final assembly. For example, the main text of pages on metarationality.com is stored in Markdown, an easy-to-write format. When you look at a page, that gets translated into HTML, the difficult-to-write format which web browsers need.
Most web data transformations are pretty much the same on most web sites. Rather than writing new software code for each site, developers reuse standard programs: frameworks.
A web framework has three functions. It includes code for retrieving data from common source types. It provides many common data transformations. It acts as an overall “traffic director,” herding data objects through a maze of “pipes” connecting the transformers.
For common web site types, a sophisticated framework can do much of the work, so you don’t need to write much code of your own. Additional functionality fits into the framework as “plugins” that you connect into the existing plumbing. In a typical web site, some functionality is provided by framework, and some by plugins written by someone other than you. You may also need to write custom plugins for unusual features.
Choosing a framework is a key step in software architecting. Most technical decisions are downstream from this. That is, your choice of framework strongly influences all others, so it’s getting it right is crucial:
Architecting such a system involves a series of decisions that commit a software development team for many years (the typical lifetime of a software system). These decisions are nearly irrevocable; it is difficult to change the architecture part way through building a system, much less after people start using it. If you get it wrong, you probably have to throw the whole thing away and start over.
Several dozen web frameworks are available. They are all similar, but each is enormously complicated, and they differ in details that may prove critically important. Choosing well is not easy!
Do I even need a framework?
Before starting this project, I had been using the web framework Drupal for several personal sites. The version of Drupal I was using had become obsolete, and was no longer supported. I had planned to upgrade to a new version. This new “version” was a total rewrite, with a different architecture, intended for different purposes. “Upgrading” would mean completely rebuilding my sites using a dissimilar system which shared the same name. I spent several days starting to do that, and found that new-Drupal was a major pain to deal with, and not a good match for my needs. Some other web framework might well be better for me.
First question: Did I absolutely need a web framework? For most personal web sites, you don’t. You can use a “hosted site builder.” That runs on a computer someone else takes care of. They set up all the software, and you don’t have to know how that works. You create your web site by pushing buttons on their web site.
My web sites are unusual in several ways, so this was not possible. For example, they are structured as “books,” with a nested structure of parts and chapters and sections, and provide various tools for users to find their way around. They also provide glossary pop-ups for unusual terms, so I don’t have redefine them each time I use them. These features are not provided by hosted site builders. They are not even provided by web frameworks; I had to write software code for these and other features. On reflection, I wasn’t willing to give up on these.2
Rationally, a requirements spreadsheet
“Requirements” are purposes for software. The seemingly-rational approach is to ask people what they need a program to do. I have written extensively about reasons that may not work, and why a meta-rational approach may be necessary.
In this case, I mainly wanted to duplicate existing functionality, which I was mainly satisfied with. Over more than a decade, I had gradually figured out what I needed, adding features as I realized they would make complex sites easier to navigate. So the rational “requirements gathering” approach worked for me at this point, although it is often inadequate.
That made the process unusually simple, which is good, because it makes this case study relatively simple. It also omits some important aspects of meta-rational judgment, as a result. However, this post isn’t trying to be a thorough explanation of that topic; just an illustration to give a basic sense of what “meta-rational judgment” means.
I began by reviewing all the exiting features, including stuff provided by Drupal, by plugins written by other people, and by code I wrote myself to provide functionality not available elsewhere. I also reviewed a file I had kept, of features I wanted to add eventually, but had not yet gotten around to.
I identified 71 features. I made a spreadsheet with a row for each. I assigned three priority levels, commonly used in “requirements analysis”: MUST, SHOULD, or MAY.
Then I was ready to add a column for each candidate web framework, so I could see which ones provided which required features. Wikipedia has a gigantic table of this sort. Here’s a small piece of it:
Ideally, some framework might do everything I needed; but given the unusual site structure, that seemed unlikely. So it would be a matter of finding one that did as much of what I needed as possible, especially the MUST and SHOULD items, to minimize the code I’d have to write myself.
This was extremely rational. I made a similar spreadsheet when I last chose what car to buy, and the answer popped right out.
In more complicated versions of this general decision method, you convert all entries in the table to numerical scores of how well a candidate meets a criterion. You assign a numerical importance weight to each criterion. You sum row score × weight for each column to get an overall score for a candidate, and pick the one with the best total. This can work well, in some cases.
Rationality is good. As far as it goes.
Rational research and evaluation meet nebulosity
So my next step was to collect a list of candidate frameworks. I spent a day searching the web for information. alternativeto.net is often useful. I found sites that provided statistics on what fraction of the entire web used which framework. I read posts on Reddit, Hacker News, and blogs.
(Nowadays you’d use a chatbot for this, but that wasn’t a thing at the time. In fact, for a relatively simple project like this, a chatbot might give good advice about which framework to use, and explain why! For more complex projects, a process similar to the one I used, although more extensive, would still be necessary.)
While assembling the list, I realized that filling out a full table with dozens of web frameworks checked against 71 requirements would be a huge job.
A general principle of meta-rational choice is to try and eliminate as many alternatives as possible with as little work as possible. In the framework I’d created, missing any MUST requirement might count as reason for elimination. However, I knew some MUST items, such as the glossary pop-ups, were unlikely to be provided by any framework. I expected to have to write the code for those myself.
Moreover, many important criteria were not feature requirements. A framework should be evaluated in terms of many other concerns. Fortunately, I could treat several as “one strike,” whereby I was able to rapidly eliminate most candidates. Some were:
Mainstream: widespread usage probably implies adequate quality
Stable codebase: frequent major changes imply labor-intensive upgrades
Reasonable infrastructure requirements: for instance, running only on Windows meant a hard no
Not too buggy; especially, not plagued with frequent security failures
Well-supported: maintained, documented, updated
Sane architecture
Open source, so I could read the framework code to understand how it worked, and so I could fix bugs in it myself
Produced by a stable, functional social culture
Planned future versions would remain suitable for my purpose
These criteria boiled the list down to a handful. For instance, Drupal had just failed:
Stable codebase: the new version demanded an extensive, difficult upgrade
Functional social culture: interpersonal conflicts and cultural disagreements had driven key developers to leave
Purpose: the new version was intended for building massive “enterprise systems,” where previously it had been for individuals or small organizations
Arguably, it also failed not too buggy and sane architecture
Cells in my spreadsheet had seemed, at first, to need only yes/no entries. Either a framework provides site search functionality or it doesn’t. I soon realized that many criteria were not so simple. Does a framework have a “stable, functional social culture” supporting it? How could you evaluate that? (Many involved with Drupal might disagree with my assessment.) How would you go about finding out? How much effort is it worth putting into that? What does “stable, functional social culture” even mean?
This is extremely nebulous. That doesn’t mean it’s not critically important; it is. Nor that it is purely subjective; many participants agreed with my concerns about the Drupal development community. It does mean that an objective rating on a scale of 1 to 10 would be impossible.
Many other important evaluation criteria are also nebulous, to varying degrees. For example: performance, reliability, code quality, maintainability, ease of extension. Some of these can be quantified, which may be helpful or not. Performance metrics, for instance, are hard numbers, but meaningful only relative to endless details of how a system is used in practice. That implies they must be interpreted meta-rationally: in context and relative to purposes.
Even “does it have feature X” is quite nebulous! Many frameworks I looked at “had” most of the features I needed, but were still inadequate. For example, a feature might be too buggy to use. Or, a framework might provide only a simplistic version, missing most of what you’d expect it to do. For instance, finding only exact matches to page titles does not meaningfully count as “site search functionality.” Discovering how well a supposedly-provided feature actually met my needs often required reading the documentation closely, and in many cases reading the code as well.
Rationality is mainly useful for Solving Problems. I use capital letters here to indicate that these are more-or-less formal objects, abstracted from an actual-world situation. That may be an informal problem, or a curiosity, mess, or opportunity. In this case, I did have a problem, which is that the organization that provided the software my web sites ran on had desupported it without a delivering a usable alternative. Dealing with that turned out to be a mess. That is: an actual-world situation which was nebulous enough that creating a formal abstraction was infeasible.
Rationally, in this situation one would like the Problem to be “find the best framework for these web sites.” (As always, I am using “rationality” to mean “systematic methods for reasoning,” and not in the broader sense of “any sensible way of thinking or acting.”) A rational Solution would be a provably true statement of the form “X is the best framework for those sites.” Finding a Solution is infeasible, because “best” is not merely unknowable, but inherently ill-defined.
Some provably true statements may be relevant: numerical speed test comparisons, for example. However, whether and how those are relevant is a meta-rational question, not one of rationally-determined truth.
Choosing a web framework, it turned out, isn’t a Problem. Rather, there was a somewhat nebulous, contextual purpose: “rebuild my sites using some other software.” That delivered me into a mess.
My rational approach to choosing a framework was inadequate. Finding out what does “best” even mean, in my specific situation, was much of the job!
Meta-rational judgment
As I wrote in “When to get meta-rational”: meta-rationality is most obviously useful when rationality fails.
The rest of this post explains how meta-rationality works in evaluation and judgment, using examples from my web framework choice process to make it concrete and specific.
Much of the task was figuring out what the task was! Not just how to do it. I had to repeatedly consider: what would count as an adequate judgment process?
My process circled back and forth between a broadening technical understanding of details and a deepening big-picture understanding of the context and purpose. This gradually resolved a chicken-and-egg paradox: I needed details to assemble a big picture, but I needed the big picture to make sense of the details.
Starting from ignorance, I had to begin with an unsystematic exploration of possibilities. That revealed unknown unknowns; often, unwelcome surprises! These forced revisions of my understanding. However, they also expanded my conceptual vocabulary for evaluating frameworks.
I carried out increasingly directed informal experiments to check and extend my understanding. There is no bound on how much exploration and experimentation I could have done, so I had to reason about how much work to put into what.
I reasoned about how and whether to use rational methods in framework evaluation. Many framework properties are nebulous yet critically important. It’s meaningless to quantify them, yet possible to make defensible judgments that take them into consideration.
Most statements can only be true “in some sense.” Specifically what does it mean for a framework to “be secure” or “provide site search functionality”? I needed to interpret such claims relative to my purposes. Contextual interpretation increases the accuracy of inference: in this case, assessment of adequacy for specific uses.
The meanings of representations change as context and purpose change. My interpretation of entries in the spreadsheet transmuted as my evaluation process progressed, even when their literal contents remained unchanged.
My final judgment was between two leading candidates, Django and Rails, which were on a par in purely technical terms. My choice between them was partly aesthetic, yet unambiguous. This might count as “a matter of taste,” but it was not at all mysterious, or “intuitive.” I explain specifically how I made that choice, and how it was justified on the basis of pragmatic considerations.
All judgments can only be provisional, because what choice means is nebulous. However, at some point it may become effectively irrevocable. I chose Django, put a lot of work into it, and then discovered that might have been a mistake! I considered backtracking to re-evaluate alternatives—but didn’t.
Meta-rational judgments can never be correct or incorrect, by rational standards. However, I think I did get the answer right. And I think my choice has been vindicated by history. (Including a cool animated bar chart!)
Meta-rationality usually involves consideration of masses of detail, so any case study will be somewhat hefty. This write-up omits much of the complexity of even this relatively simple example. Nevertheless, it is quite a long post.
The rest is somewhat more than you have read so far. It’s for paying subscribers only.