In antiquity, a similar thing happened, in a more primitive form. The Aristotelians thought tremendous progress had been made. Pyrrho came back from India and refuted Aristotelian metaphysics. In the field of medicine, this philosophical dispute created two camps: the rationalist/dogmatist camp and the skeptic/empiric camp.
Plato's Academy then re-interpreted Socrates to be like Pyrrho. For a while, the Academy was essentially Pyrrhonist. Then Carneades swaped pithanon for epoche - i.e., he retreated from suspension of judgment and allowed that one could choose on the basis of what seemed most plausible. This is the system Cicero studied and wrote about. When Cicero translated "pithanon" into Latin he used "probabilis" - the same word we derive "probability" from.
Could you give a pointer to any further reading on the connection to the 2008 financial crisis? Here, you just link to another page of yours where all that is said on the matter is "The belief that such methods and guarantees do exist has been a major cause of the 2008 financial crisis and the science replication crisis, among other catastrophes.", and the interested reader can only take that on faith :(
This was widely reported in the financial press during the aftermath. I don’t know of a good retrospective summary history off-hand, but I expect there is one!
Wikipedia’s discussion is at https://en.wikipedia.org/wiki/2008_financial_crisis#Incorrect_pricing_of_risk . It’s not very good, but note the bit about “Li's Gaussian copula formula,” a too-simple (mis)application of probability theory. “It will go down in history as instrumental in causing the unfathomable losses that brought the world financial system to its knees.” “"The super-boom got out of hand when the new products became so complicated that the authorities could no longer calculate the risks.”
I recommend this paper, been a favorite of mine since it came out: "WARNING: Physics Envy May Be Hazardous To Your Wealth!" from 2010, by Andrew W. Lo (MIT Sloan School of Management / Laboratory for Financial Engineering) and Mark T. Mueller (MIT Sloan and MIT Department of Physics / Center for Theoretical Physics)
'We conclude by offering a new interpretation of tail events, proposing an “uncertainty checklist” with which our taxonomy can be implemented, and considering the role that quants played in the current financial crisis'
As a personal note, after thoroughly understanding their “Level 4 uncertainty” items, and the structure of the mortgage market, with which institutions were lending how much to whom, and congratulating myself for all that, I was massively short mortgage lenders before the crisis (a year early, actually).
I did not anticipate their “Level 5” uncertainty, specifically the “government intervention” with “shortsales restrictions” in that column. That nearly wiped me out. A traumatic experience I haven’t recovered from financially, or emotionally either. I’ve been excessively vigilant about random tail risks ever since.
Another limitation of statistics that I think sometimes gets overlooked is that we often care about individual outcomes rather than averages.
For a gambler playing a casino game many times, the average payoff is meaningful. If you play a game only once, not so much. This is the difference between a life insurance company's perspective (their profits are based on averages) and a buyer's perspective (you only have one life to live and don't know when you will die).
Similarly, when talking about public health, we care about population-level statistics (with more vaccinations, infections go down) but this is often unsatisfying for treating individual patients. No individual can say for sure whether a vaccination saved their life, because we don't know what would have happened if we had done something different.
Also, national or worldwide statistics are often not very meaningful for individuals because large populations often include many people who aren't much like you.
A common pattern in scams and cons is, you set someone up with a "too good to be true" promise, but the trick is you know you can deliver on it, or appear to. After the shock they open up and trust you, and then you can set them up with an even bigger promise and take all their money.
Anyway, if I told a naive person that there is one simple and objectively correct mathematical way to talk about uncertainty, and all other mathematical models are inferior, it would sound pretty surprising. But it's true, as you point out in the article! If you want to use fuzzy logic or non-monotonic whatever, almost always you are making a mistake and you should have just used probability instead. Understanding probability is very powerful, it's all you need to mathematically model uncertainty.
This surprising and impressive fact is then the set up for convincing someone that probability theory is the eternal secret to all reasoning, learning, and decision making.
"The answers to these questions are matters of meta-rational judgement. Skill in answering depends on numerous, diverse considerations, maxims, and methods."
And also, that's only the beginning. Skillfully answering those questions is just one of prerequisites for the consequential one - is it, after all, your place to buck against the system?
This looks very very essential and cool, my own work is very engaged with the metarationality of probability (from a less formal critical direction) and I look forward to fully absorbing the above.
One possible tip though - these kinds of posts are hard to find time to absorb on a Monday, maybe consider sending Sunday? just my 2c.
Thanks! I do send many of them on Saturday. I've read many opinions about which time and day is best, and they're so varied that I have little confidence in any theory...
In antiquity, a similar thing happened, in a more primitive form. The Aristotelians thought tremendous progress had been made. Pyrrho came back from India and refuted Aristotelian metaphysics. In the field of medicine, this philosophical dispute created two camps: the rationalist/dogmatist camp and the skeptic/empiric camp.
Plato's Academy then re-interpreted Socrates to be like Pyrrho. For a while, the Academy was essentially Pyrrhonist. Then Carneades swaped pithanon for epoche - i.e., he retreated from suspension of judgment and allowed that one could choose on the basis of what seemed most plausible. This is the system Cicero studied and wrote about. When Cicero translated "pithanon" into Latin he used "probabilis" - the same word we derive "probability" from.
Could you give a pointer to any further reading on the connection to the 2008 financial crisis? Here, you just link to another page of yours where all that is said on the matter is "The belief that such methods and guarantees do exist has been a major cause of the 2008 financial crisis and the science replication crisis, among other catastrophes.", and the interested reader can only take that on faith :(
This was widely reported in the financial press during the aftermath. I don’t know of a good retrospective summary history off-hand, but I expect there is one!
Wikipedia’s discussion is at https://en.wikipedia.org/wiki/2008_financial_crisis#Incorrect_pricing_of_risk . It’s not very good, but note the bit about “Li's Gaussian copula formula,” a too-simple (mis)application of probability theory. “It will go down in history as instrumental in causing the unfathomable losses that brought the world financial system to its knees.” “"The super-boom got out of hand when the new products became so complicated that the authorities could no longer calculate the risks.”
I recommend this paper, been a favorite of mine since it came out: "WARNING: Physics Envy May Be Hazardous To Your Wealth!" from 2010, by Andrew W. Lo (MIT Sloan School of Management / Laboratory for Financial Engineering) and Mark T. Mueller (MIT Sloan and MIT Department of Physics / Center for Theoretical Physics)
https://arxiv.org/pdf/1003.2688
'We conclude by offering a new interpretation of tail events, proposing an “uncertainty checklist” with which our taxonomy can be implemented, and considering the role that quants played in the current financial crisis'
The Uncertainty Continuum taxonomy:
Level 1: Complete Certainty
Level 2: Risk without Uncertainty
Level 3: Fully Reducible Uncertainty
Level 4: Partially Reducible Uncertainty
Level 5: Irreducible Uncertainty
Level ∞: Zen Uncertainty
I like the “uncertainty checklist” table (p. 51)!
As a personal note, after thoroughly understanding their “Level 4 uncertainty” items, and the structure of the mortgage market, with which institutions were lending how much to whom, and congratulating myself for all that, I was massively short mortgage lenders before the crisis (a year early, actually).
I did not anticipate their “Level 5” uncertainty, specifically the “government intervention” with “shortsales restrictions” in that column. That nearly wiped me out. A traumatic experience I haven’t recovered from financially, or emotionally either. I’ve been excessively vigilant about random tail risks ever since.
Nice to see you writing more about this!
Another limitation of statistics that I think sometimes gets overlooked is that we often care about individual outcomes rather than averages.
For a gambler playing a casino game many times, the average payoff is meaningful. If you play a game only once, not so much. This is the difference between a life insurance company's perspective (their profits are based on averages) and a buyer's perspective (you only have one life to live and don't know when you will die).
Similarly, when talking about public health, we care about population-level statistics (with more vaccinations, infections go down) but this is often unsatisfying for treating individual patients. No individual can say for sure whether a vaccination saved their life, because we don't know what would have happened if we had done something different.
Also, national or worldwide statistics are often not very meaningful for individuals because large populations often include many people who aren't much like you.
A common pattern in scams and cons is, you set someone up with a "too good to be true" promise, but the trick is you know you can deliver on it, or appear to. After the shock they open up and trust you, and then you can set them up with an even bigger promise and take all their money.
Anyway, if I told a naive person that there is one simple and objectively correct mathematical way to talk about uncertainty, and all other mathematical models are inferior, it would sound pretty surprising. But it's true, as you point out in the article! If you want to use fuzzy logic or non-monotonic whatever, almost always you are making a mistake and you should have just used probability instead. Understanding probability is very powerful, it's all you need to mathematically model uncertainty.
This surprising and impressive fact is then the set up for convincing someone that probability theory is the eternal secret to all reasoning, learning, and decision making.
Nice way of explaining it!
I'd be more inclined to say "it's all you can get" rather than "all you need," though :)
"The answers to these questions are matters of meta-rational judgement. Skill in answering depends on numerous, diverse considerations, maxims, and methods."
And also, that's only the beginning. Skillfully answering those questions is just one of prerequisites for the consequential one - is it, after all, your place to buck against the system?
Good point, yes!
This looks very very essential and cool, my own work is very engaged with the metarationality of probability (from a less formal critical direction) and I look forward to fully absorbing the above.
One possible tip though - these kinds of posts are hard to find time to absorb on a Monday, maybe consider sending Sunday? just my 2c.
Thanks! I do send many of them on Saturday. I've read many opinions about which time and day is best, and they're so varied that I have little confidence in any theory...