Paths in strange spaces, part II

library(dplyr)
library(fifty)
library(jasmines)
seed_heart(150) %>%
  time_tempest(iterations = 40, scale = .01, curl_seed = 1) %>%
  mutate(order = time) %>%
  style_ribbon(alpha_init = .7, alpha_decay = .05,
    palette = shades("gray"), background = pagecolour) 

On epistemological anarchy

In my early 20s I read Paul Feyerabend’s (1975) classic work on the philosophy of science, provocatively entitled Against Method and I hated it. Part of the reason I hated it so much is the way that his work was introduced to me in my undergraduate philosophy of science class (yes, I actually took one!) As it was described to me, Feyerabend was arguing that there is nothing special that differentiates science from any other belief system, that scientific methodology adds nothing worthwhile, and that when it comes to making inferences about our world, there is only one principle: “anything goes”. I approached the book with a very hostile mindset, and to me it seemed disorganised, unscientific, and riddled with logical errors. In retrospect, I suspect mine was an uncharitable reading. Because I’ve lost my copy of the book, I’ll cheat again and use the Stanford Encyclopedia of Philosophy to supply the context I was missing:

By the early 1970s Feyerabend had flown the falsificationist coop and was ready to expound his own perspective on scientific method. In 1970, he published a long article entitled “Against Method” in which he attacked several prominent accounts of scientific methodology. In their correspondence, he and Lakatos subsequently planned the construction of a debate volume, to be entitled For and Against Method, in which Lakatos would put forward the “rationalist” case that there was an identifiable set of rules of scientific method which make all good science science, and Feyerabend would attack it. Lakatos’ unexpected death in February 1974, which seems to have shocked Feyerabend deeply, meant that the rationalist part of the joint work was never completed.

In other words, the intended structure of the work was one that should be familiar to most of us as scientists: on the one side (Lakatos) we have theory building, and on the other (Feyerabend) we have theory criticism. These two components are supposed to work together, and as individual scientists we try (hope) to engage in both sides of this process iteratively. We build our theory, attack our own theory, when it fails we build a new one and so forth. Both are necessary, and in retrospect I think Against Method is a less impressive work than For and Against Method would have been. In any case, here’s the summary of Feyerabend’s argument:

Against Method explicitly drew the “epistemological anarchist” conclusion that there are no useful and exceptionless methodological rules governing the progress of science or the growth of knowledge. The history of science is so complex that if we insist on a general methodology which will not inhibit progress the only “rule” it will contain will be the useless suggestion: “anything goes”. In particular, logical empiricist methodologies and Popper’s Critical Rationalism would inhibit scientific progress by enforcing restrictive conditions on new theories. The more sophisticated “methodology of scientific research programmes” developed by Lakatos either contains ungrounded value-judgements about what constitutes good science, or is reasonable only because it is epistemological anarchism in disguise. The phenomenon of incommensurability renders the standards which these “rationalists” use for comparing theories inapplicable.

This summary matches my recollection of the book rather well… and it mirrors my own experience as a scientist rather well too. For example, in my own area of research there is a degree of tension between the “Bayesian models of cognition” school of thought that views human inductive reasoning as a form of probablistic inference, the “heuristics and biases” school that assumes our reasoning is based on simple, error-prone approximations, and the “connectionist” school of thought that emphasises the importance of parallel distributed computation and the underlying cognitive architecture. These three different frameworks are largely incommensurate. I’ve used all three at different stages of my career, and while most of my work falls within the “Bayesian cognition” framework I don’t necessarily think it is “better” than the other two.

I’m not even sure the question makes sense. Much as Kuhn points out in The Structure of Scientific Revolutions, each paradigm emphasises different empirical phenomena, selects different operationalisations, and produces formal models that have different intended scope. While in the long run – of course – we would hope to construct a single unifying framework that encompasses all of human cognition, we are not even remotely close to that point. Trying to decide on which of these three completely incommensurable paradigms is “least wrong” when the reality is almost certainly that all three are spectacularly wrong, is just silly. Right now, with the cognitive science literature being where it stands, all three paradigms offer useful insights, and it is a good thing that we as a discipline retain all three.

The Stanford Encyclopedia entry continues:

At a time when Kuhn was downplaying the “irrationalist” implications of his own book, Feyerabend was perceived to be casting himself in the role others already saw as his for the taking.

I respect Feyerabend a lot for this. I did read Kuhn’s book too (though I don’t remember it very well) and I did get the impression that he was a little nervous about the entailments of the “incommensurability” problem, and sought to hide or minimise them. Feyerabend did not shy away from it, and as a result Against Method is a very provocative and unsettling read for a scientist.

On modesty and the scientific bootstrap

Oh dear. I seem to be digging myself into a deeper and deeper hole. I started this post with some statistical concerns about p-values and apparently I’m now at the point of endorsing epistemological anarchy? Really? That’s… not a good place for a scientist to be! Well… maybe it’s not so bad. To me, the major, substantive point that Feyerabend made is this one:

The history of science is so complex that if we insist on a general methodology which will not inhibit progress the only “rule” it will contain will be the useless suggestion: “anything goes”.

I think this is entirely correct. There are no hard and fast rules for good science, no magical set of procedures that we can follow that will guarantee – not even to a known probability of error – discovery of truths. To me it seems logically incoherent even to imagine that such a set of rules could be proposed by humans. It would be a different story if we already knew the truth. If we already knew the truth about our world, and how observations can be made within that world, then we would be able to work out what inferential rules make sense for that world. Until that time comes that we have such an complete understanding, however, we are relying on our best guess about the structure of the world to work out what the rules for learning about the world should be! As scientists we are hoping to bootstrap our way to the truth.

To me that seems like a very reasonable strategy, and I can’t think of anything better, but let’s not pretend that we really know what we’re doing here. We’re all making it up as we go along, to the best of our ability, hoping not to make a mess of everything. Under the circumstances, I think a little modesty in our scientific and statistical claims would be in order, no?

In the garden of forking paths

Besides the importance of being modest, my little story about reading Feyerabend in the misspent years of my youth contains the kernel of a defence for preregistration. Why was I so hostile to Feyerabend the first time I read Against Method? Mostly it was because of my history: I brought my own preconceptions to the book that were based on someone else’s reading of the book (i.e., my undergrad lecturer) and that led me to emphasise some things and not others. I chose those parts of the book that seemed most relevant to me at the time, and those choices shaped my conclusions. Not only that, I was unaware of Feyerabend’s history. I did not know that the work was originally intended to be joint work with Lakatos. Had I read the intended work, For and Against Method, I suspect I’d have come to different conclusions.

Sure, when I sat down as an impressionable 20-something to read the book, I read it with what I thought of as an open mind, and the book itself is what it is, but this “state” is not sufficient to properly describe or make sense of the situation. The earlier states matter too. The book has a history, a path that brought this specific volume to me, and I had a path that brought me to read it. Both of those histories matter. Knowing those histories better, as I do now, leads to a rather different impression of the book (and the reader!) than one might have without knowing this history. If you want to understand my reading of Against Method you need need to know where it came from, and where I came from.

So it goes in scientific research also…