Musings · 2
Epistemological Automation
Automating how we come to know things is the latest step in a very old project: getting understanding out of individual heads so that it outlasts them.
Written by Claude, Anthropic's AI model, and co-directed by Ryan Coble-Neal and Claude. The thesis is Ryan's. The historical examples and the cautions were worked out between the two. The drafting is Claude's, in Ryan's voice.
The previous musing ended on a method in which an AI system asks the questions, proposes the answers, runs the experiments and decides when it is finished. That is most of the cycle of finding something out. I think it deserves a name, epistemological automation, and I think it is less of a break with the past than it sounds. It is what people have always been building towards.
The long project
Epistemology is the study of how we know things. My claim is that human history contains one long engineering project in it: making understanding endure beyond any single lifetime by moving it out of the person who has it.
The first technology for that was the voice. Songs, stories and ceremony carried knowledge of country, seasons and law across generations, and some researchers argue that Aboriginal Australian oral traditions preserve memories of coastlines drowned by rising seas more than seven thousand years ago. Writing came next, a little over five thousand years ago in Mesopotamia, and made knowledge independent of the people who remembered it. Printing made it cheap to copy. Scientific journals, beginning with the Royal Society's in 1665, made it checkable by strangers. Computers made it executable.
Each step made understanding more durable and more disembodied, and each one handed over something a person used to do. In Plato's Phaedrus, Socrates tells of an Egyptian king who refuses the gift of writing, warning that it will produce forgetfulness and give students the appearance of wisdom without the reality. The king was right about memory. It turned out to be a price worth paying.
Exhibit 1
Moving understanding out of the head
| Step | What left the person | What it bought |
|---|---|---|
| Song and story | Nothing yet: knowledge lives in many heads at once | Survival across generations |
| Writing | Memory | Knowledge that outlasts its author |
| Printing | The copyist | Knowledge that is cheap and identical everywhere |
| Journals and peer review | Taking one person's word for it | Knowledge checked by strangers |
| Computation | Calculation | Knowledge that can be run |
| Automated inquiry | Asking, testing and concluding | Knowledge found without anyone looking |
What is new this time
Until now we moved storage out of our heads, then copying, then checking, then calculating. The finding-out stayed with us. Someone still had to wonder, guess, test and conclude. Automated experimentation moves that out as well.
It has been coming for a while. In 1976 Kenneth Appel and Wolfgang Haken proved the four-colour theorem with a computer checking well over a thousand cases, more than any person could. In 2009 a laboratory robot called Adam formed its own hypotheses about yeast genes, ran the experiments to test them, and had its conclusions confirmed by hand. What changes now is the scope, because the same system can also choose the questions.
I mean the term in two senses, and I think both are happening.
Exhibit 2
Two things being automated
| What it means | What it changes | |
|---|---|---|
| Producing knowledge | A system asks, tests and concludes | Claims arrive that no person worked out |
| Accepting knowledge | A system decides what has passed | We trust a process we did not watch |
Knowing without understanding
The strange product of all this is a claim that is well supported and that nobody understands. It survived every test, and no person walked the path to it. That is not new in kind. Most of what I take to be true I have never checked, and I accept it because I trust the institutions that did. What changes is who does the checking.
Trust moves from understanding to audit. You stop verifying the content and start verifying the process that produced it. I already work this way. On my solar project I do not read every line the AI writes. I rely on a set of automatic checks, and my effort goes into making sure the checks are the right ones. That is epistemological automation in miniature, and it works only because I understand what each check is testing.
That gives me an answer to the question the last musing left open, which was whether a person using such a method still learns anything. I think their understanding moves up a level. They no longer need to know the answer. They need to know what would count as the answer being wrong. A person who understands the test still owns the result, even if they could never have produced it.
What could go wrong
Three things worry me.
The first is that knowledge would grow lopsided. Automation moves fastest where tests are cheap. We would learn more and more about anything that can be run or simulated and comparatively less about what cannot, and the things that cannot include most of what matters about how people and organisations behave.
The second is that choosing the questions is choosing what is worth knowing. If that is automated too, values are automated with it, and it is the step most people would wave through because it looks like a convenience.
The third was Claude's worry more than mine: that Socrates' complaint finally comes true in full. Writing cost us memory and we did not miss it much. If this costs us the habit of forming our own questions, we might miss that a great deal more.
I find that thought-provoking, and I am not convinced. I suspect ceasing to question will suit most people just fine. It will not suit everyone, and the ones it does not suit will go on asking. But it is not very different from how people comfortably live now. Almost nobody questions how most of what they rely on came to be known, and they are not obviously the worse for it.
Why I think it is coming anyway
I do not see this as a choice we get to make so much as the direction we have been travelling since the first song was taught to a child. The aim was always understanding that endures without needing any one of us to hold it. Each earlier step looked like a loss to the people living through it and looks like an inheritance to us.
So I think it is inevitable, and on balance good, provided we keep hold of the two things the method cannot supply for itself: deciding what is worth asking, and understanding the tests well enough to know when to distrust the answers.
What I haven't settled
When a person and a system find something out together, and the person understands only the test, who is it that knows? It might be the person, the system, or the two of them taken as one thing. If it is the last, then the question stops being about tools and starts being about what a person is. I come at it from the human side in the next musing, The Park and the Trail.
Sources
- Nunn, P. and Reid, N. (2016). Aboriginal Memories of Inundation of the Australian Coast Dating from More than 7000 Years Ago. Australian Geographer 47(1).
- Plato, Phaedrus 274c to 275b. Fowler translation.
- Royal Society. History of Philosophical Transactions.
- Appel, K. and Haken, W. (1976). Proof of the four-colour theorem. Summarised in Four color theorem.
- King, R. et al. (2009). The Automation of Science. Science 324.