Cooperative Epistemology

Originally published May 27, 2016

 The mathematical theory of instrumental rationality comprises two interrelated disciplines: decision theory and game theory, i.e., individual and group optimization. These are fields for which we can say that both exist. By contrast, epistemic rationality is given its coherent mathematical treatment only concerning individual knowledge. The study of group rationality is scattered to the winds, a field so small it as yet has no name. It could be called interpersonal epistemic rationality, multi-agent epistemology, interactive epistemology, or, preferably, something more clever. I title this post cooperative epistemology because that is the ideal I want the field to be about.

Distinction-Mongering
It will prove helpful to distinguish between the formal theories of rationality, the theory of rationality in practice, the practice of rationality, and the practice of theorizing about rationality. We can further subdivide these chunks into epistemic and instrumental by prepending the respective adjective to every instance of 'rational'. I do this as a rule, but the reason I'm inflicting such onerous distinctions on you is because the theoretical study of the practice of group epistemic rationality, by philosophers and psychologists, gets plenty of attention and is given its names. What's comparatively neglected is the mathematical theory of interpersonal epistemology.

Thus social epistemology is a cool field I aim to pay attention to, but is the "Robert Nozick investigating whether induction is justified" of group rationality, I am seeking more things like Aumann's Agreement Theorem.

Aumann's Agreement Theorem
is so famous and well-known in my circles that it needs no introduction here. Nevertheless I want to consolidate some of the last many years of discussion about it.

In the beginning, Robin Hanson and Tyler Cowen coauthored a paper about how disagreements are, therefore, either irrational or dishonest.

Nevermind
I had ambitious plans for a much longer post, but I don't feel like writing this one anymore, so I'm going to truncate it here and publish it. The main upshot was probably going to be something about how Wei Dai continues to be and have been the single best contemporary thinker.

Edit:
there are so many surprising ways in which the epistemic and the instrumental are the same. i sought to articulate them all, and i failed. here's one. after posting this i remembered that this was my main motivation for wanting more mathematical theory of group epistemic rationality, and ipso facto the main reason for attempting to write this. because while it is easy to draw astonishing parallels between decision theory and the study of knowledge, to such an extent that they're inextricable, we have no such body of parallels for, say, game theory and belief-sharing. i was also thinking that epistemic selfishness/competitiveness is even worse than instrumental. well, bye

12:33:46 actually there is a ton of literature about aumann's agreement theorem
12:33:59 judea pearl also did some work on combining evidence from multiple agents
12:34:16 there's the result that it's better to share likelihood ratios than posterior probabilities
12:34:40 discussion about how, on the other side of that coin, it's easier to know what your posterior probabilities are than what your likelihood ratios are
12:35:14 sensor fusion is a relevant subfield of AI (combining data from different places into the model)
12:35:22 I should have mentioned all of these
12:35:42 wei dai wrote LW posts about group rationality (practice) and about aumann's agreement theorem (mathematical theory)