Originally published May 27, 2016
The mathematical theory of instrumental rationality comprises two
interrelated disciplines: decision theory and game theory, i.e.,
individual and group optimization. These are fields for which we can say
that both exist. By contrast, epistemic rationality is given its
coherent mathematical treatment only concerning individual knowledge.
The study of group rationality is scattered to the winds, a field so
small it as yet has no name. It could be called interpersonal epistemic
rationality, multi-agent epistemology, interactive epistemology, or,
preferably, something more clever. I title this post cooperative
epistemology because that is the ideal I want the field to be about.
will prove helpful to distinguish between the formal theories of
rationality, the theory of rationality in practice, the practice of
rationality, and the practice of theorizing about rationality. We can
further subdivide these chunks into epistemic and instrumental by
prepending the respective adjective to every instance of 'rational'. I
do this as a rule, but the reason I'm inflicting such onerous
distinctions on you is because the theoretical study of the practice of
group epistemic rationality, by philosophers and psychologists, gets
plenty of attention and is given its names. What's comparatively
neglected is the mathematical theory of interpersonal epistemology.
Thus social epistemology
is a cool field I aim to pay attention to, but is the "Robert Nozick
investigating whether induction is justified" of group rationality, I am
seeking more things like Aumann's Agreement Theorem.
Aumann's Agreement Theorem
so famous and well-known in my circles that it needs no introduction
here. Nevertheless I want to consolidate some of the last many years of
discussion about it.
In the beginning, Robin Hanson and Tyler
Cowen coauthored a paper about how disagreements are, therefore, either
irrational or dishonest.
I had ambitious plans for a much
longer post, but I don't feel like writing this one anymore, so I'm
going to truncate it here and publish it. The main upshot was probably
going to be something about how Wei Dai continues to be and have been
the single best contemporary thinker.
are so many surprising ways in which the epistemic and the instrumental
are the same. i sought to articulate them all, and i failed. here's one.
after posting this i remembered that this was my main motivation for
wanting more mathematical theory of group epistemic rationality, and
ipso facto the main reason for attempting to write this. because while
it is easy to draw astonishing parallels between decision theory and the
study of knowledge, to such an extent that they're inextricable, we
have no such body of parallels for, say, game theory and belief-sharing.
i was also thinking that epistemic selfishness/competitiveness is even
worse than instrumental. well, bye
12:33:46 actually there is a ton of literature about aumann's agreement theorem
12:33:59 judea pearl also did some work on combining evidence from multiple agents
12:34:16 there's the result that it's better to share likelihood ratios than posterior probabilities
discussion about how, on the other side of that coin, it's easier to
know what your posterior probabilities are than what your likelihood
12:35:14 sensor fusion is a relevant subfield of AI (combining data from different places into the model)
12:35:22 I should have mentioned all of these
12:35:42 wei dai wrote LW posts about group rationality (practice) and about aumann's agreement theorem (mathematical theory)