Bayesian vs. Frequentists View
2. Bayesian vs. Frequentists ViewΒΆ
In probability theory and statistics two different approaches are popular: the Bayesian and the frequentist interpretation of probability. The difference results basically from the philosophical question: What is probability? This topic is nicely explained in the following video:
These two points of view can be summarized as follows:
Bayesian perspective:
Probability is a reasonable expectation depending on the state of knowledge
Expresses a degree of belief
Makes updates of the probability sequentially in use of prior belief and additional data/evidence
Frequentists perspective:
Probability exists independently from an observer
Probability of an event is the limit of its relative frequency in infinitely many independent random experiments
In each experiment the event either occurs or not (similarly, a hypothesis is either true or not)
By assumption, a repeat of the experiments would in the limit result in the same probability
Hence, in frequentism probability is an objective absolute quantity which can be estimated by observations. In contrast, probability is subjective in the Bayesian approach and expresses a quantification of belief which is concluded from observations.
Both approaches are very useful and have their advantages as well as disadvantages. Clearly, frequentism reflects more the principles of natural sciences like reproducibility and objectivity. Moreover, it is mostly used in basic courses on statistics. Nevertheless, Bayesian probability reflects the way humans assign probabilities to events: based on prior knowledge and additional evidence.
In the following section, we illustrate the difference in use of the coin toss example.