It appears to be an empirical fact that if an experiment is continually repeated under the exact same conditions, then for any event E, the proportion of time that the outcome is contained in E approaches some constant value as the number of repetitions increases.
For instance, if a coin is continually flipped, then the proportion of flips resulting in heads will approach some value as the number of flips increases. It is this constant limiting frequency that we often have in mind when we speak of the probability of an event.
From a purely mathematical viewpoint, we shall suppose that for each event E of an experiment having a sample space S, there is a number, denoted by P(E), that is in accord with the following three axioms:
Axiom 1:
The first axiom states that the probability that the outcome of the experiment is contained in E is some number between 0 and 1. 0\leq P(E)\leq 1.
If the probability of the occurrence of an event is equal to 1 we say that the event is a sure event and if the occurrence of the event is impossible then the probability of the event is equal to 0. According, to the first axiom the probability of occurrence of any event is a non-negative real number less than or equal to 1.
Axiom 2:
Let S denote the sample space then we have that P(S) = 1. The second axiom states that with probability 1, the outcome will be a member of the sample space S. This axiom is also known as the axiom of certainty.
Axiom 3:
The third axiom is known as the axiom of additivity. It states that for any set of mutually exclusive events the probability that at least one of these events occurs is equal to the sum of their respective probabilities.
For any sequence of mutually exclusive events E_1,E_2, \ldots(that is E_i\cap E_j = \emptyset when i\neq j) we have that , P(\cup_{i=1}^{n} E_i) = \sum_{i=1}^{n}P(E_i)\text{ where } n= 1, 2, \ldots, \infty.
Probability of an Event as Relative Frequency of Occurence:
It should be noted that if we interpret P(E) as the relative frequency of the event E when a large number of repetitions of the experiment are performed, then P(E) would indeed satisfy the above axioms. For instance, the proportion (or frequency) of time that the outcome is in E is clearly between 0 and 1, and the proportion of time that it is in S is 1 (since all outcomes are in S). Also, if E and F have no outcomes in common, then the proportion of time that the outcome is in either E or F is the sum of their respective frequencies.
Two consequences of the probability axioms:
Two corollaries that follow straightforwardly from the above three axioms are:
- Let E^c denote the complement of an event E. The complement rule states that, P(E^c) = 1 - P(E).
- If E and F are two events then we have that, P(E\cup F) = P(E) + P(F) - P(E\cap F).
