Probability. Robert P. Dobrow

empirical point of view, a probability matches up with our intuition of the likelihood or “chance” that an event occurs. An event that has probability 0 “never” happens. An event that has probability 1 is “certain” to happen. In repeated coin flips, a fair coin comes up heads about half the time, and the probability of heads is equal to one-half.

      Let be an event associated with some random experiment. One way to understand the probability of is to perform the following thought exercise: imagine conducting the experiment over and over, infinitely often, keeping track of how often occurs. Each experiment is called a trial. If the event occurs when the experiment is performed, that is a success. The proportion of successes is the probability of , written .

      This is the relative frequency interpretation of probability, which says that the probability of an event is equal to its relative frequency in a large number of trials.

When the weather forecaster tells us that tomorrow there is a 20% chance of rain, we understand that to mean that if we could repeat today's conditions—the air pressure, temperature, wind speed, etc.—over and over again, then 20% of the resulting “tomorrows” will result in rain. Closer to what weather forecasters actually do in coming up with that 20% number, together with using satellite and radar information along with sophisticated computational models, is to go back in the historical record and find other days that match up closely with today's conditions and see what proportion of those days resulted in rain on the following day.

      There are definite limitations to constructing a rigorous mathematical theory out of this intuitive and empirical view of probability. One cannot actually repeat an experiment infinitely many times. To define probability carefully, we need to take a formal, axiomatic, mathematical approach. Nevertheless, the relative frequency viewpoint will still be useful in order to gain intuitive understanding. And by the end of the book, we will actually derive the relative frequency viewpoint as a consequence of the mathematical theory.

1.3 PROBABILITY FUNCTION

      We assume for the next several chapters that the sample space is discrete. This means that the sample space is either finite or countably infinite.

      A set is countably infinite if the elements of the set can be arranged as a sequence. The natural numbers is the classic example of a countably infinite set. And all countably infinite sets can be put in one-to-one correspondence with the natural numbers.

      If the sample space is finite, it can be written as . If the sample space is countably infinite, it can be written as .

      The set of all real numbers is an infinite set that is not countably infinite. It is called uncountable. An interval of real numbers, such as (0,1), the numbers between 0 and 1, is also uncountable. Probability on uncountable spaces will require differential and integral calculus and will be discussed in the second half of this book.

      A probability function assigns numbers between 0 and 1 to events according to three defining properties.

      PROBABILITY FUNCTION

      Given a random experiment with discrete sample space , a probability function is a function on with the following properties:

      1 

      2 (1.1)

      3 For all events ,(1.2)

You may not be familiar with some of the notation in this definition. The symbol means “is an element of.” So means is an element of . We are also using a generalized -notation in Equations 1.1 and 1.2, writing a condition under the to specify the summation. The notation means that the sum is over all that are elements of the sample space, , that is, all outcomes in the sample space.

      In the case of a finite sample space , Equation 1.1 becomes

      And in the case of a countably infinite sample space , this gives

      In simple language, probabilities sum to 1. The third defining property of a probability function says that the probability of an event is the sum of the probabilities of all the outcomes contained in that event. We might describe a probability function with a table, function, graph, or qualitative description. Multiple representations are possible, as shown in the next example.

       Example 1.5 A type of candy comes in red, yellow, orange, green, and purple colors. Choose a piece of candy at random. What color is it? The sample