## Probability

I have struggled with the concept of probability for a long time. Not with the maths but with the meaning. Does a probability number really mean anything at all? And if so, what? Recently, I have reached a definite position on this. Here it is.

In a metaphysical sense, it seems clear that the probability number is meaningless. Or, more accurately, that it is not a property of the event in question at all (I am using the word ‘event’ loosely to refer to anything for which a probability may be calculated). An event either occurs or does not occur. No fractions are possible. Probability therefore must be a measure of a person’s state of knowledge of the factors that determine the event in question. That is, probability is an epistemological concept rather than a metaphysical one. It originates because of the need to make choices in the face of incomplete knowledge. This is clear since probability is used not just for future events but also for past ones. A classic example of this is the use of medical tests in conjunction with statistical analyses to arrive at a probability of a patient’s having a particular disease. In reality, either the person has the disease or not. The probability assigned to the possibility of disease is merely a tool used to decide whether further investigation is warranted.

This seems to suggest that probability is a subjective rather than an objective. But the precise math used to calculate probabilities suggests otherwise. Is probability subjective or objective? To answer this, it would be useful to look at what the words subjective and objective mean. In a comment on an old post, Burgess Laughlin wrote (and I agree):

“Objective,” in my philosophy (Objectivism), has two meanings. First, in metaphysics, it means existing independent of consciousness. The redundant phrase “objective reality” captures this meaning. Second, in epistemology, “objective” refers to knowledge that is drawn (inferred) logically from facts of reality. (See “Objectivity,” The Ayn Rand Lexicon.)

Subjective, as I use the word, refers to judgements or responses that cannot be traced back to facts of reality or the thought processes of the subject (A typical example is emotions).

Probability is not objective in the metaphysical sense. In fact, without consciousness, it would not exist at all. It is also not objective in the epistemological sense since it arises only in cases where the subject does not have complete knowledge of the facts of reality. And the fact that there are precise mathematical rules to calculate probabilities means that probability is not subjective either. If probability is neither objective nor subjective, what is it? Consider the case of a coin being tossed. Lacking any knowledge of the composition and weight distribution of the coin, the velocity with which it was tossed, the composition of air, the nature of the ground etc, the probability of a particular side showing up is taken to be 0.5. Where did this number come from? It is quite clear that this choice is purely arbitrary. The entire math of probability is based on a simple principle applied consistently. Given multiple possibilities and a complete lack of quantitative knowledge of relevant causes, each possibility has an equal probability. Clearly this is arbitrary, but it is the best one can do. And applied consistently, it provides a very precise framework for quantifying a lack of knowledge. It allows quantification of that which we do not even know!

Anyone who is familiar with Rand’s philosophy should note that my use of ‘arbitrary’ is different from (though related to and inspired from) Rand’s use of the word in the classification of the epistemological status of statements as true, false or arbitrary. To apply the principle of equal likelihood, one already needs to have identified all the possibilities. This means that probability cannot be applied to arbitrary (in Rand’s sense) assertions. What about the truth status of a statement involving probability? Such a statement can be demonstrated to be true (or false) subject to the equal likelihood principle. Without the principle, it is arbitrary (in Rand’s sense).

I think this classification – objective, subjective and arbitrary – might be useful in several other areas of math as well. For example (I need to think more about this though), it can be applied to Euclid’s axioms (in geometry). These axioms could be described as arbitrary and theorems could then be considered as true subject to the axioms.

In my next post, I will try to relate my position to statistics and randomness.

### 10 Responses

1. K.M., this is a very interesting post, and I especially like your treatment of the metaphysics – that a single event has a particular outcome irrespective of any probability assignment.

I do think, though, that mathematical probability is epistemologically objective if it is regarded in the proper context. Perhaps I can convince you of this if I explain. It might clear up the apparent conflict you identified of something that has precise mathematical rules somehow not being objective.

You asked, after pointing out that the probability of a fair coin showing heads is 1/2, “Where did this number come from?” The choice of 1/2 is not arbitrary – not even in the sense you described distinguishing it from “anything goes” arbitrariness. Probability is an empirical measurement of an ensemble of events. It means: Given a set of N independent events, the probability of a specific event is, to a degree of certainty, the number of times the specific event occurred divided by the total N, as N becomes large. By “a degree of certainty,” it is meant simply that the uncertainty in the measurement can be made smaller and smaller by increasing N. (Since this is an inductive process, it has the characteristics of induction, including the requirement of objectively determining when N is large enough to achieve certainty of the probability measure.)

Thus, probability is not primarily about prediction (though the concept of “expected value” follows naturally and is generally the motivation of the measurement), nor (as you pointed out) is it an attribute of an event considered apart from an ensemble. It is primarily a method of measurement, a means of condensing the nature of complex phenomena into tractable parameters, such as mean, variance, etc. I see nothing non-objective about this. On the contrary, like concept formation in general, it has a huge epistemological payoff. I cannot think of a piece of modern technology that does not make practical use of mathematical probability in grasping certain aspects of reality. As a circuit designer in the communications industry, I could not begin to analyze a system if I had to consider every instantaneous physical influence upon each bit of information as it traveled from the power amplifier, through a transmission medium, to the receiver.

Of course, people might use the concept of probability in corrupt or arbitrary ways, but this does not negate its validity.

By the way, your post has prompted a new question in my mind that I’ll have to think a lot more about: Can probability theory ever be applied meaningfully to the actions of organisms with free will? I’m inclined to say outright, “Decidedly not!” though I can immediately come up with several trails of thought that I would need to pursue to the end in order to completely formulate and delimit my answer. In any case, let me leave the “events” in my probability definition to be restricted to processes of inanimate matter and biological processes not involving free will.

2. Stephen,
Thanks for the detailed comment. I am going to be out for a few days. I will reply once I am back next week.

3. KM,

I buy the idea that the idea of probability is not metaphysical; I agree with Steve that it is objective in an epistemologically sense though.

Am interested in knowing where you are going with this.

Anup

4. Hi Stephen,
I have written a followup post here in which I argue that even your statistical-empirical definition of probability contains an irreducible arbitrary assumption of equal likelihood.
Of course, probability has a lot of practical uses, primarily through statistics, but I will argue in a later post that those uses are somewhat of a shortcut rather than being examples of “pure” probability.
Application of probability to actions involving the exercise of free will is something I too will have to think over. My initial position is that since applications of probability require an irreducible arbitrary choice, it should be possible to apply it meaningfully even in those cases.

5. “This seems to suggest that probability is a subjective rather than an objective. But the precise math used to calculate probabilities suggests otherwise. Is probability subjective or objective?”

This can be tackled in a better way, if we think of probability as a set of rules to handle the updating of one’s belief. In a Bayesian framework, one starts with a prior belief and updates it as more information arrives. The rules of updating are objective, while the initial belief itself is subjective. It is the latter that I categorize as statistics and the former as Probability.

Why should one follows the rules of Probability to update the beliefs? well, the easy answer would be to take a formalist posture, and just say these are the rules we chose, and they seem useful. But even otherwise, it can be shown that if we put in some conditions of consistency and rationality, then we should obtain Bayesian updating.

6. “[Probability] originates because of the need to make choices in the face of incomplete knowledge. This is clear since probability is used not just for future events but also for past ones.”

Perhaps very much of the problems regarding the interpretation of probability originates from the attempts at applying probability to past events. In my opinion this is wrong and very dangerous. Probability can only be used about the future. It is meaningless to talk about the probability that this or that happened. What one actually then talks about is the probability that one will find out that this or that happened.

Probability is clearly an epistemological concept, but that does not make it arbitrary or subjective. As pointed out by Krishnamurthy:

“The rules of updating are objective, while the initial belief itself is subjective.”

However, this subjectivity is not as arbitrary as many hard-liner frequentists may think. In my field of research (target tracking), a prior distribution has to be assigned to the movement pattern of the target. How else should one quantify the obvious insight that the target most likely (although not certainly) is going to continue to move straight ahead? Parameters of this a priori distribution can, at least in retrospect, be determined perfectly objectively.

7. […] subjectivity of probability K. M. has an interesting post on the subjectivity of probability.  I argued along these lines in my last job.  We were doing statistical inference and other kinds […]

8. Starting with Bayes’ Theorem versus starting with the rule of conditional probability, though they may produce the same number in the end, are distinct processes with distinct meanings. Further, the distinction that is clear to me is this: probability implies a description that is true across all spacetime. Conditional probability does not.

http://matthewvantemple.wordpress.com/2010/02/01/the-subjectivity-of-probability/

9. Probability is nothing more than a measure of the degree of confidence that we can have / put into the likelihood of an event.

That is why there are two major schools of thought on probability, two distinct perspectives: the Bayesian and the Frequentists approach. The former looks at the variables which are known, the ones that can be measured or computed (such as the number of sides on a die, the weight distribution of a die, etc)… the latter rejects the variables that can be known but only pays attention to the recorded history (how the die has landed thus far, the distribution we have observed). A Bayesian meteorologist, for example, looks at barometer pressure and temperature while a Frequentist meteorologist looks at “this day in history”. Both use what they deem matters most in order to assess the scenario, in order to make estimations about future events. Neither perspective is wrong and both have their place.

Ultimately, though, probability is not an intrinsic quality of an event. The intrinsic qualities of reality affect the variables which influence randomness and outcome, but that is all we can say.

To say that probability is a “measure of likelihood” is a bit false, I think. Its more proper to say that it is a “measure of the degree of confidence we humans can place in the likelihood”.

Look at it from a betting perspective. If some event has a probability of 10% (1/10) and you placed bets each time we tested the event… and if you lose \$1 every time the event doesnt happen but you gain \$10 each time the event does happen… then in the long run you will break even. It is what we call a “Fair Game”. By adjusting the betting buy-in/pay-out rates such that you guarantee a long run of \$0, you can more confidently determine the likelihood of an event. That “confidence” is the probability of success.

10. I find it “distasteful” and blatantly self-contradictory that you give the word “objective” two simultaneous meanings. Which is it? How can the definition of the word “objective” be something you arbitrarily decide upon in context? By “subjectifying” the definition of objective, you make the word moot and trivial; and ultimately you fail to capture the meaning of the word.

I am an objectivist, myself, and I can easily find a single definition to fit all contexts. When and if you arrive at such a definition, that is objective. And if you fail to arrive at a single unifying definition then you havent quite yet reached an objective truth.

You cannot call yourself an objectivist if you accept multiple simultaneous but distinct answers to a single question.

Im sorry, but anyone who agrees with you on this (not barring you yourself, and no insult intended) is simply deluded on the whole philosophical perspective. Ayn Rand is included in that criticism – her perspective is insulting to anyone who calls themselves rational.

What is objectively true is true independent of human will. It is true independently of consciousness and it is true independently of logical reasoning.

Indeed, I accept the possibility that some truths are still true even if logic says otherwise, because logic is a human construct, having no foundation, built around a comfort-feeling derived from what human neuro-physiology accepts on faith as intuitive and first principally obvious. Nothing outside of this comfort feeling is proof of truth. I accept that if (hypothetically) this universe was intended to fool us then truth exists outside of all human capacity to ever know through empiricism or logic.

What is true is true independent of human will, human reason, human desire. The phrase “objective truth” is itself a redundant statement, because there are no relative truths. Relative truths are nothing but delusions, deceptions and lies that we accept in good faith as a truth. Our societies amazingly unwaveringly capacity to deem any truth as a “relative” one says more about our prideful arrogance and reluctance to accept that we are wrong than it does about any civilized realization.

“Tolerance for diversity” in truth is “fear of persecution” twisted and perverted… we tolerate only because we expect to be tolerated ourselves… with no regard for what is or is not true.