The concepts of ‘prior odds’, a.k.a., prior probabilities or simply priors, and ‘posterior odds’ come up in most discussions about the evaluation of evidence.  The significance and meaning of both terms becomes clear when viewed in the context of a “Bayesian approach”, or the logical approach, to evidence evaluation.  That approach has been discussed at length elsewhere and relates to the updating of one’s belief about events based upon new information.  A key aspect is that some existing belief, encapsulated as the ‘prior odds’ of two competing possibilities or events, will be updated on the basis of new information, encapsulated in the ‘likelihood-ratio’1 (another term you will undoubtedly have seen), to produce some new belief, encapsulated as ‘posterior odds’ about those same competing possibilities.

But what precisely do these terms, ‘prior odds’ and ‘posterior odds’, mean and how do they relate to the work of a forensic examiner?

In simple terms the odds of any two competing events, $E_{1}$ and $E_{2}$, can be written as a ratio of probabilities, as follows:
$$odds = \dfrac{p(E_{1})}{p(E_{2})}$$
where $p(E_{x})$ refers to the probability of event $E_{x}$.2  Odds consist of a single scalar value, ranging from 0 to +infinity, because it is a ratio of two probabilities, each ranging from 0 to 1.   Odds are generally expressed in relativistic terms.  For example, odds of 2 to 1 (or 2:1), mean that $p(E_{1})$ is two times $p(E_{2})$.  Alternatively, odds of 1000 (or 1000 to 1; or 1000:1) means $E_{1}$ is 1000 times more likely to be true than $E_{2}$.

At any given point in a decision-making process it is possible to describe the state-of-mind of the decision-maker in terms of odds such as these.  Please note that whether or not the decision-maker can actually express their belief in such terms is irrelevant to the matter since this form can still be used to describe that belief.  Basically, at any given point the decision-maker will have a balance of belief about the two events; it may be equal for both, or it may ‘favour’ one event or the other based upon whatever information they have been given or know at that point in time.

If one thinks about this formula it is easy to see that the odds reflect an underlying probabilistic belief about the two competing events. In practical terms the odds will favour one of these two events over the other depending upon the relative probability of each being true (or happening).  The terms ‘prior’ and ‘posterior’ mean simply ‘before’ and ‘after’, as suggested earlier. The thing that comes between them — temporally speaking — is some new information about those events.

That new information takes another, slightly different, form.  There are almost always multiple potential ‘explanations’ for this type of information.  In general, there is never a single cause or source that is absolutely guaranteed to produce the information.  Rather that information, the evidence, may occur under any number of possible scenarios — with the ones of interest are those being argued in court.

If the prior odds reflect (a temporary) state of belief in the mind of the trier, then how can an examiner use that information in their own assessment?  The answer is simple, they can’t.

This is one of the key reasons why I, and many others, believe the focus of any forensic work should be on the determination of a likelihood ratio ($LR$) which relates to the support provided by the evidence for each of the two competing propositions.  The $LR$ is usually described in terms of Bayes Theorem — being the part of the equation used to update prior belief to form some new, posterior belief.3 Using mathematical symbols, the complete odds form of the equation, with the ‘components’ labelled, looks like this:
$$ \underbrace{\dfrac{p(H_{1}|E,\textit{I})}{p(H_{2}|E,\textit{I})}}_{\text{Posterior Odds}}= \underbrace{\dfrac{p(E|H_{1},\textit{I})}{p(E|H_{2},\textit{I})}}_{\text{Likelihood Ratio}} \cdot \underbrace{\dfrac{p(H_{1}|\textit{I})}{p(H_{2}|\textit{I} )}}_{\text{Prior Odds}} $$

In this formula we can see the relationship between the prior odds, the $LR$ (belief about the evidence), and posterior odds.  However, it doesn’t matter if Bayes Theorem is used, or if some other approach is taken in the real-world.4  It is, however, important that whatever approach is used should possess certain traits (discussed more fully elsewhere).

The formula also discloses another aspect of the odds, in general.  Both sets of odds in the formula (and the $LR$ as well) are based on conditional probabilities.  Prior odds, $p(H_{1}|\textit{I}) / p(H_{2}|\textit{I} )$, are conditioned by relevant framework information, $\textit{I} $.   The $LR$, $p(E|H_{1},\textit{I}) / p(E|H_{2},\textit{I})$, is conditioned by the propositions of interest,$H_{1}$ and $H_{2}$, and that same framework information.  And the posterior odds, $p(H_{1}|E,\textit{I}) /p(H_{2}|E,\textit{I})$, end up being conditioned by the evidence, $\textit{E}$ , and the framework information.

It is also important to note that, in the forensic context, we aren’t usually working with real odds at all.  There are various reasons but the main one is that the set of events is not necessarily (in fact, rarely) exhaustive or complete.

Is it possible to estimate priors to permit a more complete application of the Theorem?  In theory, yes.  Some have advocated for this with the idea that it may help the trier to understand how the $LR$ could affect their belief (by providing various priors and explaining the effect the $LR$ would have on each of those).5  This approach is feasible but I would argue that it could be very misleading if not done carefully.

In some situations the examiner may also have knowledge or information that could be used to inform prior belief in some legitimate manner.  For example, they may have information about the frequency with which one might expect to encounter a particular type of printer in a given geographic region.  That type of base-rate information might be helpful to the trier but, in my opinion, it would be best way to provide it in some manner independent of the other evidence.

It has also been suggested that examiners could apply the concept of “equal prior odds” based on the idea that this produces a fair and unbiased starting point.  In that approach equal weighting is assigned to each of the probabilities involved.6  This is a bad idea.  The approach is neither fair nor unbiased as explained in detail by Taroni and Biedermann.7 The issue is quite complicated but one problem, for example, is the inherent implication that the population of possible perpetrators consists of only two people, one of whom is the suspect.  Such a belief is generally illogical and patently unfair if applied without anything to support this contention.

In the end the focus of any forensic work should be on the determination of a likelihood ratio ($LR$) that relates to the support provided by the evidence for each of the two competing propositions.  That is, the weight of the evidence that should be presented to the trier for their consideration and incorporation with other evidence in the matter.  Precisely how that should be communicated is a matter for further discussion.8

Footnotes:

  1. Or, very often in a forensic science context, some non-numeric construct that serves the same purpose rather than a numerically-informed likelihood-ratio.
  2. As discussed below the actual probabilities of interest in our application are a bit more complicated than this.  But the concept remains the same.
  3. It should be noted that the $LR$ is not solely a facet of Bayes Theorem.  It is an independent concept and there is no real reason to invoke Bayes Theorem to understand or use the $LR$. At the same time this is how it has been done in most of the forensic science literature to date.  For an extended discussion of this topic, see DH Kaye.  Likelihoodism, Bayesianism, and a Pair of Shoes. Jurimetrics, Fall 2012.
  4. In addition, the construct need not be a formal $LR$to work in this manner. It can be, and usually is, some expression (e.g., Bayes Factor or something similar) that is functionally equivalent to the $LR$. The key is that it serves a specific purpose and is expressed in a specific manner.
  5. See, for example, R. Meester and M. Sjerps. Why the effect of prior odds should accompany the likelihood ratio when reporting DNA evidence. Law, Probability and Risk (2004) 3, 51–62.  Their discussion focused on DNA evidence but much of it can apply to any type of evidence.
  6. See Köller et al. Probability Conclusions in Expert Opinions on Handwriting.  Substantiation and Standardization of Probability Statements in Expert Opinions. (München, 2004) – available online (this version is in both German and English).
  7. F. Taroni and A. Biedermann. Inadequacies of Posterior Probabilities for the  Assessment of Scientific Evidence. 4 Law, Prob. & Risk 89 (2005)
  8. For guidelines in that regard see the ENFSI Guideline for Evaluative Reporting in Forensic Science, v 3.0 (2015).  Available from http://enfsi.eu/wp-content/uploads/2016/09/m1_guideline.pdf.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.