Three principles… that’s all.

There are three simple principles1 necessary to ensure proper application of the logical approach to evidence evaluation. Yet, despite this elegant simplicity, some examiners feel the logical approach is too complicated, or confusing, to be used in our work.

Now, to be fair, real-world case things can be a bit complex and challenging from time to time , but that isn’t unique to the logical approach. It is simply a reflection of the nature of forensic document examination. Our work is difficult and challenging at the best of times and every properly trained document examiner must be able to deal with, and explain, complex topics to laypersons. It is the reason why forensic handwriting examination (FHE) expertise is necessary in a court of law.

When one applies the logical approach, that inherent complexity is more exposed… and that is a good thing.

In my opinion, a very strong argument can be made that the logical approach is simpler and easier than any other alternative, even in its most complicated form. That is certainly true once you get over the initial hurdle of comprehension. In my opinion, the real issue for examiners, and for others who argue against the use of the logical approach, is their lack of understanding coupled with a failure to appreciate what the logical approach does to clarify our reasoning processes, how it functions to guide those processes, and how it assists us in explaining and defending our opinions.

Perhaps most important of all, the logical approach demands a different view of the evidence and what it means. Ultimately, it also demands a different view of the role of the examiner. Hence, some people find it difficult to come to terms with the process and what it requires of them.2

I’m often asked for a simple overview or summary of the process — a short-hand description or ‘briefer’ version that explains the basics. Well, here it is…

The short version

The logical approach is a system of logical reasoning, first and foremost. The opinion that results will be a statement along the lines of “the findings provide more support for proposition X, rather than proposition Y,” accompanied by some estimate or indication of the strength of that differential support. In other words, the opinion speaks to the degree of relative support provided by a set of findings for one proposition, versus some other specified proposition (or, often, set of alternative propositions).

The main differences between this and the ‘traditional’ approach to conclusions are two-fold. First, the examiner focuses on that which they actually know — the evidence and what it means in the context at hand. As a result, the examiner can never (legitimately) be accused of over-stepping their authority by impinging on that of the court because they never speak to (probability of) the propositions directly. Second, the opinion scale itself has a clear and logical definition, though one of several depending upon the basis applied by the examiner — which, of course, should be disclosed.


In practical terms, the three basic principles that an examiner needs to remember and apply in order to use the logical approach to evidence evaluation are, as follows:

  1. Evaluation always occurs in a framework of information,
  2. Two or more competing propositions are required,
  3. The examiner must always speak to the probability of the evidence given the propositions, NOT the probability of the propositions.

These three outline the basic elements for proper and logical evidence evaluation. If an examiner keeps them firmly in mind, they will not go wrong. Of course, a thoughtful review of those principles shows there is a bit more going on here.

Any real-world process of evaluation is complex. As point #3 implies, all evaluation is probabilistic (i.e., uncertain) in nature, from which it follows that the assessment will be both conditional and personal. In addition, the process will vary according to the specific details of a given instance, at least for most types of evidence.

The question becomes how does each of these principles manifest in real-world application of the logical approach? It turns out there are many different ways to apply the approach, depending upon the nature of the evidence and the question asked of the examiner. This is clear when one reviews the ENFSI Guideline for Evaluative Reporting in Forensic Science which provides several nice (albeit non-FDE) examples.

Let’s look more closely at each of these three principles to make sure that the basics are clear.

Evaluation always occurs in a framework of information

This is not a particularly contentious idea though it tends to extend further than some examiners realize. Every case exists in some context and that overall context forms the basis of the framework.

In any given evaluation, framework information is, at a minimum, useful and, very often, necessary for proper evaluation. Put simply, when framework information is ‘relevant’ to the evaluation, it becomes necessary for a proper evaluation of the evidence. If the information is ‘irrelevant’, then it is not needed and should be avoided, if possible (or vetted out before the examiner sees or hears it).

An obvious concern with framework information is its potential to ‘bias’ an evaluation. Clearly, some information may affect an evaluation and skew the result away from the real answer, and that’s an outcome we want to avoid.3

Beyond this, avoiding any and all bias effect is very difficult (impossible?) for human beings simply because bias often works at a subconscious or subliminal level, for the most part. Different mechanisms have been proposed to address this issue. Most of them are aimed at ways of providing only relevant information to the examiner, while avoiding potentially biasing elements. However, the issue of how this is best done is not simple or trivial. In part, that’s because some types of ‘relevant’ information can also be potentially ‘biasing’.

Regardless of the solution used, it must include a directive to disclose whatever framework information was actually provided so that the trier is aware of any such issues in the evaluation.

A key point is the fact that framework is usually required to define the population of interest under the alternative proposition (at least, when that alternative invokes someone or something other than the main suspect in the matter). Framework is likely to include information that delimits or constrains the situation. As such, framework information can be critical to the evaluation process in those instances when it is both relevant and necessary for proper evaluation.

One final point is the reality that the examiner may not have access to framework information, even when it exists. When that’s the case the evaluation will necessarily be incomplete or, at least, constrained to the information that is available.

Two or more competing propositions are required

Evaluation of the findings in terms of a single proposition is possible, but the result does not, and cannot, address the probative value (i.e., weight) of the evidence. That is, any opinion addressing only a single proposition does not provide the trier with information needed to differentiate between possible theoretical (or proposed) causes. To achieve that end, the evaluation requires at least two competing propositions.

The traditional approach is similar in this regard, except that the process is much less transparent or clear. While the traditional approach is opaque in this regard, it is true that many (most?) examiners implicitly consider an alternative, although that is often a ‘standard’ (negative) option which may not be completely appropriate in any given case.

Using the logical approach, propositions should be clearly stated so they are completely clear and unambiguous. Any relevant sub-propositions should also be delineated for clarity.

The latent complexity in any evaluation, and in the ultimate opinion, derives mainly from this point. Consideration of sub-propositions add greatly to the practical complexity of the situation.

Always speak to the probability of the evidence given the propositions

And NOT the probability of the propositions. This is the fundamental core of the opinion and this approach, which recognizes the limit of what an expert can say about the situation while maintaining logical consistency and coherence.

The likelihood-ratio

How does all the above come together in practice? Through the likelihood-ratio.4 All three principles are neatly summed up in the “odds”-form of the formula for the likelihood-ratio, or $LR$, which is the basis for any conclusion produced using the logical approach. The complete odds form of the equation looks like $$\frac{p(E|H_{1},\textit{I})}{p(E|H_{2},\textit{I})}$$ Where $H_{x}$ are two competing propositions, $I$ is the framework of relevant, contextual info, and $p(E…)$ is the conditional probability of the evidence.

Principle 1 is captured in the $I$ element. Principle 2 is shown by the use of $H_{1}$ and $H_{2}$ (which is, in reality, a minimum condition).5 Principle 3 is seen in the structure of the numerator and denominator with each element describing a probability for the evidence given one of the two propositions of interest.

In practical terms there are many ways to evaluate and assign the relevant probabilities. There are also different ways to provide the resulting $LR$ (or $BF$) to the trier; including direct presentation of the numeric value, presentation of the individual probabilities, or some ‘verbal equivalent’ relating to the magnitude of the $LR$ (or $BF$).

In their 2009 paper, the Association of Forensic Science Providers outlined four elements that will ideally be fulfilled by any evaluation and reporting scheme. It’s interesting to consider how the above 3 principles relate to those 4 elements, which they described as follows:6

  • ‘Balance’ meaning that the evidence/findings should be evaluated given at least one pair of competing propositions; ideally with the first proposition based upon one party’s account of the events and the latter based upon some alternative account. This is seen in principle 2.
  • ‘Logic’ meaning that the evaluation process must be one that speaks to the probability of the evidence/findings given the propositions (plus relevant background information), and not the probability of the propositions given the evidence/findings (plus background information). This is essential to ensure there is no inappropriate or unjustified transposition of the conditional element. This involves both principle 1 and principle 3.
  • ‘Transparency’ meaning the entire process should be demonstrable and recorded so as to permit proper review and assessment. Both principle 1 and 2 comes into play here. The examiner must declare which propositions are used in the evaluation (whether there are two or more of them), and disclose any background/framework information known to them at the time of the examination.
  • ‘Robustness’ meaning simply that the evaluation process must be capable of sustaining scrutiny or review by other experts through review or cross-examination. This is best covered in principle 3 as the focus on the probability of the evidence requires an understanding of conditionality of that probability. All challenges to an examiner’s opinion are variations on the conditioning elements, and nothing more or less.

Clearly, these elements are inter-related. Robustness, for example, benefits from proper and complete disclosure, a requirement common to all forensic work. Examiners always want their work to be ‘robust’ and defensible. One of the nice benefits of the logical approach is that it makes that goal easier to attain and it does so while being overtly, and openly, balanced and logical. 

Basically, applying the logical approach properly makes it very easy to function in an open and thorough manner, disclosing all aspects of the process for review and discussion. For one thing, when an examiner applies the logical approach properly all aspects of the process will be clear to the examiner. That self-awareness permits disclosure and transparency. After all, one cannot disclose what one does not know.

Some Final PointS…

First, as it always the case, there are some exceptions to what I’ve outlined above.

The logical approach to evidence evaluation is appropriate for most types of examinations, but there are select situations where other forms of evaluation may be justified. The evaluation process outlined here does not necessarily apply for investigative, intelligence, or technical/factual examinations (and resulting reports).7

Second, the opinion that derives from this approach is straight-forward. In some instances and when appropriate, the examiner might just provide the $LR$ itself. However, some sort of “verbal equivalent” is usually provided instead.8 For that purpose, there are various choices with many conforming to the version laid out in the ENFSI Guideline document.

The wording of opinions is a matter of some debate at the present time and really a topic for discussion entirely on its own. However, this is a relatively minor issue so long as the examiner uses, and declares, some pre-defined scale that is clearly (and explicably) based on the $LR$ . That just means the examiner must disclose the basis and design of the opinion scale to the trier-of-fact so the latter understands what the opinion actually means and that’s something that should always be done anyways.

Footnotes

  1. One might also think of these as rules for the use of the approach.
  2. I will also say that some examiners are simply unwilling to even consider doing things differently than they ‘always have done’.
  3. Almost always the word ‘bias’ has a negative connotation with bias seen as negatively distorting the process.
  4. In many forensic domains the element that is actually used is more akin to a Bayes Factor, than a likelihood-ratio. But the two are functionally equivalent while requiring different explanation to ensure they are understood by the trier and used accordingly.
  5. The simple form involving only two competing propositions is used for simplicity only.
  6. Association of Forensic Science Providers, Standards for the formulation of evaluative forensic science expert opinion, Science & Justice, Vol 49, Iss 3, Sept 2009, pp 161-164, ISSN 1355-0306 (http://dx.doi.org/10.1016/j.scijus.2009.07.004)
  7. See the ENFSI guide for definitions of these types of reports.
  8. This is particularly the case when the probabilities are subjectively assigned based on the examiner’s knowledge, training, and experience which is most of the time in FDE work.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.