When an examiner expresses an opinion along the lines of ‘the findings support one proposition over another proposition’, a common question is whether this is means, ‘it is more likely than not that the favored proposition actually happened’?1 Strictly speaking, the short answer to the question is “no, it is inappropriate because these two statements are not equivalent.” In order to conclude the latter, one must consider information that goes beyond the FDE evidence. As a rule, any opinion I provide will be constrained to the probability of the findings/observations in terms of one of at least two possible explanations.2

What opinion scale do you use?
My opinion scale is provided with every report to help ensure it is properly understood. I use the logical approach to evidence evaluation, and that approach dictates a certain type of opinion wording for most cases.  Or, rather, the opinion flows directly out of the evaluation process with the wording being simply the logical expression of the outcome of that process. The following is the information I provide:
The examiner’s opinion is intended to convey the degree of support provided by the observed evidence (i.e., findings) for a specific proposition versus one or more specified alternative propositions. It is a statement of the examiner’s belief about the relative support provided by the findings for one proposition versus at least one other. The processes outlined above will result in one of several possible opinions. The opinion is expressed in qualitative terms that relate to the magnitude of the degree of support provided, using a pre-determined (quasi-logarithmic) scale, as follows:
  • “The evidence provides approximately equal support for proposition X and proposition Y” meaning the evidence does not differentiate between the propositions and the probability of the evidence given each proposition is roughly equal (e.g., both are low; both are high). This is an ‘inconclusive’ result expressed when the findings do not differentiate the propositions generally due to significant limitations in the material;
  • “The evidence provides more support for proposition X than for proposition Y and the level of that support, while stronger for X than for Y, is limited or weak” meaning the evidence favours one proposition (X) over the other (Y), but only slightly; or
  • “The evidence provides <modifier> support for proposition X over proposition Y” meaning that the evidence favours proposition X over Y to the degree indicated by <modifier>; with the <modifier> being one of the terms “moderate”, “moderately strong”, “strong”, “very strong”, or “extremely strong”.
It is important to note that either proposition, X or Y, may be presented first in any of the above phrases depending upon which proposition is better supported by the evidence. Additionally, it is important to note that all opinions expressed in a report are based upon relevant background information and specific exhibits provided to the examiner, as well as the proposition set utilized in the evaluation and as laid out in the report. As a result, should the information, exhibit material, or propositions change, the opinion may also change. In particular, if different propositions are of interest, please contact the examiner to discuss the matter further. Finally, also note the evaluation process is based upon a personal assessment by the examiner of the relevant probabilities that apply in a given situation with that assessment deriving from the examiner’s training, ongoing professional development, and experience, including proficiency testing of related skills.
As additional clarification, the following table details the underlying meaning of each verbal modifier based upon relevant likelihood ratio (LR) values (i.e., weight of the evidence).  The evaluation determines the LR (or Bayes factor) and this table is then used to determine/assign an appropriate verbal opinion. This is a convention based on the ANZPAA NIFS and ENFSI guidelines. Table: Opinion Scale using Verbal Terms (LR concordance)
Equivalent Likelihood ratio values (LR Opinion wording – from evaluation of evidence
> 100,000 Evidence provides extremely strong support for proposition X over proposition Y.
10,000-100,000 Evidence provides very strong support for proposition X over proposition Y.
1,000-10,000 Evidence provides strong support for proposition X over proposition Y.
100-1,000 Evidence provides moderately strong support for proposition X over proposition Y.
10-100 Evidence provides moderate support for proposition X over proposition Y.
2-10 Evidence provides more support for proposition X than for proposition Y and the level of support, while stronger for X than for Y, is limited/weak(See note 1 below)
1 Evidence provides roughly equal support for proposition X and proposition Y.
Note 1:  This wording for evidence that does not strongly differentiate between propositions is necessary to avoid or minimize any weak evidence effect. The same format can be used for all opinions. 
There are other forms of opinion or conclusion wording, but I do not consider them to be logically sound or justified so, as a rule, I do not use them.
  1. The opinion might be expressed with a strength that varies from limited/weak to extremely strong (link to opinion scale), but the strength does not matter.
  2. There are select types of examinations that permit an opinion about the propositions but those are very rare.

Most people think that all examiners use a standardized opinion scale. To a degree that is true, but the reality is a bit more complicated. I have discussed this elsewhere in more detail (here and here among other places) and I do not intend to repeat all of the history or discussions that have happened over time. Suffice it to say that there are two different approaches generally taken by examiners; one embodied in the SWGDOC Standard Terminology for Expressing Conclusions of Forensic Document Examiner and the other that conforms to the ENFSI Guideline for Evaluative Reporting (or something like that recommended and endorsed by the National Institute of Forensic Science in Australia (ANZPAA NIFS), or the Document Section of the Canadian Society of Forensic Science).

The former (SWGDOC) approach is the historical/traditional approach found in many labs wherein an examiner does their evaluation and formulates an opinion that speaks to the probability of a particular proposition; that is, it is a statement of belief about what actually happened to cause the observed evidence/findings.

The latter (ENFSI) approach has, in recent decades, been recognized as being more logically sound in that it constrains the resulting opinion to a statement about the relative probability of the observed evidence/findings if one of several (at least two) competing propositions is true. This approach does not speak directly to whatever may have actually happened to produce the evidence/findings; only to the degree of support provided by the evidence/findings in favour of one propositions relative to another (or others).

Without going into detail, there are a few inter-related reasons why, in my opinion, the latter is preferable.

First, the examiner clearly has knowledge that permits an opinion about the evidence, but they lack certain requisite knowledge or information required to convert that belief into one about what happened. That additional requisite knowledge relates to other aspects of the matter that lie outside the examiner’s domain of expertise.

Second, that type of knowledge is known to other parties involved; specifically, the trier of fact (and, to a lesser degree, the counsel or lawyers involved in making arguments to the trier). Furthermore, it is the role and responsibility of the trier to make a determination about ‘what happened’ as they decide about issues of guilt/innocence, or liability.

Third, it is the examiner’s role (as an expert) to assist the trier in their determinations by explaining and clarifying the meaning and value of evidence that falls within the examiner’s domain. However, the examiner must never present their opinion or belief in a manner that is overly influential in the proceedings. Their role is to assist the trier, not usurp the latter’s role by making statements about ‘what happened’.1

This distinction leads to the question from clients that appears at the start of this post.

Does an opinion along the lines of ‘the findings support one proposition over another proposition’ mean that ‘it is more likely than not that the favored proposition actually happened?’ They ask this because they 1) may never have encountered an opinion based on evaluative reasoning (what I call the ‘logical approach’), and 2) think that an examiner should be able to tell them ‘what happened’ in a given matter.

To formalize the situation, a properly formed and logically sound opinion speaks only to the findings and the probability of observing the findings under each of at least two propositions. Any opinion that speaks to the probability of the propositions (singly or in contrast to one another) is very different.

Or, more accurately, I don’t know anything about ’what happened’ because 1) I don’t have all the requisite information needed to make that assessment, and 2) more importantly, decisions about such things fall in the domain of the trier. Such decisions are not mine to make. The logical approach is aimed, in part, at ensuring the expert does not, either accidentally or intentionally, overstep their bounds as an expert in a way that might (even potentially) usurp the role of the trier.

Now, lawyers or barristers (or clients, in general) can certainly reach their own conclusion based upon their overall knowledge of the matter. In fact, it could be argued that a barrister should, and must, be doing that to fulfill their role in the situation. After all, they have to argue their side or their perspective on the matter to the trier. To do that, they need to consider all the evidence they have (including that provided the examiner/expert) and present their argument for a particular outcome.

The examiner, despite what others may expect, is not in a position to do that. Unfortunately, for decades that is exactly what they have done—unwittingly, for the most part. But past behaviour does not make this either correct or proper.

If, during testimony, the Court insists that an expert give this type of opinion, the situation changes a bit. I recommend strongly that the expert respond by saying that 1) they don’t have the necessary information, 2) they would have to apply their own personal threshold for any such decision (which anyone can do just as well since it doesn’t involve actual expertise), and 3) the reason for the conclusion format and wording is to avoid potentially over-stepping the bounds or scope of an expert. Then, should the court still insist on an answer in that form, the expert can give whatever they like as an opinion.

Personally, I would continue to refrain, asking for guidance from the Court with respect to 1) the utility equation I should be applying (i.e., what cost-benefit function be should used to assess each potential outcome), plus 2) the appropriate threshold to apply for each possible level of outcome.

Since the court is unlikely to provide either of those things, the matter should come to an end simply because I cannot do what they ask without that information.

To be clear, the reason for doing things this way is to 1) ensure the trier understands the limits of expert opinion (which they should already know, but clearly don’t at that point), and 2) provide the involved parties with the opportunity to object (and/or appeal) any outcome that results from improper, or incomplete, court guidance/direction. All of this might seem inappropriate for a witness but, in reality, doing so simply ensures the examiner fulfills their role as an expert while not going beyond the bounds of that role.

  1. Any concern about an examiner ‘usurping’ the court may seem silly to some but, in reality, individuals designated as an ‘expert’ are often viewed with great respect and given much deference by both judges and juries. The potential (if not reality) that an examiner’s opinion might be given too much weight simply because it comes from an ‘expert’ is very real. And such an outcome must be avoided.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.