ICFIS 2023

The 11th International Conference on Forensic Inference and Statistics, or ICFIS 2023, is set for June 12-15 of this year. It will be held at the Faculty of Law (Juridicum) of Lund University, Lund, Sweden. While I am saddened that I cannot attend this particular meeting, several years ago I had the pleasure of going to the 2014 International Conference on Forensic Inference and Statistics, or ICFIS which was the 9th iteration of the conference. I wrote a blog post about that meeting some time ago.

I can say, based on past experience alone, that this meeting is well worth attending. That’s particularly true if you are interested in the logical approach to evidence evaluation, but it would benefit any forensic scientist. You will not find a better collection of brilliant people all focused on forensic inference, in the broadest sense.

Forensic scientists, lawyers, academics—they will all be there.

Read more

Three principles

Three principles… that’s all.

There are three simple principles1 necessary to ensure proper application of the logical approach to evidence evaluation. Yet, despite this elegant simplicity, some examiners feel the logical approach is too complicated, or confusing, to be used in our work.

Now, to be fair, real-world case things can be a bit complex and challenging from time to time , but that isn’t unique to the logical approach. It is simply a reflection of the nature of forensic document examination. Our work is difficult and challenging at the best of times and every properly trained document examiner must be able to deal with, and explain, complex topics to laypersons. It is the reason why forensic handwriting examination (FHE) expertise is necessary in a court of law.

When one applies the logical approach, that inherent complexity is more exposed… and that is a good thing.

In my opinion, a very strong argument can be made that the logical approach is simpler and easier than any other alternative, even in its most complicated form. That is certainly true once you get over the initial hurdle of comprehension. In my opinion, the real issue for examiners, and for others who argue against the use of the logical approach, is their lack of understanding coupled with a failure to appreciate what the logical approach does to clarify our reasoning processes, how it functions to guide those processes, and how it assists us in explaining and defending our opinions.

Perhaps most important of all, the logical approach demands a different view of the evidence and what it means. Ultimately, it also demands a different view of the role of the examiner. Hence, some people find it difficult to come to terms with the process and what it requires of them.2

I’m often asked for a simple overview or summary of the process — a short-hand description or ‘briefer’ version that explains the basics. Well, here it is…

Read more

But doesn’t that mean it is ‘more likely than not’?

When an examiner expresses an opinion along the lines of ‘the findings support one proposition over another proposition’, a question often follows. Specifically, does that opinion mean ‘it is more likely than not that the favoured proposition actually happened’?3 The short answer is “no, it does not mean that.” At least, not necessarily.

In order to reach such a conclusion one must consider information that goes beyond the FDE evidence. As a rule, any opinion I provide will be constrained to the probability of the findings/observations in terms of one of at least two possible explanations.4 Ultimately, equating the two statements is inappropriate because they are not equivalent.5Read more

Is Absolute Uniqueness Necessary?

Many years ago I came across an interesting, if limited, discussion in a blog post entitled “Expert testimony in pattern evidence cases – is absolute uniqueness necessary?”6,7 That post is dated Sept 4, 2009, shortly after the publication of National Academy of Science’s report “Strengthening Forensic Science in the United States: a Path Forward”.8 But the basic question posed in it is still relevant today. I would say that most forensic practitioners today would answer the question about the necessity of ‘absolute uniqueness’ in the negative. However, their individual reason(s) for their answer will still vary.

For many people, ‘absolute uniqueness’ is mainly a ‘forbidden’ concept because of some policy they must follow, or because of a more personal recognition of an (often vague) issue relating to the ‘limits of science’. For other people the matter is a well-defined issue in science and logic dictated by the nature and limits of information (knowledge), what information can really tell us about the world, and how information can and should be used to update beliefs about the world. For the latter group (which is steadily increasing in size as awareness and understanding improves), the concept of ‘absolute uniqueness’ is neither required, nor even beneficial in forensic work.

There has been a LOT of discussion about this in recent years, but I found the blog post interesting at the time even though it focused mainly on latent print examination. I feel that not much has changed since then so, even now, it deserves recognition and consideration.  Since the blog itself is no longer active, I have reposted the complete series of messages here (pulling them from archive.org).

The topic started with a post from the moderator (Barry Fisher) who wrote:

Expert testimony in pattern evidence cases – is absolute uniqueness necessary?
What information is needed to form a conclusion about an identification? Do conclusions require statistical data, as in DNA cases, to offer an opinion? Is it possible to state that two items of evidence come from a sole source? What may an expert opine when no statistical data is readily available and only experience suggests a conclusion? The National Academy report raises some profound questions and some intriguing research possibilities. But in the interim, while we wait for academics to study the multitude of pattern evidence forensic scientists encounter in their day to day work, who may report cases and testify in court? Readers are invited to speak to these issues.

Read more

Impinging on the Court’s authority

One of the strangest things I have heard raised in argument against the logical approach is that use of the approach means the expert’s testimony will end up impinging upon the authority of the Court.9 I have heard this a few times recently. I find this particularly troubling because it has come from lawyers. Unfortunately, this has always happened in circumstances where I could not actually discuss the matter with them.

As an objection to the logical approach, this is the most unexpected thing I have ever heard, without a doubt. In reality, proper application of the logical approach is one of the few ways to ensure that this issue will not happen.

To clarify, it is important to first understand the concept of “usurping the role of the Court” which means, in essence, to improperly influence the court’s procedures and decision-making, often by speaking inappropriately to or about the ultimate issue. Or, in other words, to impinge on the Court’s authority to make decisions about the ultimate issue, or ‘what happened’. To be sure, there is a legitimate concern that this could be a problem, particularly when the court is listening to an expert. As a result, the concept has been discussed literally for years and it is not a new concern.

In fact, it can be found in various codes and directives regarding expert evidence. Indeed, Justice Sopinka noted this precise issue in the 1994 R. v. Mohan ruling when he stated, in part, “There is also a concern inherent in the application of this criterion that experts not be permitted to usurp the functions of the trier of fact.” 

Read more

Intra- vs inter-source Variation

Some time ago, in 2009 to be precise, a series of posts was made to the CLPEX.com chat board (a discussion group mainly for latent print examiners) that discussed intra-source versus inter-source variation.10 I’ve replicated key parts of the discussion below, with quotes for the original posters, interspersing some of my own thoughts.

The discussion focused on latent print examination (LPE) but many of the concepts cross over to other disciplines, like handwriting examination.

Terminology is key to understanding so this discussion is worth a review.   

The original post was by L.J. Steele who asked,

Anyone have a good set of pictures to illustrate significant intra-source variation — two good-quality rolled prints or two latents known to be from the same person that might trip up a trainee (or even a veteran)? I’m looking for something for an article and/or powerpoint to help attorneys understand what I mean when I talk about intra-source explainable differences.

There were several replies which I’ll leave out as they addressed the original question, but didn’t get into the topic explored in this post.

Then Pat A. Wertheim commented:

I don’t think I have ever heard the term “intra-source.” It is quite common to talk about “same source.” I am not even sure the meaning would be the same or whether there might be some fine distinction between the two.

Has anyone else ever used or heard the term “intra-source?” Is there any difference between that term and “same source?”

That’s where I’ll pick up the response provided by Glenn Langenburg:

(g.) Yeah in the community of folks looking at fingerprint statistics, these are commonly used terms.

I hold a much broader view on this as it applies far beyond the fingerprint realm. In reality, these terms are common to many applications and fields of study. The underlying concepts relating to the source(s) of variation are found throughout statistical theory and methods.

In fact, the differentiation of intra-source variation and inter-source variation is fundamental to most traditional parametric tests for statistical hypothesis testing; at least when it involves a comparison of means (i.e., t-test, ANOVA, etc.). For reference, I would say that the terms ‘intra-source’ and ‘inter-source’ are less often seen in the literature than the similar terms, ‘within-source’ and ‘between-source’. 

Glenn explains the terms as they pertain to the LPE realm, as follows: 

(g.) Intra-source variation is essentially represented by the concept of distortion (i.e. “how different can two impressions appear when in fact, they are from the same source skin”) versus Inter-source variation (i.e. “how similar can two impressions appear when in fact, they are from different sources)–what we might think of as close non-matches.

Given the nature of fingerprints, these fundamental concepts reduce to the points made by Glenn. However, in other domains such as handwriting comparison, the situation is a bit more complicated.  Nonetheless, exact parallels are present.11

The latter term sounds rather like a type of random match probability (RMP), doesn’t it?  What is the likelihood/probability that a set of common features would be observed, by chance alone, when the samples are in fact drawn from different sources taken from some (hopefully specified) population?  Without some estimation of the second factor (inter-source), how is it possible to determine the value of the first factor (intra-source)?  The short answer is, you can’t. 

Any given feature observed in a comparison will be ‘possible’ under either proposition; only the likelihood of observation changes.

(g.) In the statistical approaches proposed by Neumann, Champod, Mieuwly, Egli, and others, likelihood ratios represent these two competing parts: intra-source versus inter-source variations.  This is intuitive, since analysts are already doing this everytime we offer an opinion.  Everytime we report an identification, at some point we weighed the differences observed and asked ourselves, are these differences likely due to a distortion (within tolerance for Intra-source variation) or are they true discrepancies (within tolerance for Inter-source variation)?

As Glenn, notes this is all encapsulated perfectly in the concept of the likelihood-ratio used in the logical approach to evidence evaluation.

Ultimately, and in terms of the classical ‘identification’ opinion, this also means the examiner came to a conclusion that the evidence can only be explained in one way. All other possible explanations are deemed to be unreasonable to the point that they can be rejected outright. The main issue for most critics who disapprove of such opinions is the implicit application of some unknown threshold beyond which the expression of such a conclusion, an identification, can be justified. What is that threshold and how do we know it has been exceeded? Another obvious, and very important, issue is who should be making such decisions — the examiner or someone else? 

Any and all statistical methods, not just those of a ‘Bayesian’ nature, must take variation into account.12 Generally, this is done by contrasting and comparing within- and between- sources of variation. A simple truism that derives from these concepts is, as follows:

Differentiation between two potential sources can be achieved if and only if between-source variation exceeds within-source variation  

Basically, the spread between different (multiple) samples must exceed the spread for any given individual sample within the set of all possible samples. If there is too much overlap, the samples cannot be effectively distinguished from one another.

Glenn ended his comments with:

So you had experienced these concepts before, but maybe not heard these exact terms. Also, they differ from Intra-observer variation versus Inter-observer variation.  Whereas, the concept in the previous paragraph deals with how the features can present themselves in an impression (what arrangements are possible)…Inter/Intra observer variations deal with how analysts perceive features.  What features did I perceive today in an impression versus yesterday or last week (in the same impression) (INTRA-OBSERVER) v. How different are the observation from analyst to analyst all examining the same impression (INTER-OBSERVER).  I have some good data on this concept to share with the community soon (in the thesis).

I have to agree with Glenn on all his points.

The concepts of intra- versus inter- variation are both common to, and critical for, all forms of comparison (and, obviously) decision-making. This is a very interesting topic that comes into play for everything forensic examiners do on a regular basis — even though, as Glenn points out, the terms may not be particularly familiar to some people.