The expression “better late than never” applies to this post. Over the span of two days in June 2013 the Measurement Science and Standards in Forensic Handwriting Analysis (MSSFHA) conference was held. It explored the (then) current state of forensic handwriting analysis, aka, forensic handwriting examination (FHE). Presentations varied in content but most discussed recent advancements in measurement science and quantitative analyses as it relates to FHE.
The conference was organized by NIST’s Law Enforcement Standards Office (OLES) in collaboration with the AAFS — Questioned Document Section, the ABFDE, the ASQDE, the FBI Laboratory, the NIJ and SWGDOC.
Every ASQDE meeting is worth attending. They are great fun with lots of useful and interesting content. Unfortunately, I could not make it to the 2016 ASQDE conference held in Pensacola, Florida. Nonetheless I managed to participate, albeit via Skype.
One of the activities at the conference was a panel discussion discussing “Approaches to Evaluation and Reporting of Expert Evidence” and I was invited to participant with three other people. It was a very interesting session…
The concept of ‘prior odds’, a.k.a., prior probabilities or simply priors, comes up in most discussions about the evaluation of evidence. A related term, posterior odds, also arises. The significance and meaning of both these terms becomes reasonably clear when viewed in the context of a “Bayesian approach”, or logical approach, to evidence evaluation. That approach has been discussed at length elsewhere and relates to the updating of one’s belief about events based upon new information.
A key aspect is that some existing belief, encapsulated as ‘prior odds’ about conflicting possibilities, is updated on the basis of new information, encapsulated in the ‘likelihood-ratio’1 (another term you will undoubtedly have seen), to produce some new belief, encapsulated as ‘posterior odds’ about those same conflicting possibilities.
But what precisely do these terms, ‘prior odds’ and ‘posterior odds’, mean and how do they relate to the work of a forensic examiner?
Several of the posts on this blog relate to the logical approach to evidence evaluation; aka, the coherent logical approach, or the likelihood-ratio (LR) approach. In my opinion, it is the best way to evaluate evidence for forensic purposes no matter what type of evidence is being discussed. I say “best” because it is simple, logically sound, and relatively straight-forward to apply in forensic work. It helps to promote transparency through the application of a thorough and complete evaluation process (all points I have explained in other posts).
The reality is, however, that this approach is still not well understood by forensic practitioners, nor by members of the legal profession.
I hope that in time, and with education, that will change. Several workshops I have presented have been aimed at helping examiners understand what it really means, how it works, the philosophical basis behind the approach as well as the need for and benefit of doing things that particular way. It really does work to the benefit of both the examiner and their ultimate client, the court.
One recurring issue at these workshops relates to the very basic and fundamental concept of what the term “Bayesian” means. For various reasons, but mainly just misunderstanding, many people in the forensic document examination community hold the term “Bayesian” in negative regard. When the word ‘Bayes’, or any of its many derivations, come up in the conversation eyes glaze over while heads sag ever so slightly. And those are the positive people in the crowd.
I find such reactions understandable, but unfortunate. The fact is that an understanding of the term is beneficial for anyone interested in how it might be applied in a forensic evidence context, whether or not one chooses to do so. Indeed, for myself the answer to the question posed above — when is a Bayesian not a Bayesian? — lies in knowing how the overall Bayesian philosophy and theorem (or rule) differs from the more constrained and limited logical approach to evidence evaluation. These two are not the same or even close to equivalent.