Posted on by RB Ostrum

Prior odds — their meaning and significance

The concepts of ‘prior odds’, a.k.a., prior probabilities or simply priors, and ‘posterior odds’ come up in most discussions about the evaluation of evidence. The significance and meaning of both terms becomes clear when viewed in the context of a “Bayesian approach”, or the logical approach, to evidence evaluation. That approach has been discussed at length elsewhere and relates to the updating of one’s belief about events based upon new information. A key aspect is that some existing belief, encapsulated as the ‘prior odds’ of two competing possibilities or events, will be updated on the basis of new information, encapsulated in the ‘likelihood-ratio’1 (another term you will undoubtedly have seen), to produce some new belief, encapsulated as ‘posterior odds’ about those same competing possibilities.

But what precisely do these terms, ‘prior odds’ and ‘posterior odds’, mean and how do they relate to the work of a forensic examiner?
Read more

Posted on by RB Ostrum

Hilton and Mathematical Probability

In 1958 Ordway Hilton participated in Session #5 of the RCMP Seminar Series. His article was originally published in that series by the RCMP, and subsequently republished in 1995 in the International Journal of Forensic Document Examiners.1

The later republication included the following abstract:

In every handwriting identification we are dealing with the theory of probability. If an opinion is reached that two writings are by the same person, we are saying in effect that with the identification factors considered the likelihood of two different writers having this combination of writing characteristics in common is so remote that for all practical purposes it can be disregarded. Such an opinion is derived from our experience and is made without formal reference to any mathematical measure. However, the mathematician provides us with a means by which the likelihood of chance duplication can be measured. It is the purpose of this paper to explore the possibility of applying such mathematical measure to the handwriting identification problem to see how we might quantitatively measure the likelihood of chance duplication.

Hilton’s article was written in 8 main sections with references, and is followed by a discussion between seminar participants. Today’s review will discuss each section of the article in turn.
Read more

Posted on by RB Ostrum

Huber, Headrick & Bayes…

Like many document examiners I consider Huber and Headrick’s 1999 textbook, Handwriting Identification: Facts and Fundamentals, to be a seminal work.1

Huber and Headrick Handwriting Identification

In my opinion, it is the best textbook written to date on the topic of handwriting identification. The authors provide a comprehensive overview as well as some less conventional perspectives on select concepts and topics. In general, I tend to agree with their position on many things. A bit of disclosure is need here: I was trained in the RCMP laboratory system; the same system in which Huber and Headrick were senior examiners and very influential. Hence, I tend to be biased towards their point-of-view.

That does not, however, mean that I think their textbook is perfect. While it is well written and manages to present a plethora of topics in reasonable depth, some parts are incomplete or misleading; particularly when we take developments that have happened since it was written into account.

One area of particular interest to me relates to the evaluation of evidence; specifically evaluation done using a coherent logical (or likelihood-ratio) approach.2  I have posted elsewhere on the topic so I’m not going to re-hash the background or details any more than necessary.

Rev Bayes

This post will look at the topic of ‘Bayesian concepts’ as discussed by Huber and Headrick in their textbook. These concepts fall under the general topic of statistical inference found in their Chapter 4, “The Premises for the Identification of Handwriting”.  The sub-section of interest is #21 where the authors attempt to answer the question, “What Part Does Statistical Inference Play in the Identification Process?”  Much of their answer in that sub-section relates to Bayesian philosophy, in general, and the application of the logical approach to evidence evaluation. However, while they introduce some things reasonably well, the discussion is ultimately very flawed and very much in need of correction. Or, at least, clarification.

Read more
Posted on by RB Ostrum · 3 Comments

Propositions — key to the evaluation process

One of the key elements in the logical approach to evidence evaluation are the propositions used for the evaluation. They are, in a certain sense, the most important part of the whole process. At the same time, they are also one of the least understood.

Today’s post explores the concept of propositions. I will attempt to describe what they are, how they are used, why we don’t change them once set and why they matter so much, among other things… all from the perspective of forensic document examination (though also applicable to other forensic disciplines).

Read more
Posted on by RB Ostrum

R you ready for statistics?

The R program for statistics is an amazingly powerful and completely free program (under the terms of the Free Software Foundation’s GNU General Public License).  If you have any need to do statistics, then you really must take a look at R or, more formally, “The R Project for Statistical Computing“.

What exactly is R?  Simply put, “R is a language and environment for statistical computing and graphics.”  It is a special open-source implementation of S which is one of the earlier statistical programming languages.
Read more

Posted on by RB Ostrum

Forewarned…

Forewarned is forearmed or, if Latin is your thing, “praemonitus, praemunitus”. So the saying goes and clearly there is great value in knowing what lies ahead for us. If we know what is coming our way we can, in theory, prepare properly for any challenge.

Challenges are nothing new to forensic scientists. Critics routinely point out issues they have with our work. Some of those criticisms are fair and reasonable, others not so much. Much of the critical commentary affects a discipline as a whole demanding an overall, or group, response by members of each discipline. In my experience, disciplines tend to be behind the curve in their responses to critics. Nonetheless, over time some issues have been addressed, at least partially if not completely, through empirical research. Others have not. To be fair, the activities needed to properly address the critics are not trivial and require both time and resources; scarce commodities in modern forensic labs. Overall, things are improving, albeit very slowly.

Criticism takes on a whole new meaning in the context of a court of law. Indeed, I think that criticism is the essence of cross-examination — a fundamental and important aspect of any adversarial justice system. Although essential, it is rarely an enjoyable part of the proceedings for any expert.

Read more
Posted on by RB Ostrum · 1 Comment

Certification — ABFDE

What is certification? In my opinion, professional certification is a designation that indicates the holder of the certification has appropriate and adequate qualifications to do some particular, generally well-defined, job or task. As an example I am a forensic document examiner and I have received professional certification from the American Board of Forensic Document Examiners, Inc.

An internet search for ‘certification’ produces a huge list of possibilities, with more such programs being developed all the time as people become attuned to issues of quality and competency. Indeed, almost every profession has some type of certification and a few have several (consider all of the ‘certifications’ in the computing industry). Most, if not all, certification programs are aimed at improving the quality in a given profession by setting minimum standards for the job. The basic idea is that someone meeting or exceeding those standards will produce quality output on the job. Certification programs are generally created or are administered by a professional society, a college or university, or some private body set up expressly for that purpose.

Forensic Document Examination is no exception so it may be worthwhile discussing certification options as well as the pros and cons that I see for those options.

Read more
Posted on by RB Ostrum · 1 Comment

When is a ‘Bayesian’ not a ‘Bayesian’?

Several of the posts on this blog relate to the logical approach to evidence evaluation; aka, the coherent logical approach, or the likelihood-ratio (LR) approach. In my opinion, it is the best way to evaluate evidence for forensic purposes no matter what type of evidence is being discussed. I say “best” because it is simple, logically sound, and relatively straight-forward to apply in forensic work.  It helps to promote transparency through the application of a thorough and complete evaluation process (all points I have explained in other posts).

The reality is, however, that this approach is still not well understood by forensic practitioners, nor by members of the legal profession.

I hope that in time, and with education, that will change. Several workshops I have presented have been aimed at helping examiners understand what it really means, how it works, the philosophical basis behind the approach as well as the need for and benefit of doing things that particular way. It really does work to the benefit of both the examiner and their ultimate client, the court.

One recurring issue at these workshops relates to the very basic and fundamental concept of what the term “Bayesian” means. For various reasons, but mainly just misunderstanding, many people in the forensic document examination community hold the term “Bayesian” in negative regard. When the word ‘Bayes’, or any of its many derivations, come up in the conversation eyes glaze over while heads sag ever so slightly. And those are the positive people in the crowd.

I find such reactions understandable, but unfortunate.  The fact is that an understanding of the term is beneficial for anyone interested in how it might be applied in a forensic evidence context, whether or not one chooses to do so.  Indeed, for myself the answer to the question posed above — when is a Bayesian not a Bayesian? — lies in knowing how the overall Bayesian philosophy and theorem (or rule) differs from the more constrained and limited logical approach to evidence evaluation. These two are not the same or even close to equivalent.

Read more
Posted on by RB Ostrum · 1 Comment

The ‘best’ textbook ever written about forensic handwriting identification…

Huber and Headrick Handwriting Identification

Okay, determining the ‘best’ of anything is always a challenge. It is, in almost every instance, a highly subjective decision based on some set of appealing features or characteristics… appealing to the person making the determination, of course. And, because this is my blog, that person happens to be me. In fairness, there are a number of authors who have written extensively on the topic: Osborn, Ellen, Hilton, Harrison, Hilton, among others (and I apologize to those I have left off this list). I have read all of those textbooks (including most editions) and each has its strengths and weaknesses. Nonetheless, in my opinion the best general textbook written to date on the topic of handwriting identification was done by co-authors Roy A. Huber and A.M. (Tom) Headrick, both long-time document examiners in the R.C.M. Police laboratory system.

That textbook is Handwriting Identification: Facts and Fundamentals.1Read more

Posted on by RB Ostrum

Can of worms…

Worms in a can

When someone “opens a can of worms” it usually spells trouble. For many people, that phrase evokes a powerful image of a writhing mess of worms escaping from a previously-sealed, but now opened, can or container. With the result of such action being serious problems for the owner of said can, often problems of an unanticipated or uncertain nature. In the context of our work as Forensic Document Examiners, I sometimes hear this coming up in discussions of how to handle questions on the stand. The advice goes along the lines of ‘keep your answers simple and say as little as possible in order to limit any opportunity for questions from the other side.’

It is suggested that lengthy or complex answers will only lead to more questions and more discussion. The latter are the proverbial “can of worms” that one must strive to avoid opening.

That makes little sense to me.

Read more