What is the “logical approach” to evidence evaluation?

The logical approach to evidence evaluation is an assessment process that focuses on the evidence, rather than directly addressing any propositions that might ‘explain’ that evidence. In other words, the examiner uses their expert knowledge and ability to determine the likelihood of the evidence (under competing hypotheses or propositions). Likelihood in this context means the conditional probability of the evidence given the hypotheses of interest. It is a system of logical reasoning.

I’ve posted on this topic a few times. Be sure to read my “Introduction to the Logical Approach to Evidence Evaluation”, as well as “Propositions — key to the evaluation process” and “When is a ‘Bayesian’ not a ‘Bayesian’?” This approach to evidence evaluation is also explained in the ENFSI Guideline for Evaluative Reporting.

What is the HFHE report?

The Forensic Handwriting Examination and Human Factors: Improving the Practice Through a Systems Approach report, or just the HFHE Report, was produced by the Expert Working Group for Human Factors in Handwriting Examination.

I was fortunate to be invited to be a part of this expert working group. The membership spanned various scientific disciplines and included forensic document examiners, lawyers, cognitive scientists (including human factors specialists), among others.  A more lengthy discussion of this can be found here.

The EWG “…conducted a scientific assessment of the effects of human factors on forensic handwriting examination” and produced a report that “…provides a comprehensive discussion of human factors as they relate to all aspects of handwriting examination, from documenting discriminating features to reporting results and testifying in court.” Of course, the report also provides extensive recommendations aimed at improving the practice in the future.  Specifically, the work was produced “…to encourage and enhance efforts to apply human factors principles to forensic science applications, reduce the risk of error, and improve the practice of forensic handwriting examination.”

It should be noted that the EWG was supported by the National Institute of Justice (NIJ), Office of Investigative and Forensic Sciences (OIFS), and the National Institute of Standards and Technology (NIST) Special Programs Office.

Formal citation: Taylor, M., Bishop, B., Burkes, T., Caligiuri, M., Found, B., Bird, C., Grose, W., Logan, L., Melson, K., Merlino, M., Miller, L., Mohammed, L., Morris, J., Osborn, J., Osborne, N., Ostrum, R. B., Saunders, C., Shappell, S., Sheets, H., Srihari, S., Stoel, R., Vastrick, T., Waltke, H. and Will, E. (2021), Forensic Handwriting Examination and Human Factors: Improving the Practice Through a Systems Approach, NIST Interagency/Internal Report (NISTIR), National Institute of Standards and Technology, Gaithersburg, MD.

What is Evaluative reporting?

Evaluative reporting refers to the expression (written or verbal) of an opinion conforming to the logical approach to evidence evaluation.1

In short:

Evaluative reporting is a formalised thought process that enables the evaluation of scientific findings given two opposing (or competing) propositions. It is a way of providing a strength of the findings of an examination given those alternative propositions.

Ballantyne, K., et al. (2017). An Introductory Guide to Evaluative Reporting, National Institute of Forensic Science Australia New Zealand

NIFS-AU download link:  https://www.anzpaa.org.au/ArticleDocuments/220/An%20Introductory%20Guide%20to%20Evaluative%20Reporting.PDF.aspx

Another useful reference is the ENFSI Guideline for Evaluative Reporting: Strengthening the Evaluation of Forensic Results across Europe (STEOFRAE) report. It states, in part:

Evaluative reporting evaluates the forensic findings in the light of at least one pair of propositions. It is based on a likelihood ratio and conforms to the principles of evaluation. Most of the time, evaluative reporting will follow from comparative examinations between material of unknown source and reference material from one or more potential source(s) and/or associated activities. An evaluative report is any forensic expert report containing an evaluative reporting section.

Willis, S. M., et al. (2015). ENFSI Guideline for Evaluative Reporting: Strengthening the Evaluation of Forensic Results across Europe (STEOFRAE), European Network of Forensic Science Institutes.

ENFSI download link: https://enfsi.eu/wp-content/uploads/2016/09/m1_guideline.pdf