People sometimes question whether forensic work is scientific in nature. Given that the overall discipline is called ‘forensic science’ this is an interesting, if rather meaningless, question. I say ‘meaningless’ because, practically speaking, it is a non-issue.

Why? Simply because a court may choose to admit anyone as an expert, whether their expertise is scientific, purely experiential, or something else entirely. Nonetheless, it is interesting to consider the issue, if only because forensic document examination is one of those disciplines where this is a common challenge — does it involve any “science” at all?

As a result, this topic is worth some discussion.

First, let us consider what science entails. Everyone ‘knows’ what science is but, strangely enough, it is not as straight-forward to define as one might think. In fact, I would say there is no singular definition accepted by everyone.

Formal dictionary definitions, for example, are relatively vague.

Merriam-Webster, in its full definition, says:

1a: knowledge or a system of knowledge covering general truths or the operation of general laws especially as obtained and tested through scientific method

b: such knowledge or such a system of knowledge concerned with the physical world and its phenomena : NATURAL SCIENCE

2a: a department of systematized knowledge as an object of study: the science of theology

b: something (such as a sport or technique) that may be studied or learned like systematized knowledge: have it down to a science

3: a system or method reconciling practical ends with scientific laws: cooking is both a science and an art

4 capitalized : CHRISTIAN SCIENCE

5: the state of knowing : knowledge as distinguished from ignorance or misunderstanding

The Cambridge English Dictionary gives another multi-faceted and broad definition:

– (knowledge from) the careful study of the structure and behaviour of the physical world, especially by watching, measuring, and doing experiments, and the development of theories to describe the results of these activities
– a particular subject that is studied using scientific methods
– the study of science

The Oxford English Dictionary provides a general definition, followed by 3 explanatory extra bits:

The intellectual and practical activity encompassing the systematic study of the structure and behaviour of the physical and natural world through observation and experiment
– A particular area of science
– A systematically organized body of knowledge on a particular subject
– (in archaic terms) Knowledge of any kind

These three show that the word “science” has different interpretations and meanings.

Before I turn to a discussion of how FDE conforms, or does not conform, to the above definitions, I need to mention an interesting essay entitled, “THE PRINCIPAL ELEMENTS OF THE NATURE OF SCIENCE: DISPELLING THE MYTHS” by Professor William F. McComas from the Rossier School of Education, U.S.C. Though written in 1998, the essay has as much significance today as when it was written. Many years ago, I had the opportunity to hear a lecture by Professor McComas where he discussed fifteen common misconceptions and myths about science, many of which are found in science textbooks and taught in classrooms today. His essay covers the same points.

The nature and ubiquity of these myths helps us to understand why so many educated persons have a warped view of what ‘science’ is, and is not. It is a truly insightful discourse that I strongly recommend to all readers. I won’t delve into the 15 myths in this post beyond but I will list them:

• Myth 1: Hypotheses Become Theories That In Turn Become Laws
• Myth 2: Scientific Laws And Other Such Ideas Are Absolute
• Myth 3: A Hypothesis Is An Educated Guess
• Myth 4: A General And Universal Scientific Method Exists
• Myth 5: Evidence Accumulated Carefully Will Result In Sure Knowledge
• Myth 6: Science And Its Methods Provide Absolute Proof
• Myth 7: Science Is Procedural More Than Creative
• Myth 8: Science And Its Methods Can Answer All Questions.
• Myth 9: Scientists Are Particularly Objective
• Myth 10: Experiments Are The Principal Route To Scientific Knowledge
• Myth 11: Scientific Conclusions Are Reviewed For Accuracy
• Myth 12: Acceptance Of New Scientific Knowledge Is Straightforward
• Myth 13: Science Models Represent Reality
• Myth 14: Science And Technology Are Identical
• Myth 15: Science Is A Solitary Pursuit

I strongly recommend taking the time to read this essay. It is very good and you won’t think of “science” in quite the same way ever again.

Now, let us turn now to how the courts have addressed ‘science’. Science, itself, is not much of a concern for the courts (well, it is in specific cases), but the issue has come up because of the concept of the ‘expert’ — someone qualified and admitted by the court to provide ‘opinion evidence’ in some matter. There have been several rulings that touch on the topic but, in particular, we have Daubert v. Merrell Dow Pharmaceuticals, Inc. (in the USA) or R. v. Mohan (in Canada), among others.¹

Not surprisingly, the idea of ‘science’ that everyone ‘knows’ (and loves) is inextricably tied to the idea of the ‘scientific method’. Although less well-defined than many realize, the scientific method simply involves the acquisition of knowledge through empirical research (of some type). Now, the specific research design will vary by domain, but the basic process and steps are often the same.

To start things off, some conjecture is made by a scientist about the world which forms the basis of some testable hypotheses. Predictions are made based on those hypotheses, and then experiments (of some sort) are carried out. The results of the experiments will either match the predictions, or they won’t.

Now, any good scientific hypothesis must be falsifiable, which at its most basic means the possibility exists that predicted outcome might not happen; that is, some outcome might occur that runs contrary to the prediction based on a given hypothesis. If that happens, the hypothesis is ‘ruled out’ (partially or completely).

Now, it is debatable whether the scientific method requires experimentation or not (see McComas’ Myth #10), but it is certainly an approach commonly used by many scientists (while others conduct purely observational studies). In the experimental approach a scientist attempts to test a particular aspect of some specified theory. To do this, the scientist formulates a working hypothesis that contrasts from some (usually stated or obvious) alternative. They then design an experiment that generates data and information used to test which theory is correct (or, more accurately, which is better supported by the evidence).

In the ‘classical’ sense, the design often involves a null hypothesis $H_{0}$ which, if rejected on the basis of some statistical test, provides a degree of support for the alternative.² The information derived from the experiment serves to affirm or refute some aspect of the original theory resulting in an adjustment of belief about the original theory.

One problem with this classical approach to hypothesis testing is the fact that a failure to reject the null is a ‘useless’ outcome. It is inappropriate to ‘accept’ the null as true simply because the test failed to rule it out. That is true even though such an outcome actually does have meaning. There is just no way, in this scheme, to figure out what it means.

To address this issue, one might use some form of Bayesian methodology that speaks to the strength of belief provided by the results in favour of one hypothesis over the other. Interestingly, an exact parallel to this can be seen in regular, everyday forensic science work. Evaluations done in a case context can be viewed literally as a variation on this approach.

Consider a generic case where forensic document examination is involved. Let’s look at how this corresponds to the above description of the scientific method.

First, we need a theory about what happened to produce some evidence. Ideally, at least two competing (i.e., mutually-exclusive) theories.³ In a casework context, the ‘theories’ to be tested relate to the beliefs of the parties involved in the matter. Working hypotheses are propositions generated by the client or, at least, set in accordance with some beliefs about what may have happened (with those beliefs ideally corresponding to the arguments to be made in court). At a minimum, it is necessary to have at least one ‘alternative’ possibility in the form of a competing position.

So, in general, one of the hypotheses represents the belief of an immediate client (e.g., an investigator or prosecutor); while the other represents a counter-position (e.g., the defendant’s statement about what did or did not happen). Of course, there is no reason for any defendant to express their position on anything. They may simply reject the prosecution position, indicate that they were not involved, or say nothing whatsoever. In that case, the alternative position would be essentially the negation of the prosecution position.⁴ In this context, the data an examiner uses/assesses does not come from an ‘experiment’, per se.⁵ Rather, it comes from the analysis and comparison of samples provided to the examiner.

The examiner applies standard analytical methods and procedures for handwriting examination which are aimed at detecting and delineating similarities and differences in the graphical elements, from which the examiner makes inferences about the original, underlying habitual writing behaviour. Ultimately, the examiner summarizes their observations as findings that effectively describe the questioned item in detail, as well as any relationship that item has to specimen samples (or other questioned samples). In reality, the process involves another step before the examination and comparison takes place — the declaration of expectations.

In general, the examiner will express their expectations for the findings under each proposition before generating those findings. In doing so, the examiner can demonstrate their expertise and knowledge (based on their past training/experience) about what they expect to see if either proposition is true. That makes the subsequent evaluation relatively simple since the process becomes one where the findings are simply compared to the pre-declared expectations.

The ‘testing’ aspect of the process corresponds to the evaluation of the findings produced in the data generation phase. The evaluation is done to see which of the two competing hypotheses (propositions) is best supported by the data. At the same time, the examiner also determines the degree of differential in support based upon the outcome. As a rule, the strength is determined mainly by the presence/absence of limitations in the material. All of this is expressed in the form of an opinion about the evidence/findings that relates to the two propositions/hypotheses under consideration.

Now, the conclusion of this process differs from the ‘standard’ scientific method in a very significant, and important, way. In most scientific experiments, the scientist acts as the decision-maker. They consider the outcome of their experiment and combine that with other information they know about the matter, to determine whether the hypothesis should be rejected or accepted. A forensic evaluation, in contrast, is aimed at assisting another party in their decision-making. Specifically, the examiner simply helps the trier as they try to reach a decision on the matter.

In a sense, each case examination and evaluation can be viewed as a mini-experiment that tests hypotheses applicable to a specific case. The examiner conducts appropriate evaluations to inform and assist the trier so that they can reach a decision about what actually happened (i.e., which of the propositions is ‘true’). From that, a decision is made regarding guilt or liability.

Another, very different, ‘scientific’ aspect of FDE work relates to competency testing and the demonstration of applicable skill (which can be tested and studied using scientific methods). As noted at the beginning of this post, the issue of whether the method being used is ‘scientific’ or not is not a big concern. Courts may decide to admit any person as an expert if their knowledge, skills, and abilities are needed to assist the court with some difficult topic (one that is technically ‘beyond the ken’ of the average person). But all such evidence, whether scientific or not, should be assessed in terms of it’s accuracy and reliability (or validity and precision).

How the latter is assessed depends upon the trier, but competency testing has obvious application in that regard. The process of testing can, and should, be done in some rigorous manner. One that clearly conforms to the tenets in the scientific method. I’m happy to say that the FDE community has done plenty of this over the last 40 years.

So, is FDE a ‘science’ or a ‘scientific’ pursuit? I think it is, but it really depends on your perspective and view of what constitutes a science. That isn’t as simple or straight-forward as many people think.

1. These two can be viewed as seminal rulings which, over time, have been enhanced by various subsequent rulings.
2. The degree of support is determined by the specifics of the statistical test being applied.
3. People sometimes think that these theories need to be exhaustive but, in reality, that is not necessary. Nor, in most cases, is it particularly beneficial.
4. The negation of the prosecutor’s position is not necessarily the best for the defense. It is simply something to use so the evaluation can be done.
5. It is true that case-specific research or experimentation may take place, if or when appropriate. In addition, the base information about the subject matter and known to the examiner will generally derive from past experimentation as outlined in the literature.