Posted on by RB Ostrum

What is Context and Bias?

Context and bias are important concepts when discussing any type of evidence evaluation. They apply to any and all forms of forensic evidence so some awareness of them is important.

First, evidence always exists within some context, more commonly referred to as a framework (or background), that is specific to the matter at hand. That is just an unavoidable fact which must be acknowledged. Even a purported absence of information is a type of ‘framework’ — it is just an artificial and unrealistic one.1

Framework information can be relevant or irrelevant and, at the same time, either biasing or not biasing to the evaluation process. To clarify the terminology, we can use the Report of the Expert Working Group for Human Factors in Handwriting Examination which discusses all of this at length.[note]Forensic Handwriting Examination and Human Factors: Improving the Practice Through a Systems Approach. (NSITIR 8282). U.S. Department of Commerce, National Institute of Standards and Technology. https://doi.org/10.6028/NIST.IR.8282.[/note] That report provides the following definitions pertinent to this discussion:

  • Bias: A systematic pattern of deviation.
  • Cognitive Bias: A systematic pattern of deviation in human judgment.
  • Relevant Information: Information that is pertinent and applicable to the subject, material, or question being considered. The consideration may be broad (i.e., discipline level) or specific (i.e., task level).

Stoel et al provide guidelines that can be used to determine when action should be taken to manage contextual information, based on the presence/absence of a bias effect and relevance/irrelevance of the information being provided.2

The relationships of interest are shown in the following table:

Task-Relevant InformationTask-Irrelevant Information
BiasingKeep information, but take measures.Shield examiner from this information.
Not biasingUse information.Shield, if possible and efficient.  Not strictly necessary to do since it is not biasing.

The above indicates that task-relevant information should be used in the evaluation process. That makes sense since the presence or absence of that information is likely to change the outcome. If the information is not biasing (bottom left cell in the table), then it can be safely provided to the examiner and used. If the information is biasing (or potentially biasing), then it should be used but carefully, and in accordance with some type of Context Information Management (CIM) System. Obviously, if information is irrelevant then the examiner should not be given the information, particularly if it is also biasing (or potentially biasing).3

So, what is a CIM, and how does it work? The approach to this issue will differ for a public laboratory vs. a private examiner. In a public laboratory with plenty of staff there may be an examiner who reviews a case to vet the material and control task-irrelevant information before handing it over to another examiner who handles the actual examination. There may even be a dedicated unit for case receipt and review, one that covers all sections in the lab. A private examiner, on the other hand, cannot do that and can only try to avoid problems by forewarning clients of the potential risk to their case. At the same time, mechanisms like blind peer review can be used in either situation.

Second, as the above indicates, framework information is not something to be avoided at all costs. It often provides information about the perpetrator, the scenario, or the specific alternative proposition that could be key to formulating propositions or using/considering an appropriate population under the main or alternative hypothesis. The real issue is having access to the right information, at the right time. That can be difficult to achieve in some situations.

When working with a private examiner the client is best off waiting to be asked for more information. Initially, provide only the most basic information and do not provide details to the examiner, until asked. That way the examiner has a chance to guide the discussion with a view to minimizing any potential issues while getting the necessary, relevant information.

Posted on by RB Ostrum

Do you only accept cases from a lawyer?

In general, yes, I only accept a case submissions from a lawyer, solicitor, or barrister.  I will explain why that is done in a minute, but there are a few exceptions to this rule.

I may accept cases from someone other than the above (e.g., usually a party directly or personally involved in the matter) so long as the following conditions apply:

  1. It is NOT a matter before the courts,
  2. It is NOT a matter intended to go before the courts, and
  3. It is NOT a matter with any legal or judicial implications.

Now, this might seem to rule out a lot of things and it does.  The kind of submission I will accept directly from a private citizen (i.e., non-lawyer/non-barrister) are those aimed at genealogical research, personal interest or education, and similar things.  If you aren’t sure about this, please ask and I will clarify.

For most matters a submission must be arranged and executed through a lawyer, solicitor, or barrister. I will, however, consider a contract if struck with some businesses or a government department/agency of well-established reputation.

It is important that the contract for services be formed between a lawyer or law firm (as a representative) and myself because it helps to ensure:

  1. a professional relationship in all matters,
  2. an arms-length arrangement relative to any party with a personal interest in the outcome, and
  3. there is minimal and controlled interaction to avoid introducing biasing information into the process. Please note that, as a potential expert witness, I must do everything possible to minimize any bias that may affect the outcome in casework.

This might seem unnecessary and even problematic, but it really does work to the benefit of all parties. If you have any questions, please let me know.

Posted on by RB Ostrum

From whom do you accept cases? Who are your clients?

Cases are accepted from public entities (e.g., police, investigative bodies, government, etc.) as well as from private individuals.  However, clients who are private individuals must be represented by a lawyer, with very few exceptions. 

For all matters that are, or may be, related to some judicial proceedings of a criminal or civil nature, I will accept submissions only through a lawyer or solicitor representing a party. As a rule, I will not deal directly with parties or individuals personally involved in such proceedings. If you do not presently have a lawyer, I suggest you retain the services of one prior to obtaining the services of any forensic document examiner.

This approach is taken to ensure an arms-length relationship with the involved parties.  It is also important to limit exposure to unnecessary, and potentially biasing, information about the situation at play.  To ensure that such information is not provided all potential clients and their representatives must review our submission guidelines before initiating contact with this office.

For non-legal matters, such as genealogical research or similar issues, my services may be engaged directly by anyone or any institution, public or private.

In addition to the above, all service requests are reviewed before acceptance for potential ‘conflict’.  A ‘conflict check’ must be successfully completed prior to any formal engagement of services. 

Posted on by RB Ostrum

What is a ‘conflict check’?

A ‘conflict check’ is a term used for a preliminary process of checking to see if there is any conflict of interest present prior to any work agreement being reached with a client. For legal reasons, it is necessary to ensure there is no conflict of interest involved when work is commissioned. This refers to both real/substantive and perceived conflicts.

A commission for work cannot and will not be accepted if there is a real or perceived conflict in any of the following conditions:

  1. Conflict (real, perceived, or potential) with an existing (different) client who has already retained my services (i.e., one from whom I have a signed agreement and paid retainer fee), and
  2. Conflict (real, perceived, or potential) with my work as an employee of the Government of Canada; specifically, the Canada Border Services Agency, but not limited to that agency (i.e., more generally, the interests of the Government of Canada).

In order to conduct a conflict check the client must provide me with the following information:

  1. Your complete contact information,
  2. The file or case caption, in full,
  3. Names of all parties involved, including attorneys and law firms

Once the conflict check has cleared, I will provide my present fee schedule, a complete current CV and an Agreement for Expert Services, if one has been requested.

Please note that receipt of that information and form only indicates there is no conflict at the time of contact. It does not constitute ‘retained’ status. Only after receipt of the signed Agreement and nonrefundable retainer am I considered officially retained, and therefore unavailable to the other side should I be contacted by them.

I (Mr. Ostrum) may not be designated or in any way indicated as a retained expert unless payment of the nonrefundable retainer and a signed Agreement for Expert Services (available upon request) has been received.

Note: “Future” conflicts may develop which preclude the continuation of services. If such an event occurs, the existing contract will be null and void. Fees will be charged for services rendered to date, but any excess payment will be returned should that occur.

Posted on by RB Ostrum

What is the HFHE report?

The Forensic Handwriting Examination and Human Factors: Improving the Practice Through a Systems Approach report, or just the HFHE Report, was produced by the Expert Working Group for Human Factors in Handwriting Examination.

I was fortunate to be invited to be a part of this expert working group. The membership spanned various scientific disciplines and included forensic document examiners, lawyers, cognitive scientists (including human factors specialists), among others.  A more lengthy discussion of this can be found here.

The EWG “…conducted a scientific assessment of the effects of human factors on forensic handwriting examination” and produced a report that “…provides a comprehensive discussion of human factors as they relate to all aspects of handwriting examination, from documenting discriminating features to reporting results and testifying in court.” Of course, the report also provides extensive recommendations aimed at improving the practice in the future.  Specifically, the work was produced “…to encourage and enhance efforts to apply human factors principles to forensic science applications, reduce the risk of error, and improve the practice of forensic handwriting examination.”

It should be noted that the EWG was supported by the National Institute of Justice (NIJ), Office of Investigative and Forensic Sciences (OIFS), and the National Institute of Standards and Technology (NIST) Special Programs Office.

Formal citation: Taylor, M., Bishop, B., Burkes, T., Caligiuri, M., Found, B., Bird, C., Grose, W., Logan, L., Melson, K., Merlino, M., Miller, L., Mohammed, L., Morris, J., Osborn, J., Osborne, N., Ostrum, R. B., Saunders, C., Shappell, S., Sheets, H., Srihari, S., Stoel, R., Vastrick, T., Waltke, H. and Will, E. (2021), Forensic Handwriting Examination and Human Factors: Improving the Practice Through a Systems Approach, NIST Interagency/Internal Report (NISTIR), National Institute of Standards and Technology, Gaithersburg, MD.

Posted on by RB Ostrum

Can I send you something for a “quick look” before I decide about using your services?

In general, the answer to this is ‘no’.  However, there are some specific exceptions that might apply in your case.  I’ll get to those possibilities shortly, but first it is important to clarify the main issue at hand.

First, like other professionals I do not work for free and I expect to be paid fairly for my time and efforts. To protect everyone’s interests I require a  formal (signed) work agreement between the parties; specifically the client and myself. An estimate of costs will be provided before the agreement is finalized, and before actual work begins.  Finally, I require a deposit payment to be made before I commence work.

Further to this, I will be officially retained on a matter only after the agreement is signed and received at my office, and after the deposit payment has been made.

Second, there is no way to form an opinion until the work is actually done. At best, it is possible to assess potential limitations to determine what material will be required for an effective examination.

Please understand that no qualified document examiner will ever give an opinion after simply ‘eye-balling’ something; that isn’t how it works. I provide top-quality service and my reputation rests on providing the best possible analysis and opinion.

Third, most people come to me with their own opinion on a matter. That’s understandable, but my professional opinion must be, and will be, based on a detailed review of the evidence. To that end, I will conduct a complete examination and comparison before I reach a final conclusion and give you my opinion. The outcome may or may not meet your expectations or conform to your belief, particularly if you are looking for something particular or specific.

It is critical to understand the duty of an expert is to provide impartial and unbiased information to the trier of fact so that they can determine what the evidence means. I guarantee the quality of my work but that guarantee does not extend to providing any particular outcome or opinion. That is not how it works.

For all these reasons, please do not expect me to provide my opinion without first doing a proper and complete examination.

Now, there are times when a “pre-submission review” of a case submission might be acceptable. Specifically, this might be done to conduct a proper assessment of the material in situations where the matter is complicated, confusing, or it is unclear whether the material you have will be adequate.

As a rule, I provide a work estimate based on a general description of the items and the questions you need answered. Unfortunately, that means there is always some uncertainty in the quote. Hence, from time to time a preview of the material (i.e., pre-submission review) will allow a more precise estimate of the time required or the costs involved so that the estimate is more accurate.

At the same time, please note this is not a “quick look” intended to provide any sort of opinion. No actual examination will happen until my services have been formally retained, and the retainer fee has been received and acknowledged. In other words, any pre-submission is limited to a determination of the type of work to be done, the time required, and a final estimation of costs. A “pre-submission” review is not an examination and will not result in a final, or even preliminary, opinion.

Posted on by RB Ostrum

Is ‘bias’ a concern in Forensic Science?

The short answer to this question is ‘yes’, bias is a concern. However, that is far too simplistic a response.

The bottom line is that every examiner must be aware of the potential for bias in their work, the various ways in which it may manifest, and what can and should be done to minimize such concerns. At an absolute minimum, examiners must be able to explain any steps they have taken to achieve that goal so that the trier can be assess the effectiveness of any such steps. The steps taken will undoubtedly differ for a private examiner working alone, versus a larger laboratory with considerably more resources and mechanisms they might employ.

In general, ‘bias’ is a potential concern for any and every human endeavour that involves or requires evaluation and reasoning. Since every forensic evaluation is a human-based activity involving mainly subjective assessment (even in those situations where there might be an empirical basis to support that assessment), it follows that the potential for bias will be present. It also follows that every examiner needs to be aware of this prospect when doing their work.

On the positive side of things, examiners that follow published guidelines and best practice documents will be utilizing procedures that are designed to help minimize this type of concern. For example, examiners applying the logical approach to evidence evaluation must expressly consider (and disclose) relevant framework information, as well as all relevant propositions considered in the course of their evaluation. Doing so reduces the potential for bias in the evaluation process.