On May 20 of each year is a great day celebrated all over the world as World Metrology Day.  Metrology is so very important in everything, but in forensic science it is fundamental.  In honor of that day and in advance of it to hopefully bring some awareness, I will examine Metrology in this post.

What most attorneys do not appreciate or understand is precisely how a measurement is made, what its significance is and how it needs to be reported in a scientifically responsible way that acknowledges and quantifies the limitations of the measurement itself.

Any time one measures something it is a unique event in the universe, not to be exactly repeated ever again.  It depends on a lot of factors.

In binary or integer reported values arrived at by analytical measurements in forensic science (e.g., the weight of a controlled substance, the feet or meters between objects in an crime scene, angles of trajectory in anthropometric-based reconstructions, blood alcohol content), there is a need to report uncertainty to account for and recognize the limitations of each analytical measurement made.

“Huh?” you may be thinking.

Think of it this way.  Imagine your bathroom spring scale.  You stand on it in the morning, you get a result.  You frown, you step off, then you IMMEDIATELY step back on again two seconds later.  The result the second time is that you are two pounds lighter!  You perhaps think to yourself that the second measure was more accurate and move on with your day.  But if you were a metrologist or someone who is interested in analytical measurement, you would have a different world view.  We all know that in that difference of less than two seconds, there is no way that you lost two pounds.  This simple example is only but one example of the variability of a measure and how each measure is a unique event unique to itself in the universe not to be repeated again.

While it is also important to report uncertainty in subjective non-binary valued forensic science disciplines such as pattern recognition, in analytical measurement it is a must.

Everyone should agree that only honest and scientifically responsible reported results belong in a Courtroom.  No true scientist would disagree.  But is that practiced?  NO.

To report the total uncertainty budget as an expression of Bayesian-based metrology is to essentially identify and quantify all possible sources of error (both Type I and Type II) and account for it in an empirical manner.

The end result is something such as the following:

The result of the analysis is: x +/-y with a xx% coverage/confidence interval.

So, it must be expressed similar to this:   0.164g/100ml +/- .004 with a 95% coverage (confidence) interval.

Without the data being honestly and scientifically reported, it should not be accepted.

Here is why.

If I tell you that something is 3 feet in length does that really accurately describe that measure based solely upon that information?  No.

If I said that I used a yard stick to measure it, then you may have a better idea.  But if I used a yard stick that was all beat up or had been manufactured imprecisely and not accurately, then you may begin to see the problem in part.  What if I told you that when I made the call that it was exactly 3 feet that I was not looking directly on at the end of the yard stick, but rather at a significant angle so my perspective was off?  What if I told you that my yard stick only had demarcations on it for every foot instead of each inch?  What if I told you I closed one eye when I made the call?  And on it goes.

The greater part of being metrologically responsible in the reporting of uncertainty of any analytical measurement and moving closer towards compliance with ISO 17025 would be to acknowledge and attempt to quantify the sources of uncertainty.  These individual differences both infrasubject and intersubject and among samples themselves as well as among analytical devices/measurands and even the same analytical device/measurand in the case of duplicate or even replicate testing are far more important and will produce the greater range of variability than simply the machine’s stated or actual rate of analytical error as acknowledged by the manufacturer.

One needs to keep in mind that just like with the yard stick example above, this is compounding or contributory error.  They are not islands unto themselves.

To present any analytical measurement as an absolute is not only scientifically irresponsible and ignores metrology, but most importantly for our purposes in presenting truth in the COurtroom, it is unduly prejudicial as in doing so the Government benefits inappropriately from the power of “loaded language” (such as the words “identical” and “match” in pattern recognition based forensic science).  In essence, in doing so (reporting it as an absolute), it misguides the trier of fact.

The absence of reporting uncertainty in a metrologically responsible manner gives the trier of fact no useful information from which to evaluate the measure itself.  As such, it is not possible to meaningful judge the value or attach a weight to the measurement.  It has no context.

It is no different if I told you that I am 5’5″ tall but if my process and measuring device is +/- two feet with a 95% confidence level, what do you think the chances are that I am really truly 5’5″ tall?

Judges should preclude the presentation of the reported measure as an absolute.  Having come from the American Academy of Forensic Science (AAFS) annual meeting this past February, I can tell you that there is not a single scientist who disagrees that the metrologically responsible and ISO 17025 method of reporting is scientifically correct.

To be more direct, I mean that if a measure is being reported as an absolute, it should be (read as must be) precluded entirely from evidence as doing so in presenting it as an absolute without properly reported uncertainty per ISO 17025 or the well-developed body of  science known as metrology, the trier of fact does not have the relevant information to make an honest assessment of the evidence itself.  As such a reporting (as an absolute), it  is inherently and demonstrably unduly prejudicial.

Until they start reporting it in a metrologically responsible fashion it is not science, but the functional equivalent of guessing or really science fiction.  We do not invite triers of fact to guess.  Proof is demanded.  Speculation is shunned.

A total uncertainty budget must be presented with confidence intervals by the Government as the Government is the proponent of the evidence claiming it to be relevant in its reported form.

The utter lack of reporting the uncertainty in a metrologically responsible way screams unfair and NOT science and should not be allowed into Court.

Whereas it is true that science in the long term is self-correcting, we need to be on the forefront of this push.  Why wait for ISO 17025?  Let’s force it now.

The old saying is true:  “In God we trust. All others must bring data”.

_______

Special acknowledgment:  A great litigating attorney who focuses almost exclusively on this very topic by the name of Theodore “Ted” Vosk introduced me first to the concept of Uncertainty Measurement and metrology in 2009.  I thank him for the introduction as from that moment I have spent a tremendous amount of time learning about this subject as well as International Standards in general including attending other conferences and reading.  Attorney Vosk is a legal role model as he has quite successfully litigated these matters in the field of breath alcohol testing in the state of Washington.  He is available for hire to act as counsel on these matters.