We looked at the National Academy of Sciences body of work wherein it identified weaknesses in the way that forensic science in the United States is practiced.  One of the major areas that it focused on and ultimately took issue with concerned the body of forensic sciences loosely called pattern recognition.

The basic pseudo-scientific principles of pattern recognition based sciences is not unlike Sesame street in the video below.  I am being serious.

Like cookie monster we ask “Did you guess which thing is not like the other things?  Did you guess which thing with all of your might?”

In pattern recognition based forensic science disciplines, we take a known and compare it against an unknown that is typically at the scene to try to guess with all of our might.

The problem becomes fairly obvious fairly quickly.

• First, we need to prove that these knowns that we are comparing the unknown against are obtained as near as possible to the real world conditions as possible that generated the unknown so that the comparison can even begin to be valid.  For example, shooting a projectile from a gun into a body will massively deform the bullet along its path if it hits bone or dense mass, but typically the known that is generated for comparison purposes against this unknown is in fact created by firing its bullet into a water tank which creates a near pristine object, not like the deformed one from the scene;
• Second, we need a sufficient population density study to prove that the knowns and unknowns are uniquely contributed and traceable uniquely to the originating source to the exclusion of all others.  For example, we need to prove through meaningful scientific study that only this type of seed with these types of observable and quantifiable characteristics is in fact unique only to marijuana to the exclusion of all other botanical substances;
• Third, can we use points of comparison/agreement at all?  If so, what definable, measurable and empirical points constitute these points of comparison/agreement?  Can we distinguish between class, subclass and individual points of comparison/agreement based upon meaningful population density studies?
• Fourth, have the methods of analysis and the protocols been tested and validated to be declared suitable for its intended use?
• Finally, can we identify the sources of uncertainty in our analysis and can we quantify them so as to be able to report the uncertainty of our results in a metrologically responsible way?

A lot of forensic science is caught up in pattern recognition.  It includes the following general disciplines (note the list is not comprehensive):

1. Forensic Firearm Analysis
2. Forensic Toolmark Analysis
3. Forensic Odentology Analysis
4. Impression evidence such as Forensic Fingerprint Analysis, Forensic footprint analysis and Forensic Tire Mark Analysis
5. Forensic Hair Analysis
6. Forensic Fiber Analysis
7. Some forms of trace evidence analysis
8. Forensic Questioned Document analysis

The National Academy of Sciences report states that these areas of science have potential to be great tools of science and truth.  Yet, right now, the way they are studied, practiced and presented in court is based upon unproven and fundamental systemic flaws in that it is in fact based upon,  There are too many fundamental assumptions, much like the ones listed above, that have not been scientifically proven to be true or valid.

In other words, the jury is still out.