The problems with the modern practice of forensic firearm and toolmark analysis

The field is technically called a forensic firearm and toolmark analysis whose experts use an optical comparison microscope to examine the exemplar with the unknown recovered from the crime scene by a comparison criteria that is referred to as  “sufficiently consecutive matching striae” in an attempt to determine unique source.

 

The modern practice of forensic firearm and toolmark analysis leaves a lot to be desired. A lot. All pattern recognition endeavors do as well (e.g., latent fingerprint, handwriting, hair comparison, odontology-bite mark, lead bullet). It was subject to a few blog posts here not too long ago

• Forensic firearm and toolmark analysis is unscientific
• Another crime laboratory in danger: US Army

 

The notion that with no validated and standardized criteria that have been statistically proven to provide for uniqueness that an examiner who has no formal tribology background can bridge the large gap between the millions of firearms in that class down to not simply a subclass and not merely to a batch in a given day, but rather to a specific firearm that is exclusive of all other firearms on the planet earth is pure apocrypha. Fantasy really.

 

First we have to understand their historical method and their historical training:

Here is how they are typically trained to look at the casings (note that it is in 2D)

Basic nomenclature used by them:

• Striations: Lines or scratches left by the irregularities on the end of the tool as it scrapes away metal.
• Stamped or Impression markings: Toolmarks created by the striking of a cartridge primer by the firing pin, or the head of a cartridge case by the ejector.
• Stamped Striation Marks: Impressions left by the firing pin, ejector, and breechblock, if these parts have been manufactured by filing, or turned on a lathe.

 

In very brief this is what they are trained to do:

They test fire several rounds of ammunition of the same brand as crime scene ammunition through the suspect firearm. This makes the exemplars. They examine and compare test firings first. If they “match” perfectly, they examine the evidence recovered from the crime scene.

They begin their examination by first aligning the ejector and extractor marks properly. Then examine the firing pin impressions, noting the shape, position, depth and width. Then look for irregularities such as tips or depressions within the firing pin crater as a whole. Finally, look for identifiable machine toolmarks in the firing pin crater. Then examine the breech face marks, making notations regarding the location, length, width and contour of toolmark impressions. Extractor and ejector marks can also be examined and compared in a similar manner.

Here are straie in case you have never seen them:

 

There are many issues with this method of analysis and in particular its interpretation:

1. This is not the 19^th century. Firearms are now mass-manufactured, thus eliminating much of the “uniqueness” of a gun that came from being built by hand back then. There is very little hand filing if ever these days. The firearms are made using cutting tools made of High Speed Steel (HSS) (e.g., T-1, M-2, M-7, and M-42) or Carbide (made of Tungsten Carbide based alloys) such as the below:

Modern cutting tools do not wear anywhere near as rapidly as they once did. They are very rugged as they are constructed or coated with cobalt, or Titanium Nitride (TiN) which is added to HSS tools to increase hardness. Titanium Carbon-Nitride (TiCN) is added to HSS or carbide tools to increase hardness and prevent metal from adhering to the tool. Titanium Aluminum Nitride (TiAlN) increases tool life during high temperature cutting. Polycrystalline Cubic Boron Nitride (PCBN). So, if the tools that make the toolmarks in the firearm last longer and fail to deform more readily, by necessary implication the difference from firearm to firearm is less distinctive.

This all involves is the beautiful science known as tribology. In tribology there is an entire fascinating filed that involves the geometry of chip formation and Built-up edge (BUE formation) and the shear fracture left behind from the use of a tool such as this:

2. The core of the issue of real world identification is that as you fire a firearm each time it will “wear” down the barrel. Metal on metal under gas pressure will do that. This concept is well known in the true forensic science community of tribology where the literature is overwhelming that an identification of source is simply not possible if the firearm is within its first 300 or so discharges through the barrel. How does the government know that a given firearm is inside or outside of its first three hundred discharges? They don’t, they just assume. Or they don’t know this key fact.

3. There is also the concept of how precisely and under what conditions there can be toolmark transference. Toolmarks are created by metal cutting tools on the machined surfaces of firearms. These toolmarks are then transferred onto the cartridge case surfaces by the expansion of the cartridge brass in the barrel chamber, and breech face of the pistol slide. In the case of bullets, toolmarks are transferred onto the bullet surfaces by the obturation of the bullet into the rifled barrel lands and grooves. The modulus of elasticity of typical cartridge brass is 16×10^6 PSI, and the modulus of elasticity for typical gun steel is 29×10^6 PSI. Obviously the lower modulus of elasticity of brass causes the brass to expand under high pressure into the barrel chamber, and pistol breech face. Modulus of elasticity is defined as the ratio of the applied stress or force to the change in shape of an elastic body. NOT ALL TOOLMARKS TRANSFER. There is also the issue of frictional forces between the moving parts that can leave marks as well.

4. The interpretation is not standardized and non-validated. The simple point in fact is that despite what we in the lay world believe there is infrequently any criteria (e.g., 12 points of comparison) to make this all important call from class to uniqueness.

• Class Characteristics: Marks that all firearms of a given make and model will leave. These are coarser toolmark impressions.
• Subclass Characteristics: Fall under this category, are present in only some toolmarks. They are created during batch lot manufacturing of cutting tools, and are present on only those tools from a specific batch lot. These are also coarser toolmark impressions.
• Individual Characteristics: Marks specific to only one firearm. These are finer toolmark impressions.
• Accidental Characteristics: Non-reproducible toolmarks that may appear on one spent cartridge case, but not on any other spent cartridge case.

Again, no universal standards exist delineating these characteristics. If there are any, they are not statistically sound in terms of their validity.

Further, traditionally, breech face marks and firing pin impressions on a cartridge case head (and land and groove impressions on a bullet) are considered individual marks. But, not all breech face marks or land and groove impressions are in fact unique to that gun. Studies show that up to 20% matching striations may be all unique to that gun, but all common to several guns of the same make and model, or a combination of both. Again, there are no standards to determine which breech face marks (or land and groove impressions) are unique to that gun. Firearms examiners routinely just assume that any matching breech face marks identify the gun uniquely when looking at them in 2D.

5. The real problem for this science lies in the cognitive bias inherent in the examination itself. You, the examiner, fire a bullet from a known firearm. You retrieve it. You examine the seized evidence round that comes from a police investigation. Boy. Do you think that you are primed for cognitive bias?

Here is a case study of where the prosecution through PSP first examined a recovered firearm and called it a “match” to seized evidence. They had no photomicrographs, just an opinion by a high school graduate. It was reduced to this report. Note that it is specific as to source.

When challenged, they then went and took the unusual step of making photomicrographs off of their comparison microscope that resulted in this. Note it is in 2D (Of course, it was still considered a “match”) When pressed with precisely how they came up with this theory of uniqueness, it all came down to the most horrible phrase ever uttered in a courtroom “based upon my training, knowledge and experience.”

The top is the recovered evidence and the bottom is the exemplar from the recovered gun. Note they are not even at the same magnification or in the same orientation.

Instead, we used a Scanning Electron Microscope (the below results are from this same case and are of the same evidence and exemplar where PSP said that it was a “100% match”) was revealed in 3D by SEM to be far from it as seen below when examined in 3D and topographically examined when just looking at the firing pin position and characteristics in 3D alone. (Luckily for us in this case, this examiner had previously testified in another case that was provided to me by a PACDL member that if the firing pin position and characteristics were not a “match” then it was a “positive exclusion” of the firearm.)

6. All of this is why there is a big push for funding the studies to consider the issue of at what point do the similarities become enough to distinguish and form valid opinions of specificity. When studies involving white light inferometers were used to get a 3D image (instead of the 2D image that the comparison microscope uses), the revealing part was “that only 21% to 38% of the striae on pairs of bullets fired from the same” gun “matched.” Wow!

Here is a data sheet from one result:

Instead, what do we do? Often non-credentialed lay witnesses who were former traffic cops who are trained by others similarly lacking in an engineering or tribology background using a quasi-apprenticeship method, make the call of a “match.”  The term of art (or the call meaning the use of the word “match”) used by these practitioners is arrived at by these untrained microscopist looking with what amounts to their “highly trained set of eyeballs” at a three-dimensional object under a insufficiently magnified stereo comparison microscope which is a two dimensional representation to arrive at the conclusion that there is supposedly “sufficient consecutively matching striae” to warrant a conclusion that the unknown and the known are in fact a match.