I was recently asked a question which is basically this: “Justin, why don’t you stipulate to any forensic science result ever?”

My answer is simple. I haven’t found a single analytical test or a single forensic science result that does not have some area of legitimate inquiry whether it is in the data (the test itself), the QC, the traceability of the standards, the QA performed, or the technicians, the expert or the very foundational validation of the technique employed. I know that is a bold statement, but it is true. Having said that, the question becomes does the issue matter to a jury? Now that is the issue. These cases are tried by purposeful design to folks (meaning the jury) who have absolutely no idea of the subject matter. If the science was going to be evaluated by other scientists, then it would a lot easier and a lot fewer miscarriages would occur, in my humble opinion. However, they are not. This is where someone like me comes in who can use transferable concepts to bridge the gap between the world of science and the world of the jury.

A paper audit can only show so much. Even the most detailed instructions that I see still allow for some individual discretion by the user. There may very well be a disconnect between the existence of a truly validated method with robust instructions and actual execution that is not traceable by the paperwork. Knowledge of protocol and protocol adherence are two entirely different concepts. There is also the “x-factor” which is can you, as a witness who is likely very nervous in front of a jury, explain what you do, how you do it and how it results in a specific (or really a near specific) qualitative measurement with a quantitative measurement that is very free of calibration and bias error SO THAT THE JURY CAN UNDERSTAND IT.

Perhaps there will someday where crime laboratories that have truly validated methods for what they do such as what exists in the GLP or EPA regulated world using perhaps the USP guidelines of well-designed validation experiments that investigate and prove at a minimum accu­racy (bias), pre­ci­sion (calibration), speci­ficity, limit of detection, limit of quantitation, lin­ear­ity and range, rugged­ness, robust­ness and uncertainty in their qualitative and quantitative measurement that are all the while verified on the particular instruments used by the particular operators using them. Perhaps someday they will publish and make available these efforts in a true validation report that features:

• Objec­tive and scope of the method (applic­a­bil­ity, type).
• Sum­mary of methodology.
• Type of com­pounds and matrix.
• All chem­i­cals, reagents, ref­er­ence stan­dards, QC sam­ples with purity, grade, their source or detailed instruc­tions on their preparation.
• Pro­ce­dures for qual­ity checks of stan­dards and chem­i­cals used.
• Safety pre­cau­tions.
• A plan and pro­ce­dure for method imple­men­ta­tion from the method devel­op­ment lab to rou­tine analysis.
• Method para­me­ters.
• Crit­i­cal para­me­ters taken from robust­ness testing.
• List­ing of equip­ment and its func­tional and per­for­mance require­ments, e.g., integration perimeters, base­line noise and col­umn tem­per­a­ture range.
• Detailed con­di­tions on how the exper­i­ments were con­ducted, includ­ing sam­ple prepa­ra­tion. The report must be detailed enough to ensure that it can be repro­duced by a com­pe­tent tech­ni­cian with com­pa­ra­ble equipment.
• Sta­tis­ti­cal pro­ce­dures and rep­re­sen­ta­tive calculations.
• Pro­ce­dures for QC in rou­tine analy­ses, e.g., sys­tem suit­abil­ity tests.
• Rep­re­sen­ta­tive plots, e.g., chro­matograms, spec­tra and cal­i­bra­tion curves.
• Method accep­tance limit per­for­mance data.
• The expected uncer­tainty of mea­sure­ment results.
• Cri­te­ria for revalidation.
• The person(s) who devel­oped and val­i­dated the method.
• Ref­er­ences.
• Sum­mary and conclusions.
• Approval with names, titles, date and sig­na­ture of those respon­si­ble for the review and approval of the ana­lyt­i­cal test procedure.

Maybe someday the process (sample preservation, sample prep, instrument running, data collection, and data interpretation) will be done by folks who are fully trained and fluent in the underlying theory of the technique they are employing as opposed to just simply the process employed.

Maybe someday all of the instruments will be control charted.

Maybe someday there will be truly blinded proficiency tests  on each analyst on a relevant sample that contains a target analyte that is something that can be frequently misinterpreted that is intermixed with routine samples run with very tight acceptance criteria with the quantitative result not dependent on the results of other laboratories but rather on the targeted and designed value.

Maybe someday the Quality Assurance officer will be a person who is more qualified in the theory, have demonstrated greater proficiency it the assay, and can actually employ the technique better than the person who runs the assay that they are called upon to evaluate. A lot of what I see is the QA officer is not technically trained in the theory or the process of the assay they are called upon to double-check. Such a regime makes QA little more than a rubber stamp.

Maybe someday there will be verifiable information that is kept by the laboratory that they will readily make available that justifies their data and their opinions. I find that the difficulty in getting the data is due to the typical crime laboratory being ultimately administratively lead by a sworn police officer who has no technical training in science whatsoever. A promoted traffic cop, if you will. This person (who is not steeped in the scientific culture of openness or transparency or has not even been meaningfully exposed to science likely has no idea of the process involved and what constitutes relevant information) has very much a “us-versus-them” mentality. Having said that, I personally don’t have any problem getting whatever data I want because I can explain exactly what I am looking for, why it could matter in a given case, and how it can make a difference in the validity of the measure or opinion offered. I frequently tell my colleagues that if you don’t know what it is and why it matters, then don’t ask for it. I usually do it with a smile first by sending my subpoena to the laboratory directly followed up with a very nice mannerly introductory phone call to their subpoena compliance officer and the person who is in charge of the science of the laboratory. After chatting with them and getting them to understand that while I may have a JD after my name and although I freely admit that I am not a PhD that I am fairly fluent in the science of it all, they generally recognize that I am not asking them to do all of this work for no purpose or as an act of meaningless busywork all wrapped in a game of “got you” but instead in a wholly legitimate effort to meaningfully examine the validity of the opinion expressed. If they choose not to be open, then that is their choice. It is America after all. It just will be a costly and tough process for them as I will force them to be wholly open by going into Court.

Maybe, just maybe, I would consider adjusting my view.