Judge Probably Understands Probability
Carrie R. Valentine, Ph. D (biochemistry)
A recent New York Times article reports that Italy’s Court of Cassation has overturned the acquittal of Amanda Knox for the murder of Meredith Kercher.[1]
(Pictured Above: Amanda Knox post-release)
Although Schneps and Comez acknowledge that the Court has not yet publicly explained the motivations behind the ruling, they suggest that the failure of the judge to understand probability is the reason for overturning the acquittal. They report that the judge refused to allow a re-test of the knife with more recent DNA analysis techniques in 2011, the year of acquittal. The reasoning of the acquitting judge was reported to be “If the scientific community recognizes that a test on so small a sample cannot establish identity beyond a reasonable doubt, he explained, then neither could a second test on an even smaller sample.”
(Pictured above: The room where Meredith Kercher, the 21 year old British university exchange student from South London was found)
Schneps and Comez argue that repeated testing will determine accuracy just as flipping a coin will determine error if a 50/50 ratio is not found after many trials “whatever concerns the judge might have had regarding the reliability of DNA tests”.
However, the reasons for doubting the reliability of the DNA tests do bear on whether re-testing the samples could provide valid information about DNA matches. The report of the court-appointed outside experts in DNA analysis, Carla Vecchiotti and Stefano Conti[i], analyzed all the DNA matches submitted by the forensic scientist for the prosecution and concluded that only one was based on sufficient DNA to constitute a match. This sample was the DNA of Amanda Knox on the handle of a knife; however, the blood of the victim was not verified by the experts to be on the knife blade.
The conclusion of the technical consultant for the prosecution that the victim’s DNA was found on the knife blade was rejected by the experts for many reasons including inadequate analysis of low copy number DNA and the possibility of contamination from analyzing the specimen in the same workspace as samples of the victim’s DNA. International standards for the validity of interpreting results from DNA amplifications were not met with these samples.
For other samples, the experts criticized the interpretation of the analyst because poor DNA yield required interpretation of the results and potential genotypes were either included or excluded with no objective criteria. Because the analyst had knowledge of which genotypes were present in the defendant’s DNA, the choices made were those that produced a match; the obvious conclusion is that the interpretation of poor quality data was biased.
The experts were asked to determine whether they could re-test any of the samples and they did test several specimens to determine the quantity of DNA recovered. They concluded that no samples recovered sufficient DNA to allow them to proceed to amplification of the DNA because the amount recovered was less than required for reliable results according to international standards. In addition, the experts criticized the management of the crime scene for failure to control access to prevent the contamination of objects with blood from the victim.
(Pictured above: The knife found in Raffaele Sollecito’s apartment)
Analyzing a poor quality specimen can be compared to tossing a coin in such dim light that heads cannot be clearly distinguished from tails and the shadows must be ‘interpreted’, subject to bias of the expected results. Why continue tossing the coin when heads cannot be distinguished from tails beyond reasonable doubt? A better quality test is needed, not more repetitions.
The judge’s comment that the samples are now “even smaller” indicates his understanding that each attempt to remove DNA from the specimen reduces the amount of DNA remaining on it.
The issue seems to be whether techniques have improved in the recovery and analysis of DNA to allow accurate analysis on the small amounts remaining on these specimens. Even this argument does not address the issue of possible contamination. However, until the Court of Cassation releases its reasoning for the ruling, it seems to be premature to conclude that the judge erred because of his lack of understanding of probability.
[1] Schneps L. and Comez C. Justice Flunks Math. New York Times March 6, 2013. http://www.nytimes.com/2013/03/27/opinion/when-judges-cant-do-math-justice-suffers.html
[2] Komponisto, katy_did. 2011. The Conti-Vecchiotti report: the experts’ reports discrediting the DNA evidence against Amanda Knox and Raffaele Sollecito, translated into English. Retrieved April 30, 2012 from http://knoxdnareport.wordpress.com/
_________
Dr. Carrie Valentine graduated with a BA in Chemistry from the University of Oregon-Eugene. She was conferred her PhD in Biochemistry in 1975 at the Massachusetts Institute of Technology. She performed her post doctoral studies at Montana State University, Bozeman and the University of California, Davis. She formally retired from academic life in 2008. She is currently part of the Online Faculty for Grand Canyon University, Phoenix, AZ. From 1997 to 2008, she served as a Research Microbiologist, Division of Genetic and Reproductive Toxicology, National Center for Toxicological Research (NCTR), Jefferson, Arkansas. From 1992 until 1997, she was a Staff Fellow at the Division of Genetic Toxicology, NCTR, Jefferson, AR. From 1990 until 1992, she was an Assistant Professor of Pediatrics at the University of Arkansas for Medical Sciences, Little Rock, AR. She has lectured numerous times as well as published several peer review articles.
The McShane Firm | DUI Lawyers
Pennsylvania DUI Blog
brmull says:
Re: “Other samples that clearly showed the victim’s blood as well as Amanda Knox’s blood were discounted because the forensic specimens were analyzed in the same workspace as samples from the victim.”
I don’t recall anywhere that DNA results in this case were ignored by the forensic police because of laboratory contamination. Dr. Stefanoni said there was no evidence of such contamination in this case.
You argue that a result shouldn’t be considered reliable if it doesn’t meet some arbitrary guideline for collection and handling, which is the crux of the defense’s argument. But with all chain of custody issues, the proper way to handle it is to consider whether contamination is a plausible explanation for the results obtained, and then weigh that together with all the other evidence in the case in order to reach a verdict.
Michelle Moore says:
Wow, this is a really great article and I am very thankful you wrote it. I hope that the parents of the victim Meredith Kercher will read this.
They are looking for answers and I think this could help them. They have been fed BS by a corrupt Perugia Prosecutor, Giuliano Mignini. He had motive and power.
Carrie R. Valentine says:
To BRMull:
Thank you for your knowledgeable response to this post. Dr. Stefanoni was the forensic scientist for the prosecution; the experts rejected the technical consultant’s conclusion that the victim’s blood was on the knife blade for the following reasons:
“…we do not accept the conclusions about the certain attribution of the profile detected in Sample B [knife blade] to the victim Meredith Susanna Cara Kercher since the genetic profile, as obtained, is unreliable in that it is not supported by scientifically valid analytical procedures. Neither, as previously explained, can it be excluded that the result obtained from this sample may derive from contamination phenomena occurring at any stage of the collection and/or handling and/or analytical procedures performed.” Page 105, 106 of the experts’ report referenced in the article above.
The standards for reliable interpretation of genetic profiles are not arbitrary and are itemized and explained in detail in the Conti-Vecchiotti report to the court.
Carrie R. Valentine
DaveAlexander says:
Michelle Moore, could you please provide proof that this happened. I can guarantee you that the Kercher’s know a hell of a lot more about this case than you do.
Angela Smitherman says:
oh, right Dave, of course. The Kerchers know a “lot” about this case. So much that John Kercher wrote an error-filled book about it. Like, for example, in the chapter about the original trial where he states the prosecution put forth two different scenarios about what happened – specifically with respect to whether Meredith arrived home to an empty house or not (Kindle locations 2714 vs. 2750). John Kercher knows so much that he failed to include either Sollecito OR actual, sole murderer Rudy Guede in his Appendix of key case individuals! And he stated as fact that his petite daughter could not have been overpowered, sexually assaulted and murdered by one single man. Like this hasn’t already happened hundreds of thousands of times. And what convinced him that Meredith couldn’t have been killed by one taller, stronger man. Her orange belt in karate which, to put it in school terms, is the equivalent of finishing the first grade.
Harry Rag says:
Dr. Patrizia Stefanoni, Dr. Renato Biondo, the head of the DNA Unit of the Scientific Police, Professor Francesca Torricelli, former Caribinieri General Luciano Garofano and Professor Novelli have all confirmed that Meredith’s DNA was on the blade of the knife. Even Greg Hampikian and Elizabeth Johnson have acknowledged that the DNA on the blade of the knife was consistent with Meredith’s DNA.
Conti and Vecchiotti never proved there had been any contamination.
Alberto Intini, the head of the Italian police forensic science unit, pointed out that unless contamination has been proved, it doesn’t exist.
Professor Novelli also pointed out that contamination has to be proved:
“The contaminant must be demonstrated, where it originated from and where it is. The hook contaminated by dust? It’s more likely for a meteorite to fall and bring this court down to the ground.”
Conti and Vecchiotti didn’t visit the laboratories of the scientific police or ask about their cleaning procedures. They didn’t know that the negative tests had been filed with another judge. They didn’t know that Dr Stefanoni analysed the traces on the knife six days after last handling Meredith’s DNA or that she last handled Sollecito’s DNA 12 days before she analysed the bra clasp. This means that contamination couldn’t have occurred in the laboratory. Meredith had never been to Sollecito’s apartment, so contamination away from the laboratory was impossible.
Galati made the following common sense observation in his appeal:
“It is evident that the “non-exclusion” of the occurrence of a certain phenomenon is not equivalent to affirming its occurrence, nor even that the probability that it did occur.” (The Galati appeal, page 57).
He goes on to explain that unless there is proof of contamination of the knife and bra clasp, you can’t simply claim there was in order to nullify this evidence:
“…if one is not able to [67] affirm where, how and when they would have happened, they cannot enter into a logical-juridical reasoning aimed at nullifying elements already acquired, above all if scientific in nature.” (p57).
Conti and Vecchiotti regarded the downstairs flat as part of the crime scene even though no crime was committed there.
Worst of all, they didn’t carry out a new test on the knife despite the fact they were specifically instructed to do so and there are a number of laboratories that have the technology to carry out a test on the remaining the DNA.
Incidentally, Vecchiotti was appointed by a judge at the Cosenza court and the judge didn’t accept her findings. Other experts were appointed and they found incriminating DNA evidence that she had missed. The suspect admitted his guilt.
It should be noted that the Italian Scientific Police follow the guidelines of the ENFSI – European Network Forensic Science Institutes. Dr. Stefanoni observed that they followed these specific guidelines whereas Conti and Vecchiotti basically picked and mixed a random selection of international opinions:
“We followed the guidelines of the ENFSI, theirs is just a collage of different international opinions”.
Carrie R. Valentine says:
To Harry Rag:
Thank you for a highly knowledgeable and pertinent post. You quote Professor Novelli as saying,
“The contaminant must be demonstrated, where it originated from and where it is. The hook contaminated by dust? It’s more likely for a meteorite to fall and bring this court down to the ground”.
The primary source of contamination in a PCR laboratory is dust – from the aerosol released from opening the amplified PCR tubes. For this reason the opening of the tubes must be in a location removed from the processing of specimens and access between the two work areas controlled. I do not know what provisions were in the forensic laboratory to control this type of contamination. However, if negative controls were run and were negative, this is good evidence against contamination, which was not provided to Conti and Vechiotti.
Carrie R. Valentine
Alexei K. says:
First, I would like to point an inaccuracy in Prof. Valentine’s account: the prosecution’s expert never claimed that there was blood on the knife. At the first trial, in 2009, she testified that she had found no blood on the blade. The absence of blood has been accepted by both sides.
Second, I would like to point out that, contrary to Harry Rag, Hampikian and Johnson did not agree that DNA was definitely the victim’s. They did point out, as did Conti and Vecchiotti, that the “scientific police” lab was not equipped for low-template DNA testing at that time. Novelli’s claim cited by Harry Rag was about the bra clasp and not about the knife so is irrelevant to this discussion. Harry Rag makes no mention of the experts supporting defense on the DNA, such as Sarah Gino, Adriano Tagliabracci and three others.
Third, the public deserves to know that Harry Rag and Leila Schneps are frequent posters to the two anti-Knox sites which Nina Burleigh has recently labeled “The Amanda Knox Haters Society”.
Carrie R. Valentine says:
With regard to the following comment by Harry Rag:
“Worse of all, they didn’t carry out a new test on the knife despite the fact that they were specifically instructed to do so and there are a number of laboratories that have the technology to carry out a test on the remaining DNA.”
According to their report, Conti and Vecchiotti were charged by the court to ascertain:
1). “whether it is possible, by means of a new technical analysis, to identify the DNA present on items 165b (bra clasp) and 36 (knife), and to determine the reliability of any such identification” and
2). “if it is not possible to carry out a new technical analysis, shall evaluate, on the basis of the record, the degree of reliability of the genetic analysis performed by the Scientific Police on the aforementioned items, including with respect to possible contamination.”
On each area of the knife (p.9) and clasp (p. 11) indicated by Dr. Stefanoni, they performed generic tests for the presence of blood, which were all negative. Samples were taken from each of the areas designated previously on the knife blade (p. 10) and clasp (pp. 12,13) for subsequent extraction of DNA (pp. 13-15) and quantification of DNA (pp. 16-26).
The results were summarized on page 20:
“With regard to samples A-M, the real-time PCR analysis reveals a very low initial amount of DNA, with the maximum measurable DNA concentration at 0.005 ng/μL (5 pg/μL) in sample “I” [between blade and knife handle, not analyzed by Stefanoni]. The following table reports the measurements relative to the individual samples examined in triplicate. It should be noted that the analysis conducted on the internal control (IPC) does not reveal anything suggestive of an inhibition of the PCR reaction. In fact, the value of Ct relative to the IPC turns out to always lie in the interval between 28-31, in line with expectation. In other words, from the analysis performed one can rule out the presence of inhibitors of the PCR reaction capable of compromising the DNA amplification in the individual samples analyzed. The concentration of DNA in the majority [including knife blade] of the samples examined turned out to be undeterminable (Undetermined).”
Consequently they concluded (pp. 32, 33) that:
“Taking note that no DNA suitable for further laboratory investigations (amplification, electrophoresis) was present either on the swabs [tamponature] (A-B-C-D-E-F-G-H-I) taken from Exhibit 36 (knife) or on those (L-M) taken from Exhibit 165B (hooks of the bra), the experts verbally informed the consultants of the parties that they would proceed to a detailed examination of the Technical Report drawn up by the Scientific Police, as according to the task formulated during the assignment hearing.”
Although it might be possible to argue that other experts would have been able to recover more DNA or would have been able to analyze such small quantities of DNA, I credit these experts for declining to analyze specimens with insufficient DNA for reliable results based on their expertise. In my view, these experts fulfilled their charge to the best of their ability.
Carrie R. Valentine
TomG says:
To Dave Alexander
According to John Kercher’s book “Meredith” he cites multiple instances where he and his family were badly informed by their own lawyer. He also indicates clearly that he accessed the website TJMK for his sources of information. The result being a very ill-informed book. I think it is fair to say that the information that the Kerchers received from Francesco Maresca was very selective if the book is anything to go by.
Ken Peterson says:
I often see reports of experts who seem to understand the math of practically everything and the practicality of nearly nothing.
Repeating trials serve to improve the confidence in the outcome only if they are “fair trials” and “everything else is equal”, to use well known phrases.
The conclusions are only sensible if they have to do with whatever is actually at issue.
The outcome is not sensible if it is based on testing something other than the material in question or if the investigators do not know what they were actually testing.
Some years ago a large and highly respected study of the effects of drugs on hypertension was published. The conclusions were supported by impressive work on the probabilities but it seems the investigators did not know what substances were actually tested. I think of it as the great Australian National Cow Plop study because, as far as I can tell, the docs might have been dosing their patients with tablets punched out of cow manure. There was no way to tell what they actually used. (It was published in a leading U.S. general medical journal. I have never seen even one comment about the methods of the study or of the conclusions.)
Fine math can be done on non-sense numbers and the numbers will still be non-sense.