For dozens of years, criminal prosecutions have relied on junk science. Forensic science, properly applied, can actually provide matches that identify suspects. But it’s not properly applied. In the hands of the DOJ, forensic evidence examination is a closed loop. Outside scientists have been granted access to the DOJ’s DNA work, but everything else — from fingerprints to hair samples — has been locked away in the government’s database.
Still, the DOJ insists its science is solid, something it bases on confirmation bias. The matches determined in its forensic labs are “scientifically certain” because the DOJ’s expert witnesses have said so in court. Not only are outside scientists locked out of examining evidence and forensic processes, but defense lawyers are as well.
The DOJ has finally decided to dial back its “scientific certainty” a bit by issuing guidance instructing its experts to not make this claim in court. This follows years of bogus matches being presented as sure things by forensic experts in court, leading to an unknown number of false convictions. This step back is a step forward for an agency that is mostly unwilling to admit to any mistakes or wrongdoing.
This small change is likely due to a damning report [PDF] issued by the President’s Council of Advisors on Science and Technology (PCAST) that asserts that the “scientifically certain” evidence prosecutors rely on is severely flawed.
The questions that DNA analysis had raised about the scientific validity of traditional forensic disciplines and testimony based on them led, naturally, to increased efforts to test empirically the reliability of the methods that those disciplines employed. Relevant studies that followed included:
• a 2002 FBI re-examination of microscopic hair comparisons the agency’s scientists had performed in criminal cases, in which DNA testing revealed that 11 percent of hair samples found to match microscopically actually came from different individuals;
• a 2004 National Research Council report, commissioned by the FBI, on bullet-lead evidence, which found that there was insufficient research and data to support drawing a definitive connection between two bullets based on compositional similarity of the lead they contain;
• a 2005 report of an international committee established by the FBI to review the use of latent fingerprint evidence in the case of a terrorist bombing in Spain, in which the committee found that “confirmation bias”—the inclination to confirm a suspicion based on other grounds—contributed to a misidentification and improper detention; and
• studies reported in 2009 and 2010 on bitemark evidence, which found that current procedures for comparing bitemarks are unable to reliably exclude or include a suspect as a potential biter.
Beyond these kinds of shortfalls with respect to “reliable methods” in forensic feature-comparison disciplines, reviews have found that expert witnesses have often overstated the probative value of their evidence, going far beyond what the relevant science can justify. Examiners have sometimes testified, for example, that their conclusions are “100 percent certain;” or have “zero,” “essentially zero,” or “negligible,” error rate. As many reviews—including the highly regarded 2009 National Research Council study—have noted, however, such statements are not scientifically defensible: all laboratory tests and feature-comparison analyses have non-zero error rates.
Despite these conclusions, law enforcement forensic scientists, along with the FBI and DOJ are promising to continue using junk science to convict people. The Attorney General’s response to the report is basically, “Thanks for all the hard work, but we’re not changing a thing.”
In a statement reported by the Wall Street Journal, Attorney General Loretta Lynch said that the agency remains “confident that, when used properly, forensic science evidence helps juries identify the guilty and clear the innocent, and the department believes that the current legal standards regarding the admissibility of forensic evidence are based on sound science and sound legal reasoning.” As such, she said, while “we appreciate their contribution to the field of scientific inquiry, the department will not be adopting the recommendations related to the admissibility of forensic science evidence.”
The FBI’s response [PDF] is no more enthusiastic. Its one-page blast claims the entire PCAST report is flawed. It also asserts there are “multiple studies” that back up its forensics work. However, the FBI turned down an invitation to participate in the PCAST study and one of the PCAST members points out that the studies the FBI claims contradict the PCAST findings actually do no such thing.
Asked about the FBI’s complaints, Eric Lander, co-chair of the presidential council and president and founding member of the Broad Institute of MIT and Harvard, a biomedical research group, told The Intercept that the FBI is mistaken. “Neither report says that proficiency testing be used to estimate the ‘error rate’ of forensic methods,” he wrote in an email, and both reports agree that examiners should be subject to proficiency tests. And Lander said he is “not aware” of what studies the FBI believes were ignored by the report. “We specifically received FBI’s input on studies to consider and we did so.”
[…]
Lander wrote that the presidential council did in fact review the six studies the FBI complains that it missed. “However, these studies are clearly not empirical studies ‘providing support for foundational validity,’” he wrote. “Indeed, only one of the papers even reports an empirical study of current forensic method at all!”
But the most defensive response [PDF] to the PCAST study has come from the American Congress of Forensic Science Laboratories. Unsurprisingly, the group isn’t pleased that its conclusions and methodology have been questioned. Before addressing the shortcomings it feels the study contains, the ACFSL first tries to claim the whole thing is just a politically-motivated attack on the good people in law enforcement.
Interestingly, the PCAST report comes during a presidential administration that has demonstrated a deep sensitivity to the needs and demands of trial attorneys, criminal defendants, and advocates of sweeping criminal justice reform. Future administrations may take a different approach, tending to champion positions traditionally held by police and prosecutors. We have no opinion in these matters. But these swings in ideological perspective cause commensurate changes in how forensic science and its role in our criminal justice system are perceived. In the current political climate, forensic science is looked upon with far more suspicion and, in some cases, disdain than would be the case in other political circumstances. And because forensic science is both expected and apt to remain independent of these political currents, it is vulnerable to being misportrayed and even bullied in a way that compromises its occupational stability.
It’s somewhat interesting that a field where evidence has been used to deprive innocent people of their freedom — and whose courtroom assertions are treated as nearly-unassailable despite a long history of uncorrected errors — can somehow be bullied by a report both the DOJ and FBI have already indicated they’re going to ignore.
It also attacks some of the contributors for their work with the Innocence Project (Eric Lander) and their previous assertions about the intrusiveness of DNA collections (Tania Simoncelli) — all the while claiming it has no interest in “disparaging” the authors of the study. Even in this, the forensic scientists’ sample size is way too small. Attacking Lander for his Innocence Project work conveniently overlooks his considerable contributions to the scientific community.
[Innocence Project’s Chris] Fabricant said the congress’ assertions were absurd. “To suggest that the leading scientists in the country would cash in their credibility to do – what? What possible agenda could they be pursuing except scientific validity?” he asked. “I’d like to know what agenda they propose is being driven, and how somebody like Eric Lander – who mapped the human genome — is going to preside over a process that is intended to undermine the criminal justice.”
These responses indicate business — as lousy and inaccurate as it is — will continue as usual until the DOJ is forced to confront these issues, either by litigation or legislation. Not exactly heartening news for citizens who don’t like being locked up for crimes they didn’t commit.