Psychological Assessments in Legal Contexts: Are Courts Keeping "Junk Science" Out of the Courtroom?

What is it about?

In this article, we report the results of a two-part investigation of psychological assessments by psychologists in legal contexts. The first part involves a systematic review of the 364 psychological assessment tools psychologists report having used in legal cases across 22 surveys of experienced forensic mental health practitioners, focusing on legal standards and scientific and psychometric theory. The second part is a legal analysis of admissibility challenges with regard to psychological assessments. Results from the first part reveal that, consistent with their roots in psychological science, nearly all of the assessment tools used by psychologists and offered as expert evidence in legal settings have been subjected to empirical testing (90%). However, we were able to clearly identify only about 67% as generally accepted in the field and only about 40% have generally favorable reviews of their psychometric and technical properties in authorities such as the Mental Measurements Yearbook. Furthermore, there is a weak relationship between general acceptance and favorability of tools’ psychometric properties. Results from the second part show that legal challenges to the admission of this evidence are infrequent: Legal challenges to the assessment evidence for any reason occurred in only 5.1% of cases in the sample (a little more than half of these involved challenges to validity). When challenges were raised, they succeeded only about a third of the time. Challenges to the most scientifically suspect tools are almost nonexistent. Attorneys rarely challenge psychological expert assessment evidence, and when they do, judges often fail to exercise the scrutiny required by law.

Featured Image

Photo by Bill Oxford on Unsplash

Photo by Bill Oxford on Unsplash

Why is it important?

The core take-aways from this project: 1) Each year, hundreds of thousands of psychological assessments are used in court to aid judges in making legal decisions that profoundly affect people’s lives, such as child custody determinations, insanity at the time of a crime, sentencing determinations, competence to stand trial, and even eligibility for capital punishment. 2) Our new finding in this paper is that there is a lot of variability in the scientific strength of psychological tests. We were surprised to find many are scientifically unreliable. 3) Even though courts are required to screen out “junk science” per the United States Supreme Court and the Federal Rules of Evidence, one of our major findings is that nearly all psychological assessment evidence is admitted into court without even being screened. 4) In our open-access paper (free to the public at the publisher’s dedicated website www.PsychologicalScience.org/PSPI as of the time of this press briefing), we offer concrete and actionable advice for fixing these problems to members of the public interacting with psychologists in the legal system, lawyers and judges, and psychologists. Where can I (and anyone else, including attorneys) find information about psychological tests? Google Scholar or scholarly databases in libraries offer primary sources of information, which are scientific papers published about the validity and reliability of tests. Secondary sources (review or summary sources) are freely available in libraries and for purchase online. Examples include the Compendium of Neuropsychological Tests published by Oxford University Press and the Mental Measurements Yearbook (MMY) published by the non-profit Buros Center for Testing. The MMY offers professional, independent, candidly critical reviews of more than 2,800 commercially available tests. A la carte reviews for any individual test can be purchased from their Test Reviews Online website. [Conflict of Interest statement: Kurt Geisinger, co-author on the paper, directs the Buros Center]. What is good science as compared to “junk science”? The U.S. Supreme Court (Daubert v. Merrell Dow, 1993) and the Federal Rules of Evidence (FRE 702) suggest judges look to 4 factors (among other things) to make this determination and decide whether evidence is legally admissible in court. Specifically, regarding the method used by the expert: (1) Has it been tested? (2) Is it generally accepted by other experts in the same field? (3) Has it been peer-reviewed by experts in the field? and (4) What is its error rate? We used these criteria as a foundation for this paper.

Perspectives